Julia calls the old method after method is redefined

General Usage
1

Hi all. I’m not sure if this is a bug or if I’m missing something. Essentially, Julia keeps calling the old method after I have redefined it. I have a vague feeling that this may have to do with issue 265 or its fix, and it may well be the intended behavior, but what has me doubt this is that most of the time redefining methods (e.g., using Revise.jl) works just fine. Unfortunately my repro is not very minimal, because only a very specific situation seems to trigger the issue.

The following setup is needed to reproduce the issue:

   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.2 (2017-12-13 18:08 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-w64-mingw32

julia> Pkg.clone("https://github.com/s-broda/ARCH.jl")
INFO: Cloning ARCH from https://github.com/s-broda/ARCH.jl
INFO: Computing changes...
INFO: No packages to install, update or remove

julia> Pkg.checkout("ARCH", "repro")
INFO: Checking out ARCH repro...
INFO: Pulling ARCH latest repro...
INFO: No packages to install, update or remove

julia> using ARCH
INFO: Recompiling stale cache file C:\Users\broda\.julia\lib\v0.6\ARCH.ji for module ARCH.

julia> T=10^4; spec = GARCH{1, 1}; coefs = (1., .9, .05); srand(1); data = simulate(spec, T, coefs); ht = zeros(data);

julia> try AM=selectmodel(GARCH, data) end

Now call the problematic method:

julia> ARCH.arch_loglik!(spec, data, ht, [coefs...])
-29182.153014042997

Let’s redefine the method (I’ve actually discovered this using Revise.jl, but it’s not specific to that):

julia> @eval ARCH arch_loglik!{p, q, T1<:FP}(M::Type{GARCH{p,q}}, data::Vector{T1}, ht::Vector{T1}, coefs::Vector{T1}) = 1
WARNING: Method definition arch_loglik!(Type{ARCH.GARCH{p, q}}, Array{T1<:AbstractFloat, 1}, Array{T1<:AbstractFloat, 1}, Array{T1<:AbstractFloat, 1}) in module ARCH at C:\Users\broda\.julia\v0.6\ARCH\src\GARCH.jl:5 overwritten at REPL[7]:1.
arch_loglik! (generic function with 1 method)

And call it:

julia> ARCH.arch_loglik!(spec, data, ht, [coefs...])
-29182.153014042997

So Julia still calls the old method. Even if I do

julia> Base.invokelatest(ARCH.arch_loglik!, spec, data, ht, [coefs...])
-29182.153014042997

Strangely though,

julia> @code_llvm ARCH.arch_loglik!(spec, data, ht, [coefs...])

; Function Attrs: uwtable
define i64 @"julia_arch_loglik!_64535"(i8**, i8** dereferenceable(40), i8** dereferenceable(40), i8** dereferenceable(40)) #0 !dbg !5 {
top:
  ret i64 1
}

My versioninfo:

julia> versioninfo()
Julia Version 0.6.2
Commit d386e40c17* (2017-12-13 18:08 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, haswell)

Any advice would be appreciated. If this is a bug, then I’m happy to file an issue.
Thanks!
Simon

2

Because the function already exists, you can replace the method via
function Arch.arch_loglik!... and thereby avoid @eval.
I don’t know if this will help solve your problem, but something may shake loose.

3

Every time something like this has happened to me it has been because the method I re-defined wasn’t actually the method that was being called (i.e. because I had the wrong type signature somewhere). I looked at your code and that doesn’t seem to be the case here, but perhaps there’s something I’m missing?

I use Revise all the time and find it to be extremely reliable, so there’s probably something fishy about this particular instance, but I wasn’t able to spot whatever it is by glancing at it casually.

4

Thanks both for your input.

@jlapeyre I’m using eval here to replicate what Revise does. It’s not so much a problem for my code, but rather for my workflow. I change the file, and the changes don’t take effect.

@ExpandingMan This example doesn’t in fact use Revise. Also, the function only has one method. More worryingly, the behavior depends on running the code wrapped in the try block. Without it, I get the expected behavior.

That’s why I believe this may be a bug.
Cheers
Simon

5

Oooh, I see. Yes, that does dramatically increase the chances that it might be a bug in my mind. Any chance you are able to try this on 0.7?

Also, for the hell of it, can you try it with Revise?

6

I’ve tried with revise, that’s how I ran into it. I’ll try 0.7 when I get a chance, probably after the weekend.

7

@ExpandingMan

I can confirm that the problem persists with the latest Windows nightly (Commit 78c7d87369*), and also on Linux. I’ve also been able to minimize the repro somewhat; no more cloning, or dependence on external packages, or @eval.

Setup:

abstract type VolatilitySpec end
struct GARCH{p,q} <: VolatilitySpec end
struct ARCHModel{VS<:VolatilitySpec}
    ARCHModel{VS}() where {VS}=new()
end
ARCHModel(VS::Type{T}) where {T<:VolatilitySpec} = ARCHModel{VS}()

loglikelihood(am::ARCHModel{T}) where{T} = arch_loglik!(T)

function arch_loglik!(M::Type{GARCH{p,q}}) where {p, q}
    return 5.
end

function selectmodel()
    res = Array{ARCHModel, 2}(4, 4)
    for p = 0:3, q = 0:3
        res[p+1, q+1] = ARCHModel(GARCH{p, q})
    end
    crits = loglikelihood.(res)
end

Then:


julia> arch_loglik!(GARCH{1, 1})
5.0

julia> selectmodel();

julia> arch_loglik!{p, q}(M::Type{GARCH{p,q}}) = 1
arch_loglik! (generic function with 1 method)

julia> arch_loglik!(GARCH{1, 1})
5.0

julia> @code_llvm arch_loglik!(GARCH{1, 1})

; Function Attrs: uwtable
define i64 @"julia_arch_loglik!_63633"(i8**) #0 !dbg !5 {
top:
  ret i64 1
}

I believe this might indeed be a bug.

8

Something is very wrong here, but I can’t yet tell what. For one thing, it seems to be overwriting the method without even giving a warning. The fact that this behavior only seems to occur when selectmodel is called is beyond bizarre.

I don’t have the time to dig any deeper into this right now, but at this point I suggest you open an issue on github. Even if it’s not a bug, the behavior is so strange that it definitely warrants investigation.

9

https://github.com/JuliaLang/julia/issues/26820

1 Like