World age error when parsing function from String

General Usage
1

Hi all,

I am using DataFrames and wanting to specify intra-row constraints in YAML, which can then be parsed and applied to each row of the data.

As an example, the following code works:

d = DataFrame(
    id = UInt.([1,2,3]),
    dob=Date.(["1992-10-01", "1988-03-23", "1983-11-18"]),
    date_of_marriage=[Date("2015-09-13"), missing, Date("1981-11-01")]
)

s = "(r) -> r[:date_of_marriage] > r[:dob]"   # Constraint. To be loaded from a YAML file.

f = eval(parse(s))
for r in eachrow(d)
    result = f(r)
    ismissing(result) && continue  # f returns missing
    result && continue  # f returns true
    println(r)
end

However, when I include the same code in a function it fails. That is, this code:

function myfunc()
    d = DataFrame(
        id = UInt.([1,2,3]),
        dob=Date.(["1992-10-01", "1988-03-23", "1983-11-18"]),
        date_of_marriage=[Date("2015-09-13"), missing, Date("1981-11-01")]
    )

    s = "(r) -> r[:date_of_marriage] > r[:dob]"

    f = eval(parse(s))
    for r in eachrow(d)
        result = f(r)
        ismissing(result) && continue  # f returns missing
        result && continue  # f returns true
        println(r)
    end
end

…has the following error:

ERROR: MethodError: no method matching (::##1#2)(::DataFrames.DataFrameRow{DataFrames.DataFrame}) The applicable method may be too new: running in world age 21858, while current world is 21859.

I have read past discussions of the world age problem but I must admit I just don’t get it. Can someone please explain the problem with the example above?

Also, tips welcome on the above implementation. Ideally I’d prefer to parse :date_of_marriage > :dob instead of r[:date_of_marriage] > r[:dob], but my metaprogramming skills are pretty much non-existent.

Thanks,
Jock

2

I can’t provide too much actual insight but I implemented something similar a few days ago and got it working with what in this code should correspond to result = @eval $f($r).

3

As for meta-programming, this might give you a starting point:

function interpolate_values!(e::Expr, d)
    for k = 1:length(e.args)
        e.args[k] = interpolate_values!(e.args[k], d)
    end
    return e
end

function interpolate_values!(s::Symbol, d)
    if haskey(d, s)
        return d[s]
    end
    return s
end

interpolate_values!(x, d) = x

d = Dict(:a => 1, :b => 2, :c => 3)
r = "b < a * c"
expr = parse(r)
dump(expr)
interpolate_values!(expr, d)
dump(expr)
eval(expr)

Notice that this doesn’t go through a function and thus has no world age problems. Since all values have been interpolated into the Expr it’s fine to just eval it in a global context.

Someone who is skilled with MacroTools can probably replace the interpolate_values! function with a one-liner.

4

The evaluated function in your function only becomes available after execution hits the global scope again. Only then the method table gets updated. Consider:

julia> f(i) = (@eval _f()=$i; _f())                                                                                                                                      
f (generic function with 1 method)                                                                                                                                       

julia> f(1)                                                                                                                                                              
ERROR: MethodError: no method matching _f()                                                                                                                              
The applicable method may be too new: running in world age 21849, while current world is 21850.
Closest candidates are:
  _f() at REPL[1]:1 (method too new to be called from this world context.)                                                                                               
Stacktrace:                                                                                                                                                              
 [1] f(::Int64) at ./REPL[1]:1                                                                                                                                           

julia> f(2) # now it accesses the evaled definition from before                                                                                                          
1                                                                                                                                                                        

julia> f(3) # again, it accesses the evaled definition from before                                                                                                          
2

(If you use a anonymous function in that example instead you get errors as the anonymous function effectively creates a new function on each invocation.)

Use Base.invokelatest, which updates the global method table before running a function:

julia> f(i) = (@eval _f()=$i; Base.invokelatest(_f))                                                                                                                     
f (generic function with 1 method)                                                                                                                                       

julia> f(1)                                                                                                                                                              
1                                                                                                                                                                        

julia> f(2)                                                                                                                                                              
2                                                                                                                                                                        

julia> f(i) = (ff = @eval ()->$i; Base.invokelatest(ff))  # with an anonymous function                                                                                                                
f (generic function with 1 method)                                                                                                                                                                                                                                                                                                            

julia> f(1)                                                                                                                                                              
1
1 Like
5

Thank you both for your suggestions.
They work, and also give me a line of experimentation to pursue.