# Weird scoping issue/misunderstanding in JuMP.jl

**URL:** <https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604>\
**Category:** General Usage\
**Tags:** scope\
**Created:** [September 10, 2019, 1:34pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604 "2019-09-10T13:34:18Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![tamasgal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamasgal/32/27946_2.png) [@tamasgal](https://discourse.julialang.org/u/tamasgal)\
**Post date:** [September 10, 2019, 1:34pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604/1 "2019-09-10T13:34:18Z")

</div>

I spent the last few hours hunting a bug and boiled it down to a behaviour which I do not understand at all. Here is an MWE which basically reflects the core of the issue I had with my package:

```julia
using JuMP
using Ipopt

function wtf()
    f(x) = (x+2)^2
    
    model = Model(with_optimizer(Ipopt.Optimizer, print_level=0))
    register(model, :f, 1, f, autodiff=true)
    @variable(model, x)
    @NLobjective(model, Min, f(x))
    optimize!(model)

    retval = value(x)
    
    if false
        f(x) = (x-10)^2
    end
    
    return retval
end

```

Gives `10` instead of `-2`.

So what kind of magic is going on there? Is `value` reevaluating `f(x)` which is redefined by the compiler even if the body of the `if-block` is not reachable? Whaaat?

Edit: to avoid any confusion, I originally wanted to rerun `optimize!` with an updated quality function (`f(x)` in the example) only when some conditions are met, but it obviously always gets overwritten. So any advise how to rerun an optimisation procedure without all the boiler plate (`@register`, `@NLobjective` etc.) and only with a new function is really welcome!

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 10, 2019, 1:43pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604/2 "2019-09-10T13:43:12Z")

</div>

This is [disallow methods of local functions in different blocks · Issue #15602 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/15602).

Solution is to use an anonymous function `f = x -> (x-10)^2`.

And yes, it’s pretty bad.

---

<div class="post-metadata">

**Author:** ![tamasgal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamasgal/32/27946_2.png) [@tamasgal](https://discourse.julialang.org/u/tamasgal)\
**Post date:** [September 10, 2019, 1:48pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604/3 "2019-09-10T13:48:55Z")

</div>

Uff, using an anonymous function is not easy in my case. I do things like this to avoid ugly closures:

```julia
struct SingleDUMinimiser <: Function
    z_positions::Vector{Float64}
    times::Vector{Float64}
    pmt_directions::Vector{Direction}
    multiplicities::Vector{Int}
end

```

and then

```julia
function SingleDUMinimiser(hits::Vector{CalibratedHit}, triggered_hits::Vector{CalibratedHit})
    n = length(hits)
    n_triggered = length(triggered_hits)
    z_positions = Vector{Float64}()
    times = Vector{Float64}()
    multiplicities = Vector{Int32}()
    pmt_directions = Vector{Direction}()
    sizehint!(z_positions, n)
    sizehint!(times, n)
    sizehint!(multiplicities, n)
    sizehint!(pmt_directions, n_triggered)
    for i ∈ 1:n
        hit = hits[i]
        push!(z_positions, hit.pos.z)
        push!(times, hit.t)
        push!(multiplicities, hit.multiplicity.count)
    end
    for i ∈ 1:n_triggered
        push!(pmt_directions, triggered_hits[i].dir)
    end
    SingleDUMinimiser(z_positions, times, pmt_directions, multiplicities)
end

function (s::SingleDUMinimiser)(d_closest, t_closest, z_closest, dir_z, ϕ, t₀)
    n = length(s.times)

    d_γ, ccalc = make_cherenkov_calculator(d_closest, t_closest, z_closest, dir_z, t₀)
    expected_times = ccalc.(s.z_positions)

    max_multiplicity = maximum(s.multiplicities)
    Q = 0.0
    for i ∈ 1:n
        t = s.times[i]
        z = s.z_positions[i]
        m = s.multiplicities[i]
        t_exp = ccalc(z)
        Δt = abs(t - t_exp)
        Q += Δt^2 * m / max_multiplicity
    end

    Δts = abs.(s.times - expected_times)
    # Δϕs = filter(!isnan, azimuth.(s.pmt_directions)) .- ϕ

    # return sum(Δts .^2) + sum(Δϕs.^2)/length(Δϕs)
    # return sum(Δts .^2)
    return Q
end

```

Long story short, I don’t see how this could be done using anonymous functions 😕

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 10, 2019, 1:55pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604/4 "2019-09-10T13:55:20Z")

</div>

You _define_ those functions conditionally inside other functions?

---

<div class="post-metadata">

**Author:** ![tamasgal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamasgal/32/27946_2.png) [@tamasgal](https://discourse.julialang.org/u/tamasgal)\
**Post date:** [September 10, 2019, 1:59pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604/5 "2019-09-10T13:59:00Z")

</div>

I define an initial quality function inside the function which performs the minimisation, like this:

```julia
qfunc = SingleDUMinimiser(shits, filter(h->h.triggered, du_hits))

```

and then the whole `JuMP` stuff with `model`, `register`, `@NLobjective` and `@variable` etc.

Then I call `optimize!(model)` and given some conditions I need to adjust `qfunc` and rerun `optimize!(model)` again.

So yes, I need to redefine the function inside that wrapper function conditionally…

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 10, 2019, 2:04pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604/6 "2019-09-10T14:04:01Z")

</div>

That shouldn’t be a problem. Can you repro your issue using something similar to what you now describe.

---

<div class="post-metadata">

**Author:** ![tamasgal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamasgal/32/27946_2.png) [@tamasgal](https://discourse.julialang.org/u/tamasgal)\
**Post date:** [September 10, 2019, 2:04pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604/7 "2019-09-10T14:04:27Z")

</div>

Sure, let me hack…

---

<div class="post-metadata">

**Author:** ![tamasgal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamasgal/32/27946_2.png) [@tamasgal](https://discourse.julialang.org/u/tamasgal)\
**Post date:** [September 10, 2019, 2:17pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604/8 "2019-09-10T14:17:26Z")

</div>

OK Kristoffer, you are right, I probably boiled too much, that is not the root of my optimisation problem:

```julia
using JuMP
using Ipopt

struct FMinimiser <: Function
    a
end

function(f::FMinimiser)(x)
    (x + f.a)^2
end

function optimisation_procedure(reoptimize::Bool)
    f = FMinimiser(2)
    
    model = Model(with_optimizer(Ipopt.Optimizer, print_level=0))
    register(model, :f, 1, f, autodiff=true)
    @variable(model, x)
    @NLobjective(model, Min, f(x))
    optimize!(model)
    
    if reoptimize
        f = FMinimiser(10)
        optimize!(model)
    end
    
    value(x)
end

```

This gives

```julia
> optimisation_procedure(false)
-2.0

```

However, it also returns `-2.0` for `optimisation_procedure(true)`.

Is the only way to let `JuMP` recognize the new function calling `register` and `@NLobjective` again, like this?

```julia
function optimisation_procedure(reoptimize::Bool)
    f = FMinimiser(2)
    
    model = Model(with_optimizer(Ipopt.Optimizer, print_level=0))
    register(model, :f, 1, f, autodiff=true)
    @variable(model, x)
    @NLobjective(model, Min, f(x))
    optimize!(model)
    
    if reoptimize
        f2 = FMinimiser(10)
        register(model, :f2, 1, f2, autodiff=true)
        @NLobjective(model, Min, f2(x))
        optimize!(model)
    end
    
    value(x)
end

```

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 10, 2019, 2:33pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604/9 "2019-09-10T14:33:42Z")

</div>

> [@tamasgal](#):
>
> f = FMinimiser(10)

There is no way that rebinding `f` can have any influence on what `value(x)` returns. You are not passing in the variable `f` to `register` you are passing in the `object` that `f` “points” or “maps” to which is the first instance of `FMinimiser`. Rebinding `f` does absolutely nothing to that object.

---

<div class="post-metadata">

**Author:** ![tamasgal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamasgal/32/27946_2.png) [@tamasgal](https://discourse.julialang.org/u/tamasgal)\
**Post date:** [September 10, 2019, 3:51pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604/10 "2019-09-10T15:51:25Z")

</div>

Yes, that I understand. So I need to reregister the function again and define the objective…

What’s a bit annoying is that I need a new name (symbol) every time I register a new one. I thought there is something to replace or reevaluate the registered function. I guess I have to ask some `JuMP` users

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [September 10, 2019, 4:12pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604/11 "2019-09-10T16:12:55Z")

</div>

`JuMP.register` creates a mapping between a symbol and a Julia function. At least in the global scope, you can change the Julia function without needing to re-register or reset the objective:

```nohighlight
julia> model = Model(with_optimizer(Ipopt.Optimizer, print_level = 0))
A JuMP Model
Feasibility problem with:
Variables: 0
Model mode: AUTOMATIC
CachingOptimizer state: EMPTY_OPTIMIZER
Solver name: Ipopt

julia> @variable(model, x >= 0)
x

julia> f(x) = (x - 2)^2
f (generic function with 1 method)

julia> JuMP.register(model, :foo, 1, f, autodiff=true) # No need to name :f

julia> @NLobjective(model, Min, f(x)) # Using function doesn't work
ERROR: Unrecognized function "f" used in nonlinear expression.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] top-level scope at /Users/oscar/.julia/packages/JuMP/iGamg/src/parse_nlp.jl:73
 [3] top-level scope at /Users/oscar/.julia/packages/JuMP/iGamg/src/macros.jl:1497

julia> @NLobjective(model, Min, foo(x)) # Registered symbol, not Julia function

julia> optimize!(model)

******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit http://projects.coin-or.org/Ipopt
******************************************************************************

julia> value(x)
2.00000000062657

julia> f(x) = (x + 10)^2
f (generic function with 1 method)

julia> optimize!(model)

julia> value(x)
0.0

julia> objective_value(model)
99.99999980250593

```

You would have to experiment to see what the interaction is between JuMP and the Julia “feature” in function scopes.

---

<div class="post-metadata">

**Author:** ![tamasgal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamasgal/32/27946_2.png) [@tamasgal](https://discourse.julialang.org/u/tamasgal)\
**Post date:** [September 10, 2019, 5:05pm UTC](https://discourse.julialang.org/t/weird-scoping-issue-misunderstanding-in-jump-jl/28604/12 "2019-09-10T17:05:16Z")

</div>

Yes exactly, it only works if I redefine the function, but also only in the global scope.  
But I am using a functor and assign it to a “variable”, so I do not have access to the connection anymore (it seems).  
If I was using function definitions then I am again at the original problem [disallow methods of local functions in different blocks · Issue #15602 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/15602) were it simply does not work ☹