Warning: Method definition f(Any) overwritten

[afyon]

Hi everyone,

I am trying to make a function that would look like this:

function create_f(flag::Bool)
   if flag
      f(x) = x^2
   else
      f(x) = x^3
   end
   return f
end

And, when I create this function, I got the warning that f is overwritten. I do understand the warning but I do not know why I got it, since both definitions of f happen in mutually exclusive blocks of code.

Then, when I execute f = create_f(true), f is the cubic function while f = create_f(false) gives me the error that f is undefined. Could someone help me make this code work or at least explain to me why it cannot work ?

I know that something like:

function create_f(flag::Bool)
   if flag
      f1(x) = x^2
      return f1
   else
      f2(x) = x^3
      return f2
   end
end

does work fine but it is not very elegant.

Thanks a lot!

[algunion]

You might want to try this:

function create_f(flag::Bool)
    flag && return x -> x^2
    return x -> x^3
end

[abraemer]

It works fine if you do

function create_f(flag::Bool)
   if flag
      f = x->x^2
   else
      f = x->x^3
   end
   return f
end

@algunion 's solution is the same but shortened by use of short circuiting.

Somehow in your original version, Julia seems to think that both f(x) denote the same function and then tries to add methods to it. That explains the warning because then the second f(x) = ... overwrites the first definition. I am not sure whether this should be treated as a bug or whether the f(x) syntax should only be used at toplevel or something.

[algunion]

if block does not introduce a new scope.

Here is a version where f is defined in a hard scope for both if/else branches:

function create_f(flag::Bool)
    function locf end
    if (flag)
        let f(x) = x^2
            locf = f
        end
    else
        let f(x) = x^3
            locf = f
        end
    end
    return locf
end

Now, both create_f(true)(x) and create_f(false)(x) are going to behave as expected.

Also, consider the following:

function nested(flag::Bool)
    f(x) = x
    if flag
        f(x) = x + 1
    end
    return f
end

f = nested(false)
f(2) # will produce 3

The thing is that the method/function definition takes place before the runtime call. Also, the method overriding takes place because if doesn’t introduce a new scope.

However, I am not (yet) able to explain why the else (flag = false) branch in the original function throws UndefVarError: f not defined.

Later addition:

So there might be a bug after all:

function create_f(flag::Bool)
    if flag
        f(x) = x^2
    else
        f(x) = x^3
    end
    return f
end

@code_lowered create_f(true)
@code_lowered create_f(false)

@code_lowered produces:

CodeInfo(
1 ─     Core.NewvarNode(:(f))
└──     goto #3 if not flag
2 ─     f = %new(Main.:(var"#f#7"))
└──     goto #3
3 ┄     return f
)

So, when flag = false, it just skips the f definition and attempts to return it. This behavior has nothing to do with overriding the method in the else branch (the same issue occurs if a different method signature is used).

We can compare with the following naive example:

function notbug(flag::Bool)
    if flag
        x = 3
    else
        x = 4
    end
    return x
end
@code_lowered notbug(true)
@code_lowered notbug(false)

… which produces (as expected):

CodeInfo(
1 ─     Core.NewvarNode(:(x))
└──     goto #3 if not flag
2 ─     x = 3
└──     goto #4
3 ─     x = 4
4 ┄     return x
)

[DNF]

For dynamic creation of functions, use anonymous functions. In your case, the name f is also hidden inside the outer function, so this is clearly a job for a lambda.

[algunion]

@DNF

For dynamic creation of functions, use anonymous functions. In your case, the name f is also hidden inside the outer function, so this is clearly a job for a lambda.

Agree, but despite of best practice recommendation, I think there is a bug. The definition of f is simply skipped in the else branch - even if I try to force it like this:

function create_f(flag::Bool)
    if flag
        f(x) = x^2
        z = f(3)
    else
        f(x) = x^3
        z = f(3)
    end
    return f, z
end

@code_lowered create_f(true)
@code_lowered create_f(false)

create_f(true) # (var"#f#8"(), 27)
create_f(false) # UndefVarError: `f` not defined

@code_lowered output:

CodeInfo(
1 ─      Core.NewvarNode(:(z))
│        Core.NewvarNode(:(f))
└──      goto #3 if not flag
2 ─      f = %new(Main.:(var"#f#8"))
│        z = (f)(3)
└──      goto #4
3 ─      z = (f)(3)
4 ┄ %8 = Core.tuple(f, z)
└──      return %8
)

P. S. And I think I know what is going on: in the else branch, only a method push takes place (and before runtime), with no definition for f (because the actual name was already generated by gensym). So the runtime bug consists in failing to link f to the symbol generated by gensym when flag=false.

[nsajko]

You want an anonymous function:

Simple solution:

function create_f(flag::Bool)
  if flag
    x -> x^2
  else
    x -> x^3
  end
end

The above solution is, however, not type stable:

julia> using Test

julia> @inferred create_f(true)
ERROR: return type var"#1#3" does not match inferred return type Union{var"#1#3", var"#2#4"}

A type stable solution:

function create_f(flag::Bool)
  m = flag ? 2 : 3
  let n = m
    x -> x^n
  end
end

Now the type is uniquely inferred:

julia> using Test

julia> @inferred create_f(true)
#1 (generic function with 1 method)

[mikmoore]

It’s been discussed at previous times, but I’ll bring it back up. We probably want to disallow function definitions within control flow (e.g., conditional or loop blocks). This would not apply to anonymous functions, of course, which are the correct solution for such cases.

This could be considered breaking within version 1.x, although we might lawyer it through as a “bug fix” since such function definitions do not behave how one would expect or desire. The only breakage of “correct” code would be in cases where a function was incidentally defined within control flow but there was no competing definition to cause ambiguity.

[Benny]

@algunion

Agree, but despite of best practice recommendation, I think there is a bug. The definition of f is simply skipped in the else branch - even if I try to force it like this:

This is issue #15602. Closures’ underlying top-level method tables are made when the surrounding function is defined because a function call that rebuilds a method table (which we can do to global scope functions with @eval) is a lot slower than one that conditionally assigns one of several existing functions to a variable. At least now we get a warning, and it should be heeded because we could either get the “all definitions run” effect or the “redefinitions become failed assignments” effect.

@mikmoore

We probably want to disallow function definitions within control flow (e.g., conditional or loop blocks).

It’s fine in the global scope, it’s just a footgun during a repeatable function call.

@nsajko

A type stable solution:

function create_f(flag::Bool)
  m = flag ? 2 : 3
  let n = m
    x -> x^n
  end
end

Incidentally, the let block isn’t necessary here, x -> x^m stably captures the value because there’s only 1 assignment with a type-stable value. if flag m=2 else m=3 end would be problematic though, in which case the let block helps.

[algunion]

Thanks, @Benny. I was unsuccessfully searching for the issue - but I restrained from creating another one (this seemed too significant not to be already reported).

[algunion]

@afyon, thanks for selecting a solution and helping others that will stumble upon this topic.

You selected my post as the solution - and I think it answers the OP well. However, I think @nsajko’s solution is more complete - since it also addresses the type-stability issue (e.g., it will be more instructive for others in the future).

So, I switched the solution to @nsajko’s post. I hope that my decision resonates with you.