Bad performance of anonymous functions even when they are type-stable

[frankwswang]

MWE:

julia> f1 = function (x)
                x/2
            end
#3 (generic function with 1 method)

julia> function f1_2(x)
           x/2
       end
f1_2 (generic function with 1 method)

julia> f2 = function (x)
                x
            end
#5 (generic function with 1 method)

julia> function f2_2(x)
           x
       end
f2_2 (generic function with 1 method)

julia> using BenchmarkTools

julia> @benchmark f1(0.1)
BenchmarkTools.Trial: 10000 samples with 999 evaluations.
 Range (min … max):  16.517 ns … 360.561 ns  ┊ GC (min … max): 0.00% … 90.23%
 Time  (median):     19.019 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   20.266 ns ±   9.995 ns  ┊ GC (mean ± σ):  0.44% ±  1.54%

     ▁    █
  ▂▄▆█▃▅▆▆█▃▂▃▄▄▄▃▃▃▃▃▃▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▂▂▁▂▂▂ ▃
  16.5 ns         Histogram: frequency by time         34.9 ns <

 Memory estimate: 16 bytes, allocs estimate: 1.

julia> @benchmark f1_2(0.1)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  0.800 ns … 237.800 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     0.800 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   0.904 ns ±   3.317 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  █    ▇    ▂                                                 ▁
  █▁▁▁▁█▁▁▁▁█▁▁▁▁▁▄▁▁▁▁▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▇ █
  0.8 ns       Histogram: log(frequency) by time       1.9 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

julia> @benchmark f2(0.1)
BenchmarkTools.Trial: 10000 samples with 999 evaluations.
 Range (min … max):  11.712 ns …  1.559 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     13.614 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   15.054 ns ± 19.157 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

      ██  ▇▂
  ▂▃▄▆██▆▅██▅▄▄▂▂▂▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▄▂▃▂▂▂▂▂▁▂▂▁▂▂▂▂▂▂▂▂▂▂▂ ▃
  11.7 ns         Histogram: frequency by time        27.8 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

julia> @benchmark f2_2(0.1)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  0.800 ns … 277.500 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     0.800 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   0.917 ns ±   3.086 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  █                            ▄
  █▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃ ▂
  0.8 ns          Histogram: frequency by time           1 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

Even though all functions are type-stable, the anonymous versions are drastically slower than the non-anonymous ones. Aren’t they supposed to be as generic as the latter?

BTW for both Julia 1.9.4 and 1.10.0 on my local machine, the same issue occurs.

Hence, if I have to generate functions on the fly (which is why I want to use anonymous functions), how can I achieve the same performance as the non-anonymous function versions, assuming I can ensure type stability?

Thanks!

[mkitti]

You are introducing type instability by not making the global anonymous functions const.

julia> f1 = function (x)
           x/2
       end
#3 (generic function with 1 method)

julia> function f1_2(x)
           x/2
       end
f1_2 (generic function with 1 method)

julia> const f1_const = function (x)
           x/2
       end
#5 (generic function with 1 method)

julia> using BenchmarkTools

julia> @benchmark f1(0.1)
BenchmarkTools.Trial: 10000 samples with 998 evaluations.
 Range (min … max):  15.499 ns … 816.685 ns  ┊ GC (min … max): 0.00% … 94.69%
 Time  (median):     15.917 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   16.896 ns ±  11.634 ns  ┊ GC (mean ± σ):  1.11% ±  1.62%

   ▃▅█▇█▅▄▁▁               ▁   ▃▄▄▅▃▄▂▂ ▁▁▃▂▃▂▃▁▁              ▂
  ▇█████████▇▇▃▅▄▅▁▃▁▄▄▇████▇████████████████████▆█▇▇▆█▆▇▆▆▄▅▆ █
  15.5 ns       Histogram: log(frequency) by time      20.2 ns <

 Memory estimate: 16 bytes, allocs estimate: 1.

julia> @benchmark f1_2(0.1)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  1.770 ns … 21.875 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.771 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.803 ns ±  0.264 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

   █                                                       ▅
  ▅█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃█ ▂
  1.77 ns        Histogram: frequency by time        1.82 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

julia> @benchmark f1_const(0.1)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  1.770 ns … 32.813 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.823 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.830 ns ±  0.443 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  █             █
  █▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▄▁▁▁▁▁▁▁▁▁▁▁▁▁▅▁▁▁▁▁▁▁▁▁▁▁▁▁▂ ▂
  1.77 ns        Histogram: frequency by time        1.98 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

julia> isconst(Main, :f1)
false

julia> isconst(Main, :f1_2)
true

julia> isconst(Main, :f1_const)
true

Essentially global values are type unstable because their types can change at any time. Another solution is to introduce some kind of local scope.

julia> let f1_let = function (x)
               x/2
           end
           @benchmark $f1_let(1.2)
       end
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  1.406 ns … 64.479 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.458 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.470 ns ±  0.666 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

                              █▁
  ▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁██▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂ ▂
  1.41 ns        Histogram: frequency by time        1.51 ns <

julia> g() = begin
           f1_local = x->x/2
           @benchmark $f1_local(8.2)
       end
g (generic function with 1 method)

julia> g()
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  1.406 ns … 10.000 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     1.458 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.451 ns ±  0.139 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▁                           █▁
  █▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁██▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂ ▂
  1.41 ns        Histogram: frequency by time        1.51 ns <

[frankwswang]

Thanks for the reply! This makes sense to me now. Even though the anonymous function has a specific type, when assigned to a global variable f1 or f2, these variables are not type-stable because, unlike the function name for non-anonymous functions, they can be reassigned with other types of values.

Based on this understanding, I interpolated the value of f1 before running the benchmark, and they became as performant as the non-anonymous counterparts:

julia> @benchmark ($f1)(0.1)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  0.800 ns … 126.300 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     0.900 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   1.113 ns ±   1.667 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  █   █         ▁                                  ▁    ▆   ▂ ▂
  █▁▁▁█▁▁▁▁█▁▁▁▁█▁▁▁▁▁▁▁▁▁▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁█▁▁▁█ █
  0.8 ns       Histogram: log(frequency) by time         2 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

julia> @benchmark ($f2)(0.1)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max):  0.800 ns … 196.200 ns  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     0.800 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   0.926 ns ±   2.960 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  █             ▇              ▃              ▁               ▁
  █▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁█ █
  0.8 ns       Histogram: log(frequency) by time       1.2 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.