You’re right, I forgot to include the time taken to execute the function (and thus finish the compilation). Also, @btime
has been making the Fix
functors appear artificially good compared to lambda due to running multiple times:
Subsequent runs are faster after the first time
julia> @time eval(:( Map(Fix2(*,2))([1,2,3]) ))
0.036643 seconds (24.55 k allocations: 1.414 MiB, 93.90% compilation time)
3-element Vector{Int64} |>
Map(Fix{typeof(*), (2,), 2, Tuple{Int64}, NamedTuple{(), Tuple{}}}(*, (2,), NamedTuple()))
julia> @time eval(:( Map(Fix2(*,2))([1,2,3]) ))
0.000170 seconds (102 allocations: 5.578 KiB)
3-element Vector{Int64} |>
Map(Fix{typeof(*), (2,), 2, Tuple{Int64}, NamedTuple{(), Tuple{}}}(*, (2,), NamedTuple()))
julia> @time eval(:( Map(Fix2(*,2))([1,2,3]) ))
0.000207 seconds (102 allocations: 5.578 KiB)
3-element Vector{Int64} |>
Map(Fix{typeof(*), (2,), 2, Tuple{Int64}, NamedTuple{(), Tuple{}}}(*, (2,), NamedTuple()))
julia> @btime eval(:( Map(Fix2(*,2))([1,2,3]) ))
82.500 μs (93 allocations: 5.23 KiB)
3-element Vector{Int64} |>
Map(Fix{typeof(*), (2,), 2, Tuple{Int64}, NamedTuple{(), Tuple{}}}(*, (2,), NamedTuple()))
I don’t know why I thought quoting the expression would cause it to recompile from scratch. Brain fart.
Meanwhile, constructing on a lambda is consistently slow
julia> @time eval(:( Map(x->2*x)([1,2,3]) ))
0.024207 seconds (11.55 k allocations: 672.857 KiB, 92.00% compilation time)
3-element Vector{Int64} |>
Map(Main.λ❓)
julia> @time eval(:( Map(x->2*x)([1,2,3]) ))
0.021667 seconds (11.55 k allocations: 672.857 KiB, 91.32% compilation time)
3-element Vector{Int64} |>
Map(Main.λ❓)
julia> @time eval(:( Map(x->2*x)([1,2,3]) ))
0.035034 seconds (11.55 k allocations: 672.857 KiB, 86.82% compilation time)
3-element Vector{Int64} |>
Map(Main.λ❓)
julia> @btime eval(:( Map(x->2*x)([1,2,3]) ))
18.818 ms (11541 allocations: 672.51 KiB)
3-element Vector{Int64} |>
Map(Main.λ❓)
This is because every time, a new lambda is created from scratch and given a new type
julia> typeof(x->2*x)
var"#9489#9490"
julia> typeof(x->2*x)
var"#9491#9492"
julia> typeof(x->2*x)
var"#9493#9494"
julia> typeof(x->2*x)
var"#9495#9496"
I need to do benchmarking less cavalierly!
Less ambitious compile-time testing will just focus on creation of the functor object or lambda, use @time
instead of @btime
, and focus on only the first run calling a fresh function on fresh types:
`Fix` type
julia> @time FixFirst(+, 1.0)(2.0)
0.000003 seconds
3.0
julia> @time FixFirst(+, 1.0)(2.0)
0.000001 seconds
3.0
julia> @time FixFirst(+, 1.0)(2.0)
0.000001 seconds
3.0
julia> @btime FixFirst(+, 1.0)(2.0)
1.000 ns (0 allocations: 0 bytes)
3.0
julia>
julia> @time FixFirst(+, 1im)(2)
0.000003 seconds
2 + 1im
julia> @time FixFirst(+, 1im)(2)
0.000001 seconds
2 + 1im
julia> @time FixFirst(+, 1im)(2)
0.000001 seconds
2 + 1im
julia> @btime FixFirst(+, 1im)(2)
1.000 ns (0 allocations: 0 bytes)
2 + 1im
lambda
julia> @time (x->1.0+x)(2.0)
0.002635 seconds (679 allocations: 41.471 KiB, 98.27% compilation time)
3.0
julia> @time (x->1.0+x)(2.0)
0.002937 seconds (679 allocations: 41.471 KiB, 98.22% compilation time)
3.0
julia> @time (x->1.0+x)(2.0)
0.002534 seconds (679 allocations: 41.471 KiB, 98.21% compilation time)
3.0
julia>
julia> @time (x->1im+x)(2)
0.016275 seconds (3.41 k allocations: 196.206 KiB, 99.59% compilation time)
2 + 1im
julia> @time (x->1im+x)(2)
0.005604 seconds (3.37 k allocations: 194.331 KiB, 98.65% compilation time)
2 + 1im
julia> @time (x->1im+x)(2)
0.005470 seconds (3.37 k allocations: 194.253 KiB, 98.83% compilation time)
2 + 1im
julia> @btime (x->1im+x)(2) # of course after it's compiled it's fast
1.500 ns (0 allocations: 0 bytes)
2 + 1im
The Fix
functor is still better at compile-time than a lambda, by about three orders of magnitude. It’s just harder and less convenient to measure compile time with precision than I had hoped. No it’s not, see below.
Is there a good way to “un-compile” a method for the sake of compile-time benchmarking? 