The compilation time of max with dot distribution is kinda insane. workarounds?

General Usage
1

I have a function that returns a tuple of two ints, and I want to find the largest int that shows up in each index. we’ve got this function in base called max which will do this for you, but the compilation time is kinda crazy:

julia> @time max.((0,0), [(1,1) for i=1:100]...)
 47.772725 seconds (14.32 M allocations: 549.282 MiB, 2.91% gc time, 99.41% compilation time)
(1, 1)

julia> # this is logically equivalent, but >500x faster
julia> @time (max(0, [1 for i=1:100]...), max(0, [1 for i=1:100]...))
  0.078314 seconds (68.24 k allocations: 3.324 MiB, 99.28% compilation time)
(1, 1)

it also recompiles for each new set of arguments, so max.((0,0), [(1,1) for i=1:101]...) will take a long time again, and then so will max.((0,0), [(1,1) for i=1:102]...) after those two.

is there some way I can get max to not compile that long, other than the obvious

julia> arr1 = arr2 = zeros(101)
julia> for (ind, i) in enumerate([(0, 0); [(1,1) for i=1:100]])
           arr1[ind], arr2[ind] = i
       end
julia> max(arr1...), max(arr2...)

I don’t want to run the tuple function (what I’ve been representing as [(0, 0); [(1,1) for i=1:100]]) more than I need to

note maximum([(0,0); [(1,1) for i=1:100]]) has different semantics. observe the difference here:

julia> max.((0,0), [([1:100;][i],[100:-1:1;][i]) for i=1:100]...)
(100, 100)

julia> maximum([(0,0); [([1:100;][i],[100:-1:1;][i]) for i=1:100]])
(100, 1)
2

aww, and here I was, thinking I was so smart. zip has the same issue:

julia> @time collect(zip((0,0), [(1,1) for i=1:100]...))
203.997187 seconds (53.47 M allocations: 2.030 GiB, 2.30% gc time, 100.00% compilation time)
2-element Vector{NTuple{101, Int64}}:
 (0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1)
 (0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1)
3

You’re splatting a 100 element long vector, which means you’re asking the compiler to create & optimize a function with 100 individual arguments. That’s likely the cause of the slowdown, not any individual function like zip or max.

4

then yes, that’s what I’m asking how to not do. max works just fine on thousands of arguments if you don’t distribute, so I feel like there has to be a way to make this work…

julia> @time max(1, [1 for i=1:10000]...)
  0.059525 seconds (34.08 k allocations: 2.040 MiB, 94.97% compilation time)
1
5

Maybe maximum(first, x) and maximum(last, x)

6

nice! that’s much closer to being inline, gets rid of the big compile time

julia> @time (let x=(([1:1000;][i],[1000:-1:1;][i]) for i=1:1000); maximum(first, x; init=0), maximum(last, x; init=0) end)
  0.225317 seconds (220.40 k allocations: 40.799 MiB, 14.21% gc time, 82.82% compilation time)
(1000, 1000)

got it!

julia> @time maximum.((first, last), Ref(([1:1000;][i],[1000:-1:1;][i]) for i=1:1000); init=0)
  0.157316 seconds (197.25 k allocations: 39.709 MiB, 12.08% gc time, 84.11% compilation time)
(1000, 1000)
7

maximum is the answer, but in general, this kind of problem is well solved by reduce and mapreduce. For instance: mapreduce(first, max, [(1,1) for i=1:100], init=0)