Huge difference in performance between two programmes

General Usage Performance
1

I’m sorry for the not very descriptive title but I couldn’t find a better way to phrase it

I have two similar programmes with a huge difference in performance

let
	function useless()
	end
	
	@time let
		s = 0
		for n in 1:100000
			s += n^10
		end
		s
	end
end

let
	@time let
		s = 0
		for n in 1:100000
			s += n^10
		end
		s
	end
end

The first gives me 0.221958 seconds (498.22 k allocations: 9.129 MiB, 3.08% gc time, 1.90% compilation time) and the second fives me 0.000540 seconds

The difference in performance is the same as when using global variables where normally everything is hard scoped with the let keyword.

So what is the source of the problem and how to avoid this situation ?

2

The difference you are seeing seems to be spurious, it’s just one-time compilation/startup costs. If I use @btime from BenchmarkTools.jl instead of @time, it reports exactly the same time (389µs) in both cases. (The @btime macro performs the measurement several times and reports the smallest, and it is generally far more reliable than @time for drawing conclusions about performance.)

1 Like
3

Something does seem weird given that @time reports only 1.9% compilation time. Not sure what to do about it though.

4

In Julia 1.6, I just get “0.160966 seconds (498.20 k allocations: 9.128 MiB)” with nothing about compilation time.

5

I guess @btime has to do some magic stuff that doesn’t allow you to see the problem. But when I found the problem it was because a program that usually ran in one second was taking several minutes.

By multiplying the number of iterations of the loop by 100 and removing the @time the first program takes about a 10s while the second is instantaneous.

So the problem remains and is very similar to the performance problem of global variables

2 Likes
6

Replacing the let block with a function is sufficient to completely eliminate the problem for me. That seems like it would also solve the issue in your actual program, since it’s easy to turn a bare let block into a function.

1 Like
7

Yes, it solves the problem but I would like to understand why there is a problem

3 Likes
8

Probably you should report an issue about that. Clearly something strange is happening there. The same happens with begin...end blocks:

start julia and:

julia> @time begin
               f() = nothing
                let
                       s = 0
                       for n in 1:100000
                               s += n^10
                       end
                       s
               end
       end
  0.142710 seconds (500.56 k allocations: 9.295 MiB, 4.71% gc time, 5.31% compilation time)
-5724124746025484656



julia> @time begin
               f() = nothing
                let
                       s = 0
                       for n in 1:100000
                               s += n^10
                       end
                       s
               end
       end
  0.146696 seconds (498.21 k allocations: 9.129 MiB)
-5724124746025484656

restart julia and:

julia> @time begin
               let
                       s = 0
                       for n in 1:100000
                               s += n^10
                       end
                       s
               end
       end
  0.000568 seconds
-5724124746025484656