Wrong results when I use @btime

Dear all,

I am quite baffled by the results I get when using @btime. I was following this simple lecture: https://cuda.juliagpu.org/stable/tutorials/introduction/ and just by copying and pasting from this page to a Julia script, got the following:

using Test                                                                       
using BenchmarkTools                                                             
N = 2^20                                                                         
x = fill(1.0f0, N)  # a vector filled with 1.0 (Float32)                         
y = fill(2.0f0, N)  # a vector filled with 2.0                                   
function sequential_add!(a, b)                                                   
  for i in eachindex(a, b)                                                       
    @inbounds a[i] += b[i]                                                       
  return nothing                                                                 
# This will pass the test                                                        
fill!(y, 2.0f0)                                                                  
sequential_add!(y, x)                                                            
@test all(y .== 3.0f0)                                                           
# This will fail the test                                                        
fill!(y, 2.0f0)                                                                  
@btime sequential_add!($y, $x)                                                   
@test all(y .== 3.0f0)

As the comments I inserted above indicate, after the first call to sequential_add! I get correct results, but after the second which goes through @btime macro, the test doesn’t pass and I get rather strange results for y, it is filled with 37032.0 or 35204.0, depending on the run.

There is no creativity from my side in the above example. Except for the two comments and the @test in the last line, all is copied and pasted from the link I give above.

Does anyone have an idea what is going on here? I am using Julia 1.6.3 and am launching it with julia --check-bounds=no --threads 6, although the command line options don’t change the outcome.


This thread might be of interest for benchmarking your in-place functions.


In short: @btime runs the function several times and thus updates y several times.


Indeed, changing the last few lines to:

@btime begin                                                                     
  fill!(y, 2.0f0)                                                                
  sequential_add!($y, $x)                                                        

fixed the thing :slight_smile:

1 Like

Although, to do the testing properly you should look at the link posted by @rafael.guerra


I see it now, thanks a lot.