Retain computation results with @btime

New to Julia
1

I’m trying to time a function and keep its return value:
b = @btime y = foo(x)
but at the end y is not returned. How can get @btime to also return my result?

1 Like
2

Use @belapsed instead.

1 Like
3
1 Like
4

I’m not clear on why you need this. I use

bar = foo(spam);

when I want results and

@btime foo($spam);

when I want to see how fast my function is. These are different missions for me.

1 Like
5

@belapsed did not seem to work, @elapsed did. Moreover, @btime provides more info than just time and is supposed to be more accurate ?

6

What did not work? Can you show a reproducible example and tell us your expected behaviour?

1 Like
7

It sounds like you just want

y = @btime foo(x)

However, I don’t see how this is useful. If this is part of a computation where you use y later, you probably don’t want to run foo(x) multiple times (which is what @btime does).

4 Likes
8

This is what I meant. @belapsed cannot return the computation result as I needed.

image
image679×811 17.7 KB

9

What doesn’t work in the link I shared above?

julia> @eval BenchmarkTools macro btimed(args...)
           _, params = prunekwargs(args...)
           bench, trial, result = gensym(), gensym(), gensym()
           trialmin, trialallocs = gensym(), gensym()
           tune_phase = hasevals(params) ? :() : :($BenchmarkTools.tune!($bench))
           return esc(quote
               local $bench = $BenchmarkTools.@benchmarkable $(args...)
               $BenchmarkTools.warmup($bench)
               $tune_phase
               local $trial, $result = $BenchmarkTools.run_result($bench)
               local $trialmin = $BenchmarkTools.minimum($trial)
               $result, $BenchmarkTools.time($trialmin)
           end)
       end
@btimed (macro with 1 method)

julia> b, y = BenchmarkTools.@btimed sin(12.3)
(-0.26323179136580094, 1.371)

julia> b
-0.26323179136580094

julia> y
1.371

Isn’t this what you want?

4 Likes
10

Yes. I replied to the wrong post

11

it would be great to have a way to do both though. Otherwise it’s just a waste of compute resources. BenchmarkTools.jl already calculates the result, why not also return it if needed ?

Tha solution from @giordano is appropriate for cases where the result is deterministic, and it would be nice to see it in BenchmarkTools.jl. (plus the variation to get the whole @benchmark metrics).
However, I am mostly preoccupied with cases where there is some stochasticity in the results, i.e. they’re not always the same. In these cases, it would be great to also return a vector of all calculated results (of the same length as the samples).

The docs in BenchmarkTools.jl specify the user should strive for reproducibility by e.g. using the same seed. Sometimes the nature of benchmarking itself is explorative and a variety of the results it’s expected. At that point, it would be nice to have the computed result accompanying the performance benchmark.

I can see that the motivation might be a bit shady, because now it’s not only about benchmarking performance of a function but also the content and functionality, but overall I think it would make a worthwhile addition.

12

May I ask why is this macro not shipped inside BenchmarkTools.jl?

13

I guess because no one bothered to submit a PR?

14

If that’s the case, shall I do that?

2 Likes
15

I guess you can if you’re motivated, yes.

2 Likes
16

I’m a little confused. The @btime macro does return the result of the expression being timed. It just doesn’t set global variables. For example:

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.1 (2024-10-16)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.11) pkg> st BenchmarkTools
Status `~/.julia/environments/v1.11/Project.toml`
  [6e4b80f9] BenchmarkTools v1.5.0

julia> using BenchmarkTools

julia> foo(x) = x + 1
foo (generic function with 1 method)

julia> x = 3
3

julia> b = @btime y = foo(x)
  7.750 ns (0 allocations: 0 bytes)
4

julia> b
4

julia> y
ERROR: UndefVarError: `y` not defined in `Main`
Suggestion: check for spelling errors or missing imports.

The result of the computation appears to be returned, and this has been the behavior for at least the past 7 years: Blaming BenchmarkTools.jl/src/execution.jl at b789f01a9069950f7eb49991e290895472896294 · JuliaCI/BenchmarkTools.jl · GitHub

This directly mimics the behavior of @time.

1 Like
17

I presume OP wanted to keep both the timing and the result. That’s what Base.@timed does, and the above proposed @btimed does as well to mimic Base.@timed.

1 Like
18

Sounds good! The PR is merged: Add `@btimed` and `@ballocations` macros by singularitti · Pull Request #383 · JuliaCI/BenchmarkTools.jl · GitHub