Output of @benchmark to string or table?

New to Julia
1

Hi all!

This is a very simple question but I can’t find an easy answer. Any help appreciated!

I am trying to @benchmark the speed of a function over several orders-of-magnitude of inputs. I would like to just do a for-loop and then capture the benchmark output in a string (or, better, would be an object that I could query).

When I store the output of @benchmark in a variable x, though, I get some raw data that I’d rather not have to process. (Yes, in this case I can figure it out, but in general, sometimes I’d like to capture the screen summary output of a function in a string.)

Simple example code below. Any help welcome!!
Cheers, Nick

using BenchmarkTools # for @benchmark
using Statistics     # for mean

function example_add(a,b)
	c = a+b
end

x = @benchmark example_add(1, 2)

x

# Screen output -- how do I capture to a string or other variable?
#
# BenchmarkTools.Trial: 
#   memory estimate:  0 bytes
#   allocs estimate:  0
#   --------------
#   minimum time:     0.025 ns (0.00% GC)
#   median time:      0.029 ns (0.00% GC)
#   mean time:        0.029 ns (0.00% GC)
#   maximum time:     6.074 ns (0.00% GC)
#   --------------
#   samples:          10000
#   evals/sample:     1000

minimum(x.times)
mean(x.times)
maximum(x.times)
1 Like
2 
julia> using BenchmarkTools

julia> b = @benchmark 2+3
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     0.035 ns (0.00% GC)
  median time:      0.036 ns (0.00% GC)
  mean time:        0.038 ns (0.00% GC)
  maximum time:     12.984 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000

julia> io = IOBuffer()
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=Inf, ptr=1, mark=-1)

julia> show(io, "text/plain", b)

julia> s = String(take!(io))
"BenchmarkTools.Trial: \n  memory estimate:  0 bytes\n  allocs estimate:  0\n  --------------\n  minimum time:     0.035 ns (0.00% GC)\n  median time:      0.036 ns (0.00% GC)\n  mean time:        0.038 ns (0.00% GC)\n  maximum time:     12.984 ns (0.00% GC)\n  --------------\n  samples:          10000\n  evals/sample:     1000"

julia> typeof(s)
String

julia> println(s)
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     0.035 ns (0.00% GC)
  median time:      0.036 ns (0.00% GC)
  mean time:        0.038 ns (0.00% GC)
  maximum time:     12.984 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000
5 Likes
3

Thanks very much! That’s a great solution!
Cheers, Nick

4

The return value of @benchmark is an object that you can query:

julia> using BenchmarkTools, Statistics

julia> x = @benchmark sin(3)
BenchmarkTools.Trial: 
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     0.036 ns (0.00% GC)
  median time:      0.039 ns (0.00% GC)
  mean time:        0.039 ns (0.00% GC)
  maximum time:     0.083 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000

julia> minimum(x.times)
0.036

julia> maximum(x.times)
0.083

julia> mean(x.times)
0.03916590000000001

julia> median(x.times)
0.039

julia> length(x.times)
10000

You can do dump(x) to see the fields of this data structure listed explicitly.

3 Likes