# How to plot benchmarks

**URL:** <https://discourse.julialang.org/t/how-to-plot-benchmarks/104243>\
**Category:** General Usage\
**Tags:** question, plotting, benchmarktools\
**Created:** [September 25, 2023, 8:00pm UTC](https://discourse.julialang.org/t/how-to-plot-benchmarks/104243 "2023-09-25T20:00:52Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [September 25, 2023, 8:00pm UTC](https://discourse.julialang.org/t/how-to-plot-benchmarks/104243/1 "2023-09-25T20:00:52Z")

</div>

Hi all,

I would like to plot some benchmarks, but its not clear to me what approach to take. As a simple example, suppose I have:

```julia
using BenchmarkTools
using DataFrames
using Statistics 

SUITE = BenchmarkGroup()
SUITE[:benchmark] = BenchmarkGroup()

n1s = [10,100,1000]
n2s = n1s

for n1 ∈ n1s 
    for n2 ∈ n2s 
        SUITE[:benchmark][n1,n2] = @benchmarkable(
            rand($n1, $n2),
            evals = 10,
            samples = 1000)
    end
end

results = run(SUITE)

```

How do I plot the means as a bar chart (or use violin plots for the distributions)?

My initial inclination was to convert to a dataframe and use StatsPlots.

```julia
means = mean(results[:benchmark])
df = DataFrame(mean(results[:benchmark]))
transform!(df, :first => AsTable) # how do I name the columns?

```

There are two problems. First, the means are TrialEstimate types. Second, the units are not constant.

```julia
 Row │ first second x1 x2    
     │ Tuple… TrialEst… Int64 Int64 
─────┼───────────────────────────────────────────────────────
   1 │ (10, 1000) TrialEstimate(11.478 μs) 10 1000
   2 │ (1000, 100) TrialEstimate(106.119 μs) 1000 100
   3 │ (100, 100) TrialEstimate(13.545 μs) 100 100
   4 │ (10, 10) TrialEstimate(442.292 ns) 10 10
   5 │ (100, 1000) TrialEstimate(109.565 μs) 100 1000
   6 │ (1000, 1000) TrialEstimate(1.636 ms) 1000 1000
   7 │ (10, 100) TrialEstimate(1.706 μs) 10 100
   8 │ (1000, 10) TrialEstimate(12.028 μs) 1000 10
   9 │ (100, 10) TrialEstimate(1.913 μs) 100 10

```

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [September 25, 2023, 8:35pm UTC](https://discourse.julialang.org/t/how-to-plot-benchmarks/104243/2 "2023-09-25T20:35:05Z")

</div>

This should work:

```julia
transform(df,
          :first => ByRow(identity) => [:n1, :n2],
          :second => (t -> time.(t)) => "Time [ns]")

```

---

<div class="post-metadata">

**Author:** ![gustaphe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gustaphe/32/18174_2.png) [@gustaphe](https://discourse.julialang.org/u/gustaphe)\
**Post date:** [September 25, 2023, 8:46pm UTC](https://discourse.julialang.org/t/how-to-plot-benchmarks/104243/3 "2023-09-25T20:46:56Z")

</div>

No need to cross the stream for water:

```julia
using StatsPlots
res = run(SUITE)
plot(res)

```

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [September 25, 2023, 9:42pm UTC](https://discourse.julialang.org/t/how-to-plot-benchmarks/104243/4 "2023-09-25T21:42:10Z")

</div>

Thank you for your replies. The solution provided by bertschi works. I think I can extend it to violin plots, or something else to represent distributional information.

@gustaphe, I was hoping your solution would work, but unfortunately it resulted in the following error:

```julia
julia> plot(results)
ERROR: Cannot convert BenchmarkGroup to series data for plotting
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] _prepare_series_data(x::BenchmarkGroup)
    @ RecipesPipeline ~/.julia/packages/RecipesPipeline/BGM3l/src/series.jl:8
  [3] _series_data_vector(x::BenchmarkGroup, plotattributes::Dict{Symbol, Any})
    @ RecipesPipeline ~/.julia/packages/RecipesPipeline/BGM3l/src/series.jl:36
  [4] macro expansion
    @ ~/.julia/packages/RecipesPipeline/BGM3l/src/series.jl:129 [inlined]
  [5] apply_recipe(plotattributes::AbstractDict{Symbol, Any}, #unused#::Type{RecipesPipeline.SliceIt}, x::Any, y::Any, z::Any)
    @ RecipesPipeline ~/.julia/packages/RecipesBase/BRe07/src/RecipesBase.jl:300
  [6] _process_userrecipes!(plt::Any, plotattributes::Any, args::Any)
    @ RecipesPipeline ~/.julia/packages/RecipesPipeline/BGM3l/src/user_recipe.jl:38
  [7] recipe_pipeline!(plt::Any, plotattributes::Any, args::Any)
    @ RecipesPipeline ~/.julia/packages/RecipesPipeline/BGM3l/src/RecipesPipeline.jl:72
  [8] _plot!(plt::Plots.Plot, plotattributes::Any, args::Any)
    @ Plots ~/.julia/packages/Plots/sxUvK/src/plot.jl:223
  [9] #plot#188
    @ ~/.julia/packages/Plots/sxUvK/src/plot.jl:102 [inlined]
 [10] plot(args::Any)
    @ Plots ~/.julia/packages/Plots/sxUvK/src/plot.jl:93
 [11] top-level scope
    @ REPL[17]:1

```

---

<div class="post-metadata">

**Author:** ![gustaphe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gustaphe/32/18174_2.png) [@gustaphe](https://discourse.julialang.org/u/gustaphe)\
**Post date:** [September 26, 2023, 5:38am UTC](https://discourse.julialang.org/t/how-to-plot-benchmarks/104243/5 "2023-09-26T05:38:38Z")

</div>

Sorry, I though we had changed that to an extension, you also need `using BenchmarkPlots`

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [September 26, 2023, 8:12am UTC](https://discourse.julialang.org/t/how-to-plot-benchmarks/104243/6 "2023-09-26T08:12:47Z")

</div>

Do you think it would make sense as an extension?

---

<div class="post-metadata">

**Author:** ![Christopher\_Fisher](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christopher_fisher/32/26132_2.png) [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Post date:** [September 26, 2023, 9:49am UTC](https://discourse.julialang.org/t/how-to-plot-benchmarks/104243/7 "2023-09-26T09:49:07Z")

</div>

Thank you. That worked.

Note for someone who is using `BenchmarkGroup`, you need to call the group:

`plot(results[:group])`

---

<div class="post-metadata">

**Author:** ![gustaphe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gustaphe/32/18174_2.png) [@gustaphe](https://discourse.julialang.org/u/gustaphe)\
**Post date:** [September 26, 2023, 10:11am UTC](https://discourse.julialang.org/t/how-to-plot-benchmarks/104243/8 "2023-09-26T10:11:24Z")

</div>

Yeah, loading Plots and BenchmarkTools should just work. But the deprecation process would be a pita 😃