# How to asychronously build a benchmark group
**URL:** https://discourse.julialang.org/t/how-to-asychronously-build-a-benchmark-group/106335
**Category:** Profiling
**Tags:** benchmark, benchmarktools
**Created:** [November 16, 2023, 9:24pm UTC](https://discourse.julialang.org/t/how-to-asychronously-build-a-benchmark-group/106335 "2023-11-16T21:24:28Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@filchristou](https://discourse.julialang.org/u/filchristou)
#### Post date: [November 16, 2023, 9:24pm UTC](https://discourse.julialang.org/t/how-to-asychronously-build-a-benchmark-group/106335/1 "2023-11-16T21:24:29Z")
Can I build several inter-depending benchmarks from a serial procedure in “one go” using BenchmarkTools.jl ?
Say I have the following functionality laid out in a function
```julia
function mymutatingfunc!(x)
mutfunc1!(x)
mutfunc2!(x)
mutfunc3!(x)
end
```
and `x` needs to be initialized `x = init()`
All `mutfunc!`s need to be called in that order, otherwise they don’t make sense.
E.g., `mutfunc3!` can operate normally only if `mutfunc1!`,`mutfunc2!` are called in beforehand.
## Current solution
I would like to benchmark all `mutfunc!`s individually.
This is complex because it would be something like
```julia
x = init()
SUITE["muttfunc1"] = @benchmarkable muttfunc1!(y) setup=(y = x |> deepcopy)
SUITE["muttfunc2"] = @benchmarkable muttfunc2!(y) setup=(y = x |> deepcopy |> mutfunc1!)
SUITE["muttfunc3"] = @benchmarkable muttfunc3!(y) setup=(y = x |> deepcopy |> mutfunc1! |> mutfunc2!)
```
This looks like a waste of resources.
Repeatedly calling `mtfunc!` is also not possible as the content changes significantly.
E.g. I **cannot** do something like
```julia
x = init()
SUITE["muttfunc1"] = @benchmarkable muttfunc1!(y) setup=(y = x |> deepcopy)
muttfunc1!(x)
SUITE["muttfunc2"] = @benchmarkable muttfunc2!(y) setup=(y = x |> deepcopy)
muttfunc2!(x)
SUITE["muttfunc3"] = @benchmarkable muttfunc3!(y) setup=(y = x |> deepcopy)
```
## Desirable
Could I “asychronously” build the benchmarks by sampling in a round-robin fashion.
E.g., it would be awesome if I could do something like
```julia
x = init()
@startbenchmark SUITE setup=(y=deepcopy(x))
@yieldbenchmeasure "muttfunc1" muttfunc1!(y)
@yieldbenchmeasure "muttfunc2" muttfunc2!(y)
@yieldbenchmeasure "muttfunc3" muttfunc3!(y)
end
```
Ofc it’s understandable they will all need to have the same benchmark config (e.g. `samples`, `evals`).
I hope my imaginery code is understandable of what it is meant to accomplish.
---
### Author:  [@filchristou](https://discourse.julialang.org/u/filchristou)
#### Post date: [November 17, 2023, 10:13am UTC](https://discourse.julialang.org/t/how-to-asychronously-build-a-benchmark-group/106335/2 "2023-11-17T10:13:56Z")
I keep on daydreaming here.
It might be interesting to think about it in terms of defining a `BenchmarkFunctionModel`, sort of like `Turing.jl` prepends `@model` in a function it samples.
E.g, something like
```julia
@benchmarkablegroup function myfunc(arg1, arg2, arg3)
operations1(arg1, arg2, arg3)
@yieldbenchmeasure "op2" operations2(arg1, arg2, arg3)
operations3(arg1, arg2, arg3)
@yieldbenchmeasure "op4" operations4(arg1, arg2, arg3)
end
```
So the previous desirable code could be stated as
```julia
@benchmarkablegroup function mybenchfunc(x)
@yieldbenchmeasure "muttfunc1" muttfunc1!(x)
@yieldbenchmeasure "muttfunc2" muttfunc2!(x)
@yieldbenchmeasure "muttfunc3" muttfunc3!(x)
end
```
and theoretically you would call it with something like
```julia
julia> mybenchfunc(init())
3-element BenchmarkTools.BenchmarkGroup:
tags: []
"muttfunc1" => Benchmark(evals=1, seconds=5.0, samples=10000)
"muttfunc2" => Benchmark(evals=1, seconds=5.0, samples=10000)
"muttfunc3" => Benchmark(evals=1, seconds=5.0, samples=10000)
```
So, as I believe that such functionality is not implemented I speculated a bit on a possible interface about it.
I’ll try to stop my delirium here. Let me know what you think
---
### Author:  [@gdalle](https://discourse.julialang.org/u/gdalle)
#### Post date: [November 17, 2023, 3:12pm UTC](https://discourse.julialang.org/t/how-to-asychronously-build-a-benchmark-group/106335/3 "2023-11-17T15:12:26Z")
> [@filchristou](#):
>
> I would like to benchmark all `mutfunc!`s individually.
This sounds more like [GitHub - KristofferC/TimerOutputs.jl: Formatted output of timed sections in Julia](https://github.com/KristofferC/TimerOutputs.jl)
---
### Author:  [@filchristou](https://discourse.julialang.org/u/filchristou)
#### Post date: [November 17, 2023, 8:35pm UTC](https://discourse.julialang.org/t/how-to-asychronously-build-a-benchmark-group/106335/4 "2023-11-17T20:35:04Z")
That’s a cool package and fairly close to what I am asking. But I think it doesn’t work with `BenchmarkTools.jl` (?)