# Use tags, setup, teardown with the \`@benchmarkset\` macro

**URL:** <https://discourse.julialang.org/t/use-tags-setup-teardown-with-the-benchmarkset-macro/98936>\
**Category:** General Usage\
**Tags:** benchmarktools\
**Created:** [May 16, 2023, 3:10pm UTC](https://discourse.julialang.org/t/use-tags-setup-teardown-with-the-benchmarkset-macro/98936 "2023-05-16T15:10:24Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![AlexanderNenninger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alexandernenninger/32/24602_2.png) [@AlexanderNenninger](https://discourse.julialang.org/u/AlexanderNenninger)\
**Post date:** [May 16, 2023, 3:10pm UTC](https://discourse.julialang.org/t/use-tags-setup-teardown-with-the-benchmarkset-macro/98936/1 "2023-05-16T15:10:24Z")

</div>

Is there a way to use tags with the `@benchmarkset` macro? What about `setup` and `teardown`? I’ve been searching the docsumentation but to no avail.

---

<div class="post-metadata">

**Author:** ![filchristou](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/filchristou/32/26760_2.png) [@filchristou](https://discourse.julialang.org/u/filchristou)\
**Post date:** [November 17, 2023, 9:50am UTC](https://discourse.julialang.org/t/use-tags-setup-teardown-with-the-benchmarkset-macro/98936/2 "2023-11-17T09:50:28Z")

</div>

[API Reference](https://juliaci.github.io/BenchmarkTools.jl/stable/reference/#BenchmarkTools.@case-Tuple%7BAny,%20Vararg%7BAny%7D%7D) mentions `setup` in `@case`. Is that what you want?

But you are right. The `@benchmarkset` is a bit of a mystery and I even think it malfunctions a bit.  
E.g. the example code doesn’t produce 5 cases as I would expect but just one:

```julia
julia> @benchmarkset "suite" for k in 1:5
           @case "case $k" rand($k, $k)
       end
1-element BenchmarkTools.BenchmarkGroup:
  tags: []
  "suite" => 1-element BenchmarkTools.BenchmarkGroup:
	 tags: []
	 "case 5" => Benchmark(evals=1, seconds=5.0, samples=10000)

```

I would expect it to be similar to the following code block:

```julia
julia> bg = BenchmarkGroup([], "suite" => BenchmarkGroup([]));

julia> foreach(1:5) do k
           bg["suite"]["case $k"] = @benchmarkable rand($k, $k)
       end

julia> bg
1-element BenchmarkTools.BenchmarkGroup:
  tags: []
  "suite" => 5-element BenchmarkTools.BenchmarkGroup:
	 tags: []
	 "case 2" => Benchmark(evals=1, seconds=5.0, samples=10000)
	 "case 4" => Benchmark(evals=1, seconds=5.0, samples=10000)
	 "case 5" => Benchmark(evals=1, seconds=5.0, samples=10000)
	 "case 3" => Benchmark(evals=1, seconds=5.0, samples=10000)
	 "case 1" => Benchmark(evals=1, seconds=5.0, samples=10000)

```

In the end, you can ofc use the second approach to reliably do all you `setup`, `teardown` stuff.

EDIT: opened an issue [is `@benchmarkset` usable ? · Issue #343 · JuliaCI/BenchmarkTools.jl · GitHub](https://github.com/JuliaCI/BenchmarkTools.jl/issues/343)

---

<div class="post-metadata">

**Author:** ![AlexanderNenninger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alexandernenninger/32/24602_2.png) [@AlexanderNenninger](https://discourse.julialang.org/u/AlexanderNenninger)\
**Post date:** [November 17, 2023, 10:17am UTC](https://discourse.julialang.org/t/use-tags-setup-teardown-with-the-benchmarkset-macro/98936/3 "2023-11-17T10:17:07Z")

</div>

Yeah, that’s what I’ve been doing. It works quite well with functions that generate the benchmark cases.