# "Zipped" loops in test sets: a bad design pattern?

**URL:** <https://discourse.julialang.org/t/zipped-loops-in-test-sets-a-bad-design-pattern/57685>\
**Category:** General Usage\
**Tags:** testing, design-pattern\
**Created:** [March 22, 2021, 6:39am UTC](https://discourse.julialang.org/t/zipped-loops-in-test-sets-a-bad-design-pattern/57685 "2021-03-22T06:39:07Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jollywatt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jollywatt/32/202198_2.png) [@Jollywatt](https://discourse.julialang.org/u/Jollywatt)\
**Post date:** [March 22, 2021, 6:39am UTC](https://discourse.julialang.org/t/zipped-loops-in-test-sets-a-bad-design-pattern/57685/1 "2021-03-22T06:39:07Z")

</div>

I’m (ab)using Julia’s testing interface to verify mathematical propositions for many randomly generated samples. I find myself using the following pattern a lot:

```julia
@testset "definitions" begin
	for x in rand(N)
		...
		@test cos(x) ≈ (exp(x*im) + exp(-x*im))/2
	end
end
@testset "basic identities" begin
	for x in rand(N)
		...
		@test tan(x) ≈ sin(x)/cos(x)
	end
end
@testset "complex identities" begin
	for x in rand(N)
		...
		@test sec(x)^2 == tan(x)^2 + 1
	end
end

```

This involves much code repetition when there is lots of set-up code (indicated by the `...`).  
The set-up code doesn’t lend well to being packaged into a function, because each `...` declares many local variables used all throughout the tests, and each successive `...` is different (but usually a superset of the last).

I find myself wanting to structure my code like this:

```julia
for x in rand(N)
	...
	@testset "definitions" begin
		@test cos(x) ≈ (exp(x*im) + exp(-x*im))/2
	end
	...
	@testset "basic identities" begin
		@test tan(x) ≈ sin(x)/cos(x)
	end
	...
	@testset "higher identities" begin
		@test sec(x)^2 ≈ tan(x)^2 + 1
	end
end

```

where each `...` e.g., introduces new local variables used in the next set of tests.

This of course generates three new test sets for each sample, whereas I’d like to preserve the nicely collated test set output as in the original snippet.

Is there a way I could achieve the best of both worlds? E.g., by “merging” all test sets which are instantiated with the same label?

---

<div class="post-metadata">

**Author:** ![Ian\_Slagle](https://avatars.discourse-cdn.com/v4/letter/i/b5a626/32.png) [@Ian\_Slagle](https://discourse.julialang.org/u/Ian_Slagle)\
**Post date:** [March 22, 2021, 6:41am UTC](https://discourse.julialang.org/t/zipped-loops-in-test-sets-a-bad-design-pattern/57685/2 "2021-03-22T06:41:33Z")

</div>

I’m a bit new to the testing side of things, but are your local variables in the setup overriding one another or can they all be done ahead of one loop?

---

<div class="post-metadata">

**Author:** ![Jollywatt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jollywatt/32/202198_2.png) [@Jollywatt](https://discourse.julialang.org/u/Jollywatt)\
**Post date:** [March 22, 2021, 7:01am UTC](https://discourse.julialang.org/t/zipped-loops-in-test-sets-a-bad-design-pattern/57685/3 "2021-03-22T07:01:01Z")

</div>

@Ian_Slagle Essentially, no; they could all be done at once. Risking bad question etiquette, here’s a wall of my actual code with `@test`s omitted.

```julia
@testset "bipolar" begin
	for σ in samples(N)
		ρ, φ = bipolar(σ)
		...
	end
end
@testset "bisplitunits" begin
	for σ in samples(N)
		β, σ̂ = bisplitunits(σ)
		...
	end
end
@testset "bisplit" begin
	for σ in samples(N)
		σ₊, σ₋ = bisplit(σ)
		...
	end
end
@testset "principle_generator" begin
	for σ in samples(N)
		σ′ = principle_generator(σ)
		...
	end
end
@testset "trivial identities" begin
	for σ₀ in samples(N)
		σ = principle_generator(σ₀)
		R = exp(σ)
		α = scalar_part(R)
		γ = pseudoscalar_part(R)
		...
	end
end
@testset "R = α + β₊u₊ + β₋u₋ + γI" begin
	for σ₀ in samples(N)
		σ = principle_generator(σ₀)
		R = exp(σ)
		α = scalar_part(R)
		γ = pseudoscalar_part(R)
		b = bivector_part(R)
		ρ, φ = bipolar(b)
		β, σ̂ = bisplitunits(b)
		δ, σ̂′ = bisplitunits(σ)
		...
	end
end

```

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [March 22, 2021, 7:16am UTC](https://discourse.julialang.org/t/zipped-loops-in-test-sets-a-bad-design-pattern/57685/4 "2021-03-22T07:16:54Z")

</div>

This looks fine to me, there isn’t a lot of actual repetition (unless I am missing something). The only thing you are repeating is `for σ in samples(N)`.

While reusing code is generally advisable, I usually strive to keep tests simple and readable.

---

<div class="post-metadata">

**Author:** ![Skoffer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skoffer/32/378_2.png) [@Skoffer](https://discourse.julialang.org/u/Skoffer)\
**Post date:** [March 22, 2021, 7:55am UTC](https://discourse.julialang.org/t/zipped-loops-in-test-sets-a-bad-design-pattern/57685/5 "2021-03-22T07:55:33Z")

</div>

Only thing that can be done here (but I am unsure whether it makes things better or worse) is to wrap reusable generations of variables in a function. So your last two testsets can look like this

```julia
function trivial_identities_variables(σ₀)
    σ = principle_generator(σ₀)
    R = exp(σ)
    α = scalar_part(R)
    γ = pseudoscalar_part(R)
    
    return (; σ, R, α, γ)
end

function alphabetagamma_variables(σ₀)
    σ, R, α, γ = trivial_identities_variables(σ₀)
    b = bivector_part(R)
    ρ, φ = bipolar(b)
    β, σ̂ = bisplitunits(b)
    δ, σ̂′ = bisplitunits(σ)

    return (; σ, R, α, γ, b, ρ, φ, β, σ̂ , δ, σ̂′)
end

...

@testset "trivial identities" begin
	for σ₀ in samples(N)
		σ, R, α, γ = trivial_identities_variables(σ₀)
		...
	end
end
@testset "R = α + β₊u₊ + β₋u₋ + γI" begin
	for σ₀ in samples(N)
		σ, R, α, γ, b, ρ, φ, β, σ̂ , δ, σ̂′ = alphabetagamma_variables(σ₀)
		...
	end
end

```

And use can use UnPack.jl to extract only needed subset of variables.

---

<div class="post-metadata">

**Author:** ![DrChainsaw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/drchainsaw/32/8497_2.png) [@DrChainsaw](https://discourse.julialang.org/u/DrChainsaw)\
**Post date:** [March 22, 2021, 8:43am UTC](https://discourse.julialang.org/t/zipped-loops-in-test-sets-a-bad-design-pattern/57685/6 "2021-03-22T08:43:28Z")

</div>

Am I the only one who likes for loops to create test sets? Makes it easier to spot what iteration caused the failure:

```julia
julia> using Test

julia> @testset "Main test" begin
       @testset "Test for n=$n" for n in 1:5
           @testset "sum" begin
               @test sum(1:n) < 100
           end
           @testset "prod" begin
               @test prod(1:n) < 100
           end
       end
       end
prod: Test Failed at REPL[4]:7
  Expression: prod(1:n) < 100
   Evaluated: 120 < 100
Stacktrace:
    .....
Test Summary: | Pass Fail Total
Main test | 9 1 10
  Test for n=1 | 2 2
  Test for n=2 | 2 2
  Test for n=3 | 2 2
  Test for n=4 | 2 2
  Test for n=5 | 1 1 2
    sum | 1 1
    prod | 1 1
ERROR: Some tests did not pass: 9 passed, 1 failed, 0 errored, 0 broken.

```

---

<div class="post-metadata">

**Author:** ![Jollywatt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jollywatt/32/202198_2.png) [@Jollywatt](https://discourse.julialang.org/u/Jollywatt)\
**Post date:** [March 22, 2021, 9:25pm UTC](https://discourse.julialang.org/t/zipped-loops-in-test-sets-a-bad-design-pattern/57685/7 "2021-03-22T21:25:27Z")

</div>

@DrChainsaw This is true.  
However, looped test sets are not so appropriate when iterating through a very high number of (random) samples, like my case here. It is not so important which sample is responsible for the error, either: what is important is whether the tests pass for all the samples, or fail for some.

---

<div class="post-metadata">

**Author:** ![Jollywatt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jollywatt/32/202198_2.png) [@Jollywatt](https://discourse.julialang.org/u/Jollywatt)\
**Post date:** [March 22, 2021, 9:35pm UTC](https://discourse.julialang.org/t/zipped-loops-in-test-sets-a-bad-design-pattern/57685/8 "2021-03-22T21:35:14Z")

</div>

@Skoffer True, but it’s perhaps harder to maintain: there is now an ordering for the variables to be aware of, and their definitions are hidden…

Like the original snippet, it still suffers from code repetition (in this example, recalculating the common variables `σ, R, α, γ`) for each test set. That’s what is tempting about the linear structure:

```julia
for σ₀ in samples(N) # arbitrarily large number of trials
	a, b, c, d = ...
	@testset "first set" begin
		@test F(a, b, c, d)
	end
	e, f, g, h = ...
	@testset "more complex set" begin
		# tests which assume the previous test sets have passed
		@test G(a, b, c, d, e, f, g, h)
	end
end

```

---

<div class="post-metadata">

**Author:** ![Skoffer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skoffer/32/378_2.png) [@Skoffer](https://discourse.julialang.org/u/Skoffer)\
**Post date:** [March 22, 2021, 9:41pm UTC](https://discourse.julialang.org/t/zipped-loops-in-test-sets-a-bad-design-pattern/57685/9 "2021-03-22T21:41:47Z")

</div>

Not quite true: definition is hidden, that’s correct, but ordering is unimportant. There is a semicolon at the tuple definition, which means that this is a `NamedTuple`. UnPack can handle order of the `NamedTuple`

```julia
using UnPack

function f()
    a = 1
    b = 2
    (; a, b)
end

@unpack b, a = f()
julia> a
1

julia> b
2

```

I actually do not see code repetition in my example: variables are different because they depend on variable `σ₀`. So it’s either you put everything in a huge `for` block and reuse variables in different tests, or you calculate everything inside each loop.

I think the main problem is functions that accept a huge list of variables. It’s hard to find the simple syntax for them anyway.