# How can you guarantee that all the functions / branches / loops in your package has been tested?
**URL:** https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835
**Category:** General Usage
**Tags:** question, testing
**Created:** [February 23, 2021, 9:57am UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835 "2021-02-23T09:57:55Z")
**Posts on this page:** 12
**Page:** 1
### Author:  [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)
#### Post date: [February 23, 2021, 9:57am UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835/1 "2021-02-23T09:57:55Z")
Is there any automated testing method for a rather complicated package? Probably a macro that injects some logging commands to each function / if-else branch / for-loop and produce a statistical analysis of the coverage of a testing procedure? I know the macro `@profile` can record execution time, but how about coverage ?
---
### Author:  [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [February 23, 2021, 10:04am UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835/2 "2021-02-23T10:04:46Z")
Coverage is usually handled by continuous integration tools, e.g., [codecov](https://codecov.io/). You can setup you github repo with automatic hooks to codecov such that you get a warning if code coverage decreased by pulling a PR. You can even prevet a PR from being merged unless codecov returned the desired status.
You find more info at
[https://github.com/JuliaCI/Coverage.jl](https://github.com/JuliaCI/Coverage.jl)
---
### Author:  [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)
#### Post date: [February 23, 2021, 12:05pm UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835/3 "2021-02-23T12:05:19Z")
Concerning the testing and coverage handled in Continuous Integration tools that @baggepinnen was referring to, you can have a look on [this tutorial](https://syl1.gitbook.io/julia-language-a-concise-tutorial/language-core/11-developing-julia-packages) to learn how to applying it to Julia packages…
---
### Author:  [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)
#### Post date: [February 23, 2021, 12:47pm UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835/4 "2021-02-23T12:47:44Z")
> [@sylvaticus](#):
>
> sting and coverage handled in Con
This is exactly what I wanted. Thanks !
---
### Author:  [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)
#### Post date: [February 23, 2021, 12:48pm UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835/5 "2021-02-23T12:48:08Z")
The tutorial is very elaborate. Thank you very much!
---
### Author:  [@giordano](https://discourse.julialang.org/u/giordano)
#### Post date: [February 23, 2021, 1:55pm UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835/6 "2021-02-23T13:55:31Z")
However note that services like codecov only show you _line_ code coverage, but tell you nothing about _paths_ of the code: conditionals, loops, and other control flow statements can make the behaviour of a line very different if it’s hit from different places.
---
### Author:  [@adolgert](https://discourse.julialang.org/u/adolgert)
#### Post date: [March 4, 2021, 12:05am UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835/7 "2021-03-04T00:05:08Z")
@giordano mentioned that there are different kinds of coverage. You have to figure out what arguments to the function will give you good coverage, and there are four ways to do that.
1. Construct them by hand, as in a decision table.
2. Randomly choose each argument from a set of values, including edge cases. Run the randomized tests for a while.
3. Use an all-pairs strategy, from UnitTestDesign.jl.
4. Use property-based testing, from randomizedpropertytest.jl.
Which method works best will depend on how long the function runs, how many arguments it has, and how much risk is involved. Good luck!
---
### Author:  [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)
#### Post date: [March 4, 2021, 6:06am UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835/8 "2021-03-04T06:06:19Z")
trying them out, thank you !
---
### Author:  [@rikh](https://discourse.julialang.org/u/rikh)
#### Post date: [March 8, 2021, 4:23pm UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835/9 "2021-03-08T16:23:13Z")
In line with Mosè Giordano, what is the point of _guaranteeing_ that all functions / branches / loops in your package have been tested?
In the words of Dijkstra ([1969](http://homepages.cs.ncl.ac.uk/brian.randell/NATO/nato1969.PDF)): _“Testing shows the presence, not the absence of bugs.”_
EDIT: I mean, Dijkstra doesn’t say that testing is pointless, but the efforts spent on guaranteeing that everything is covered by tests is probably better spent on other verification techniques or refactoring and documenting.
---
### Author:  [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)
#### Post date: [March 9, 2021, 2:43am UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835/10 "2021-03-09T02:43:38Z")
Yes I agree. Now I realized that unit test + coverage doesn’t guarantee good code. I should simplify the internal logic of the code.
---
### Author:  [@Stephen\_Vavasis](https://discourse.julialang.org/u/Stephen_Vavasis)
#### Post date: [March 9, 2021, 3:15am UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835/11 "2021-03-09T03:15:57Z")
I wrote a package to do this exactly this kind of testing. It works as follows: it swallows the whole program, parses out all the conditionals, and then spits out the code again as a giant ‘eval’ with an invocation-counter-increment call attached to each conditional. Obviously, this makes execution slower. It was written for an old version of Julia, possibly 0.4, so most likely it won’t work any more but you are welcome to have a look. [GitHub - StephenVavasis/microcoverage](https://github.com/StephenVavasis/microcoverage)
---
### Author:  [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)
#### Post date: [March 9, 2021, 3:17am UTC](https://discourse.julialang.org/t/how-can-you-guarantee-that-all-the-functions-branches-loops-in-your-package-has-been-tested/55835/12 "2021-03-09T03:17:34Z")
OUAH !
Thank a lot !