# ANN: Automatic test-case reduction

**URL:** <https://discourse.julialang.org/t/ann-automatic-test-case-reduction/17700>\
**Category:** Tooling\
**Tags:** testing, debug\
**Created:** [November 19, 2018, 7:31am UTC](https://discourse.julialang.org/t/ann-automatic-test-case-reduction/17700 "2018-11-19T07:31:45Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [November 19, 2018, 7:31am UTC](https://discourse.julialang.org/t/ann-automatic-test-case-reduction/17700/1 "2018-11-19T07:31:45Z")

</div>

Hi all,

> C-Reduce is a tool that takes a large C, C++, or OpenCL file that has a property of interest (such as triggering a compiler bug) and automatically produces a much smaller C/C++ file that has the same property.

I’ve used [C-Reduce](https://embed.cs.utah.edu/creduce/) for reducing complicated Julia test-cases in the past (eg [#28595](https://github.com/JuliaLang/julia/issues/28595) and [#26278](https://github.com/JuliaLang/julia/issues/26278)), and figured it might be useful for other people. A repo with instructions and support scripts is up at [maleadt/creduce\_julia](https://github.com/maleadt/creduce_julia), and depends on Julia 1.0+ and C-Reduce 2.8.0+ (both being available on PATH).

Basically, the process boils down to:

- provide a script that has a certain property (eg. it crashes) in `main.jl`
- edit `run` to catch that property (eg. `julia main.jl |& grep "he's dead, jim"`)
- run `reduce`

```julia
$ cat main.jl
error("example error message")

$ tail -n1 run
julia --compiled-modules=no -O0 main.jl |& grep "example error message"

# verify the `run` script works properly (should initially return 0)
$ ./run
ERROR: LoadError: example error message
$ echo $?
0

$ ./reduce
Reducing the following sources:
- main.jl
===< 7281 >===
...
===================== done ====================

```

If your script depends on certain packages, you can install them in the `depot` directory: Start by sourcing the `activate` script, which sets `JULIA_DEPOT_PATH` and `JULIA_PKG_DEVDIR`. You can now install packages which will end up in the `depot` directory, and will get picked up by the `reduce` script:

```julia
$ cat main.jl
using Compat
error("example error message")

$ source activate
$ julia
$ julia -e 'using Pkg; Pkg.add("Compat")'

# speed-up the process by removing useless files
$ rm -rf depot/packages/**/test

$ ./reduce
Reducing the following sources:
- main.jl
- depot/packages/Compat/HVYNa/src/Compat.jl
- depot/packages/Compat/HVYNa/src/arraymacros.jl
- depot/packages/Compat/HVYNa/src/compatmacro.jl
- depot/packages/Compat/HVYNa/src/deprecated.jl
===< 12424 >===
...
===================== done ====================

```

The process uses all available processors, but can still take a while (obviously depending on the run-time complexity, but think hours for single-file test-cases and days for multi-package setups). I’ve also used the process in conjunction with automated test-case generation through fuzzing, see [this blog post](https://blog.maleadt.net/2018/11/16/julia_bugs/) for details.

---

<div class="post-metadata">

**Author:** ![adolgert](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adolgert/32/20286_2.png) [@adolgert](https://discourse.julialang.org/u/adolgert)\
**Post date:** [October 24, 2020, 9:20pm UTC](https://discourse.julialang.org/t/ann-automatic-test-case-reduction/17700/2 "2020-10-24T21:20:54Z")

</div>

I know this is a year old now, but this technique is really useful in a number of contexts:

- Creating a smaller example to report a bug.
- What unit testers call “fault-finding”, the process of narrowing down where a fault is located.
- In line with the book on Legacy Code, that says, “make regression testing easy,” this would make regression tests faster.