# Define test error message programmatically

**URL:** <https://discourse.julialang.org/t/define-test-error-message-programmatically/116142>\
**Category:** General Usage\
**Tags:** testing, error-message\
**Created:** [June 24, 2024, 1:39pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142 "2024-06-24T13:39:47Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [June 24, 2024, 1:39pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/1 "2024-06-24T13:39:47Z")

</div>

Is there a way for a test to display a custom error message, without giving up on the nice display of numerical values equality / comparison tests?  
The workaround in [Custom error message from @test macro](https://discourse.julialang.org/t/custom-error-message-from-test-macro/2299) is only a partial solution for this:

```julia
julia> using Test

julia> x, y = 1, 2;

julia> msg = "The test failed for this reason: ...";

julia> @test x == y # numerical values, no message displayed
Test Failed at REPL[16]:1
  Expression: x == y
   Evaluated: 1 == 2

ERROR: There was an error during testing

julia> @test x == y || msg # only symbols, message is displayed
Error During Test at REPL[19]:1
  Expression evaluated to non-Boolean
  Expression: x == y || msg
       Value: "The test failed for this reason: ..."
ERROR: There was an error during testing

```

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [June 24, 2024, 2:02pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/2 "2024-06-24T14:02:07Z")

</div>

```julia
x == y || @warn "failure for this reason…"
@test x == y # more error message

```

---

<div class="post-metadata">

**Author:** ![oOosys](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ooosys/32/209566_2.png) [@oOosys](https://discourse.julialang.org/u/oOosys)\
**Post date:** [June 24, 2024, 2:14pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/3 "2024-06-24T14:14:09Z")

</div>

```julia
using Test
x, y = 1, 2
#x, y = 1, 1
msg=" Custom Test ERROR Message"
x==y || println("$msg")
@test x == y ;

```

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [June 24, 2024, 2:19pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/4 "2024-06-24T14:19:01Z")

</div>

Thanks, I don’t know why I was looking for something more complicated.

The second solution works too but here I think using `@warn` is better than `println` because it is same logging system used to display the actual error (so there is no risk of desynchronization for example).

---

<div class="post-metadata">

**Author:** ![oOosys](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ooosys/32/209566_2.png) [@oOosys](https://discourse.julialang.org/u/oOosys)\
**Post date:** [June 24, 2024, 2:21pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/5 "2024-06-24T14:21:01Z")

</div>

Both solutions give in my tests:

```julia
ERROR: LoadError: There was an error during testing
in expression starting at /home/o/oOo/o/delMe.jl:6

```

Why?

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [June 24, 2024, 2:27pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/6 "2024-06-24T14:27:27Z")

</div>

> [@gdalle](#):
>
> ` Expression evaluated to non-Boolean`

Check the error message, it’s probably the same as mine ⬆ That happens because `x == y` is a `Bool` but `msg` is not, so I can’t compute an OR with it

---

<div class="post-metadata">

**Author:** ![oOosys](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ooosys/32/209566_2.png) [@oOosys](https://discourse.julialang.org/u/oOosys)\
**Post date:** [June 24, 2024, 2:37pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/7 "2024-06-24T14:37:42Z")

</div>

`if( x !=y ) ; println("$msg"); end` gives the same effect and does not solve the issue … So there is something else going wrong and printing an additional final ERROR message making both up to now solutions having an unnecessary side-effect.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [June 24, 2024, 2:43pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/8 "2024-06-24T14:43:05Z")

</div>

The solution given by Steven is correct and works on my machine, so my initial problem is solved.  
I’m not sure what goes wrong in your case so if you need help please post the complete code through which you arrive at the error.

---

<div class="post-metadata">

**Author:** ![oOosys](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ooosys/32/209566_2.png) [@oOosys](https://discourse.julialang.org/u/oOosys)\
**Post date:** [June 24, 2024, 3:08pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/9 "2024-06-24T15:08:24Z")

</div>

> [@gdalle](#):
>
> I’m not sure what goes wrong in your case so if you need help please post the complete code through which you arrive at the error.

The complete code is given in my answer and results in printing the ERROR if run as script from within Geany text editor. In other words, you don’t have any additional output except this one of @test and the custom error message on your system?

This code runs without any side-effect of an additional ERROR message:

```julia
using Test
x, y = 1, 2
#x, y = 1, 1
msg = " Custom ERROR message "
try
    @test x == y
catch e
    println(msg)
    flush(stdout) # Ensure the message is flushed immediately
    exit()
end
println("END")

```

on my system if run from within Geany.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [June 24, 2024, 3:10pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/10 "2024-06-24T15:10:02Z")

</div>

This is what I get with your code, as expected. The error here is because the test itself failed.

```julia
julia> using Test

julia> x, y = 1, 2
       #x, y = 1, 1
(1, 2)

julia> msg=" Custom Test ERROR Message"
" Custom Test ERROR Message"

julia> x==y || println("$msg")
 Custom Test ERROR Message

julia> @test x == y ;
Test Failed at REPL[5]:1
  Expression: x == y
   Evaluated: 1 == 2

ERROR: There was an error during testing

```

---

<div class="post-metadata">

**Author:** ![oOosys](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ooosys/32/209566_2.png) [@oOosys](https://discourse.julialang.org/u/oOosys)\
**Post date:** [June 24, 2024, 3:11pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/11 "2024-06-24T15:11:06Z")

</div>

OK … so you are in the REPL … What happens if you run this code as script outside the REPL? Maybe you need then the code I have provided which uses `try/catch` to avoid an additional ERROR message printed at the end of the output ?

By the way: my question what is needed to arrive while running a script at the behavior in the REPL is still open and without a detailed answer.

In other words what you marked as solution does not work for me if run as script code (i.e. not in the REPL).

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [June 24, 2024, 3:27pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/12 "2024-06-24T15:27:32Z")

</div>

> [@oOosys](#):
>
> Both solutions give in my tests:
> 
> ```julia
> ERROR: LoadError: There was an error during testing
> in expression starting at /home/o/oOo/o/delMe.jl:6
> 
> ```
> 
> Why?

It’s by design. A failed test is supposed to throw an error.

---

<div class="post-metadata">

**Author:** ![oOosys](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ooosys/32/209566_2.png) [@oOosys](https://discourse.julialang.org/u/oOosys)\
**Post date:** [June 24, 2024, 4:34pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/13 "2024-06-24T16:34:01Z")

</div>

> [@GunnarFarneback](#):
>
> It’s by design. A failed test is supposed to throw an error.

OK … but then in addition to the message printed by `@test` there will be finally an additional error message while using

```julia
x == y || @warn "failure for this reason…"
@test x == y # more error message

```

in a Julia script run as script (not inside a REPL). The double messaging can be avoided using `try/catch` without re-throwing the error and exiting the script instead (see my another reply).

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [June 24, 2024, 5:46pm UTC](https://discourse.julialang.org/t/define-test-error-message-programmatically/116142/16 "2024-06-24T17:46:21Z")

</div>

If that is what you want to achieve, by all means do so. One difference that might not matter for your use case but is somewhat important in CI scripts is the return code of the script.