# Documenter.jl doctest does not apply filter as expected

**URL:** <https://discourse.julialang.org/t/documenter-jl-doctest-does-not-apply-filter-as-expected/136706>\
**Category:** General Usage\
**Tags:** documenter, regex, doctest\
**Created:** [April 13, 2026, 10:17pm UTC](https://discourse.julialang.org/t/documenter-jl-doctest-does-not-apply-filter-as-expected/136706 "2026-04-13T22:17:03Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![phx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/phx/32/219940_2.png) [@phx](https://discourse.julialang.org/u/phx)\
**Post date:** [April 13, 2026, 10:17pm UTC](https://discourse.julialang.org/t/documenter-jl-doctest-does-not-apply-filter-as-expected/136706/1 "2026-04-13T22:17:03Z")

</div>

Hi all,  
I am writing the documentation for a package of mine, and in a doctest I have this output:

```plaintext
11-element Vector{ComplexF64}:
                     0.0 + 0.0im
   -1.617576049847552e-6 + 6.617439891157473e-23im
   -2.216802827443102e-6 - 2.602082008249062e-18im
  -2.1754485086580383e-5 + 5.293471330973909e-22im
    4.696594721193055e-5 + 6.511993073259912e-21im
   -6.763856924121541e-5 + 3.4410971976147374e-21im
   -3.686889866871368e-5 - 3.599741391814458e-21im
    2.176614750375809e-5 + 2.6487551060169705e-22im
   -6.419493464380101e-5 - 2.8434333556510326e-25im
 -0.00010698407746021099 + 6.775843525379591e-21im
    4.065995056214414e-5 + 6.776282561183226e-21im

```

a pretty standard array of complex numbers.  
Now, I don’t care that much about the actual numbers, I just want to show a list of eleven complex numbers, so I assign the following substitution rule to the `filter` keyword of the doctest, :

```julia-auto
r"^(\s+)?[-+]?([0-9]*[.])?[0-9]+([eE][-+]?\d+)? [-+] [-+]?([0-9]*[.])?[0-9]+([eE][-+]?\d+)?im"m => "### + ###im"

```

and set the expected output to

```plaintext
11-element Vector{ComplexF64}:
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im

```

As you can see, I have to deal not only with the actual floating-point numbers, but also to the variable white space at the beginning of each line.  
That’s why I composed the regex with this structure in mind: “(variable amount of whitespace at beginning of line) (floating-point number) [-+] (floating-point number)im”. I put the `m` flag at the end of the raw string so that the `^` matches the start of each line instead of the start of the whole string only (it shouldn’t matter for the doctest, since, I think, each line is matched separately).

This works as expected in the REPL:

```julia-repl
julia> str = """11-element Vector{ComplexF64}:
                            0.0 + 0.0im
          -1.617576049847552e-6 + 6.617439891157473e-23im
          -2.216802827443102e-6 - 2.602082008249062e-18im
         -2.1754485086580383e-5 + 5.293471330973909e-22im
           4.696594721193055e-5 + 6.511993073259912e-21im
          -6.763856924121541e-5 + 3.4410971976147374e-21im
          -3.686889866871368e-5 - 3.599741391814458e-21im
           2.176614750375809e-5 + 2.6487551060169705e-22im
          -6.419493464380101e-5 - 2.8434333556510326e-25im
        -0.00010698407746021099 + 6.775843525379591e-21im
           4.065995056214414e-5 + 6.776282561183226e-21im""";

julia> replace(str, r"^(\s+)?[-+]?([0-9]*[.])?[0-9]+([eE][-+]?\d+)? [-+] [-+]?([0-9]*[.])?[0-9]+([eE][-+]?\d+)?im"m => "### + ###im") |> print
11-element Vector{ComplexF64}:
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im
### + ###im

```

and yet the debug messages of Documenter show this:

```plaintext
Evaluated output (filtered):

11-element Vector{ComplexF64}:
                     0.0 + 0.0im
   -1.617576049847552e-6 + 6.617439891157473e-23im
   -2.216802827443102e-6 - 2.602082008249062e-18im
  -2.1754485086580383e-5 + 5.293471330973909e-22im
    4.696594721193055e-5 + 6.511993073259912e-21im
   -6.763856924121541e-5 + 3.4410971976147374e-21im
   -3.686889866871368e-5 - 3.599741391814458e-21im
    2.176614750375809e-5 + 2.6487551060169705e-22im
   -6.419493464380101e-5 - 2.8434333556510326e-25im
 -0.00010698407746021099 + 6.775843525379591e-21im
    4.065995056214414e-5 + 6.776282561183226e-21im

```

meaning that the regex filer didn’t work as expected.

I’m at a loss here, can some please help me and show me what I’m doing wrong?

---

<div class="post-metadata">

**Author:** ![lilachint](https://avatars.discourse-cdn.com/v4/letter/l/ee7513/32.png) [@lilachint](https://discourse.julialang.org/u/lilachint)\
**Post date:** [April 15, 2026, 12:20am UTC](https://discourse.julialang.org/t/documenter-jl-doctest-does-not-apply-filter-as-expected/136706/2 "2026-04-15T00:20:20Z")

</div>

I checked Documenter.jl source code, and apparently it only replaces the results IF both the expected output and the actual output match the regex: [filter\_doctests](https://github.com/JuliaDocs/Documenter.jl/blob/b28c6732925bc101645d07a4da3c1d0b075a877f/src/doctests.jl#L359). Would you mind sharing both your jldoctest block and the output?

---

<div class="post-metadata">

**Author:** ![phx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/phx/32/219940_2.png) [@phx](https://discourse.julialang.org/u/phx)\
**Post date:** [April 15, 2026, 7:00am UTC](https://discourse.julialang.org/t/documenter-jl-doctest-does-not-apply-filter-as-expected/136706/3 "2026-04-15T07:00:30Z")

</div>

I managed to reproduce my problem just by drawing some random numbers, so that I don’t have to post the whole calculation that I had in my actual doctests.

This fails:

````plaintext
```jldoctest; filter = r"^(\s+)?[-+]?([0-9]*[.])?[0-9]+([eE][-+]?\d+)? [-+] [-+]?([0-9]*[.])?[0-9]+([eE][-+]?\d+)?im"m => "### + ###im"
julia> rand(ComplexF64, 7)                                                       
7-element Vector{ComplexF64}:                                                    
### + ###im                                                                      
### + ###im                                                                      
### + ###im                                                                      
### + ###im                                                                      
### + ###im                                                                      
### + ###im                                                                      
### + ###im                                                                      
```          

````

Documenter outputs this debug message:

````plaintext
┌ Debug: Verifying doctest at docs/testrand.md
│ 
│ ```jldoctest; filter = r"^(\s+)?[-+]?([0-9]*[.])?[0-9]+([eE][-+]?\d+)? [-+] [-+]?([0-9]*[.])?[0-9]+([eE][-+]?\d+)?im"m => "### + ###im"
│ julia> rand(ComplexF64, 7)
│ 7-element Vector{ComplexF64}:
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ```
│ 
│ Subexpression:
│ 
│ rand(ComplexF64, 7)
│ 
│ Evaluated output:
│ 
│ 7-element Vector{ComplexF64}:
│ 0.3666757812315634 + 0.5514356003090427im
│ 0.8281494833007406 + 0.6011516785255695im
│ 0.9634363838391149 + 0.24998991917806657im
│ 0.7728861352300147 + 0.22854726370723033im
│ 0.29978341620462245 + 0.5506865946612612im
│ 0.5720278400709893 + 0.8265610770601598im
│ 0.9441303898009322 + 0.4246940993558108im
│ 
│ Expected output:
│ 
│ 7-element Vector{ComplexF64}:
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ 
│ 1 doctest filter was applied:
│ 
│ r"^(\s+)?[-+]?([0-9]*[.])?[0-9]+([eE][-+]?\d+)? [-+] [-+]?([0-9]*[.])?[0-9]+([eE][-+]?\d+)?im"m => "### + ###im"
│ 
│ Evaluated output (filtered):
│ 
│ 7-element Vector{ComplexF64}:
│ 0.3666757812315634 + 0.5514356003090427im
│ 0.8281494833007406 + 0.6011516785255695im
│ 0.9634363838391149 + 0.24998991917806657im
│ 0.7728861352300147 + 0.22854726370723033im
│ 0.29978341620462245 + 0.5506865946612612im
│ 0.5720278400709893 + 0.8265610770601598im
│ 0.9441303898009322 + 0.4246940993558108im
│ 
│ Expected output (filtered):
│ 
│ 7-element Vector{ComplexF64}:
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ### + ###im
│ ### + ###im
└ @ Documenter /opt/julia/packages/Documenter/AXNMp/src/doctests.jl:394

````

If I don’t pre-emptively filter the output myself, everything works fine: the following doctest compiles without errors, both expected and evaluated output are filtered producing the output with the #s.

````plaintext
```jldoctest; filter = r"^(\s+)?[-+]?([0-9]*[.])?[0-9]+([eE][-+]?\d+)? [-+] [-+]?([0-9]*[.])?[0-9]+([eE][-+]?\d+)?im"m => "### + ###im"
julia> rand(ComplexF64, 7)           
7-element Vector{ComplexF64}:        
               0.0 + 0.0im
    -1.61757652e-6 + 6.6174473e-4im
    -2.21680202e-6 - 2.6020062e-5im
   -2.175448383e-5 + 5.2933909e-4im
     4.69659475e-5 + 6.5119912e-5im
    -6.76385641e-5 + 3.44107374e-6im
    -3.68688968e-5 - 3.5997458e-5im
```

````

(I realised then that this second doctest is actually what I want, because I don’t want to show the ###s in the documentation, but the actual numbers. Still, I expect Documenter not to throw an error in the first case…)

---

<div class="post-metadata">

**Author:** ![lilachint](https://avatars.discourse-cdn.com/v4/letter/l/ee7513/32.png) [@lilachint](https://discourse.julialang.org/u/lilachint)\
**Post date:** [April 15, 2026, 2:49pm UTC](https://discourse.julialang.org/t/documenter-jl-doctest-does-not-apply-filter-as-expected/136706/4 "2026-04-15T14:49:20Z")

</div>

The reason that Documenter.jl throws an error is because `filter` is only meant to help with testing non-deterministic functions. In your first docstring, since the actual REPL outputs the numbers, but your expected output is ###s, it would error as expected.

The use case for `filter` is not for making documents aesthetically pleasing, for that you might as well consider using `[...]` to truncate output or use something else. It is used for things like:

```jldoctest
julia> @time exp_func()
  0.03 seconds

```

Since the output of `@time` is non-deterministic, since I am testing the reliability of `@time` but not the speed of `exp_func()`, how many seconds it actually takes, in the expected output or in the evaluated output, doesn’t matter. Therefore `filter = r"[0-9\.]+" => "***"` could be used to mask (or remove) the parts i.e. the time that is not fixed across runs. So it would actually validate e.g. `*** seconds` against `*** seconds` instead of `0.03 seconds` against `0.05 seconds`.

Basically, `filter` is not intended for aesthetics but testing functions with unpredictable behaviour or special scenarios.

---

<div class="post-metadata">

**Author:** ![phx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/phx/32/219940_2.png) [@phx](https://discourse.julialang.org/u/phx)\
**Post date:** [April 16, 2026, 9:57am UTC](https://discourse.julialang.org/t/documenter-jl-doctest-does-not-apply-filter-as-expected/136706/5 "2026-04-16T09:57:45Z")

</div>

I see. So basically the filter regex has to match in both expected and evaluated cases. Thanks!