How to avoid `using MyPackage` in every doctest?

I have a jldoctest entry which is:

julia> using CellListMap

julia> box = Box([100,70,130],12)
Box{OrthorhombicCell, 3, Float64, 9}
  unit cell matrix: [100.0 0.0 0.0; 0.0 70.0 0.0; 0.0 0.0 130.0]
  cutoff: 12.0
  number of computing cells on each dimension: [10, 7, 12]
  computing cell sizes: [12.5, 14.0, 13.0] (lcell: 1)
  Total number of cells: 840

Do I really need using CellListMap there? I am getting Box undefined if I do not put it there, but that is a function of the package and exported by the package. I am using:

using Test, Documenter, CellListMap
doctest(CellListMap)

It seems that I need to use this command:

DocMeta.setdocmeta!(MyPackage, :DocTestSetup, :(using MyPackage); recursive=true)

Is that the only solution? Where can I put that such that I don’t have to type it every time? I tried adding it to docs/make.jl but that didn’t work. That help entry, in particular, is within the comment block above the corresponding function.

2 Likes

Actually the whole jldoctest workflow is not working for me. My runtests.jl starts with:

using CellListMap
using StaticArrays
using Test
using Documenter

DocMeta.setdocmeta!(
    CellListMap,
    :DocTestSetup,
    :(using CellListMap);
    recursive=true
)
doctest(CellListMap)

But I wtill get Box undefined in a the jldoctest that is:

julia> box = Box([100,70,130],12)
Box{OrthorhombicCell, 3, Float64, 9}
  unit cell matrix: [100.0 0.0 0.0; 0.0 70.0 0.0; 0.0 0.0 130.0]
  cutoff: 12.0
  number of computing cells on each dimension: [10, 7, 12]
  computing cell sizes: [12.5, 14.0, 13.0] (lcell: 1)
  Total number of cells: 840

and this one is in a regular .md file of the docs.

Also, I get loads of errors of the sort:

β”Œ Error: doctest failure in ~/.julia/dev/CellListMap/src/testing.jl
β”‚ 
β”‚ ```jldoctest
β”‚ julia> box = Box([ 50.  0. 00. 
β”‚                     0. 30. 30.          
β”‚                     0. 00. 50. ],  2.)
β”‚ 
β”‚ julia> x = 100*rand(SVector{3,Float64},10000);
β”‚ 
β”‚ julia> p = [ CellListMap.wrap_to_first(x,box) for x in x ];
β”‚ 
β”‚ julia> scatter(Tuple.(p),aspect_ratio=1,framestyle=:box,label=:none)
β”‚ ```
β”‚ 
β”‚ Subexpression:
β”‚ 
β”‚ x = 100*rand(SVector{3,Float64},10000);
β”‚ 
β”‚ Evaluated output:
β”‚ 
β”‚ ERROR: UndefVarError: SVector not defined
β”‚ Stacktrace:
β”‚  [1] top-level scope
β”‚    @ none:1
β”‚ 
β”‚ Expected output:
β”‚ 
β”‚ 
β”‚ 
β”‚   diff =
β”‚    ERROR: UndefVarError: SVector not defined
β”‚    Stacktrace:
β”‚     [1] top-level scope
β”‚       @ none:1
β”” @ Documenter.DocTests ~/.julia/packages/Documenter/R2HVS/src/DocTests.jl:385

But none of these fields are jldoctests, so they should not be evaluated. What even stranger is that the file where the error is indicated, testing.jl has nothing to do with that code block, which is another file (pbc.md), and is a standard julia block:

#```julia
julia> box = Box([ 50.  0. 00. 
                    0. 30. 30.          
                    0. 00. 50. ],  2.)

julia> x = 100*rand(SVector{3,Float64},10000);

julia> p = [ CellListMap.wrap_to_first(x,box) for x in x ];

julia> scatter(Tuple.(p),aspect_ratio=1,framestyle=:box,label=:none)
#```

I would also appreciate if some points to a package where jldoctests are being used both in the documentation and on function comments, such that I can copy them. I looked at some examples suggested in the Documenter docs page, but at least the ones I have checked didn’t seem to be using these features.

See
https://github.com/Nosferican/WebDriver.jl/blob/master/docs/make.jl
or
https://github.com/uva-bi-sdad/GHOST.jl/blob/main/test/runtests.jl
https://github.com/uva-bi-sdad/GHOST.jl/blob/main/src/01_BaseUtils.jl

Those are a few examples of DocMeta.setdocmeta!

2 Likes

you know, include is just like pasting code

using foo
include("do1.jl")

@Rratic What you mean by that exactly in this context?

@Nosferican, thanks for the examples. At a first sight I do not see any difference relative to what I’m doing…

If you put that part in the make.jl file, it automatically runs before the tests. Why is that a problem still?

Is Box exported? Otherwise, use MyModule.Box. The idea is that doctests look like it would look for an user, so the code is evaluated outside MyModule.

Also add using SVector to setdocmeta!.

1 Like

Because it does not work :frowning: (see below)

Yes.

That block is was not a jldoctest block, I was not expecting it to be tested. And the doctests are trying to run many blocks that are not jldoctest blocks and throwing a lot of errors. The errors also, strangely, keep referencing my testing.jl file, which not only does not contain any jdoctest, but neither the blocks it is complaining about. Is a file named testing.jl a problem in principle for any reason? (edit: now I changed all julia blocks to julia-repl blocks, and it seems that this errors disappeared. Are julia blocks ran by doctests?

This is my current make.jl:

import Pkg
Pkg.add("Documenter")
using Documenter
using CellListMap
DocMeta.setdocmeta!(CellListMap, :DocTestSetup, :(using CellListMap); recursive=true)
push!(LOAD_PATH,"../src/")
makedocs(
    modules=[CellListMap],
    sitename="CellListMap.jl",
    pages = [
        "Overview" => "index.md",
        "Examples" => "examples.md",
        "Periodic conditions" => "pbc.md",
        "Parallelization" => "parallelization.md",
        "Performance" => "performance.md",
        "Reference" => "reference.md",
        "Help entries" => "help.md",
    ]
)
deploydocs(
    repo = "github.com/m3g/CellListMap.jl.git",
    target = "build",
    branch = "gh-pages",
    versions = ["stable" => "v^", "v#.#" ],
)

This is what I’m getting if I try to run the doctests (this is the same error I get if I run include("make.jl"), with the make.jl file above:

julia> using Documenter, CellListMap
[ Info: Precompiling CellListMap [69e1c6dd-3888-40e6-b3c8-31ac5f578864]

julia> using Test

julia> DocMeta.setdocmeta!(CellListMap, :DocTestSetup, :(using CellListMap); recursive=true)

julia> doctest(CellListMap)
[ Info: SetupBuildDirectory: setting up build directory.
[ Info: Doctest: running doctests.
β”Œ Error: doctest failure in ~/.julia/dev/CellListMap/docs/src/pbc.md:7-15
β”‚ 
β”‚ ```jldoctest
β”‚ julia> box = Box([100,70,130],12)
β”‚ Box{OrthorhombicCell, 3, Float64, 9}
β”‚   unit cell matrix: [100.0 0.0 0.0; 0.0 70.0 0.0; 0.0 0.0 130.0]
β”‚   cutoff: 12.0
β”‚   number of computing cells on each dimension: [10, 7, 12]
β”‚   computing cell sizes: [12.5, 14.0, 13.0] (lcell: 1)
β”‚   Total number of cells: 840
β”‚ ```
β”‚ 
β”‚ Subexpression:
β”‚ 
β”‚ box = Box([100,70,130],12)
β”‚ 
β”‚ Evaluated output:
β”‚ 
β”‚ ERROR: UndefVarError: Box not defined
β”‚ Stacktrace:
β”‚  [1] top-level scope
β”‚    @ none:1
β”‚ 
β”‚ Expected output:
β”‚ 
β”‚ Box{OrthorhombicCell, 3, Float64, 9}
β”‚   unit cell matrix: [100.0 0.0 0.0; 0.0 70.0 0.0; 0.0 0.0 130.0]
β”‚   cutoff: 12.0
β”‚   number of computing cells on each dimension: [10, 7, 12]
β”‚   computing cell sizes: [12.5, 14.0, 13.0] (lcell: 1)
β”‚   Total number of cells: 840
β”‚ 
β”‚   diff =
β”‚    Box{OrthorhombicCell, 3, Float64, 9}
β”‚      unit cell matrix: [100.0 0.0 0.0; 0.0 70.0 0.0; 0.0 0.0 130.0]
β”‚      cutoff: 12.0
β”‚      number of computing cells on each dimension: [10, 7, 12]
β”‚      computing cell sizes: [12.5, 14.0, 13.0] (lcell: 1)
β”‚      Total number of cells: 840ERROR: UndefVarError: Box not defined
β”‚    Stacktrace:
β”‚     [1] top-level scope
β”‚       @ none:1
β”” @ Documenter.DocTests ~/.julia/packages/Documenter/R2HVS/src/DocTests.jl:385
β”Œ Error: Doctesting failed
β”‚   exception =
β”‚    `makedocs` encountered a doctest error. Terminating build
β”‚    Stacktrace:
β”‚      [1] error(s::String)
β”‚        @ Base ./error.jl:33
β”‚      [2] runner(#unused#::Type{Documenter.Builder.Doctest}, doc::Documenter.Documents.Document)
β”‚        @ Documenter.Builder ~/.julia/packages/Documenter/R2HVS/src/Builder.jl:217
β”‚      [3] dispatch(#unused#::Type{Documenter.Builder.DocumentPipeline}, x::Documenter.Documents.Document)
β”‚        @ Documenter.Utilities.Selectors ~/.julia/packages/Documenter/R2HVS/src/Utilities/Selectors.jl:170
β”‚      [4] #2
β”‚        @ ~/.julia/packages/Documenter/R2HVS/src/Documenter.jl:257 [inlined]
β”‚      [5] cd(f::Documenter.var"#2#3"{Documenter.Documents.Document}, dir::String)
β”‚        @ Base.Filesystem ./file.jl:106
β”‚      [6] makedocs(; debug::Bool, format::Documenter.Writers.HTMLWriter.HTML, kwargs::Base.Iterators.Pairs{Symbol, Any, NTuple{6, Symbol}, NamedTuple{(:root, :source, :sitename, :doctest, :modules, :doctestfilters), Tuple{String, String, String, Symbol, Vector{Module}, Vector{Regex}}}})
β”‚        @ Documenter ~/.julia/packages/Documenter/R2HVS/src/Documenter.jl:256
β”‚      [7] (::Documenter.var"#all_doctests#32"{Bool, Vector{Regex}, Vector{Module}})()
β”‚        @ Documenter ~/.julia/packages/Documenter/R2HVS/src/Documenter.jl:849
β”‚      [8] macro expansion
β”‚        @ ~/.julia/packages/Documenter/R2HVS/src/Documenter.jl:870 [inlined]
β”‚      [9] macro expansion
β”‚        @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1151 [inlined]
β”‚     [10] doctest(source::String, modules::Vector{Module}; fix::Bool, testset::String, doctestfilters::Vector{Regex})
β”‚        @ Documenter ~/.julia/packages/Documenter/R2HVS/src/Documenter.jl:870
β”‚     [11] doctest(package::Module; manual::Bool, testset::Nothing, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
β”‚        @ Documenter ~/.julia/packages/Documenter/R2HVS/src/Documenter.jl:805
β”‚     [12] doctest(package::Module)
β”‚        @ Documenter ~/.julia/packages/Documenter/R2HVS/src/Documenter.jl:792
β”‚     [13] top-level scope
β”‚        @ REPL[4]:1
β”‚     [14] eval
β”‚        @ ./boot.jl:360 [inlined]
β”‚     [15] eval
β”‚        @ ./Base.jl:39 [inlined]
β”‚     [16] repleval(m::Module, code::Expr, #unused#::String)
β”‚        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.3.30/scripts/packages/VSCodeServer/src/repl.jl:157
β”‚     [17] (::VSCodeServer.var"#69#71"{Module, Expr, REPL.LineEditREPL, REPL.LineEdit.Prompt})()
β”‚        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.3.30/scripts/packages/VSCodeServer/src/repl.jl:123
β”‚     [18] with_logstate(f::Function, logstate::Any)
β”‚        @ Base.CoreLogging ./logging.jl:491
β”‚     [19] with_logger
β”‚        @ ./logging.jl:603 [inlined]
β”‚     [20] (::VSCodeServer.var"#68#70"{Module, Expr, REPL.LineEditREPL, REPL.LineEdit.Prompt})()
β”‚        @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-1.3.30/scripts/packages/VSCodeServer/src/repl.jl:124
β”‚     [21] #invokelatest#2
β”‚        @ ./essentials.jl:708 [inlined]
β”‚     [22] invokelatest(::Any)
β”‚        @ Base ./essentials.jl:706
β”‚     [23] macro expansion
β”‚        @ ~/.vscode/extensions/julialang.language-julia-1.3.30/scripts/packages/VSCodeServer/src/eval.jl:34 [inlined]
β”‚     [24] (::VSCodeServer.var"#53#54")()
β”‚        @ VSCodeServer ./task.jl:411
β”” @ Documenter ~/.julia/packages/Documenter/R2HVS/src/Documenter.jl:859
Doctests: CellListMap: Test Failed at /home/leandro/.julia/packages/Documenter/R2HVS/src/Documenter.jl:870
  Expression: all_doctests()
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/Documenter/R2HVS/src/Documenter.jl:870 [inlined]
 [2] macro expansion
   @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1151 [inlined]
 [3] doctest(source::String, modules::Vector{Module}; fix::Bool, testset::String, doctestfilters::Vector{Regex})
   @ Documenter ~/.julia/packages/Documenter/R2HVS/src/Documenter.jl:870
Test Summary:         | Fail  Total
Doctests: CellListMap |    1      1
ERROR: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken.

julia> 
1 Like

Hmm, I should have taken a look at my pc but am at phone now. Is it maybe something with the syntax? Like missing backticks or so?

From reading the conversation, you want to make sure that you are exposing all the elements of the namespace you are referring to. For example, if you have using MyPkg and try calling magic which is not exposed by having export magic in MyPkg.jl you would still get that error.

What I tend to do is to run build the docs with the jldoctest as part of my test suite. For the documentation step of CI I just refer to the docs built during the testing.

1 Like

It is exported. I have no idea what’s wrong. For the moment I removed the tests. I tried to add a @meta block in the page as well, but that didn’t work either.

Looking at this error, the doctest is in a Markdown file? In that case, you need a DocTestSetup in an at-meta block in the same .md file before the doctest. That should fix this particular error at least.

2 Likes

I tried that as well and got the same error. I will have to build a mwe when I have time.

1 Like