To create the doc of Muscade.jl, I use Literate.jl to run an example, which generates graphics (GLMakie.jl) and *.md files, that Documenter.jl compiles into *.html.
This all works fine on my computer, but the documentation.yml job on GITHUB’s server fails: Failed to precompile GLMakie, because dependency X11 (see line 800) is absent on a “headless server”.
My attempts to correct documentation.yml to install X11 have not born fruit (I don’t speak Yaml…). Ask the pros, I looked at how the doc of Interpreter.jl, which contains a figure, is generated: but I can’t find the relevant source code, and its documentation.yml does not refer to X11 at all.
So I try the other way, even though it’s less elegant (updating the doc no longer automatically updates the figure in the doc): Pregenerate the figures, and modify the example to show the code for graphic generation, but not actually execute it. This almost works. Using the #md markup, I get Interpreter.jl not to execute the code… but it generates an @example that Documenter.jl executes - so GLMakie, and hence X11 are still required…
So two separate questions, either of which can solve my problem - and teach me something.
What is wrong with my documentation.yml, how do I get it to install X11 on the server?
Is there an elegant way to get Interpreter.jl to a) typeset a snippet of code b) not execute it (as #md does) and c) generate MD that will not cause Documenter.jl to execute said snippet?
Interesting point: the plots we generate for now are 2D and could probably be handled by CairoMakie.jl, and you imply this would not require dependencies to be installed on the server: I haven’t tried.
But we will be creating 3D plots and interactive plots sooner or later, so a solution for GLMakie.jl specifically would be more useful.
The following 1 direct dependency failed to precompile:
GLMakie
Failed to precompile GLMakie [e9467ef8-e4e7-5192-8a1a-b1aee30e663a] to "/home/runner/.julia/compiled/v1.10/GLMakie/jl_KqVOX0".
┌ Warning: OpenGL/GLFW wasn't loaded correctly or couldn't be initialized.
│ This likely means, you're on a headless server without having OpenGL support setup correctly.
│ Have a look at the troubleshooting section in the readme:
│ https://github.com/MakieOrg/Makie.jl/tree/master/GLMakie#troubleshooting-opengl.
└ @ GLMakie ~/.julia/packages/GLMakie/TH3rf/src/gl_backend.jl:4
ERROR: LoadError: InitError: Exception[GLFW.GLFWError(65550, "X11: The DISPLAY environment variable is missing"), ErrorException("glfwInit failed")]
The above contains a Warning and an ERROR, and it is unclear which one causes the other. There is a complaint about DISPLAY not set, in spite of the instruction DISPLAY=:0.
Ah that’s a different step than the one you added the prefix to. So during precompilation after installation, you get the same problem when GLMakie is loaded. Just add the prefix to that command, too, and you should be good.
OK, here my ignorance of YAML, and not least, of what it expresses, handicaps me. I am afraid you will have to feed me with a teaspoon.
Ah that’s a different step than the one you added the prefix to. So during precompilation after installation, you get the same problem when GLMakie is loaded. Just add the prefix to that command, too, and you should be good.
Trying to interpret you: (I make a list of statement you could please comment with “true” or “false”
Steps have distinct environments.
DISPLAY is local to an environment
The following is a step
run: >
cd docs;
DISPLAY=:0 xvfb-run -s '-screen 0 1024x768x24'
julia --color=yes make.jl
This step executes without error, because I define DISPLAY in it
The following is a step
- name: Upload site as artifact
uses: actions/upload-artifact@v4
with:
name: Docs build
path: ./docs/build
But if fails because I do not define DISPLAY in it
Each run specifies a shell command, if you want to run that shell command through xvfb-run then you need to prefix it with that. If you take a look at your yaml:
- name: Install dependencies
run: julia --project=docs/ -e 'using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.instantiate()'
- name: Build and deploy docs
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: >
cd docs;
DISPLAY=:0 xvfb-run -s '-screen 0 1024x768x24'
julia --color=yes make.jl
then the Build and deploy docs step runs two separate commands. The > means all following lines are concatenated into one big line. So you have cd docs and then the command DISPLAY=:0 xvfb-run -s '-screen 0 1024x768x24' julia --color=yes make.jl which sets DISPLAY and then uses xvfb-run to run julia. Those are not separate commands even though they appear on two different lines.
But in the previous step Install dependencies, you have run: julia --project=docs/ -e 'using Pkg... so this runs julia without the xvfb-run wrapper. When this commands loads GLMakie in precompilation, it fails. So change that one to DISPLAY=:0 xvfb-run -s '-screen 0 1024x768x24' julia --project=docs/ ... as well.
The Upload site as artifact step is unrelated to these problems.
YES!!! That definitely helped. Thanks in particular for your explanation.
Having corrected part of my documentation.yml to
- name: Install Julia dependencies
run: >
DISPLAY=:0 xvfb-run -s '-screen 0 1024x768x24'
julia --project=docs/ -e 'using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.instantiate()'
I get the output
GLMakie Waiting for background task / IO / timer.
[pid 7431] waiting for IO to finish:
Handle type uv_handle_t->data
timer 0x2807dec0->0x7fcfff185120
This means that a package has started a background task or event source that has not finished running. For precompilation to complete successfully, the event source needs to be closed explicitly. See the developer documentation on fixing precompilation hangs for more help.
69831.9 ms ✓ GLMakie
268 dependencies successfully precompiled in 338 seconds. 9 already precompiled.
2 dependencies had output during precompilation:
┌ MKL_jll
│ Downloading artifact: IntelOpenMP
└
┌ GLMakie
│ [pid 7431] waiting for IO to finish:
│ Handle type uv_handle_t->data
│ timer 0x2807dec0->0x7fcfff185120
│ This means that a package has started a background task or event source that has not finished running. For precompilation to complete successfully, the event source needs to be closed explicitly. See the developer documentation on fixing precompilation hangs for more help.
└
No error here, but a somewhat ominous warning about “waiting for IO to finish”.
Executiom of my make.jl still has a trivial error “file not found” that I will fix, I am curious whether the not-completed compilation will have an impact: I will update.
Update:
The doc compiles on the server! Thank you so much!
That definitely helped. Thanks in particular for your explanation.