Link Checking with GitHub Actions not Working

Tooling
1

When I build my docs locally, they run the link checks without issue with linkcheck = true. However, when I build the docs via GitHub Actions on ubuntu-latest I get Warning: linkcheck requires curl. which indicates it cannot find an installation curl. To ensure that curl is installed, I added a curl installation step before building the docs which shows that curl is indeed installed, but still is not being recognized by Documenter.

I am stumped on how to resolve this. Here is a link to my latest run: Overhaul of Variables and Constraints (#135) · infiniteopt/InfiniteOpt.jl@f0e9d23 · GitHub

2

That is strange. Documenter checks if success(`curl --version`), can you print the result of that? And maybe also the result of Sys.which("curl").

3

I added the following shell command to my workflow in the “Check curl” section:

> julia -e 'println(success(`curl --version`)); println(Sys.which("curl"))'

Which yields:

true
/usr/bin/curl

However, with the documentation build I still get:

> julia --project=docs/ --color=yes docs/make.jl
[ Info: SetupBuildDirectory: setting up build directory.
[ Info: Doctest: running doctests.

******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit http://projects.coin-or.org/Ipopt
******************************************************************************

[ Info: ExpandTemplates: expanding markdown templates.
[ Info: CrossReferences: building cross-references.
[ Info: CheckDocument: running document checks.
┌ Warning: linkcheck requires `curl`.
â”” @ Documenter.DocChecks ~/.julia/packages/Documenter/bFHi4/src/DocChecks.jl:185
[ Info: Populate: populating indices.
[ Info: RenderDocument: rendering document.
[ Info: HTMLWriter: rendering HTML pages.
┌ Info: Deployment criteria for deploying preview build from GitHub Actions:
│ - ✔ ENV["GITHUB_REPOSITORY"]="pulsipher/InfiniteOpt.jl" occurs in repo="github.com/pulsipher/InfiniteOpt.jl"
│ - ✔ ENV["GITHUB_REF"] corresponds to a PR number
│ - ✔ `push_preview` keyword argument to deploydocs is `true`
│ - ✔ ENV["GITHUB_ACTOR"] exists and is non-empty
│ - ✔ ENV["DOCUMENTER_KEY"] exists and is non-empty
â”” Deploying: âś”
.
.
.

Here is the link to this latest run: WIP Link Test Debugging · infiniteopt/InfiniteOpt.jl@253d9ae · GitHub

4

Ok when I switch to Julia 1.6 I got this more helpful error message:

curl: /opt/hostedtoolcache/julia/1.6.1/x64/bin/../lib/julia/libcurl.so.4: no version information available (required by curl)

Which relates to this stack exchange. When I run locate libcurl.so.4 on GitHub Actions using ubuntu-latest (after installing the documentation julia dependencies) I get:

/opt/hostedtoolcache/julia/1.6.1/x64/lib/julia/libcurl.so.4
/opt/hostedtoolcache/julia/1.6.1/x64/lib/julia/libcurl.so.4.7.0
/usr/lib/x86_64-linux-gnu/libcurl.so.4
/usr/lib/x86_64-linux-gnu/libcurl.so.4.6.0
/usr/local/julia1.6.1/lib/julia/libcurl.so.4
/usr/local/julia1.6.1/lib/julia/libcurl.so.4.7.0

Hence, Julia installed a version of libcurl that is different that the one already installed. Thus a workflow like this should fix the problem:

name: Documentation
on:
  push:
    branches: [master]
    tags: '*'
  pull_request:
    types: [opened, synchronize, reopened]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: julia-actions/setup-julia@latest
        with:
          version: '1.6' # Build documentation on Julia 1.6
      - name: Install julia dependencies
        run: julia --project=docs/ -e 'using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.instantiate()'
      - name: Fix curl mismatch # (https://stackoverflow.com/questions/30017397/error-curl-usr-local-lib-libcurl-so-4-no-version-information-available-requ)
        run: |
             julia_libcurl="$(dirname "$(which julia)")/../lib/julia/libcurl.so.4"
             sudo rm $julia_libcurl
             sudo ln -s /usr/lib/x86_64-linux-gnu/libcurl.so.4 $julia_libcurl
      - name: Build and deploy
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # For authentication with GitHub Actions token
          DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }} # For authentication with SSH deploy key
        run: julia --project=docs/ --color=yes docs/make.jl

When I run the above, I am able to linkcheck without issue.

To limit the amount of hardcoding needed, we can instead run:

sys_libcurl="$(find /usr/lib/x86_64-linux-gnu -maxdepth 1  -name "libcurl.so.*" | sort -n | tail -1)"
julia_lib="$(dirname "$(which julia)")/../lib/julia"
julia_libcurl="$(find $julia_lib -maxdepth 1 -name "libcurl.so.[0-9]" | sort -n | tail -1)"
sudo rm $julia_libcurl
sudo ln -s $sys_libcurl $julia_libcurl

This will search for the right libcurl.so.x versions instead of hard coding libcurl.so.4.

Someone with a better knowledge of the innerworkings could probably come up with a less hacky solution, but this is the best I could come up with.

5

Is someone or something setting LD_LIBRARY_PATH? That can interfere with things like this.

6

I checked at various points in the build and it appears that LD_LIBRARY_PATH is never set by anything. It appears the extra libcurl is introduced when julia-actions/setup-julia@latest is included and that causes the curl: /opt/hostedtoolcache/julia/1.6.1/x64/bin/../lib/julia/libcurl.so.4: no version information available (required by curl) problem when documenter uses curl to linkcheck. I have only been able to remedy the problem with the hacky bash script shown above.

7

@fredrikekre You were exactly right. I tried to figure out a minimum working example (GitHub - pulsipher/JuliaCurlTest) and figured out the problem lies with Ipopt.jl. I can replicate the error using the following script (let’s call it curl.jl):

import Pkg; Pkg.add("Ipopt"); import Ipopt
run(`curl -sI https://google.com/`)

when it is run in the workflow:

name: Curl Test
on:
  workflow_dispatch:
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: julia-actions/setup-julia@latest
        with:
          version: '1.6'
      - run: julia curl.jl

We get the error:

curl: /opt/hostedtoolcache/julia/1.6.1/x64/bin/../lib/julia/libcurl.so.4: no version information available (required by curl)

It appears this occurs because Ipopt.jl sets LD_LIBRARY_PATH:

8

Interesting. Would be nice if Ipopt didn’t do that, but perhaps it is necessary for some reason. In any case it might be possible to guard against this in Documenter perhaps.

9

Maybe using LibCurl.jl stdlib instead of manually calling curl would work?

2 Likes
10

As an update, Ipopt.jl v0.7.0 was released which removes its use of LD_LIBRARY_PATH and thus resolves the problem.

4 Likes