Help with hosting documentation using Documenter.jl

Tooling Package Management
1

Hi, I’m in desperate need of help: I don’t even know where to start.

I’m developing this package https://github.com/sergio-santos-group/ProtoSyn.jl/tree/dev

(I’m linking to the dev branch, this is what I’m trying to merge to the master branch).

However, I’m not the GitHub owner (it’s my supervisor who has recently left the academia, so contact with him has been limited. This may be important later on).

As you can probably see, I have a docs folder full with documentation. Everything runs ok on my machine, I can build the documentation. However, when running through Travis CI (you can check the travis.yml), I get some errors and the documentation is not uploaded. From previous releases, we already have a gh-pages branch, which I tried to update manually (by pushing the build folder to this branch), but that didn’t update the documentation webpage. Another “weird” aspect of my package is that it connects to Python packages using PyCall.jl. In this case, it’s the torchani package, I tried setting up a conda environment to download all Python-side dependencies automatically, see the protosyn.yml file in the project’s GitHub page.

I can share the big Travis CI log, if that helps, but I’ll just paste here some of the important parts (I think they’re important …):

  • Conda creates environment from file OK, activates it.
  • Python imports the torchani library OK (downloads ANI parameters, this is a “first step” when using torchani for the first time):
$ python -c "import torchani; torchani.models.ANI2x()"
Downloading ANI model parameters ...
  • The before_install build of PyCall.jl works OK:
$ julia -e 'using Pkg; Pkg.add.(["PyCall"]);Pkg.build("PyCall");using PyCall;println("Python for Julia $(PyCall.python) ver. $(PyCall.pyversion)")'
(... installs all dependencies ...)
Python for Julia /home/travis/.julia/conda/3/bin/python ver. 3.9.13
  • It seems that the PyCall.jl is pointing torwards the correct python (from the previously built conda environment).

Problem 1:
If I just run this (as previously described), I get an error stating that, when developing the docs, ProtoSyn can’t find torchani. Well, to me it seems like PyCall is pointing torwards the correct Python and that just doing python -c "import torchani; torchani.models.ANI2x()" during the before_install stage is able to find torchani just fine, from the conda environment. I have no idea why later, in the doc building step, it suddently can’t find the installed packages. Is there any extra setting I need to set? Perhaps re-load the conda env during this step, somehow?

Continuing …

I’m able to bypass this issue by adding this bit to the end of the PyCall.jl build step (described earlier):

run(`$(PyCall.python) -m pip install torch`);run(`$(PyCall.python) -m pip install torchani`)

I’m basically re-installing torch and torchani using pip. For some reason, Travis CI can now find this during the doc building step. Using this “trick”:

  • Documenter.jl and ProtoSyn.jl (my package) load OK:
The command "julia --project=docs/ -e 'using Pkg; Pkg.add("Documenter"); Pkg.instantiate(); Pkg.develop(PackageSpec(path=pwd()))'" exited with 0.
  • The next step (julia --project=docs/ docs/make.jl) FAILS.

Problem 2:

Downloading ANI model parameters ...
free(): invalid pointer
signal (6): Aborted
in expression starting at none:0
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7f603a3e5836)
unknown function (ip: 0x7f603a3ec8b9)
cfree at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_ZN4llvm2cl3optINS_15FunctionSummary23ForceSummaryHotnessTypeELb1ENS0_6parserIS3_EEED2Ev at /home/travis/julia/bin/../lib/julia/libLLVM-12jl.so (unknown line)
__cxa_finalize at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
__do_global_dtors_aux at /home/travis/julia/bin/../lib/julia/libLLVM-12jl.so (unknown line)
_fini at /home/travis/julia/bin/../lib/julia/libLLVM-12jl.so (unknown line)
unknown function (ip: 0x7f603a39f030)
exit at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
main at julia (unknown line)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x400808)
Allocations: 193211636 (Pool: 193157427; Big: 54209); GC: 171
/home/travis/.travis/functions: line 109:  7337 Aborted                 julia --project=docs/ docs/make.jl
The command "julia --project=docs/ docs/make.jl" exited with 134.

You can also check previous commits to see all the options I’ve already tried. Another problem that may be happening is failure to identify gen-keys, or something like that. As I’ve said, I don’t really have access to this GitHub repository settings …

Any idea on how to help me? Please?

2

Another attempt was the following:

jobs:
  include:
    - stage: "Documentation"
      julia: 1.7.2
      os: linux
      script:
        - wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh;
        - bash miniconda.sh -b -p $HOME/miniconda
        - export PATH="$HOME/miniconda/bin:$PATH"
        - conda env create -f protosyn.yml
        - source $(conda info --root)/etc/profile.d/conda.sh
        - conda activate protosyn
        - python -c "import torchani; torchani.models.ANI2x()"
        - julia -e 'using Pkg; Pkg.add.(["PyCall"]);Pkg.build("PyCall");using PyCall;println("Python for Julia $(PyCall.python) ver. $(PyCall.pyversion)");'
        - julia --project=docs/ -e 'using Pkg; Pkg.add("Documenter"); Pkg.instantiate(); Pkg.develop(PackageSpec(path=pwd()))'
        - julia --project=docs/ docs/make.jl
      after_success: skip

So, essentially, download conda env dependencies on the docuemntation build stage. Travis CI was not able to identify torch or torchani, this way.

┌ Warning: failed to run `@setup` block in src/protosyn-api/core/calculators/calculators-section.md
│ ```@setup calculators
│ using ProtoSyn
│ using ProtoSyn.Peptides
│ using ProtoSyn.Calculators
│ pose = ProtoSyn.build(Peptides.grammar, seq"GME")
│ torchani = Calculators.TorchANI.get_default_torchani_model()
│ energy_function = Calculators.EnergyFunction([torchani])
│ energy_function(pose)
│ ```
│   exception =
│    LoadError: ArgumentError: ref of NULL PyObject
│    in expression starting at string:7
└ @ Documenter.Expanders ~/.julia/packages/Documenter/MLty7/src/Utilities/Utilities.jl:34
The command "julia --project=docs/ docs/make.jl" exited with 0.

However, this build passes Travis CI (for some reason). When I merge with the master branch, however, the Travis CI fails. The error isn’t very explicit, but it does point to a Decumenter.jl file at the line with the following:

Git failed to fetch $upstream This can be caused by a DOCUMENTER_KEY variable that is not correctly set up. Make sure that the environment variable is properly set up as a Base64-encoded string of the SSH private key. You may need to re-generate the keys with DocumenterTools.

This is why I think it may also be a problem the keys on GitHub. Is there a way to bypass this check, somehow?