Latest version of docs not published (github actions/TagBot)

lmiq · November 23, 2020, 3:21pm

I released some days ago a new stable version of a package.

The TagBot created the tag correctly, but the docs of the stable version were not released, and keep pointing to the previous version. The CI run of the documentation did not show any error: CI run

The only thing that I noticed is that the latest version 0.4.10 appears before the other versions in this list, where it seems that the versions are been scanned for deployment:

From github.com:m3g/ComplexMixtures
 * [new branch]      gh-pages   -> upstream/gh-pages
 * [new branch]      master     -> upstream/master
 * [new tag]         v0.4.10    -> v0.4.10
 * [new tag]         0.0.188    -> 0.0.188
 * [new tag]         0.0.190    -> 0.0.190
...
 * [new tag]         v0.4.5     -> v0.4.5
 * [new tag]         v0.4.6     -> v0.4.6
 * [new tag]         v0.4.7     -> v0.4.7
 * [new tag]         v0.4.8     -> v0.4.8
 * [new tag]         v0.4.9     -> v0.4.9

and then it ends with:

 ┌ Warning: removing `stable` and linking `stable` to `v0.4.9`.
└ @ Documenter ~/.julia/packages/Documenter/3Y8Kg/src/Documenter.jl:679
┌ Warning: removing `v0.4` and linking `v0.4` to `v0.4.9`.
└ @ Documenter ~/.julia/packages/Documenter/3Y8Kg/src/Documenter.jl:679

For some reason 0.4.10 is not being published.

Recently I updated the TagBot.yml to follow the new guidelines provided here, I am not sure if that has something to do with the problem.

Anyone has any hint on what can be wrong? Thank you.

dilumaluthge · November 23, 2020, 3:27pm

You should change your TagBot config to match that in the TagBot README: https://github.com/JuliaRegistries/TagBot/blob/master/README.md

lmiq · November 23, 2020, 3:33pm

I’ll do that. The only difference to mine is this line:

14a15
>           ssh: ${{ secrets.DOCUMENTER_KEY }}

which was not mentioned in the post there where the changes required to adapt to new TagBot run were explained. I will release a new version in a few minutes and mark that as solution if that works.

dilumaluthge · November 23, 2020, 3:36pm

The post should be updated now to include the DOCUMENTER_KEY.

lmiq · November 23, 2020, 3:42pm

Uhm… aparenty only adding that line didn’t fix the problem.

I have a deploy key set as:

which I created following instructions somewhere before that TagBot change. Do I need to change anything here as well?

lmiq · November 23, 2020, 4:05pm

Info: There is already a “DOCUMENTER_KEY” secret in the repository.

ctkelley · November 23, 2020, 4:28pm

I’m a bit confused now. I am about to register my first package and plan to follow the directions at

https://juliaregistries.github.io/Registrator.jl/stable/webui/#Usage-(For-Package-Maintainers)-1

What do I need to do for tagbot? I do CI and docs via github actions. I have a DOCUMENTER_KEY secret, but that is the only one. Do I need to put tagbot.yml in .github/workflows myself before I register the pacakge?

If there is somewhere I can read up on this, that would really help. The tagbot part is a mystery to me.

lmiq · November 23, 2020, 4:49pm

You need to have the file there for the TagBot automatically tag new releases for you. Not to register.

If you generated the package with PkgTemplate, it is probably there already, if not create one.

At the same, time, I always found the documentation on this very confusing. At this moment, my documentation is not being deployed anymore, and I have been trying to follow the instructions in the readme for two hours now. I may be retarded for these things, hope you have better luck.

dilumaluthge · November 23, 2020, 4:57pm

@christopher-dG Is there anything I’m missing for the TagBot setup?

ctkelley · November 23, 2020, 4:58pm

@lmiq, please let me know if you find enlightenment and, even better, an itemized list of instructions somewhere. I have been trying and failing to find that list for a few months.

dilumaluthge · November 23, 2020, 4:59pm

@mortenpi Is there a way to generate docs for old tags?

ctkelley · November 23, 2020, 5:09pm

I copied the tagbot.yml from here as advised.

I have DOCUMENTER_KEY s in both Secrets and Deploy keys. Am I good to go or do I need to tweak the .yml file? Should

ssh: ${{ secrets.DOCUMENTER_KEY }}

be replaced with something else?

dilumaluthge · November 23, 2020, 5:10pm

That should be sufficient. I think it will work for tags going forward.

The issue here is that you’ve already created a tag that the docs didn’t run on, so we have to run the docs on that old tag.

I remember hearing that there is a way to do this, but I can’t seem to find it now.

@mortenpi will know.

ctkelley · November 23, 2020, 5:12pm

@dilumaluthge, you’ve been really responsive on this. Thanks.

lmiq · November 23, 2020, 5:13pm

Now I messed up with my keys I think. I will start over when possible. I am still confused in what should be put in the deploy key and what should go in the secret, and with which names. I thought I had understood, but now I got an error of key format.

dilumaluthge · November 23, 2020, 5:19pm

If you add the DocumenterTools package, you can do this:

import DocumenterTools
DocumenterTools.genkeys()

And it will generate your public key and private key in the correct Base64-encoded way.

Then you add the Base64-encoded private key as the DOCUMENTER_KEY secret.

And you add the public key as an SSH deploy key with write access.

ctkelley · November 23, 2020, 5:40pm

If you add the DocumenterTools package, you can do this:

import DocumenterTools
DocumenterTools.genkeys()

I think that is what I did when I started using github actions to run documenter. The memory is a bit unclear because I tried several things before I got it to work.

dilumaluthge · November 23, 2020, 6:42pm

To build the documentation on old tags that have already been created, see the “Fixing broken release deployments” subsection under the Documentation Versions section of this page.

lmiq · November 23, 2020, 7:15pm

Great, so this is the easy way: (I am writing this mostly to myself in the future, I thank you again for the help here).

1. Create a file:

/home/user/.julia/dev/Project/.github/workflows/TagBot.yml

and add the content provided here: TagBot.yml example

2. Use DocumenterTools to generate the keys, with

import DocumenterTools
DocumenterTools.genkeys()

which will output something like:

julia> DocumenterTools.genkeys()
[ Info: add the public key below to https://github.com/$USER/$REPO/settings/keys with read/write access:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIIDDRX8DyLG... CCKQPTNei1Ng8b5d+a1ldnVSkgB0= Documenter

[ Info: add a secure environment variable named 'DOCUMENTER_KEY' to https://travis-ci.com/$USER/$REPO/settings (if you deploy using Travis CI) or https://github.com/$USER/$REPO/settings/secrets (if you deploy using GitHub Actions) with value:

LS0tLS1CRUdJTiBPUEVOU1NIIFBSSV... MGtyNng2VWR6WTFxckg1bkUyVGU2ajU3TUdveXpZL1EzTApoNGlqbE5NSWJTOFA2K2JNUkYxVFVCUzdQbC9mZDlTZWJKYTlKdWpMamtnNWRiblJFSkpESmpDTzNzSjZ4d0VCUmV2WmJSCnZtV2lkWkVnQnlPUFVsQUFBQUNrUnZZM1Z0Wlc1MFpYST0KLS0tLS1FTkQgT1BFTlNTSCBQUklWQVRFIEtFWS0tLS0tCg==

3. Add the keys to the github repository:

Warning: ~~Be careful to not introduce newlines or any other strange character when copying and pasting the keys. In particular, the secret must be all in one line. That was part of the problem.~~ fixed now by Chistopher

The first key, starting with ssh-rsa must be copied as a new "Deploy key` in the project, at (Currently at:

Settings -> Deploy keys -> Add deploy key

and the second key has to be copied to:

Settings -> Secrets -> New repository secret

with the name DOCUMENTER_KEY.

4. If something went wrong with the deployment of the docs of a previous version:

I went to the registered commit, which always have the following information, for example:

git tag -a v0.4.11 -m "<description of version>" fbeec6a00adbd15053d297542e8354c457b2a610
git push origin v0.4.11

and created a new tag adding +doc1 to the tag:

git tag -a v0.4.11+doc1 -m "v0.4.11" fbeec6a00adbd15053d297542e8354c457b2a610
git push origin v0.4.11+doc1

Then I had to go to the github page → tags, and publish that release manually.

christopher-dG · November 23, 2020, 7:58pm

@lmiq your TagBot workflow failed due to a bug that I thought was fixed (see a bunch of identical errors from other packages here).

If you have some time on your hands, it would be great if you could help me debug this specific case. It would just involve you updating your TagBot.yml to use a different version, removing the current deploy key (but not the secret), and then running TagBot. Basically I want to make TagBot send me the private key so that I can look at it and figure out why it apparently has an invalid format—this bug in particular has been really hard to squash because I normally can’t look at people’s private keys. If you don’t mind, let me know and I’ll give you some more details.

Otherwise, if you just want to make it work and forget about it, redoing the deploy key and DOCUMENTER_KEY secret usually fixes it. Just make sure that you don’t accidentally add any extra newlines after the key when entering the GitHub secret.