How to configure an artifact to use a header during package download?

Tooling Package Management
,
1

While trying to configure an artifact for a package stored on a private GitLab instance, I tested the requests using a private token to retrieve the file, which worked fine with Downloads.download by passing a headers=Dict("PRIVATE-TOKEN"=><private token>) to it.

While this works when defining the artifact (with some modification of ArtifactUtils.add_artifact!) it is useless when downloading the final package. In fact, at that point there does not seem to be a way to pass a token to the artifact download call.

I then tried to use a .netrc file based on @StefanKarpinski 's advice, but that seems to be somewhat ignored by curl as a call with the same address works when the token is specified but fails if the .netrc is used.

My .netrc has the following structure

machine <target server>
login <username that is used to connect with GitLab GUI, also tried oauth2>
password <GitLab issued private token>

Here is the result of a Downloads.download call

* Connected to <servername> (<serverip>) port 443 (#0)
* schannel: disabled automatic use of client certificate
* Server auth using Basic with user '<username>'
> GET /api/v4/projects/<filepath> HTTP/1.1
Host: <servername>
Authorization: Basic <encoded username:password>
Accept: */*
User-Agent: curl/7.84.0 julia/1.8

* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< Server: nginx
< Date: Tue, 14 Mar 2023 14:18:05 GMT
< Content-Type: application/json
< Content-Length: 35
< Connection: keep-alive
< Cache-Control: no-cache
< Vary: Origin
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-Request-Id: 01GVG7RH33HTJY9EVE071PW6MK
< X-Runtime: 0.035986
<
* Connection #0 to host <servername> left intact

I’ve tried a lot of solutions proposed here or on stackoverflow, libcurl site and all, but I did not achieve anything except by manually providing the token to curl.

EDIT the credentials in the .netrc seem to be recognized since the encoded the Authorization: Basic ... matches with the information in the .netrc. However these are somhow lost in the process which does not happen with the token in the header solution.

1 Like
2

For the general information of people that may stumble upon this topic, the GitLab API seems to only support token identification through headers at this point supports downloading file with authentication. I use this with a self hosted gitlab instance using API v4.

my artifact has its source url as
url = "<gitlab instance>/api/v<version>/projects/<project id>/repository/files/<path to file in repo with escaped special characters>/raw?ref=master&lfs=true&private_token=<project token>"
so that a user has no need to be authenticated to pull the artifact.

That of course works only if there is not too much security control there. If a user-by-user authentication must be used, I don’t think it is possible at the moment.

3

Relevant Pkg issue: Feature request: override artifact downloading for specific URLs · Issue #1888 · JuliaLang/Pkg.jl · GitHub.

4

Yep, I saw that one, but the fact is that for Gitlab StefanKarpinsky’s answer doesn’t work because .netrc is ignored it’s either in the request of with a header, nothing else seems to work.

I’ve closed Add a way to modify `download` options · Issue #18 · simeonschaub/ArtifactUtils.jl · GitHub to support that but only when Pkg will support this.