Artifacts: Can basic authentication be used?

General Usage
,
1

I am using the Julia Artifacts system and setting up a Artifacts.toml file where the binaries are for internal private usage. We are using something like JFrog Artifactory, where one typically does Basic authentication upon requests. Is it possible to add basic authentication somehow to the Artifacts.toml file without exposing username and password?

The idea is to be able to do something akin to this when Julia uses artifact"someresource" to access some resources mentioned in the Artifacts.toml file:

julia> r = HTTP.request(:GET, "https://erik:qwerty@mickeymouse.jfrog.io/artifactory/api/storage/stuff/myfile.txt");

The normal examples of Artifacts.toml files don’t show any examples using authentication. Here is just an example:

[socrates]
git-tree-sha1 = "43563e7631a7eafae1f9f8d9d332e3de44ad7239"
lazy = true

    [[socrates.download]]
    url = "https://github.com/staticfloat/small_bin/raw/master/socrates.tar.gz"
    sha256 = "e65d2f13f2085f2c279830e863292312a72930fee5ba3c792b14c33ce5c5cc58"

The idea is that instead of this GitHub URL I would use some JFrog Artifactory URL which might require authentication.

2

Note that Artifacts.toml can only handle tarballs with these extensions. For your use-case I think DataDeps.jl would be a better fit, which I believe also supports authentication

1 Like
3

Okay, my example was perhaps a bit bad, as I am actually intending to download binaries for different architectures. But I suppose what specifically you are downloading with DataDeps.jl doesn’t matter. I looked at BinDeps.jl but this seems to be about defining building rules. That might be interesting as well. But at the moment, I primarily looking for a way to fetch binaries stored on JFrog, which is used by my package. Basically I am trying to hack together a solution for OpenCV since BinaryBuilder is not up and running for it yet.

4

BinDeps.jl isn’t really developed anymore, lots of maintainability issues. Artifacts are good if you’re OK with their restrictions (only tarballs, no way to handle authentication, etc…), otherwise I’d use DataDeps.jl

1 Like
5

Also faced this problem trying to use private JLL-packages as a way of cross-platform binary management.

I think there are two problems here:

  1. Unable to configure auth information together with protocol / download method. I don’t know what would be simpler to use and provide for different cases from local machines to workflow runners - OAuth / PAT tokens / username:password / SSH keys. I would only prefer something similar to git, because you already have to use it.

  2. Current binary storage (github) doesn’t follow standard auth protocol: github browser URLs are not working with auth header for private repos - you are forced to use github API.

So you must first request a list of all releases:

curl -L \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer ghp_***"\
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/repos/USER/REPO_jll.jl/releases

Then find appropriate asset id, and form a second request to download it:

curl -L \
  -H "Accept: application/octet-stream" \
  -H "Authorization: Bearer ghp_***" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  -o wfdb.tar.gz \
  https://api.github.com/repos/USER/REPO_jll.jl/releases/assets/ASSET_ID

So, there are no private JLLs yet.