Unable to automatically install artifact

General Usage
1

I’m trying to setup an automatic julia artifact download for brainflow. However I am encountering a .tar unpacking error. Let me elaborate.

Here is my script to generate the artifact in a folder with an Artifacts.toml folder:

using SHA
using Pkg
using Pkg.Artifacts
using Tar

url = "https://github.com/brainflow-dev/brainflow/releases/download/3.7.2/compiled_libs.tar"
download_path = "$(tempname())-download.tar"
download(url, download_path)

brainflow_hash = create_artifact() do artifact_dir 
    Tar.extract(download_path, artifact_dir)
end

artifact_name = "brainflow"
artifacts_toml = abspath(joinpath(@__DIR__, "Artifacts.toml"))
Pkg.Artifacts.bind_artifact!(artifacts_toml, artifact_name, brainflow_hash, force=true)

# now the artifact already works
brainflow_artifact_path = artifact"brainflow"

# adding the download info
function file_hash(path)
    return open(path) do file
        bytes2hex(sha2_256(file))
    end
end
tar_hash_sha256 = file_hash(download_path)
Pkg.Artifacts.bind_artifact!(artifacts_toml, artifact_name, brainflow_hash; download_info=[(url, tar_hash_sha256)], force=true)

Now I remove the artifact and try to download it again, then I get an error:

julia> remove_artifact(brainflow_hash)
julia> brainflow_artifact_path = artifact"brainflow"
ERROR: Unable to automatically install 'brainflow' from 'C:\Users\matcox\Documents\Julia\Artifacts.toml'

Following this discourse discussion on Artifacts I tried to debug:

julia> download_artifact(brainflow_hash, url, tar_hash_sha256; verbose=true)
[ Info: No hash cache found
[ Info: Calculated hash 92bd7485de8ccbd19d27a9b203e8176806269415b1087c767bb36539efe063d9 for file C:\Users\matcox\AppData\Local\Temp\jl_A2FMwHFjeJ-download.tar
[ Info: Unpacking C:\Users\matcox\AppData\Local\Temp\jl_A2FMwHFjeJ-download.tar into C:\Users\matcox\.julia\artifacts\1681bc5c836c0465e02c42dce71db9572d723b6e...
false

So it goes wrong in unpack:

Pkg.PlatformEngines.unpack(download_path, brainflow_artifact_path; verbose=verbose)
ERROR: Could not unpack C:\Users\matcox\AppData\Local\Temp\jl_PJRsMp1PNn-download.tar into C:\Users\matcox\.julia\artifacts\1681bc5c836c0465e02c42dce71db9572d723b6e

unpack calls a command like such:

julia> excludelist = nothing # I confirmed this was indeed set to nothing inside unpack()
julia> cmd = Pkg.PlatformEngines.gen_unpack_cmd(download_path, brainflow_artifact_path, excludelist)
julia> run(cmd)
ERROR: failed process: Process(`'C:\Users\matcox\AppData\Local\Programs\Julia-1.5.1\bin\..\libexec\7z.exe' x 'C:\Users\matcox\AppData\Local\Temp\jl_PJRsMp1PNn-download.tar' -y -so`, ProcessExited(2)) [2]

I tried typing this line on the command line

C:\>C:\Users\matcox\AppData\Local\Programs\Julia-1.5.1\bin\..\libexec\7z.exe x C:\Users\matcox\AppData\Local\Temp\jl_PJRsMp1PNn-download.tar -y

7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21

Scanning the drive for archives:
1 file, 36666880 bytes (35 MiB)

Extracting archive: C:\Users\matcox\AppData\Local\Temp\jl_PJRsMp1PNn-download.tar
--
Path = C:\Users\matcox\AppData\Local\Temp\jl_PJRsMp1PNn-download.tar
Type = tar
Physical Size = 36666880
Headers Size = 23040
Code Page = UTF-8

ERROR: Can not open output file : Access is denied. : .\BoardController.dll
ERROR: Can not open output file : Access is denied. : .\BoardController.lib
...

Now I’m stuck. Any help would be much appreciated. I’d really like to get automatic artifact deployment working.

2

If you want to create artifacts yourself, I recommend using ArtifactUtils.jl. It’s a nicer interface than low-level Pkg commands, users often do some mistakes when calling them.

However in your case you seem to have some permissions issues :thinking:

1 Like
3

ArtifactUtils.jl looks great. I wish I’d known about it before :wink:

Just to confirm, similar error here:

using ArtifactUtils, Pkg.Artifacts
add_artifact!(
                  "Artifacts.toml",
                  "brainflow",
                  "https://github.com/brainflow-dev/brainflow/releases/download/3.7.2/compiled_libs.tar",
                  force=true,
              )
ERROR: Could not unpack C:\Users\matcox\AppData\Local\Temp\jl_h0ifO0k6hB into C:\Users\matcox\.julia\artifacts\jl_4t8KRo
4

For your info, I’m on a company laptop which has no windows admin rights. However, this is the first time I encounter permission issues with unpacking files.

I’m a little sad, because Tar.extract works perfectly fine.

5

Someone else with admin rights got the same unpack error on Windows: https://github.com/brainflow-dev/brainflow/issues/150#issuecomment-738496467

Should I create an issue somewhere?

6

I’ve solved the issue for now by writing my own custom artifact download function that uses Tar.extract(). It feels somewhat imperfect.

However, it also gave me the opportunity to add a developer feature to the package: to optionally use a custom compiled library in a fixed location. This was requested by the package owner. I would not know how to easily enable this with artifacts. I believe artifacts are there to simplify the life of users who are unaware of the fact that external libraries are used.