Unable to automatically install artifact

I’m trying to setup an automatic julia artifact download for brainflow. However I am encountering a .tar unpacking error. Let me elaborate.

Here is my script to generate the artifact in a folder with an Artifacts.toml folder:

using SHA
using Pkg
using Pkg.Artifacts
using Tar

url = "https://github.com/brainflow-dev/brainflow/releases/download/3.7.2/compiled_libs.tar"
download_path = "$(tempname())-download.tar"
download(url, download_path)

brainflow_hash = create_artifact() do artifact_dir 
    Tar.extract(download_path, artifact_dir)

artifact_name = "brainflow"
artifacts_toml = abspath(joinpath(@__DIR__, "Artifacts.toml"))
Pkg.Artifacts.bind_artifact!(artifacts_toml, artifact_name, brainflow_hash, force=true)

# now the artifact already works
brainflow_artifact_path = artifact"brainflow"

# adding the download info
function file_hash(path)
    return open(path) do file
tar_hash_sha256 = file_hash(download_path)
Pkg.Artifacts.bind_artifact!(artifacts_toml, artifact_name, brainflow_hash; download_info=[(url, tar_hash_sha256)], force=true)

Now I remove the artifact and try to download it again, then I get an error:

julia> remove_artifact(brainflow_hash)
julia> brainflow_artifact_path = artifact"brainflow"
ERROR: Unable to automatically install 'brainflow' from 'C:\Users\matcox\Documents\Julia\Artifacts.toml'

Following this discourse discussion on Artifacts I tried to debug:

julia> download_artifact(brainflow_hash, url, tar_hash_sha256; verbose=true)
[ Info: No hash cache found
[ Info: Calculated hash 92bd7485de8ccbd19d27a9b203e8176806269415b1087c767bb36539efe063d9 for file C:\Users\matcox\AppData\Local\Temp\jl_A2FMwHFjeJ-download.tar
[ Info: Unpacking C:\Users\matcox\AppData\Local\Temp\jl_A2FMwHFjeJ-download.tar into C:\Users\matcox\.julia\artifacts\1681bc5c836c0465e02c42dce71db9572d723b6e...

So it goes wrong in unpack:

Pkg.PlatformEngines.unpack(download_path, brainflow_artifact_path; verbose=verbose)
ERROR: Could not unpack C:\Users\matcox\AppData\Local\Temp\jl_PJRsMp1PNn-download.tar into C:\Users\matcox\.julia\artifacts\1681bc5c836c0465e02c42dce71db9572d723b6e

unpack calls a command like such:

julia> excludelist = nothing # I confirmed this was indeed set to nothing inside unpack()
julia> cmd = Pkg.PlatformEngines.gen_unpack_cmd(download_path, brainflow_artifact_path, excludelist)
julia> run(cmd)
ERROR: failed process: Process(`'C:\Users\matcox\AppData\Local\Programs\Julia-1.5.1\bin\..\libexec\7z.exe' x 'C:\Users\matcox\AppData\Local\Temp\jl_PJRsMp1PNn-download.tar' -y -so`, ProcessExited(2)) [2]

I tried typing this line on the command line

C:\>C:\Users\matcox\AppData\Local\Programs\Julia-1.5.1\bin\..\libexec\7z.exe x C:\Users\matcox\AppData\Local\Temp\jl_PJRsMp1PNn-download.tar -y

7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21

Scanning the drive for archives:
1 file, 36666880 bytes (35 MiB)

Extracting archive: C:\Users\matcox\AppData\Local\Temp\jl_PJRsMp1PNn-download.tar
Path = C:\Users\matcox\AppData\Local\Temp\jl_PJRsMp1PNn-download.tar
Type = tar
Physical Size = 36666880
Headers Size = 23040
Code Page = UTF-8

ERROR: Can not open output file : Access is denied. : .\BoardController.dll
ERROR: Can not open output file : Access is denied. : .\BoardController.lib

Now I’m stuck. Any help would be much appreciated. I’d really like to get automatic artifact deployment working.

If you want to create artifacts yourself, I recommend using ArtifactUtils.jl. It’s a nicer interface than low-level Pkg commands, users often do some mistakes when calling them.

However in your case you seem to have some permissions issues :thinking:

1 Like

ArtifactUtils.jl looks great. I wish I’d known about it before :wink:

Just to confirm, similar error here:

using ArtifactUtils, Pkg.Artifacts
ERROR: Could not unpack C:\Users\matcox\AppData\Local\Temp\jl_h0ifO0k6hB into C:\Users\matcox\.julia\artifacts\jl_4t8KRo

For your info, I’m on a company laptop which has no windows admin rights. However, this is the first time I encounter permission issues with unpacking files.

I’m a little sad, because Tar.extract works perfectly fine.

Someone else with admin rights got the same unpack error on Windows: https://github.com/brainflow-dev/brainflow/issues/150#issuecomment-738496467

Should I create an issue somewhere?

I’ve solved the issue for now by writing my own custom artifact download function that uses Tar.extract(). It feels somewhat imperfect.

However, it also gave me the opportunity to add a developer feature to the package: to optionally use a custom compiled library in a fixed location. This was requested by the package owner. I would not know how to easily enable this with artifacts. I believe artifacts are there to simplify the life of users who are unaware of the fact that external libraries are used.