# Cannot build CUDAnative on login node (works on compute node)

**URL:** <https://discourse.julialang.org/t/cannot-build-cudanative-on-login-node-works-on-compute-node/27860>\
**Category:** GPU\
**Tags:** question\
**Created:** [August 22, 2019, 4:58pm UTC](https://discourse.julialang.org/t/cannot-build-cudanative-on-login-node-works-on-compute-node/27860 "2019-08-22T16:58:35Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![samo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samo/32/35398_2.png) [@samo](https://discourse.julialang.org/u/samo)\
**Post date:** [August 22, 2019, 4:58pm UTC](https://discourse.julialang.org/t/cannot-build-cudanative-on-login-node-works-on-compute-node/27860/1 "2019-08-22T16:58:35Z")

</div>

# Context

I am preparing a site-wide installation of Julia with the CUDA packages (CUDAdrv, CUDAnative and CuArrays) on our GPU supercomputer. To this purpose, I am creating automated installation recipes as for all our software stack. These recipes execute in an automatic fashion **on the login nodes** , which do not have any GPUs.

# Problem

`CUDAnative` does not build on the login nodes. The reason seems to be that the installation procedure relies on being able to initialize `CUDAdrv` at the beginning. The error message is the following:

```julia
(v1.0.4-gpu) pkg> add CUDAnative
  Updating registry at `./registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed TimerOutputs ─────── v0.5.0
 Installed Adapt ────────────── v1.0.0
 Installed CEnum ────────────── v0.2.0
 Installed Crayons ──────────── v4.0.0
 Installed OrderedCollections ─ v1.1.0
 Installed DataStructures ───── v0.17.0
 Installed CUDAnative ───────── v2.2.1
 Installed LLVM ─────────────── v1.2.0
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  [be33ccc6] + CUDAnative v2.2.1
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Manifest.toml`
  [79e6a3ab] + Adapt v1.0.0
  [fa961155] + CEnum v0.2.0
  [be33ccc6] + CUDAnative v2.2.1
  [a8cc5b0e] + Crayons v4.0.0
  [864edb3b] + DataStructures v0.17.0
  [929cbde3] + LLVM v1.2.0
  [bac558e1] + OrderedCollections v1.1.0
  [a759f4b9] + TimerOutputs v0.5.0
  [2a0f44e3] + Base64 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [37e2e46d] + LinearAlgebra 
  [d6f4376e] + Markdown 
  [9a3f8284] + Random 
  [9e88b42a] + Serialization 
  [6462fe0b] + Sockets 
  [8dfed614] + Test 
  Building CUDAnative → `/scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAnative/nItlk/deps/build.log`
┌ Error: Error building `CUDAnative`: 
│ ERROR: LoadError: InitError: CUDA error: unknown error (code #999, ERROR_UNKNOWN)
│ Stacktrace:
│ [1] macro expansion at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAdrv/ADRHQ/src/base.jl:145 [inlined]
│ [2] init(::Int64) at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAdrv/ADRHQ/src/init.jl:10 (repeats 2 times)
│ [3] __init__ () at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAdrv/ADRHQ/src/CUDAdrv.jl:83
│ [4] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:633
│ [5] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:713
│ [6] _require(::Base.PkgId) at ./loading.jl:937
│ [7] require(::Base.PkgId) at ./loading.jl:858
│ [8] require(::Module, ::Symbol) at ./loading.jl:853
│ [9] include at ./boot.jl:317 [inlined]
│ [10] include_relative(::Module, ::String) at ./loading.jl:1044
│ [11] include(::Module, ::String) at ./sysimg.jl:29
│ [12] include(::String) at ./client.jl:392
│ [13] top-level scope at none:0
│ during initialization of module CUDAdrv
│ in expression starting at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAnative/nItlk/deps/build.jl:2
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1096

```

Note that `CUDAdrv` itself builds properly on the login nodes. `CUDAnative` builds only correctly on the compute nodes. For details, see the shell output in the appendix at the end of this post.

# Question

**Can we avoid the initialization of `CUDAdrv` when building `CUDAnative` in order to enable building it on the login nodes?** – Having to build `CUDAnative` on the compute nodes would be a severe complification of the automated site-wide installation with some additional drawbacks.

# Appendix

Shell output of 1) successful installation of `CUDAdrv` on login node, 2) successful precompilation of `CUDAdrv` on compute node, 3) failing installation of `CUDAnative` on login node, 4) failing `using CUDAdrv` on login node, 5) successful build of `CUDAnative` on compute node, 6) successful precompilation of `CUDAnative` on compute node:

```julia
omlins@daint101:~> module use /scratch/snx3000/omlins/9_soft_install_julia/modules/all
omlins@daint101:~> module load julia/1.0.4 
omlins@daint101:~> cd /scratch/snx3000/omlins/julia_cuda_test_install
omlins@daint101:/scratch/snx3000/omlins/julia_cuda_test_install> export JULIA_DEPOT_PATH=.
omlins@daint101:/scratch/snx3000/omlins/julia_cuda_test_install> julia
(v1.0.4-gpu) pkg> status
    Status `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  (empty environment)

(v1.0.4-gpu) pkg> add CUDAdrv
   Cloning default registries into /scratch/snx3000/omlins/julia_cuda_test_install/registries
   Cloning registry General from "https://github.com/JuliaRegistries/General.git"
 Resolving package versions...
 Installed CUDAapi ─ v1.0.1
 Installed CUDAdrv ─ v3.1.0
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  [c5f51814] + CUDAdrv v3.1.0
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Manifest.toml`
  [3895d2a7] + CUDAapi v1.0.1
  [c5f51814] + CUDAdrv v3.1.0
  [8f399da3] + Libdl 
  [56ddb016] + Logging 
  [de0858da] + Printf 
  [4ec0a83e] + Unicode 

(v1.0.4-gpu) pkg> 
omlins@daint101:/scratch/snx3000/omlins/julia_cuda_test_install> srun -Cgpu -pdebug --pty bash
srun: job 16276058 queued and waiting for resources
srun: job 16276058 has been allocated resources
omlins@nid03511:/scratch/snx3000/omlins/julia_cuda_test_install> module unload julia
omlins@nid03511:/scratch/snx3000/omlins/julia_cuda_test_install> module load julia/1.0.4 
omlins@nid03511:/scratch/snx3000/omlins/julia_cuda_test_install> export JULIA_DEPOT_PATH=.
omlins@nid03511:/scratch/snx3000/omlins/julia_cuda_test_install> julia
(v1.0.4-gpu) pkg> status
    Status `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  [c5f51814] CUDAdrv v3.1.0

julia> using CUDAdrv
[Info: Precompiling CUDAdrv [c5f51814-7f29-56b8-a69c-e4d8f6be1fde]

julia> 
omlins@nid03511:/scratch/snx3000/omlins/julia_cuda_test_install> exit
omlins@daint101:/scratch/snx3000/omlins/julia_cuda_test_install> julia
(v1.0.4-gpu) pkg> status
    Status `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  [c5f51814] CUDAdrv v3.1.0

(v1.0.4-gpu) pkg> add CUDAnative
  Updating registry at `./registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed TimerOutputs ─────── v0.5.0
 Installed Adapt ────────────── v1.0.0
 Installed CEnum ────────────── v0.2.0
 Installed Crayons ──────────── v4.0.0
 Installed OrderedCollections ─ v1.1.0
 Installed DataStructures ───── v0.17.0
 Installed CUDAnative ───────── v2.2.1
 Installed LLVM ─────────────── v1.2.0
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  [be33ccc6] + CUDAnative v2.2.1
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Manifest.toml`
  [79e6a3ab] + Adapt v1.0.0
  [fa961155] + CEnum v0.2.0
  [be33ccc6] + CUDAnative v2.2.1
  [a8cc5b0e] + Crayons v4.0.0
  [864edb3b] + DataStructures v0.17.0
  [929cbde3] + LLVM v1.2.0
  [bac558e1] + OrderedCollections v1.1.0
  [a759f4b9] + TimerOutputs v0.5.0
  [2a0f44e3] + Base64 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [37e2e46d] + LinearAlgebra 
  [d6f4376e] + Markdown 
  [9a3f8284] + Random 
  [9e88b42a] + Serialization 
  [6462fe0b] + Sockets 
  [8dfed614] + Test 
  Building CUDAnative → `/scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAnative/nItlk/deps/build.log`
┌ Error: Error building `CUDAnative`: 
│ ERROR: LoadError: InitError: CUDA error: unknown error (code #999, ERROR_UNKNOWN)
│ Stacktrace:
│ [1] macro expansion at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAdrv/ADRHQ/src/base.jl:145 [inlined]
│ [2] init(::Int64) at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAdrv/ADRHQ/src/init.jl:10 (repeats 2 times)
│ [3] __init__ () at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAdrv/ADRHQ/src/CUDAdrv.jl:83
│ [4] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:633
│ [5] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:713
│ [6] _require(::Base.PkgId) at ./loading.jl:937
│ [7] require(::Base.PkgId) at ./loading.jl:858
│ [8] require(::Module, ::Symbol) at ./loading.jl:853
│ [9] include at ./boot.jl:317 [inlined]
│ [10] include_relative(::Module, ::String) at ./loading.jl:1044
│ [11] include(::Module, ::String) at ./sysimg.jl:29
│ [12] include(::String) at ./client.jl:392
│ [13] top-level scope at none:0
│ during initialization of module CUDAdrv
│ in expression starting at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAnative/nItlk/deps/build.jl:2
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1096

julia> using CUDAdrv
ERROR: InitError: CUDA error: unknown error (code #999, ERROR_UNKNOWN)
Stacktrace:
 [1] macro expansion at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAdrv/ADRHQ/src/base.jl:145 [inlined]
 [2] init(::Int64) at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAdrv/ADRHQ/src/init.jl:10 (repeats 2 times)
 [3] __init__ () at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAdrv/ADRHQ/src/CUDAdrv.jl:83
 [4] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:633
 [5] _require_search_from_serialized(::Base.PkgId, ::String) at ./loading.jl:713
 [6] _require(::Base.PkgId) at ./loading.jl:937
 [7] require(::Base.PkgId) at ./loading.jl:858
 [8] require(::Module, ::Symbol) at ./loading.jl:853
during initialization of module CUDAdrv

julia> 
omlins@daint101:/scratch/snx3000/omlins/julia_cuda_test_install> srun -Cgpu -pdebug --pty bash
srun: job 16276186 queued and waiting for resources
srun: job 16276186 has been allocated resources
omlins@nid03511:/scratch/snx3000/omlins/julia_cuda_test_install> module unload julia
omlins@nid03511:/scratch/snx3000/omlins/julia_cuda_test_install> module load julia/1.0.4 
omlins@nid03511:/scratch/snx3000/omlins/julia_cuda_test_install> export JULIA_DEPOT_PATH=.
omlins@nid03511:/scratch/snx3000/omlins/julia_cuda_test_install> julia
(v1.0.4-gpu) pkg> status
    Status `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  [c5f51814] CUDAdrv v3.1.0
  [be33ccc6] CUDAnative v2.2.1

(v1.0.4-gpu) pkg> build CUDAnative
  Building CUDAnative → `/scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAnative/nItlk/deps/build.log`

(v1.0.4-gpu) pkg> 

julia> using CUDAnative
[Info: Precompiling CUDAnative [be33ccc6-a3ff-5ff2-a52e-74243cff1e17]

julia> 

```

---

<div class="post-metadata">

**Author:** ![samo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samo/32/35398_2.png) [@samo](https://discourse.julialang.org/u/samo)\
**Post date:** [August 23, 2019, 8:44am UTC](https://discourse.julialang.org/t/cannot-build-cudanative-on-login-node-works-on-compute-node/27860/2 "2019-08-23T08:44:18Z")

</div>

I could get CUDAnative to build on the login node by modifying the file `packages/CUDAnative/nItlk/deps/build.jl` as follows:

1. remove `using CUDAdrv`
2. replace `CUDAdrv.configured` with `true`
3. replace `CUDAdrv.version()` with `v"9.2.0"`

@maleadt, could you modify the build procedure of CUDAnative to obtain this values without requiring to initialize CUDAdrv?

Shell output shared here for completeness:

```julia
omlins@daint105:/scratch/snx3000/omlins/julia_cuda_test_install> cp /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAnative/nItlk/deps/build.jl .
omlins@daint105:/scratch/snx3000/omlins/julia_cuda_test_install> rm -rf compiled/ conda/ environments/ logs/ packages/ prefs/ registries/
omlins@daint105:/scratch/snx3000/omlins/julia_cuda_test_install> julia
(v1.0.4-gpu) pkg> status
    Status `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  (empty environment)

(v1.0.4-gpu) pkg> add CUDAdrv
   Cloning default registries into /scratch/snx3000/omlins/julia_cuda_test_install/registries
   Cloning registry General from "https://github.com/JuliaRegistries/General.git"
 Resolving package versions...
 Installed CUDAapi ─ v1.0.1
 Installed CUDAdrv ─ v3.1.0
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  [c5f51814] + CUDAdrv v3.1.0
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Manifest.toml`
  [3895d2a7] + CUDAapi v1.0.1
  [c5f51814] + CUDAdrv v3.1.0
  [8f399da3] + Libdl 
  [56ddb016] + Logging 
  [de0858da] + Printf 
  [4ec0a83e] + Unicode 

(v1.0.4-gpu) pkg> add CUDAnative
 Resolving package versions...
 Installed Adapt ────────────── v1.0.0
 Installed Crayons ──────────── v4.0.0
 Installed OrderedCollections ─ v1.1.0
 Installed CEnum ────────────── v0.2.0
 Installed TimerOutputs ─────── v0.5.0
 Installed DataStructures ───── v0.17.0
 Installed CUDAnative ───────── v2.2.1
 Installed LLVM ─────────────── v1.2.0
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  [be33ccc6] + CUDAnative v2.2.1
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Manifest.toml`
  [79e6a3ab] + Adapt v1.0.0
  [fa961155] + CEnum v0.2.0
  [be33ccc6] + CUDAnative v2.2.1
  [a8cc5b0e] + Crayons v4.0.0
  [864edb3b] + DataStructures v0.17.0
  [929cbde3] + LLVM v1.2.0
  [bac558e1] + OrderedCollections v1.1.0
  [a759f4b9] + TimerOutputs v0.5.0
  [2a0f44e3] + Base64 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [37e2e46d] + LinearAlgebra 
  [d6f4376e] + Markdown 
  [9a3f8284] + Random 
  [9e88b42a] + Serialization 
  [6462fe0b] + Sockets 
  [8dfed614] + Test 
  Building CUDAnative → `/scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAnative/nItlk/deps/build.log`
┌ Error: Error building `CUDAnative`: 
│ ERROR: LoadError: InitError: CUDA error: unknown error (code #999, ERROR_UNKNOWN)
│ Stacktrace:
│ [1] macro expansion at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAdrv/ADRHQ/src/base.jl:145 [inlined]
│ [2] init(::Int64) at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAdrv/ADRHQ/src/init.jl:10 (repeats 2 times)
│ [3] __init__ () at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAdrv/ADRHQ/src/CUDAdrv.jl:83
│ [4] _include_from_serialized(::String, ::Array{Any,1}) at ./loading.jl:633
│ [5] _require_from_serialized(::String) at ./loading.jl:684
│ [6] _require(::Base.PkgId) at ./loading.jl:967
│ [7] require(::Base.PkgId) at ./loading.jl:858
│ [8] require(::Module, ::Symbol) at ./loading.jl:853
│ [9] include at ./boot.jl:317 [inlined]
│ [10] include_relative(::Module, ::String) at ./loading.jl:1044
│ [11] include(::Module, ::String) at ./sysimg.jl:29
│ [12] include(::String) at ./client.jl:392
│ [13] top-level scope at none:0
│ during initialization of module CUDAdrv
│ in expression starting at /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAnative/nItlk/deps/build.jl:2
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1096

shell> cp -f build.jl /scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAnative/nItlk/deps/build.jl

(v1.0.4-gpu) pkg> build CUDAnative
  Building CUDAnative → `/scratch/snx3000/omlins/julia_cuda_test_install/packages/CUDAnative/nItlk/deps/build.log`

(v1.0.4-gpu) pkg> 
omlins@daint105:/scratch/snx3000/omlins/julia_cuda_test_install> srun -Cgpu -pdebug --pty bash

srun: job 16300107 queued and waiting for resources
srun: job 16300107 has been allocated resources

omlins@nid03510:/scratch/snx3000/omlins/julia_cuda_test_install> 
omlins@nid03510:/scratch/snx3000/omlins/julia_cuda_test_install> module unload julia
omlins@nid03510:/scratch/snx3000/omlins/julia_cuda_test_install> module load julia/1.0.4 
omlins@nid03510:/scratch/snx3000/omlins/julia_cuda_test_install> export JULIA_DEPOT_PATH=.
omlins@nid03510:/scratch/snx3000/omlins/julia_cuda_test_install> julia
(v1.0.4-gpu) pkg> status
    Status `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  [c5f51814] CUDAdrv v3.1.0
  [be33ccc6] CUDAnative v2.2.1

julia> using CUDAdrv

julia> using CUDAnative
[Info: Precompiling CUDAnative [be33ccc6-a3ff-5ff2-a52e-74243cff1e17]

julia>

```

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [August 23, 2019, 8:46am UTC](https://discourse.julialang.org/t/cannot-build-cudanative-on-login-node-works-on-compute-node/27860/3 "2019-08-23T08:46:31Z")

</div>

> [@samo](#):
>
> @maleadt, could you modify the build procedure of CUDAnative to obtain this values without requiring to initialize CUDAdrv?

I’ve recently done exactly that: [Replace Pkg.build with runtime initialization by maleadt · Pull Request #435 · JuliaGPU/CUDAnative.jl · GitHub](https://github.com/JuliaGPU/CUDAnative.jl/pull/435)

---

<div class="post-metadata">

**Author:** ![samo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samo/32/35398_2.png) [@samo](https://discourse.julialang.org/u/samo)\
**Post date:** [August 23, 2019, 10:34am UTC](https://discourse.julialang.org/t/cannot-build-cudanative-on-login-node-works-on-compute-node/27860/4 "2019-08-23T10:34:23Z")

</div>

Thanks @malead! Installing the master version of the CUDA packages worked on the login nodes out of the box without any patching [1] 🙂  
Unfortunately, there comes up another problem, which is: [Precompiling CUDAnative on login node (works on compute node)](https://discourse.julialang.org/t/precompiling-cudanative-on-login-node-works-on-compute-node/27886).

[1] Shell output of successful installation of the CUDA packages on the login node:

```julia
omlins@daint104:~> module use /scratch/snx3000/omlins/9_soft_install_julia/modules/all
omlins@daint104:~> module load julia/1.0.4 
omlins@daint104:~> cd /scratch/snx3000/omlins/julia_cuda_test_install
omlins@daint104:/scratch/snx3000/omlins/julia_cuda_test_install> export JULIA_DEPOT_PATH=.
omlins@daint104:/scratch/snx3000/omlins/julia_cuda_test_install> julia
(v1.0.4-gpu) pkg> status
    Status `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  (empty environment)

(v1.0.4-gpu) pkg> add CUDAdrv#master
   Cloning default registries into /scratch/snx3000/omlins/julia_cuda_test_install/registries
   Cloning registry General from "https://github.com/JuliaRegistries/General.git"
   Cloning git-repo `https://github.com/JuliaGPU/CUDAdrv.jl.git`
  Updating git-repo `https://github.com/JuliaGPU/CUDAdrv.jl.git`
 Resolving package versions...
 Installed CUDAapi ─ v1.0.1
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  [c5f51814] + CUDAdrv v3.1.0 #master (https://github.com/JuliaGPU/CUDAdrv.jl.git)
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Manifest.toml`
  [3895d2a7] + CUDAapi v1.0.1
  [c5f51814] + CUDAdrv v3.1.0 #master (https://github.com/JuliaGPU/CUDAdrv.jl.git)
  [8f399da3] + Libdl 
  [56ddb016] + Logging 
  [de0858da] + Printf 
  [4ec0a83e] + Unicode 

(v1.0.4-gpu) pkg> add CUDAnative#master
   Cloning git-repo `https://github.com/JuliaGPU/CUDAnative.jl.git`
  Updating git-repo `https://github.com/JuliaGPU/CUDAnative.jl.git`
 Resolving package versions...
 Installed Adapt ────────────── v1.0.0
 Installed CEnum ────────────── v0.2.0
 Installed TimerOutputs ─────── v0.5.0
 Installed Crayons ──────────── v4.0.0
 Installed OrderedCollections ─ v1.1.0
 Installed DataStructures ───── v0.17.0
 Installed LLVM ─────────────── v1.2.0
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  [be33ccc6] + CUDAnative v2.2.1 #master (https://github.com/JuliaGPU/CUDAnative.jl.git)
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Manifest.toml`
  [79e6a3ab] + Adapt v1.0.0
  [fa961155] + CEnum v0.2.0
  [be33ccc6] + CUDAnative v2.2.1 #master (https://github.com/JuliaGPU/CUDAnative.jl.git)
  [a8cc5b0e] + Crayons v4.0.0
  [864edb3b] + DataStructures v0.17.0
  [929cbde3] + LLVM v1.2.0
  [bac558e1] + OrderedCollections v1.1.0
  [a759f4b9] + TimerOutputs v0.5.0
  [2a0f44e3] + Base64 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [37e2e46d] + LinearAlgebra 
  [d6f4376e] + Markdown 
  [9a3f8284] + Random 
  [9e88b42a] + Serialization 
  [6462fe0b] + Sockets 
  [8dfed614] + Test 

(v1.0.4-gpu) pkg> add CuArrays#master
   Cloning git-repo `https://github.com/JuliaGPU/CuArrays.jl.git`
  Updating git-repo `https://github.com/JuliaGPU/CuArrays.jl.git`
 Resolving package versions...
 Installed Reexport ─────── v0.2.0
 Installed Requires ─────── v0.5.2
 Installed VersionParsing ─ v1.1.3
 Installed NNlib ────────── v0.6.0
 Installed Tokenize ─────── v0.5.6
 Installed FFTW ─────────── v0.3.0
 Installed GPUArrays ────── v1.0.0
 Installed StaticArrays ─── v0.11.0
 Installed BinaryProvider ─ v0.5.6
 Installed AbstractFFTs ─── v0.4.1
 Installed Compat ───────── v2.1.0
 Installed JSON ─────────── v0.21.0
 Installed FillArrays ───── v0.6.4
 Installed CSTParser ────── v0.6.2
 Installed Conda ────────── v1.3.0
 Installed Parsers ──────── v0.3.6
 Installed MacroTools ───── v0.5.1
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Project.toml`
  [3a865a2d] + CuArrays v1.1.0 #master (https://github.com/JuliaGPU/CuArrays.jl.git)
  Updating `/scratch/snx3000/omlins/julia_cuda_test_install/environments/v1.0.4-gpu/Manifest.toml`
  [621f4979] + AbstractFFTs v0.4.1
  [b99e7846] + BinaryProvider v0.5.6
  [00ebfdb7] + CSTParser v0.6.2
  [34da2185] + Compat v2.1.0
  [8f4d0f93] + Conda v1.3.0
  [3a865a2d] + CuArrays v1.1.0 #master (https://github.com/JuliaGPU/CuArrays.jl.git)
  [7a1cc6ca] + FFTW v0.3.0
  [1a297f60] + FillArrays v0.6.4
  [0c68f7d7] + GPUArrays v1.0.0
  [682c06a0] + JSON v0.21.0
  [1914dd2f] + MacroTools v0.5.1
  [872c559c] + NNlib v0.6.0
  [69de0a69] + Parsers v0.3.6
  [189a3867] + Reexport v0.2.0
  [ae029012] + Requires v0.5.2
  [90137ffa] + StaticArrays v0.11.0
  [0796e94c] + Tokenize v0.5.6
  [81def892] + VersionParsing v1.1.3
  [ade2ca70] + Dates 
  [8bb1440f] + DelimitedFiles 
  [76f85450] + LibGit2 
  [a63ad114] + Mmap 
  [44cfe95a] + Pkg 
  [3fa0cd96] + REPL 
  [ea8e919c] + SHA 
  [1a1011a3] + SharedArrays 
  [2f01184e] + SparseArrays 
  [10745b16] + Statistics 
  [cf7118a7] + UUIDs 
  Building Conda → `/scratch/snx3000/omlins/julia_cuda_test_install/packages/Conda/kLXeC/deps/build.log`
  Building FFTW ─→ `/scratch/snx3000/omlins/julia_cuda_test_install/packages/FFTW/2okGQ/deps/build.log`

```