# How to add CUDA as optional dependency?

**URL:** <https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077>\
**Category:** GPU\
**Tags:** question, gpu, pkg\
**Created:** [December 14, 2021, 2:54pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077 "2021-12-14T14:54:00Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [December 14, 2021, 2:54pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/1 "2021-12-14T14:54:00Z")

</div>

Suppose I have a package and that I am interested in providing an option to end users to speed up their calculations in the presence of a NVIDIA GPU. Not everyone will benefit from this option and so I can’t add CUDA as a dependency. Is there a solution specifically designed for the CUDA.jl stack that allows package developers to add the stack as an optional dependency?

Note: I am aware of Requires.jl but I can’t afford the lack of support for semantic versioning.

---

<div class="post-metadata">

**Author:** ![josuagrw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josuagrw/32/1015_2.png) [@josuagrw](https://discourse.julialang.org/u/josuagrw)\
**Post date:** [December 14, 2021, 3:56pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/2 "2021-12-14T15:56:55Z")

</div>

This is probably not what you are looking for but in principle you could split up your code into three parts: general, GPU-specific, and CPU-specific.

Then users could do something like:

```julia
using GeneralPackage, GPUPackage

result = GeneralPackage.calculate(inputs, GPUPackage.Engine())

```

or conversely

```julia
using GeneralPackage, CPUPackage

result = GeneralPackage.calculate(inputs, CPUPackage.Engine())

```

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [December 14, 2021, 4:07pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/3 "2021-12-14T16:07:35Z")

</div>

Thank you @josuagrw , I will implement the proposed solution if there is no alternative.

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [December 14, 2021, 4:43pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/4 "2021-12-14T16:43:35Z")

</div>

Currently, `Pkg.jl` does not support conditional provisioning.

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [December 14, 2021, 5:13pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/5 "2021-12-14T17:13:37Z")

</div>

Is it a problem to include CUDA as a dependency?  
If there is no GPU available, you can then decide what to do.  
That is also mentioned in the [CUDA.jl docs](https://cuda.juliagpu.org/dev/installation/conditional/#Scenario-2:-GPU-is-optional).

---

<div class="post-metadata">

**Author:** ![abulak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abulak/32/28314_2.png) [@abulak](https://discourse.julialang.org/u/abulak)\
**Post date:** [December 14, 2021, 5:18pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/6 "2021-12-14T17:18:34Z")

</div>

You can use `Requires.jl` for this purpose, eg:  
[https://github.com/jump-dev/SCS.jl/blob/12f994e316f6b2d5c522ca2e603c964edaf42db9/src/SCS.jl#L30](https://github.com/jump-dev/SCS.jl/blob/12f994e316f6b2d5c522ca2e603c964edaf42db9/src/SCS.jl#L30)

But that’s a very limited scope solution, unfortunately.

---

<div class="post-metadata">

**Author:** ![samo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samo/32/35398_2.png) [@samo](https://discourse.julialang.org/u/samo)\
**Post date:** [December 14, 2021, 6:08pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/7 "2021-12-14T18:08:10Z")

</div>

There is a simple solution to this, which has been adopted by many packages already:

1. add CUDA.jl as a normal dependency
2. do `using CUDA` in your package
3. check at runtime if CUDA is functional or not (with `CUDA.functional()`) and use the GPU in function of this.

The installation of CUDA.jl as well as `using CUDA` and compiling CUDA.jl code will work without error on a system without GPU. Only at runtime, trying to use the GPU for real, it would fail. But, this you will avoid by not doing so if `CUDA.functional()` returns `false`.

Example:

- [here](https://github.com/eth-cscs/ImplicitGlobalGrid.jl/blob/master/src/shared.jl#L6-L16) is how it is done in ImplicitGlobalGrid

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [December 14, 2021, 6:36pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/8 "2021-12-14T18:36:41Z")

</div>

Any system that runs Julia will successfully build CUDA.jl? How often the build fails on different OSes, platforms, etc? Ideally I would like to avoid an extra dependency, specially if it has some potential to reduce the number places where a package can run.

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [December 14, 2021, 6:41pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/9 "2021-12-14T18:41:08Z")

</div>

We put a lot of effort in making sure that CUDA.jl can be loaded, even if CUDA is not available. You can use `CUDA.functional()` to check if your system has CUDA available.

```julia
julia> using CUDA

julia> CUDA.functional()
false

```

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [December 15, 2021, 6:29am UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/10 "2021-12-15T06:29:53Z")

</div>

CUDA.jl does nothing before its first use. Even artifacts are downloaded at run time. So the only ‘cost’ is additional dependencies, and package load time. The latter can still be optimized, so if somebody’s familiar with the recent techniques for improving latency (optimizing precompilation, avoiding invalidations) that would be a valuable contribution.

---

<div class="post-metadata">

**Author:** ![abulak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abulak/32/28314_2.png) [@abulak](https://discourse.julialang.org/u/abulak)\
**Post date:** [December 16, 2021, 11:15am UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/11 "2021-12-16T11:15:12Z")

</div>

yeah, the question is only do you want to pull a few GB of `CUDA.jl` for a small package when the user may not even have CUDA enabled gpu… 😉

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [December 16, 2021, 11:47am UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/12 "2021-12-16T11:47:16Z")

</div>

Yes, that is a big concern I have as well. Downloading a bunch of binaries and packages _always_ seems too heavy, specially if you take into account countries under development with poor internet connection. Even though `CUDA.functional()` is **wonderful** (thanks for sharing) it still doesn’ t fully solve the issue from a software engineering perspective.

Maybe someday in the future we could have a `julia --gpu` flag or something cooked into the language like we do for threads and distributed computing? 🙂 I am just wondering if that would make sense at all. No need to install CUDA, OpenCL, … dependencies manually, the language itself would have GPU support baked in.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [December 16, 2021, 1:11pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/13 "2021-12-16T13:11:28Z")

</div>

> [@abulak](#):
>
> yeah, the question is only do you want to pull a few GB of `CUDA.jl` for a small package when the user may not even have CUDA enabled gpu…

Where are those gigabytes going to come from?

> [@maleadt](#):
>
> Even artifacts are downloaded at run time.

Just make sure you also call `CUDA.functional()` lazily. You shouldn’t do that during ` __init__ ()`, and definitely not globally.

For an example, look, at Flux.jl: [https://github.com/FluxML/Flux.jl/blob/e1268a6467ed78e1009b60972a1b9195525a0e0f/src/functor.jl#L175-L192](https://github.com/FluxML/Flux.jl/blob/e1268a6467ed78e1009b60972a1b9195525a0e0f/src/functor.jl#L175-L192)

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [December 16, 2021, 1:24pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/14 "2021-12-16T13:24:06Z")

</div>

@maleadt can you comment on this current list of dependencies in CUDA.jl’s Project.toml?

```julia
[deps]
AbstractFFTs = "621f4979-c628-5d54-868e-fcf4e3e8185c"
Adapt = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
BFloat16s = "ab4f0b2a-ad5b-11e8-123f-65d77653426b"
CEnum = "fa961155-64e5-5f13-b03f-caf6b980ea82"
CompilerSupportLibraries_jll = "e66e0078-7015-5450-92f7-15fbd957f2ae"
ExprTools = "e2ba6199-217a-4e67-a87a-7c52f15ade04"
GPUArrays = "0c68f7d7-f131-5f86-a1c3-88cf8149b2d7"
GPUCompiler = "61eb1bfa-7361-4325-ad38-22787b887f55"
LLVM = "929cbde3-209d-540e-8aea-75f648917ca0"
LazyArtifacts = "4af54fe1-eca0-43a8-85a7-787d91b784e3"
Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"
Printf = "de0858da-6303-5e67-8744-51eddeeeb8d7"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
Random123 = "74087812-796a-5b5d-8853-05524746bad3"
RandomNumbers = "e6cf234a-135c-5ec9-84dd-332b85af5143"
Reexport = "189a3867-3050-52da-a836-e630ba90ab69"
Requires = "ae029012-a4dd-5104-9daa-d747884805df"
SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
SpecialFunctions = "276daf66-3868-5448-9aa4-cd146d93841b"
TimerOutputs = "a759f4b9-e2f1-59dc-863e-4aeb61b1ea8f"

```

From heaviest to lightest, which ones could potentially introduce non-trivial issues during installation? I see a lot of compiler packages, but I don’t understand how challenging it is to install these packages in old machines (thinking of students in countries under development).

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [December 16, 2021, 9:39pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/15 "2021-12-16T21:39:43Z")

</div>

No, most of those packages are fairly small. We’ve taken care to isolate larger dependencies in separate packages (like NNlibCUDA.jl) or by using Requires.jl  
There’s a couple of packages that require binary dependencies, but those work well enough nowadays to not be a problem.

---

<div class="post-metadata">

**Author:** ![abulak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abulak/32/28314_2.png) [@abulak](https://discourse.julialang.org/u/abulak)\
**Post date:** [December 16, 2021, 10:42pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/16 "2021-12-16T22:42:45Z")

</div>

from the fact that one of the binary libraries may link against CUDA\_jll.

SCS is a tiny (100sKB) C-library; we ship separate SCS\_GPU\_jll which has CUDA\_full\_jll as build dep only. The aim is to add the gpu functionality to the package only conditionally, when CUDA\_full is already present.

Or maybe is there a better way of doing this?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [December 21, 2021, 8:13am UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/17 "2021-12-21T08:13:15Z")

</div>

> [@abulak](#):
>
> The aim is to add the gpu functionality to the package only conditionally, when CUDA\_full is already present.

You mean CUDA\_jll? CUDA\_full\_jll should never be present, as you mention it’s a build dep only.

That said, currently the way CUDA.jl handles its CUDA toolkit dependency is entirely separate from CUDA\_jll.jl. We hope to unify that in the future, but as of now you can’t have both loaded at the same time.

---

<div class="post-metadata">

**Author:** ![abulak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abulak/32/28314_2.png) [@abulak](https://discourse.julialang.org/u/abulak)\
**Post date:** [December 26, 2021, 12:58am UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/18 "2021-12-26T00:58:38Z")

</div>

We use `CUDA_full_jll` as build dep and require users to `using CUDA_jll` before `using SCS`;  
This way we can define gpu-related functionality in `SCS.jl` through `Requires` and only `include` it when CUDA is present. This e.g. defines some constants in the package.

However, maybe it’s time to revisit the solution. What do you suggest?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [December 28, 2021, 8:35am UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/19 "2021-12-28T08:35:22Z")

</div>

> [@abulak](#):
>
> We use `CUDA_full_jll` as build dep and require users to `using CUDA_jll` before `using SCS`

That’s the correct set-up indeed, but you mentioned `CUDA_full` twice in your previous post, so I assume you meant `CUDA_jll`. This solution doesn’t integrate with CUDA.jl yet, but I hope to take another look at this in the new year.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [January 5, 2022, 3:44pm UTC](https://discourse.julialang.org/t/how-to-add-cuda-as-optional-dependency/73077/20 "2022-01-05T15:44:46Z")

</div>

Adding another dimension to the question, I wonder what is the future of

[https://github.com/timholy/ComputationalResources.jl](https://github.com/timholy/ComputationalResources.jl)

It seemed like an interesting idea to ask for computational resources explicitly in user scripts. If the language had this notion of resource built-in we could switch between different resources without depending on packages I guess? Just a set of abstract types defining the resources and a convention to implement all Base methods with resources as the first argument when possible. Falling back to the CPU.