# CUDNN in Julia

**URL:** <https://discourse.julialang.org/t/cudnn-in-julia/84330>\
**Category:** GPU\
**Tags:** cuda\
**Created:** [July 16, 2022, 7:17pm UTC](https://discourse.julialang.org/t/cudnn-in-julia/84330 "2022-07-16T19:17:08Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jonas208](https://avatars.discourse-cdn.com/v4/letter/j/85f322/32.png) [@Jonas208](https://discourse.julialang.org/u/Jonas208)\
**Post date:** [July 16, 2022, 7:17pm UTC](https://discourse.julialang.org/t/cudnn-in-julia/84330/1 "2022-07-16T19:17:08Z")

</div>

**Is there any way - or what is _the best_ way - to use CUDNN in Julia?**

**Details:**  
I am interested in that topic because I want do add GPU-acceleration (Nvidia) to my own little Deep Learning module (it is not available anywere yet). I found out that CUDNN is a common choice for GPU-accelaration for neuronal networks and now I would like to find more about that. I know that it is like a part of CUDA and that it was originally written for C/C++.

**Is CUDNN accesibal with using CUDA.jl or is there a good wrapper of CUDNN to Julia?**

Best Regards,

Jonas S.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [July 17, 2022, 7:22am UTC](https://discourse.julialang.org/t/cudnn-in-julia/84330/2 "2022-07-17T07:22:47Z")

</div>

Should be, yes. Try `using CUDA.CUDNN`.

```julia
julia> using CUDA.CUDNN

julia> CUDNN.cudnn

cudnnActivationBackward
cudnnActivationDescriptor
cudnnActivationDescriptorCache
cudnnActivationDescriptorCacheLock
cudnnActivationDescriptor_t
cudnnActivationForward
cudnnActivationForward!
cudnnActivationForwardAD
cudnnActivationForwardWithDefaults
cudnnActivationMode_t
cudnnActivationStruct
cudnnAddTensor
cudnnAddTensor!
cudnnAdvInferVersionCheck
cudnnAdvTrainVersionCheck
cudnnAlgorithmDescriptor_t
cudnnAlgorithmPerformanceStruct
cudnnAlgorithmPerformance_t
cudnnAlgorithmStruct
cudnnAlgorithm_t
cudnnAttnDescriptor
cudnnAttnDescriptorCache
cudnnAttnDescriptorCacheLock
cudnnAttnDescriptor_t
cudnnAttnOutput
cudnnAttnQueryMap_t
cudnnAttnStruct
cudnnBatchNormMode_t
cudnnBatchNormOps_t
cudnnBatchNormalizationBackward
cudnnBatchNormalizationBackwardEx
cudnnBatchNormalizationForwardInference
cudnnBatchNormalizationForwardTraining
cudnnBatchNormalizationForwardTrainingEx
[...]

```

(Subdirectory of the CUDA.jl repo: [CUDA.jl/lib/cudnn at v3.12.0 · JuliaGPU/CUDA.jl · GitHub](https://github.com/JuliaGPU/CUDA.jl/tree/v3.12.0/lib/cudnn))

---

<div class="post-metadata">

**Author:** ![Jonas208](https://avatars.discourse-cdn.com/v4/letter/j/85f322/32.png) [@Jonas208](https://discourse.julialang.org/u/Jonas208)\
**Post date:** [July 17, 2022, 10:31am UTC](https://discourse.julialang.org/t/cudnn-in-julia/84330/4 "2022-07-17T10:31:58Z")

</div>

Thank you! But I try to find good tutorials or documentation about CUDNN but I had no succes yet.  
I think there are no articles about CUDNN directly in Julia. However, I’m sure there is helpful information about using CUDNN in general.  
Do you have any helpful information about this?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [July 17, 2022, 10:59am UTC](https://discourse.julialang.org/t/cudnn-in-julia/84330/5 "2022-07-17T10:59:44Z")

</div>

There’s several levels of abstraction at which you can use CUDNN:

- at the lowest level, there’s just the CUDNN C API functions, all of which you can use and are part of the CUDA.CUDNN submodule
- the same module also has slightly higher-level wrappers (bit more idiomatic, but still true to the CUDNN API). Those sadly aren’t really documented, but there’s a design document by @denizyuret who wrote these wrappers: [CUDA.jl/lib/cudnn at master · JuliaGPU/CUDA.jl · GitHub](https://github.com/JuliaGPU/CUDA.jl/tree/master/lib/cudnn)
- you can also use NNlib, which provides a common API of high-level neural network primitives. There’s a CUDA back-end for these that uses CUDNN, NNlibCUDA.jl

---

<div class="post-metadata">

**Author:** ![Jonas208](https://avatars.discourse-cdn.com/v4/letter/j/85f322/32.png) [@Jonas208](https://discourse.julialang.org/u/Jonas208)\
**Post date:** [July 17, 2022, 6:00pm UTC](https://discourse.julialang.org/t/cudnn-in-julia/84330/7 "2022-07-17T18:00:09Z")

</div>

Thanks! I think I will take a look at all of these options.  
So if I am using the CUDNN low-level functions provided by CUDA.jl.CUDNN should I use the original docs from Nvidia? [NVIDIA Deep Learning cuDNN Documentation](https://docs.nvidia.com/deeplearning/cudnn/index.html)

_By the way, I am brand new to this forum and I am amazed that (my) questions were answered so quickly._

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [July 17, 2022, 6:14pm UTC](https://discourse.julialang.org/t/cudnn-in-julia/84330/8 "2022-07-17T18:14:47Z")

</div>

> [@Jonas208](#):
>
> So if I am using the CUDNN low-level functions provided by CUDA.jl.CUDNN should I use the original docs from Nvidia?

Yes. The Julia functions generally are just wrappers around the C functions.

> [@Jonas208](#):
>
> By the way, I am brand new to this forum and I am amazed that (my) questions were answered so quickly.

Glad to hear that. We try our best 🙂

---

<div class="post-metadata">

**Author:** ![christiangnrd](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christiangnrd/32/215467_2.png) [@christiangnrd](https://discourse.julialang.org/u/christiangnrd)\
**Post date:** [February 25, 2025, 5:05pm UTC](https://discourse.julialang.org/t/cudnn-in-julia/84330/9 "2025-02-25T17:05:17Z")

</div>

For future reference, as of at least CUDA.jl 4.0, cuDNN wrappers/functionality were split off into their own package.

To use the functionality, `cuDNN` must be added to your environment and imported.