Should be, yes. Try using CUDA.CUDNN.
julia> using CUDA.CUDNN
julia> CUDNN.cudnn
cudnnActivationBackward
cudnnActivationDescriptor
cudnnActivationDescriptorCache
cudnnActivationDescriptorCacheLock
cudnnActivationDescriptor_t
cudnnActivationForward
cudnnActivationForward!
cudnnActivationForwardAD
cudnnActivationForwardWithDefaults
cudnnActivationMode_t
cudnnActivationStruct
cudnnAddTensor
cudnnAddTensor!
cudnnAdvInferVersionCheck
cudnnAdvTrainVersionCheck
cudnnAlgorithmDescriptor_t
cudnnAlgorithmPerformanceStruct
cudnnAlgorithmPerformance_t
cudnnAlgorithmStruct
cudnnAlgorithm_t
cudnnAttnDescriptor
cudnnAttnDescriptorCache
cudnnAttnDescriptorCacheLock
cudnnAttnDescriptor_t
cudnnAttnOutput
cudnnAttnQueryMap_t
cudnnAttnStruct
cudnnBatchNormMode_t
cudnnBatchNormOps_t
cudnnBatchNormalizationBackward
cudnnBatchNormalizationBackwardEx
cudnnBatchNormalizationForwardInference
cudnnBatchNormalizationForwardTraining
cudnnBatchNormalizationForwardTrainingEx
[...]
(Subdirectory of the CUDA.jl repo: CUDA.jl/lib/cudnn at v3.12.0 · JuliaGPU/CUDA.jl · GitHub)