I found a relevant part in the Flux documentation:
NVIDIA GPU support requires the packages
CUDA.jlandcuDNN.jlto be installed in the environment. In the julia REPL, type] add CUDA, cuDNNto install them.
Not sure what was wrong, but for some reason nothing happened when I tried to install cuDNN only by pkg> add cuDNN at the time of the original posting. After finding out the above part in the documentation, I executed pkg> add CUDA, cuDNN as instructed, and it started installing cuDNN.