GPU support in Knet: Cannot allocate before CuArrays has been initialized

Peter.Zentis
Fri, 08 Nov 2019 13:07:26 +0000
New to Julia

Hi all,
I am trying to work through the Knet tutorial (https://github.com/denizyuret/Knet.jl/blob/master/tutorial/50.cnn.ipynb) which worked great, just until I needed/wanted to use gpu support i.e. use a KnetArray. I suppose the problem is related to my CUDA setup. The output of nvidia-smi looks good to me and I do not know what else to look at:

nvidia-smi
Fri Nov  8 13:39:57 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.87.00    Driver Version: 418.87.00    CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:03:00.0  On |                  N/A |
|  0%   43C    P8    14W / 250W |    233MiB / 11169MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      2606      G   /usr/bin/X                                    38MiB |
|    0      2805      G   /usr/bin/gnome-shell                          47MiB |
|    0     14149      C   /opt/julia-1.2.0/bin/julia                   135MiB |
+-----------------------------------------------------------------------------+

Also I successfully tested nvcc as described here: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#compiling-examples

I am not sure whether this is relevant for this issue, but I am on a Opensuse Leap 15.0 system, and tried to install CUDA according to the official instructions.

Anyway, this line from the Knet tutorial fails with an error:

x = KnetArray(reshape([1.0:6.0...], (6,1,1,1)))

And I get the following error:

ERROR: AssertionError: Cannot allocate before CuArrays has been initialized.
Stacktrace:
 [1] CuArrays.CuArray{UInt8,1,P} where P(::UndefInitializer, ::Tuple{Int64}) at /root/.julia/packages/CuArrays/BcKUl/src/memory.jl:120
 [2] Type at /root/.julia/packages/CuArrays/BcKUl/src/array.jl:98 [inlined]
 [3] Type at /root/.julia/packages/CuArrays/BcKUl/src/array.jl:99 [inlined]
 [4] KnetPtrCu(::Int64) at /root/.julia/packages/Knet/HRYiN/src/cuarray.jl:90
 [5] Type at /root/.julia/packages/Knet/HRYiN/src/kptr.jl:102 [inlined]
 [6] Type at /root/.julia/packages/Knet/HRYiN/src/karray.jl:95 [inlined]
 [7] KnetArray(::Array{Float64,4}) at /root/.julia/packages/Knet/HRYiN/src/karray.jl:93
 [8] top-level scope at REPL[4]:1

Apparently, the problem is due to my CUDA setup?! At least I get a bunch of errors on pkg> test CuArrays:

Test Summary:                                   | Pass  Fail  Error  Total
CuArrays                                        |   63     1   1124   1188
  GPUArrays test suite                          |   25          431    456
    construction                                |   15            7     22
      constructors + similar                    |                 1      1
      comparison against Array                  |   15            1     16
      conversion                                |                 1      1
      value constructors                        |                 1      1
      iterator constructors                     |                 3      3

  ...

    Batchnorm                                   |                 1      1
  CUTENSOR                                      |                    No tests
  ForwardDiff                                   |   10            9     19
    UNARY                                       |                    No tests
    POW                                         |                 8      8
    LITERAL_POW                                 |   10                  10
    Broadcast Fix                               |                 1      1
ERROR: LoadError: Some tests did not pass: 63 passed, 1 failed, 1124 errored, 0 broken.
in expression starting at /home/kzentis/.julia/packages/CuArrays/BcKUl/test/runtests.jl:23
ERROR: Package CuArrays errored during testing

This is just the end of the output, I can post it all if required.
It would be really great, if someone could point me where to search for the underlying cause of my problem.

Thanks, Peter

3 posts - 2 participants