Need help with Flux CUDA.jl warning and error message

General Usage
, ,
1

I need some help understanding the following Flux problem and hopefully finding a fix for it.

When I run using Flux on my desktop system I get the following warning and error messages:

┌ Warning: This version of CUDA.jl only supports NVIDIA drivers for CUDA 10.1 or higher (yours is for CUDA 9.1.0)
└ @ CUDA ~/.julia/packages/CUDA/0p5fn/src/initialization.jl:111
ERROR: InitError: Could not find a suitable CUDA installation

However running the same on my netbook runs normally without warnings or errors.

Both systems run Manjaro, an Arch based Linux operating system, and are fully updated. The desktop CPU is Intel with an NVIDIA based graphics card:

$ inxi -G
Graphics:  Device-1: NVIDIA GF119 [GeForce GT 610] driver: nvidia v: 390.132 
           Display: x11 server: X.Org 1.20.9 driver: nvidia s-res: 1920x1200 
           OpenGL: renderer: GeForce GT 610/PCIe/SSE2 v: 4.6.0 NVIDIA 390.132

Whilst the netbook is AMD based with an AMD CPU and Radeon graphics chip:

$ inxi -G
Graphics:  Device-1: AMD Wrestler [Radeon HD 6250] driver: radeon v: kernel 
           Device-2: Chicony CNF9055 Toshiba Webcam type: USB driver: uvcvideo 
           Display: x11 server: X.Org 1.20.9 driver: radeon s-res: 1024x600 
           OpenGL: renderer: AMD PALM (DRM 2.50.0 / 5.8.18-1-MANJARO LLVM 10.0.1) 
           v: 3.3 Mesa 20.2.1

I do not know if the above is relevant to the problem and I am uncertain where the problem lies and seek your guidance as to the cause, further investigation and if it may be fixed ?

2 Likes
2

Flux depends on CUDA.jl by default for GPU acceleration, so using Flux will always try to find and load GPU-related functionality. On a machine without nvidia drivers installed (e.g. your netbook), this will fail gracefully and not attempt to run any more CUDA-specific code. Think of it as the “CPU-only” mode.

When your machine does have nvidia drivers installed (presumably because there’s an nvidia GPU attached to it), CUDA.jl will try to download and run an appropriate SDK based on your current driver version. I’m not sure what the minimal driver version is, but 390.132 is definitely too old (nvidia-smi reports 450.51.06 on my system). Can you update your driver at all? Assuming your GPU is supported on a relatively recent one, Flux + CUDA should work OOTB.

1 Like
3

Much appreciate the clarification and enlightenment.

Yes the GPU is old and Manjaro will be dropping support for the 390 drivers in the coming months and recommend switching to the Nouveau driver, I will try that soon, it may be my work around for the Flux and CUDA.jl problem for now, other than removing the graphics card (a bit too drastic).

My whole desktop system is old, circa 2008, the latest Raspberry Pi may give me similar or better performance! So I am acutely aware I need new or more recent hardware, which is another way to fix the Flux problem, but that will have to wait until next year.

4

Switching to Nouveau may actually resolve the error, since CUDA.jl looks for the proprietary driver IIRC. In the meantime, try setting the environment variable CUDA_VISIBLE_DEVICES="" and see if that works.

1 Like
5

Hi. I too am on a system with old hardware which I won’t be able to change for a while. I have CUDA 6.5 running fine on this machine with the 390 driver. Are you saying that I could run this (or another) CUDA and be Julia-compatible if I switched to the Nouveau driver?
BTW ‘using CUDA’ doesn’t give errors for me at the moment - just
‘using Flux’.
Thanks for any advice

6

You are asking a bit of a novice at this at the moment, so others may give a better answer. It seems there are three different CUDAs involved here,

  • CUDA.jl
  • NVIDIA drivers for CUDA
  • and CUDA, NVIDIA’s GPU programming toolkit

I am uncertain as to which CUDA you are refering to for ‘Julia-compatible’. For me it is the drivers for CUDA that have caused my using Flux problem. Since my original post I have switched to using Nouveau and now there are no warnings or errors with using Flux, as @ToucheSir explains, my machine no longer has NVIDIA drivers installed and Flux becomes “CPU-only” mode.

However despite the fine work of the Nouveau developers, video playback often flickers, jitters and judders on my system so I will likely switch back to the NVIDIA driver and:

I hope my response is of some help.

7

Thank you. Yes, I have successfully employed the last suggestion you quoted.

1 Like
8

I had to learn a little more about using shell and environment variables in Linux to get this to work for me. Having now reverted to using the NVIDIA drivers, for smooth graphics and video, I had entered
only CUDA_VISIBLE_DEVICES="" on the command line, which sets a shell variable, and using Flux still gave the same warning and error. I discovered I needed to enter

export CUDA_VISIBLE_DEVICES=""

which sets it as an environment variable and using Flux runs without warnings or errors.
(Thanks to this page for educating me)