Disable all GPU support when using Lux

Specific Domains GPU
1

I have an old GPU which is no longer supported and would like to run Lux anyway using just the CPU.

$ nvidia-smi -L
GPU 0: NVIDIA GeForce GTX 680 (UUID: GPU-49fc79e3-b888-22fe-dc46-f3233d654e1b)

I tried the suggestion from the documentation:

$ CUDA_VISIBLE_DEVICES='-1' julia --project
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.1 (2023-06-07)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using Lux
This fails with several errors 
[ Info: Precompiling Lux [b2108857-7c20-44ae-9111-449ecde12c47]
ERROR: LoadError: InitError: CUDA error (code 101, CUDA_ERROR_INVALID_DEVICE)
Stacktrace:
  [1] throw_api_error(res::CUDA.cudaError_enum)
    @ CUDA ~/.julia/packages/CUDA/pCcGc/lib/cudadrv/libcuda.jl:27
  [2] check
    @ ~/.julia/packages/CUDA/pCcGc/lib/cudadrv/libcuda.jl:34 [inlined]
  [3] runtime_version()
    @ CUDA ~/.julia/packages/CUDA/pCcGc/lib/cudadrv/version.jl:39
  [4] __init__()
    @ CUDA ~/.julia/packages/CUDA/pCcGc/src/initialization.jl:102
  [5] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1115
  [6] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
    @ Base ./loading.jl:1061
  [7] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
    @ Base ./loading.jl:1506
  [8] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1783
  [9] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1660
 [10] macro expansion
    @ ./loading.jl:1648 [inlined]
 [11] macro expansion
    @ ./lock.jl:267 [inlined]
 [12] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1611
 [13] include
    @ ./Base.jl:457 [inlined]
 [14] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
    @ Base ./loading.jl:2045
 [15] top-level scope
    @ stdin:3
during initialization of module CUDA
in expression starting at /home/user/.julia/packages/LuxCUDA/CsMat/src/LuxCUDA.jl:1
in expression starting at stdin:3
ERROR: LoadError: Failed to precompile LuxCUDA [d0bbae9a-e099-4d5b-a835-1c6931763bda] to "/home/user/.julia/compiled/v1.9/LuxCUDA/jl_RobAW1".
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:2296
  [3] compilecache
    @ ./loading.jl:2163 [inlined]
  [4] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1805
  [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1660
  [6] macro expansion
    @ ./loading.jl:1648 [inlined]
  [7] macro expansion
    @ ./lock.jl:267 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1611
  [9] include
    @ ./Base.jl:457 [inlined]
 [10] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2045
 [11] top-level scope
    @ stdin:3
in expression starting at /home/user/.julia/packages/Lux/71GtU/src/Lux.jl:1
in expression starting at stdin:3
ERROR: Failed to precompile Lux [b2108857-7c20-44ae-9111-449ecde12c47] to "/home/user/.julia/compiled/v1.9/Lux/jl_SLZBwA".
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:2296
 [3] compilecache
   @ ./loading.jl:2163 [inlined]
 [4] _require(pkg::Base.PkgId, env::String)
   @ Base ./loading.jl:1805
 [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
   @ Base ./loading.jl:1660
 [6] macro expansion
   @ ./loading.jl:1648 [inlined]
 [7] macro expansion
   @ ./lock.jl:267 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1611

Is there a way to disable loading of CUDA alltogether?

2

You should set CUDA_VISIBLE_DEVICES to an empty string, not to -1.

However, it’d be better if Lux left that up to the user, by relying on Requires or package extensions. In that case, you’d choose whether to install CUDA.jl or not, rather than it getting auto-loaded by Lux.

1 Like
3

I agree. I have been working on migrating Lux to rely on accelerators via Pkg extensions. Most of the things are already in place, but we need some additional features to be finalized before cutting a breaking release.

2 Likes
4

I get the same error with an empty string.

$ CUDA_VISIBLE_DEVICES='' julia --project
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.1 (2023-06-07)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(cuda) pkg> st
Status `~/Desktop/Julia/cuda/Project.toml`
  [052768ef] CUDA v4.3.2

julia> using CUDA
ERROR: InitError: CUDA error (code 101, CUDA_ERROR_INVALID_DEVICE)
Stacktrace:
  [1] throw_api_error(res::CUDA.cudaError_enum)
    @ CUDA ~/.julia/packages/CUDA/pCcGc/lib/cudadrv/libcuda.jl:27
  [2] check
    @ ~/.julia/packages/CUDA/pCcGc/lib/cudadrv/libcuda.jl:34 [inlined]
  [3] runtime_version()
    @ CUDA ~/.julia/packages/CUDA/pCcGc/lib/cudadrv/version.jl:39
  [4] __init__()
    @ CUDA ~/.julia/packages/CUDA/pCcGc/src/initialization.jl:102
  [5] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1115
  [6] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
    @ Base ./loading.jl:1061
  [7] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
    @ Base ./loading.jl:1506
  [8] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1783
  [9] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1660
 [10] macro expansion
    @ ./loading.jl:1648 [inlined]
 [11] macro expansion
    @ ./lock.jl:267 [inlined]
 [12] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1611
during initialization of module CUDA

Edit:

I think this must be a CUDA (not CUDA.jl) problem or something obnoxiously specific to my machine. According to the CUDA documentation either empty string or invalid index should disable all GPUs.

5

That’s curious. I’ll have a look.

6

Lux#main (v0.5) has dropped direct dependencies on all GPU backends. I will tag a release within a few weeks, just testing out edge cases if we missed any. See GPU Management - Lux.jl

1 Like