Inverting ComplexF32 with Metal.jl throws an error

General Usage
,
1

Hi everyone! I’m working with Metal.jl on an Apple Silicon GPU and I’ve encountered the following issue. When computing the inverse of ComplexF32 numbers, some Float64 numbers seem to sneak in and cause Metal.jl to throw an error.

Here’s a minimal example:

using Metal

function mwe(T)
    d = mtl(rand(T, 2, 2))
    return inv.(d)
end

If T = Float32, everything works fine. If T = ComplexF32 instead, I get the following error:

julia> mwe(ComplexF32)
ERROR: InvalidIRError: compiling MethodInstance for (::Metal.var"#broadcast_2d#_copyto!##2")(::MtlDeviceMatrix{ComplexF32, 1}, ::Base.Broadcast.Broadcasted{Metal.MtlArrayStyle{…}, Tuple{…}, typeof(inv), Tuple{…}}) resulted in invalid LLVM IR
Reason: unsupported use of double value
Stacktrace:
  [1] Float64
    @ ./float.jl:345
  [2] convert
    @ ./number.jl:7
  [3] Complex
    @ ./complex.jl:14
  [4] Complex
    @ ./complex.jl:43
  [5] convert
    @ ./number.jl:7
  [6] widen
    @ ./operators.jl:946
  [7] inv
    @ ./complex.jl:474
  [8] _broadcast_getindex_evalf
    @ ./broadcast.jl:699
  [9] _broadcast_getindex
    @ ./broadcast.jl:672
 [10] _getindex
    @ ./broadcast.jl:620
 [11] getindex
    @ ./broadcast.jl:616
 [12] broadcast_2d
    @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
  [1] Float64
    @ ./float.jl:345
  [2] convert
    @ ./number.jl:7
  [3] Complex
    @ ./complex.jl:14
  [4] Complex
    @ ./complex.jl:43
  [5] convert
    @ ./number.jl:7
  [6] widen
    @ ./operators.jl:946
  [7] inv
    @ ./complex.jl:474
  [8] _broadcast_getindex_evalf
    @ ./broadcast.jl:699
  [9] _broadcast_getindex
    @ ./broadcast.jl:672
 [10] _getindex
    @ ./broadcast.jl:620
 [11] getindex
    @ ./broadcast.jl:616
 [12] broadcast_2d
    @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
 [1] abs
   @ ./float.jl:699
 [2] isinf
   @ ./float.jl:724
 [3] inv
   @ ./complex.jl:475
 [4] _broadcast_getindex_evalf
   @ ./broadcast.jl:699
 [5] _broadcast_getindex
   @ ./broadcast.jl:672
 [6] _getindex
   @ ./broadcast.jl:620
 [7] getindex
   @ ./broadcast.jl:616
 [8] broadcast_2d
   @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
 [1] isinf
   @ ./float.jl:724
 [2] inv
   @ ./complex.jl:475
 [3] _broadcast_getindex_evalf
   @ ./broadcast.jl:699
 [4] _broadcast_getindex
   @ ./broadcast.jl:672
 [5] _getindex
   @ ./broadcast.jl:620
 [6] getindex
   @ ./broadcast.jl:616
 [7] broadcast_2d
   @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
 [1] abs
   @ ./float.jl:699
 [2] isinf
   @ ./float.jl:724
 [3] inv
   @ ./complex.jl:475
 [4] _broadcast_getindex_evalf
   @ ./broadcast.jl:699
 [5] _broadcast_getindex
   @ ./broadcast.jl:672
 [6] _getindex
   @ ./broadcast.jl:620
 [7] getindex
   @ ./broadcast.jl:616
 [8] broadcast_2d
   @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
 [1] isinf
   @ ./float.jl:724
 [2] inv
   @ ./complex.jl:475
 [3] _broadcast_getindex_evalf
   @ ./broadcast.jl:699
 [4] _broadcast_getindex
   @ ./broadcast.jl:672
 [5] _getindex
   @ ./broadcast.jl:620
 [6] getindex
   @ ./broadcast.jl:616
 [7] broadcast_2d
   @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
 [1] muladd
   @ ./float.jl:500
 [2] inv
   @ ./complex.jl:476
 [3] _broadcast_getindex_evalf
   @ ./broadcast.jl:699
 [4] _broadcast_getindex
   @ ./broadcast.jl:672
 [5] _getindex
   @ ./broadcast.jl:620
 [6] getindex
   @ ./broadcast.jl:616
 [7] broadcast_2d
   @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
 [1] *
   @ ./float.jl:497
 [2] literal_pow
   @ ./intfuncs.jl:437
 [3] inv
   @ ./complex.jl:476
 [4] _broadcast_getindex_evalf
   @ ./broadcast.jl:699
 [5] _broadcast_getindex
   @ ./broadcast.jl:672
 [6] _getindex
   @ ./broadcast.jl:620
 [7] getindex
   @ ./broadcast.jl:616
 [8] broadcast_2d
   @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
 [1] muladd
   @ ./float.jl:500
 [2] inv
   @ ./complex.jl:476
 [3] _broadcast_getindex_evalf
   @ ./broadcast.jl:699
 [4] _broadcast_getindex
   @ ./broadcast.jl:672
 [5] _getindex
   @ ./broadcast.jl:620
 [6] getindex
   @ ./broadcast.jl:616
 [7] broadcast_2d
   @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
 [1] /
   @ ./float.jl:498
 [2] inv
   @ ./number.jl:255
 [3] inv
   @ ./complex.jl:476
 [4] _broadcast_getindex_evalf
   @ ./broadcast.jl:699
 [5] _broadcast_getindex
   @ ./broadcast.jl:672
 [6] _getindex
   @ ./broadcast.jl:620
 [7] getindex
   @ ./broadcast.jl:616
 [8] broadcast_2d
   @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
 [1] -
   @ ./float.jl:493
 [2] inv
   @ ./complex.jl:477
 [3] _broadcast_getindex_evalf
   @ ./broadcast.jl:699
 [4] _broadcast_getindex
   @ ./broadcast.jl:672
 [5] _getindex
   @ ./broadcast.jl:620
 [6] getindex
   @ ./broadcast.jl:616
 [7] broadcast_2d
   @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
 [1] *
   @ ./float.jl:497
 [2] inv
   @ ./complex.jl:477
 [3] _broadcast_getindex_evalf
   @ ./broadcast.jl:699
 [4] _broadcast_getindex
   @ ./broadcast.jl:672
 [5] _getindex
   @ ./broadcast.jl:620
 [6] getindex
   @ ./broadcast.jl:616
 [7] broadcast_2d
   @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
  [1] Float32
    @ ./float.jl:342
  [2] convert
    @ ./number.jl:7
  [3] Complex
    @ ./complex.jl:14
  [4] Complex
    @ ./complex.jl:43
  [5] convert
    @ ./number.jl:7
  [6] oftype
    @ ./essentials.jl:666
  [7] inv
    @ ./complex.jl:477
  [8] _broadcast_getindex_evalf
    @ ./broadcast.jl:699
  [9] _broadcast_getindex
    @ ./broadcast.jl:672
 [10] _getindex
    @ ./broadcast.jl:620
 [11] getindex
    @ ./broadcast.jl:616
 [12] broadcast_2d
    @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
 [1] *
   @ ./float.jl:497
 [2] inv
   @ ./complex.jl:477
 [3] _broadcast_getindex_evalf
   @ ./broadcast.jl:699
 [4] _broadcast_getindex
   @ ./broadcast.jl:672
 [5] _getindex
   @ ./broadcast.jl:620
 [6] getindex
   @ ./broadcast.jl:616
 [7] broadcast_2d
   @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Reason: unsupported use of double value
Stacktrace:
  [1] Float32
    @ ./float.jl:342
  [2] convert
    @ ./number.jl:7
  [3] Complex
    @ ./complex.jl:14
  [4] Complex
    @ ./complex.jl:43
  [5] convert
    @ ./number.jl:7
  [6] oftype
    @ ./essentials.jl:666
  [7] inv
    @ ./complex.jl:477
  [8] _broadcast_getindex_evalf
    @ ./broadcast.jl:699
  [9] _broadcast_getindex
    @ ./broadcast.jl:672
 [10] _getindex
    @ ./broadcast.jl:620
 [11] getindex
    @ ./broadcast.jl:616
 [12] broadcast_2d
    @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:119
Hint: catch this exception as `err` and call `code_typed(err; interactive = true)` to introspect the erroneous code with Cthulhu.jl
Stacktrace:
  [1] check_ir(job::GPUCompiler.CompilerJob{GPUCompiler.MetalCompilerTarget, Metal.MetalCompilerParams}, args::LLVM.Module)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/QgckP/src/validation.jl:170
  [2] emit_llvm(job::GPUCompiler.CompilerJob; kwargs::@Kwargs{})
    @ GPUCompiler ~/.julia/packages/GPUCompiler/QgckP/src/driver.jl:455
  [3] emit_llvm(job::GPUCompiler.CompilerJob)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/QgckP/src/driver.jl:200
  [4] compile_unhooked(output::Symbol, job::GPUCompiler.CompilerJob; kwargs::@Kwargs{})
    @ GPUCompiler ~/.julia/packages/GPUCompiler/QgckP/src/driver.jl:101
  [5] compile_unhooked
    @ ~/.julia/packages/GPUCompiler/QgckP/src/driver.jl:80 [inlined]
  [6] #compile#118
    @ ~/.julia/packages/GPUCompiler/QgckP/src/driver.jl:67 [inlined]
  [7] compile(target::Symbol, job::GPUCompiler.CompilerJob)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/QgckP/src/driver.jl:55
  [8] #invoke_frozen#19
    @ ~/.julia/packages/Metal/TF981/src/initialization.jl:15 [inlined]
  [9] invoke_frozen
    @ ~/.julia/packages/Metal/TF981/src/initialization.jl:13 [inlined]
 [10] macro expansion
    @ ~/.julia/packages/Metal/TF981/src/compiler/compilation.jl:436 [inlined]
 [11] macro expansion
    @ ~/.julia/packages/ObjectiveC/ectoz/src/os.jl:264 [inlined]
 [12] (::Metal.var"#compile##0#compile##1"{GPUCompiler.CompilerJob{GPUCompiler.MetalCompilerTarget, Metal.MetalCompilerParams}})(ctx::LLVM.Context)
    @ Metal ~/.julia/packages/Metal/TF981/src/compiler/compilation.jl:435
 [13] JuliaContext(f::Metal.var"#compile##0#compile##1"{GPUCompiler.CompilerJob{GPUCompiler.MetalCompilerTarget, Metal.MetalCompilerParams}}; kwargs::@Kwargs{})
    @ GPUCompiler ~/.julia/packages/GPUCompiler/QgckP/src/driver.jl:34
 [14] JuliaContext(f::Function)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/QgckP/src/driver.jl:25
 [15] compile(job::GPUCompiler.CompilerJob)
    @ Metal ~/.julia/packages/Metal/TF981/src/compiler/compilation.jl:434
 [16] actual_compilation(cache::Dict{…}, src::Core.MethodInstance, world::UInt64, cfg::GPUCompiler.CompilerConfig{…}, compiler::typeof(Metal.compile), linker::typeof(Metal.link))
    @ GPUCompiler ~/.julia/packages/GPUCompiler/QgckP/src/execution.jl:245
 [17] cached_compilation(cache::Dict{…}, src::Core.MethodInstance, cfg::GPUCompiler.CompilerConfig{…}, compiler::Function, linker::Function)
    @ GPUCompiler ~/.julia/packages/GPUCompiler/QgckP/src/execution.jl:159
 [18] macro expansion
    @ ~/.julia/packages/Metal/TF981/src/compiler/execution.jl:199 [inlined]
 [19] macro expansion
    @ ./lock.jl:376 [inlined]
 [20] mtlfunction(f::Metal.var"#broadcast_2d#_copyto!##2", tt::Type{Tuple{MtlDeviceMatrix{…}, Base.Broadcast.Broadcasted{…}}}; name::Nothing, kwargs::@Kwargs{})
    @ Metal ~/.julia/packages/Metal/TF981/src/compiler/execution.jl:194
 [21] mtlfunction
    @ ~/.julia/packages/Metal/TF981/src/compiler/execution.jl:192 [inlined]
 [22] macro expansion
    @ ~/.julia/packages/Metal/TF981/src/compiler/execution.jl:89 [inlined]
 [23] _copyto!
    @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:128 [inlined]
 [24] copyto!
    @ ~/.julia/packages/Metal/TF981/src/broadcast.jl:52 [inlined]
 [25] copy
    @ ~/.julia/packages/GPUArrays/VNJ6B/src/host/broadcast.jl:34 [inlined]
 [26] materialize(bc::Base.Broadcast.Broadcasted{Metal.MtlArrayStyle{2, Metal.PrivateStorage}, Nothing, typeof(inv), Tuple{MtlMatrix{ComplexF32, Metal.PrivateStorage}}})
    @ Base.Broadcast ./broadcast.jl:894
 [27] mwe(T::Type)
    @ Main ~/Projects/mwe.jl:4
 [28] top-level scope
    @ REPL[3]:1
 [29] top-level scope
    @ ~/.julia/packages/Metal/TF981/src/initialization.jl:110
Some type information was truncated. Use `show(err)` to see complete types.

The stacktrace is huge, but “Reason: unsupported use of double value” appears frequently. I guess a double-precision float, which I know Metal.jl can’t use, is created somewhere along the way. Is this a solvable issue or a bug?

2

The explanation of the error is right there in the stacktrace: inv calls widen:

Metal.jl would need to overly that method with a different one which doesn’t involve going through double precision.

3

You can just use @fastmath inv.(d) to avoid widening (sacrificing a little accuracy).