Slow LSTM on GPU in Flux

Specific Domains Machine Learning
, ,
21

And there is an alternative to Flux:

22

Hey Hi @avikpal
Thanks for posting this code! I am trying to use LSTM with explicit parameters in Lux.
Hence your code seems very relevant.

However, I tried running your code, and it spits out error. Sorry I am not very good at reading Julia errors. Could you please help me on this?

MethodError: no method matching (::Lux.LSTMCell{true, false, false, Tuple{typeof(WeightInitializers.zeros32), typeof(WeightInitializers.zeros32), typeof(WeightInitializers.ones32), typeof(WeightInitializers.zeros32)}, NTuple{4, typeof(WeightInitializers.glorot_uniform)}, typeof(WeightInitializers.zeros32), typeof(WeightInitializers.zeros32)})(::Tuple{Matrix{Float32}, Matrix{Float32}, Tuple{Matrix{Float32}, Matrix{Float32}}}, ::@NamedTuple{weight_i::Matrix{Float32}, weight_h::Matrix{Float32}, bias::Matrix{Float32}}, ::@NamedTuple{rng::Random.TaskLocalRNG})

Closest candidates are:
  (::Lux.LSTMCell{use_bias, false, false})(!Matched::AbstractMatrix, ::Any, ::NamedTuple) where use_bias
   @ Lux C:\Users\.julia\packages\Lux\5xfGO\src\layers\recurrent.jl:397
  (::Lux.LSTMCell{true})(!Matched::Tuple{AbstractMatrix, Tuple{AbstractMatrix, AbstractMatrix}}, ::Any, ::NamedTuple)
   @ Lux C:\Users\.julia\packages\Lux\5xfGO\src\layers\recurrent.jl:436

I am on Julia_v1.10 and Lux@ v0.5.16