# Having issues with running NeuralPDE on GPU

**URL:** <https://discourse.julialang.org/t/having-issues-with-running-neuralpde-on-gpu/108581>\
**Category:** GPU\
**Tags:** question, gpuarrays, sciml, neural-network\
**Created:** [January 9, 2024, 6:12pm UTC](https://discourse.julialang.org/t/having-issues-with-running-neuralpde-on-gpu/108581 "2024-01-09T18:12:35Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![tajimura](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tajimura/32/206064_2.png) [@tajimura](https://discourse.julialang.org/u/tajimura)\
**Post date:** [January 9, 2024, 6:12pm UTC](https://discourse.julialang.org/t/having-issues-with-running-neuralpde-on-gpu/108581/1 "2024-01-09T18:12:35Z")

</div>

Was trying to train a PINN for NLSE using NeuralPDE. Everything runs fine on CPU, but when I try to move calculations to GPU, I get the following error:

```julia
ERROR: CuArray only supports element types that are allocated inline.
Real is not allocated inline

```

My code is below. The error is raised at `prob=discretize...` line.

```julia
using NeuralPDE, Lux, LuxCUDA, Random, ComponentArrays
using Optimization
using OptimizationOptimisers
import ModelingToolkit: Interval

@parameters t x
@variables u(..)
Dt = Differential(t)
Dxx = Differential(x)^2
L = 10.0
w = 0.2
A = 1/w
ic(x) = A / cosh(x/w)^2

eq = Dt(u(t, x)) ~ 0.5 * Dxx(u(t, x)) + u(t, x) * abs(u(t, x))^2
bcs = [u(t, 0) ~ u(t, L), u(0, x) ~ ic(x)]
domains = [t ∈ IntervalDomain(0.0, 10.0), x ∈ IntervalDomain(-L/2, L/2)]

chain = Lux.Chain(Lux.Dense(2, 16, Lux.σ), Lux.Dense(16, 16, Lux.σ), Lux.Dense(16, 1))

strategy = GridTraining(0.05)

ps_cpu = Lux.setup(Random.default_rng(), chain)[1]
ps = ps_cpu |> ComponentArray |> Lux.gpu_device() .|> Float64
discretization = PhysicsInformedNN(chain, strategy, init_params=ps)

@named pde_system = PDESystem(eq, bcs, domains, [t, x], [u(t,x)])

prob = discretize(pde_system, discretization)

callback = function (p, l)
    #println("Current loss is: $l")
    return false
end

res = Optimization.solve(prob, Adam(0.01); callback = callback, maxiters = 2500)

```

**EDIT:** Updated the code to my last version. Also ignore the equation being wrong (NLSE has imaginary unit multiplied by time derivative, mine doesn’t)

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 9, 2024, 7:53pm UTC](https://discourse.julialang.org/t/having-issues-with-running-neuralpde-on-gpu/108581/2 "2024-01-09T19:53:21Z")

</div>

Make sure everything is either Float64 or Float32. I see some Float64 literals in `0.5` and `0.05`.

---

<div class="post-metadata">

**Author:** ![tajimura](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tajimura/32/206064_2.png) [@tajimura](https://discourse.julialang.org/u/tajimura)\
**Post date:** [January 9, 2024, 10:17pm UTC](https://discourse.julialang.org/t/having-issues-with-running-neuralpde-on-gpu/108581/3 "2024-01-09T22:17:51Z")

</div>

Already tried, getting the same error message

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 10, 2024, 12:20am UTC](https://discourse.julialang.org/t/having-issues-with-running-neuralpde-on-gpu/108581/4 "2024-01-10T00:20:02Z")

</div>

> [@tajimura](#):
>
> `IntervalDomain(-L/2, L/2)`

Everywhere? Things like this would give Float64s that would give the sampling some issues. What is your Float32 everywhere code?

---

<div class="post-metadata">

**Author:** ![tajimura](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tajimura/32/206064_2.png) [@tajimura](https://discourse.julialang.org/u/tajimura)\
**Post date:** [January 10, 2024, 1:57am UTC](https://discourse.julialang.org/t/having-issues-with-running-neuralpde-on-gpu/108581/5 "2024-01-10T01:57:40Z")

</div>

i switched to Float64. If Lux doesn’t sneak any Float32s internally, everything should be Float64.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 22, 2024, 9:49am UTC](https://discourse.julialang.org/t/having-issues-with-running-neuralpde-on-gpu/108581/6 "2024-01-22T09:49:58Z")

</div>

We just released an update, NeuralPDE.jl v5.10, which should fix this issue. GPU tests are passing.