# Checking that work is being sent to processors: GPU vs Multiple CPUs

**URL:** <https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179>\
**Category:** New to Julia\
**Created:** [September 13, 2022, 2:24am UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179 "2022-09-13T02:24:29Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![yoshi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoshi/32/38259_2.png) [@yoshi](https://discourse.julialang.org/u/yoshi)\
**Post date:** [September 13, 2022, 2:24am UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/1 "2022-09-13T02:24:30Z")

</div>

In the [Parameter Parallelism section of the DiffEqGPU.jl repo’s documentation](https://github.com/SciML/DiffEqGPU.jl#parameter-parallelism-with-gpu-ensemble-methods) there is some code. I got it to work. I want to check that its running on the processing units of the GPU. Is there someway to see workload is running on the individual processing units?

I also want to compare this workload running solve with EnsembleThreads() instead. In particular, I want to see that workloads are being sent to all my CPUs. Is there a way to see this in real time? I checked top expecting to see multiple instances of Julia running but I didn’t see this.

---

<div class="post-metadata">

**Author:** ![jmair](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmair/32/35117_2.png) [@jmair](https://discourse.julialang.org/u/jmair)\
**Post date:** [September 13, 2022, 5:42am UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/2 "2022-09-13T05:42:09Z")

</div>

If you are using NVIDIA gpus you can check `nvidia-smi` in a console, which should say how much memory each process is using (similar to `top` but for the GPU). Usually this is enough for me, but there are more sophisticated tools available for CUDA.

---

<div class="post-metadata">

**Author:** ![yoshi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoshi/32/38259_2.png) [@yoshi](https://discourse.julialang.org/u/yoshi)\
**Post date:** [September 13, 2022, 7:30pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/3 "2022-09-13T19:30:53Z")

</div>

Do you have any ideas about the CPUs? Top shows me the % used by each process. But I have a 40 core processor that I’m testing on and I only see ~500% usage when I should expect to see \>3000%…

---

<div class="post-metadata">

**Author:** ![yoshi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoshi/32/38259_2.png) [@yoshi](https://discourse.julialang.org/u/yoshi)\
**Post date:** [September 13, 2022, 10:23pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/4 "2022-09-13T22:23:51Z")

</div>

Also when I check nvdia-smi I don’t see any processes

---

<div class="post-metadata">

**Author:** ![jmair](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmair/32/35117_2.png) [@jmair](https://discourse.julialang.org/u/jmair)\
**Post date:** [September 13, 2022, 10:37pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/5 "2022-09-13T22:37:19Z")

</div>

I am not 100% sure but usually the diagnostic tools external to Julia are not totally accurate. Maybe someone else knows a good tool for profiling.

Personally, I test the scaling of functions to know what I can expect which is done with some simple benchmarking. Either I change the number of threads used and look at the speedup for the same workload, or I change the problem size to see how much more performance you can get with more cores.  
For example, if you were to plot the parallel speedup vs problem size, you would expect to see something like this, which hopefully shows a speedup equal to the number of threads for large enough problem sizes:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/8/3/83fbe2d0da2a0107295fe3835388686822f35ba7.png)

Obviously, you don’t need to go to this much effort, as you can just manually run some quick checks in the REPL.

---

<div class="post-metadata">

**Author:** ![jmair](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmair/32/35117_2.png) [@jmair](https://discourse.julialang.org/u/jmair)\
**Post date:** [September 13, 2022, 10:38pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/6 "2022-09-13T22:38:11Z")

</div>

This likely means that the program doesn’t use the GPU at all

---

<div class="post-metadata">

**Author:** ![yoshi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoshi/32/38259_2.png) [@yoshi](https://discourse.julialang.org/u/yoshi)\
**Post date:** [September 13, 2022, 10:46pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/7 "2022-09-13T22:46:52Z")

</div>

uh oh – but this package is specifically designed to run on GPUs. The code I ran is an example they share with us.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [September 13, 2022, 10:58pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/8 "2022-09-13T22:58:47Z")

</div>

What are you running?

```julia
using DiffEqGPU, OrdinaryDiffEq
function lorenz(du,u,p,t)
    du[1] = p[1]*(u[2]-u[1])
    du[2] = u[1]*(p[2]-u[3]) - u[2]
    du[3] = u[1]*u[2] - p[3]*u[3]
end

u0 = Float32[1.0;0.0;0.0]
tspan = (0.0f0,100.0f0)
p = [10.0f0,28.0f0,8/3f0]
prob = ODEProblem(lorenz,u0,tspan,p)
prob_func = (prob,i,repeat) -> remake(prob,p=rand(Float32,3).*p)
monteprob = EnsembleProblem(prob, prob_func = prob_func, safetycopy=false)

@time sol = solve(monteprob,Tsit5(),EnsembleGPUArray(),trajectories=10_000,saveat=1.0f0)
@time sol = solve(monteprob, GPUTsit5(), EnsembleGPUKernel(), trajectories = 10_000, adaptive = true, dt = 0.1f0, save_everystep = false)

```

Those last two calls will definitely use GPUs and you’ll see it in `nvidia-smi` calls via the utilization percentages. Though note those examples will only run for like \<1 second IIRC, so you’ll need to be fast (or make the example bigger)

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [September 13, 2022, 11:00pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/9 "2022-09-13T23:00:56Z")

</div>

> [@yoshi](#):
>
> In particular, I want to see that workloads are being sent to all my CPUs. Is there a way to see this in real time? I checked top expecting to see multiple instances of Julia running but I didn’t see this.

Multiple instances is different from multiple threads. Multiple instances would be the result of multiprocessing, i.e. using Distributed. If you’re using `top`, you’ll just see \>100% CPU utilization (using `htop` is usually a lot nicer for investigating this kind of thing)

---

<div class="post-metadata">

**Author:** ![yoshi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoshi/32/38259_2.png) [@yoshi](https://discourse.julialang.org/u/yoshi)\
**Post date:** [September 13, 2022, 11:08pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/10 "2022-09-13T23:08:14Z")

</div>

Ya I’m running this –

For the last two lines:

- I changed modified the penultimate line so that trajectories=1000000, saveat=200.0f.0  
I checked nvidia-smi and saw nothing.

- In the last time, GPUTsit5(), didn’t load. I am using the DiffEqGPU library, so I’m unsure of why this is giving me an error. I’m getting the error:  
UndefVarError: GPUTsit5 not defined

---

<div class="post-metadata">

**Author:** ![yoshi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoshi/32/38259_2.png) [@yoshi](https://discourse.julialang.org/u/yoshi)\
**Post date:** [September 13, 2022, 11:10pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/11 "2022-09-13T23:10:18Z")

</div>

I did not know this – I’ll check htop, thanks!

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [September 13, 2022, 11:11pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/12 "2022-09-13T23:11:52Z")

</div>

How long did it take to solve? It can be hard to see with nvidia-smi if it only lasts a second.

> [@yoshi](#):
>
> In the last time, GPUTsit5(), didn’t load. I am using the DiffEqGPU library, so I’m unsure of why this is giving me an error. I’m getting the error:  
> UndefVarError: GPUTsit5 not defined

What package version?

---

<div class="post-metadata">

**Author:** ![yoshi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoshi/32/38259_2.png) [@yoshi](https://discourse.julialang.org/u/yoshi)\
**Post date:** [September 13, 2022, 11:17pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/13 "2022-09-13T23:17:13Z")

</div>

1. I kept re-running nvidia-smi while solve was running. I kept seeing nothing. So I terminated solve call. Does solve need to complete running for me to see something in nvidia-smi?

2. I’m running julia 1.6.7. Is this what you mean?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [September 13, 2022, 11:26pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/14 "2022-09-13T23:26:03Z")

</div>

> [@yoshi](#):
>
> - I kept re-running nvidia-smi while solve was running. I kept seeing nothing. So I terminated solve call. Does solve need to complete running for me to see something in nvidia-smi?

Did you terminate it when it was compiling?

---

<div class="post-metadata">

**Author:** ![yoshi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoshi/32/38259_2.png) [@yoshi](https://discourse.julialang.org/u/yoshi)\
**Post date:** [September 13, 2022, 11:37pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/15 "2022-09-13T23:37:23Z")

</div>

oh hm, maybe.

Since this exchange, I’ve been letting solve run without terminating it (trajectories=1000000,saveat=200.0f0). So the call has been running at least 10 mins. I still don’t see anything in nvidia-smi.

I don’t know how compilation is supposed to work in Julia. Am I waiting for Julia to compile 10^6 kernels (I think this is the right term for this) to send to the graphics processing units?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [September 13, 2022, 11:38pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/16 "2022-09-13T23:38:44Z")

</div>

> [@yoshi](#):
>
> Since this exchange, I’ve been letting solve run without terminating it (trajectories=1000000,saveat=200.0f0). So the call has been running at least 10 mins. I still don’t see anything in nvidia-smi.

That’s odd. It should be a few seconds or so.

> [@yoshi](#):
>
> I don’t know how compilation is supposed to work in Julia. Am I waiting for Julia to compile 10^6 kernels (I think this is the right term for this) to send to the graphics processing units?

It just compiles one kernel function.

---

<div class="post-metadata">

**Author:** ![yoshi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoshi/32/38259_2.png) [@yoshi](https://discourse.julialang.org/u/yoshi)\
**Post date:** [September 13, 2022, 11:41pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/17 "2022-09-13T23:41:13Z")

</div>

Is there a way to check if I have everything configured correctly?

I’m running Julia 1.6.7 (downloaded from website and run directly from the bin folder). I then added the packages DiffEqGPU, OrdinaryDiffEq.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [September 13, 2022, 11:42pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/18 "2022-09-13T23:42:18Z")

</div>

> [@yoshi](#):
>
> I’m running Julia 1.6.7 (downloaded from website and run directly from the bin folder). I then added the packages DiffEqGPU, OrdinaryDiffEq.

You probably want to use v1.8.1 with this. I haven’t tested LTS at least, but I know v1.8.1 should be fine? If that works, I can probably pin down what’s going on with LTS, but most things using advanced compiler toolchains run best with the latest release.

---

<div class="post-metadata">

**Author:** ![yoshi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoshi/32/38259_2.png) [@yoshi](https://discourse.julialang.org/u/yoshi)\
**Post date:** [September 13, 2022, 11:45pm UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/19 "2022-09-13T23:45:48Z")

</div>

okay let me try this with 1.8.1

---

<div class="post-metadata">

**Author:** ![yoshi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yoshi/32/38259_2.png) [@yoshi](https://discourse.julialang.org/u/yoshi)\
**Post date:** [September 14, 2022, 12:49am UTC](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179/20 "2022-09-14T00:49:11Z")

</div>

Same thing is occurring with 1.8.1: solve hasn’t completed running for \> 10 mins when trajectories=1000000,saveat=200.0f0 and nvidia-smi shows nothing. Any ideas?

Update: It took a half hour to run the first time. I tried running it a second time, in case this decreased the compilation time because something was stored in memory. I checked nvidia-smi again and saw nothing. It took about the same time to run the second time.

[Next page](https://discourse.julialang.org/t/checking-that-work-is-being-sent-to-processors-gpu-vs-multiple-cpus/87179.md?page=2)