# CUDAnative use multiple GPUs

**URL:** <https://discourse.julialang.org/t/cudanative-use-multiple-gpus/8274>\
**Category:** GPU\
**Tags:** gpu, cudanative, parallel\
**Created:** [January 10, 2018, 7:57pm UTC](https://discourse.julialang.org/t/cudanative-use-multiple-gpus/8274 "2018-01-10T19:57:56Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![fnoelscher](https://avatars.discourse-cdn.com/v4/letter/f/87869e/32.png) [@fnoelscher](https://discourse.julialang.org/u/fnoelscher)\
**Post date:** [January 10, 2018, 7:57pm UTC](https://discourse.julialang.org/t/cudanative-use-multiple-gpus/8274/1 "2018-01-10T19:57:56Z")

</div>

Hi,

I am using julia on docker from maleadt/juliagpu and I’d like to use multiple GPUs, instead of one.  
The docs mention this:

```julia
dev = CuDevice(0)
CuContext(dev) do ctx
    # allocate things in this context
    @cuda ...
end

```

but it does not seem to work. I have this block two times, but no matter which device number I choose, it always uses a single GPU instead. Any help is greatly appreciated.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [January 11, 2018, 2:41am UTC](https://discourse.julialang.org/t/cudanative-use-multiple-gpus/8274/2 "2018-01-11T02:41:57Z")

</div>

Not sure, but you might need an `@async`. That `do`-block syntax may be blocking until the first device finishes its task.

---

<div class="post-metadata">

**Author:** ![fnoelscher](https://avatars.discourse-cdn.com/v4/letter/f/87869e/32.png) [@fnoelscher](https://discourse.julialang.org/u/fnoelscher)\
**Post date:** [January 11, 2018, 10:28am UTC](https://discourse.julialang.org/t/cudanative-use-multiple-gpus/8274/3 "2018-01-11T10:28:30Z")

</div>

My code looks like this:

```julia
function calculateStuff(gpuId)
  dev = CuDevice(gpuId)
  CuContext(dev) do ctx
    @cuda (threads, blocks) expensiveFunction(...)
    synchronize()
  end
end

@spawn calculateStuff(0)
@spawn calculateStuff(1)

```

I execute julia with two worker processes so I guess that should do the trick? Nonetheless, only one GPU is used.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [January 11, 2018, 3:23pm UTC](https://discourse.julialang.org/t/cudanative-use-multiple-gpus/8274/4 "2018-01-11T15:23:50Z")

</div>

I don’t have a system with multiple GPUs, so I haven’t really worked on a decent multi-GPU API.  
But what I assume is happening here, is that you aren’t executing this code in separate processes. CUDA is an API with global state, and the `CuContext(dev)` call sets the global context for all subsequent API calls.

Maybe try the following (again, untested, but I think it should work):

```julia
using Distributed

@everywhere using CUDAdrv, CUDAnative

@everywhere function expensiveFunction()
    # ...
end

@everywhere function calculateStuff(gpuId)
    dev = CuDevice(gpuId)
    CuContext(dev) do ctx
        return expensiveFunction()
    end
end

s1 = @spawnat 1 calculateStuff(0)
s2 = @spawnat 2 calculateStuff(1)

fetch(s1)
fetch(s2)

```

I’m not too familiar with `Distributed`, so `@`everyone feel free to correct my use of the library.

---

<div class="post-metadata">

**Author:** ![fnoelscher](https://avatars.discourse-cdn.com/v4/letter/f/87869e/32.png) [@fnoelscher](https://discourse.julialang.org/u/fnoelscher)\
**Post date:** [January 11, 2018, 3:53pm UTC](https://discourse.julialang.org/t/cudanative-use-multiple-gpus/8274/5 "2018-01-11T15:53:23Z")

</div>

I resolved the issue. In fact, the problem was that all workers executed the same code - when you let workers preload files, they will execute everything that’s not within a function definition, for example.  
I put everything GPU-related in a module and moved it to a separate file, which is loaded by each worker (-L module.jl). Then, the “main” file executes functions from the module using @spawn and everything works as expected. Thanks for the help!

---

<div class="post-metadata">

**Author:** ![floswald](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/floswald/32/195_2.png) [@floswald](https://discourse.julialang.org/u/floswald)\
**Post date:** [March 24, 2018, 1:15pm UTC](https://discourse.julialang.org/t/cudanative-use-multiple-gpus/8274/6 "2018-03-24T13:15:32Z")

</div>

Hi  
That sounds a lot like a problem I am having. Would you mind posting a gist with a small example of your setup? Thanks a million!