# Creating global variable in a worker
**URL:** https://discourse.julialang.org/t/creating-global-variable-in-a-worker/102150
**Category:** New to Julia
**Tags:** parallel
**Created:** [July 27, 2023, 10:27am UTC](https://discourse.julialang.org/t/creating-global-variable-in-a-worker/102150 "2023-07-27T10:27:42Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@Jad\_Zeitouni](https://discourse.julialang.org/u/Jad_Zeitouni)
#### Post date: [July 27, 2023, 10:27am UTC](https://discourse.julialang.org/t/creating-global-variable-in-a-worker/102150/1 "2023-07-27T10:27:42Z")
I have a vector `chains` which contains a `mutable struct` called `Chain` and a function `f!(chain::Chain)` which alters `chain`. I want to distribute each element of `chains` to a worker so that the worker could return to me a new `chain`, which I would later combine into the new `chains`.
Currently, I’m using a function as follows to distribute the work.
```julia
function g(chains::Vector{Chain})
futures = []
@assert nworkers() == length(chains)
for i in 1:nworkers()
push!(futures, @spawnat workers()[i] f!(chains[i]))
return [fetch(future) for future in futures]
end
```
Currently `f!` also returns `chain` in addition to altering the input, but I worry that might consume extra memory in the worker, since I’m not sure how variable assignment works in the worker (i.e. I worry that a new variable is being created containing the returned value of `f!` in addition to the altered input).
I want to have a way to create a variable `chain` in each worker that is altered and that I can fetch, so that I don’t have let `f!` return any value. Then I could do something like `chains = [@fetchfrom i chain for i in workers()]`.
Is this possible in any way? I tried adding `chain = chains[i]` in the `for` and feeding `chain` to `f!` but it doesn’t show up in the worker, I presume because it is in a local scope. If I declare it to be a global and I fetch it, it reassigns the value in the worker since its value changes in the for loop.
---
### Author:  [@Jeff\_Emanuel](https://discourse.julialang.org/u/Jeff_Emanuel)
#### Post date: [July 27, 2023, 4:29pm UTC](https://discourse.julialang.org/t/creating-global-variable-in-a-worker/102150/2 "2023-07-27T16:29:44Z")
Try using a RemoteChannel for communication.
---
### Author:  [@Jad\_Zeitouni](https://discourse.julialang.org/u/Jad_Zeitouni)
#### Post date: [July 31, 2023, 7:53am UTC](https://discourse.julialang.org/t/creating-global-variable-in-a-worker/102150/3 "2023-07-31T07:53:26Z")
This was my first thought.
I’ve changed the function to as follows:
```julia
@everywhere function h!(channel::RemoteChannel)
chain = take!(channel)
f!(chain)
put!(channel, chain)
chain = nothing # I added this in the hopes that the worker doesn't retain chain in memory after put!
end
function g(chains::Vector{Chain})
@assert length(chains) == nworkers()
channel = RemoteChannel(()->Channel{Chain}(length(chains)), 1)
for chain in chains
put!(channel, chain)
end
@sync for i in 1:nworkers()
@spawnat workers()[i] h!(channel)
end
chains = []
for i in 1:nworkers()
push!(chains, take!(channel))
end
return chains
end
```
However, even after calling the garbage collector, there seems to be double the amount of chains in memory. Each worker still has memory occupied worth one chain as I can see on my system monitor, and the main process also has a all the chains separately in memory.
---
### Author:  [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)
#### Post date: [July 31, 2023, 8:31am UTC](https://discourse.julialang.org/t/creating-global-variable-in-a-worker/102150/4 "2023-07-31T08:31:27Z")
Have you seen this
> **[GitHub - ChrisRackauckas/ParallelDataTransfer.jl: A bunch of helper functions...](https://github.com/ChrisRackauckas/ParallelDataTransfer.jl)**
>
> A bunch of helper functions for transferring data between worker processes - GitHub - ChrisRackauckas/ParallelDataTransfer.jl: A bunch of helper functions for transferring data between worker proce...
---
### Author:  [@Jad\_Zeitouni](https://discourse.julialang.org/u/Jad_Zeitouni)
#### Post date: [July 31, 2023, 10:04am UTC](https://discourse.julialang.org/t/creating-global-variable-in-a-worker/102150/5 "2023-07-31T10:04:16Z")
Thanks for this. It really helped but I still need to figure out how to pass data from the workers to the main process without duplicating it in memory since I’m close to memory limit. I guess I can write it to disk?
So after I do
```julia
for i in 1:nworkers()
sendto(i+1, chain=chains[i])
end
for i in 1:nwokers()
@spawnat i+1 f!(chain)
end
```
I do have exactly what I want, each worker has a chain they sampled independently and in parallel. However, I would like to pass it to the main process without having it duplicated in memory. I could potentially copy them one by one from each worker and then killing the worker but I don’t like this approach.
Edit: I tried setting `chain` to `nothing` at each worker and then running `@everywhere GC.gc()` but this did not clear memory as I had expected it to.
---
### Author:  [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)
#### Post date: [July 31, 2023, 1:41pm UTC](https://discourse.julialang.org/t/creating-global-variable-in-a-worker/102150/6 "2023-07-31T13:41:01Z")
Sorry, no idea how to do this. Would not be better to use multi-threadding in this case?
---
### Author:  [@Jad\_Zeitouni](https://discourse.julialang.org/u/Jad_Zeitouni)
#### Post date: [July 31, 2023, 1:42pm UTC](https://discourse.julialang.org/t/creating-global-variable-in-a-worker/102150/7 "2023-07-31T13:42:57Z")
Yes it would be simpler indeed, but for some reason it’s about 50 percent slower, so I wanted to see if I could gain in time without sacrificing memory.
---
### Author:  [@dlakelan](https://discourse.julialang.org/u/dlakelan)
#### Post date: [July 31, 2023, 2:01pm UTC](https://discourse.julialang.org/t/creating-global-variable-in-a-worker/102150/8 "2023-07-31T14:01:43Z")
> [@Jad\_Zeitouni](#):
>
> about 50 percent slower,
Do you have hyperthreading enabled? You may want to use ThreadPinning.jl I’ve got that in my startup.jl so it’s just always pinning threads to cores.
---
### Author:  [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)
#### Post date: [July 31, 2023, 7:47pm UTC](https://discourse.julialang.org/t/creating-global-variable-in-a-worker/102150/9 "2023-07-31T19:47:54Z")
Does your code allocate a lot? If yes, either get rid of allocations, or `Distributed` might be better. Of not, Threading should be better. In 1.10 the GC should be hyper-threaded, which might help, but I have not tested that.