Using @distributed for loop

New to Julia
, ,
1

Hello everyone,

Can someone point out why following setup is not working ? Issue is that I want to copy function result in xm variable that is defined by @everywhere to not have issue with workers. But when I run loop it prints results perfectly but its copying locally beacuse everytime I try to execute xm on RFPL for its values. Its always all zeros.

@everywhere xm = zeros(10)

@sync @distributed for i in 1:10

    p = [0.18, 4.0, 2.0]
    xm .=  copy(sim_func(p))              
    println(xm)
    println(i)

end
2

I guess answer is SharedArray.

3

Hi, your problem is very similar to another recent thread where I explained the issue:

So yes a SharedArray is a valid answer, if all processes live on the same physical machine. However if that is the case, do you have a reason to use processes instead of threads? Threads have generally less overhead and are easier to work with (e.g. your code example would’ve just worked).

4

Yea, its on same machine. I’m new to parallel processing. I didn’t know that, I will give it a shot. :slight_smile: Thank you

5

I made this small example for same but it doesn’t seems to be working . Here I would expect to get m = 1.0 in end but it gives m = 0.0 which is originally assigned value. Though it prints 1.0 inside loop.

function test(x)
    x +=  1
    return x
end

m = 0.0

Threads.@threads for i in 1:1029092090

    m = test(0.0)
    println(m)

end

This one works…

function test(x)
    x +=  1
    return x
end


Threads.@threads for i in 1:1029092090

    global m = test(0.0)

end
6

This is purely an issue because you used global scope. This also works:

julia> function foo()
       m = 0
       Threads.@threads for i in 1:1029092090
           m = test(0.0)
           println(m)
       end
       return m
       end
7

Outside of question by any chance due you know any source from where I can get intuitive idea of threading and memory allocation in threading process? I read Julia’s documentation but still I feel gap.

Thank you :slight_smile:

8

Maybe Thread-Safe Storage · OhMyThreads.jl could be of interest to you.

1 Like
9

Sorry I don’t really understand what you are asking for.

Threads share the same memory space and all that memory is managed by the same garbage collector (GC). This can lead to bottlenecks if you have a lot of threads that allocate a lot of memory. So to get good scaling it is important to try and reduce allocations to a minimum (this is generally a good idea if you want to get maximal performance). In recent Julia version the GC gained the ability to work in parallel as well which helps.

Process on the other hand have totally separate memory spaces. As they are completely separate Julia processes each has their own GC which can help with allocation pressure. On the other hand it means that each process needs to load all required resource for themselves (like libraries), compile code for themselves and communication between them is not as easy.

1 Like
10

Thank you:) Another quick question - So, if Im using a function which has some multithreaded loop inside (like one in example here) in my code and that code need to run this function multiple times which leads to consumption of alot of memory. Is there a way that memory can be cleaned up after each round function was used so we go back to original memory and process repeats?

11

This question is too general to be meaningful. Memory use per se is no problem: Julia has a garbage collector and will clean up/free unused memory on its own. However it can be that the way you wrote your code introduces additional, unnecessary allocations (e.g. due to the known issues with captured variables).
If you have a piece of code that you think has inefficiencies, then I’d encourage you to post a self-contained, executable snippet to Performance :slight_smile:

1 Like