Problems using Thread.@thread

Hi,

I am trying to apply multi-threading to a for loop running a Monte Carlo simulation, but unsuccessfully.
So, let’s consider the following simple example:


using Base.Threads

a = 0
b = zeros(10,1)
Threads.@threads for i in 1:10
    a = a+1
    b[i] = rand
end

When I run this piece of code, I get the following errors:

Error thrown in threaded loop on thread 0: UndefVarError(var=:a)
Error thrown in threaded loop on thread 1: UndefVarError(var=:a)

Does anyone know how to solve this?
Also, I was wondering if I should somehow specify the number of threads at the beginning of the code.

Thank you in advance!

Should instead be:

Threads.@threads for i in 1:100 
global a = a+1
b[i] = rand()
end

Explanation:

  1. When executing within the REPL, you need to mark a as being a global, otherwise scoping issues will cause an UndefVarError.
  2. rand is a function, so to invoke it you need to add parens: rand()

Let me also add that with a being marked as global, this loop is going to be very slow (due to type instability). I recommend placing this into a function instead. Although either way, you’re going to get an incorrect result for a because of a lack of atomicity on read/write operations on a.

Also note that using atomic variable for these is almost never the right solution. The right solution is context dependent and in this particular case the right solution is a = i;

1 Like

@jpsamaroo thank you for your feedback.
The solution you suggested in the first reply works. However, I’m not sure I understood correctly how to use a function to do the aforementioned operations. What I’ve tried is:

using Base.Threads
a = 0
b = zeros(10)

function some_operations(a,b,i)
    a = a+1;
    b[i] = rand();
    return a, b
end

Threads.@threads for i in 1:10 
    global a; global b = some_operations(a,b,i)
end

But I get the following errors:

Error thrown in threaded loop on thread 1: Base.MethodError(f=typeof(Base.setindex!)(), args=((1, Array{Float64, 1}[0.343375, 0, 0, 0, 0, 0.253604, 0, 0, 0, 0]), 0.0313713, 7), world=0x00000000000055bb)
Error thrown in threaded loop on thread 0: Base.MethodError(f=typeof(Base.setindex!)(), args=((1, Array{Float64, 1}[0.343375, 0, 0, 0, 0, 0.253604, 0, 0, 0, 0]), 0.351986, 2), world=0x00000000000055bb)

What could be the reason?

What’s the point of the variable a here? I can barely understand this code. Do you have a single-threaded version that works? Could you post that (or a shorter version of it)?

Also note that rand() is not thread safe. See my recent post and suggested workaround here.

Close, but this won’t actually update a outside the function (the global a you define) because a is passed by-value (not by reference, like b is). An alternative is to do:

using Base.Threads
# Make a,b global const for performance
global const a = Ref{Int}(0) # Make into a `Ref` so it's mutable by reference
global const b = zeros(10)

function some_operations(a,b,i)
    a[] = a[] + 1 # or do: a[] += 1
    b[i] = rand();
    # Removed return because a and b will be updated by reference
end

function do_threaded_ops(a,b)
    Threads.@threads for i in 1:10 
        some_operations(a,b,i)
    end
end

do_threaded_ops(a,b)

Note that you would still need synchronization/atomics to make this safe and correct. Or, as @yuyichao rightly indicated, you can just do a = 10 outside of the loop, since that’s what a would become if run in serial, or in parallel with proper synchronization.

EDIT: “by-value” and “by-reference” are probably the wrong terminology here; instead, a and b, when passed into a function as-is, simply become locally-scoped within the function, and so reassigning them (like a = a + 1) will only re-assign a to a+1 in the local scope, not in any outer scope, like the global scope of the REPL.

@bennedich thank you for the link.
To answer your question, this code was just a super simple example I made up to understand the basics of multi-threading. The original problem I posted is available at: https://discourse.julialang.org/t/problems-using-pmap-and-doubt-about-the-number-of-workers-processes-to-use/20510
(See also my second comment on it, in which I provide further details).

About the link you posted, let me ask you the following. You clearly distinguish between either:

multi-threading (single computer, multiple cores), or parallel computing (cluster, several distributed machines)

In my case I am using one multi-core computer. Does that mean that I cannot use parallel computing at all (and thus pmap())?

I haven’t used the parallel functionality much, but my understanding is that it will work just fine on a multi-core computer, but will be more complicated to set up and use (e.g. no shared memory), and may be less efficient due to additional overhead compared to just using threads.

1 Like

Forgetting about the aforementioned maybe non-relevant example, I have now re-written my code, which has the following general structure:

using Base.Threads

...body...
Threads.@threads for i in 1:10
    ...body...
end

I get the following errors:

Error thrown in threaded loop on thread 0: Base.MethodError(f=typeof(Base.getindex)(), args=(typeof(Juno.input)(), 1), world=0x00000000000055cf)
Error thrown in threaded loop on thread 1: Base.MethodError(f=typeof(Base.getindex)(), args=(typeof(Juno.input)(), 1), world=0x00000000000055cf)

Does anyone have any idea about what they stand for? Any help would be much appreciated!