Multithreading agents.jl with @threads for all agents

My agents.jl users are loosely coupled. It should be possible to run them mostly in parallel. The only congestion point is the model variable, where their computational results are summed up.

So I introduced @threads to the for loop over all the agents and got this stacktrace:

ERROR: TaskFailedException
 [1] wait
   @ ./task.jl:345 [inlined]
 [2] threading_run(fun::var"#13#threadsfor_fun#8"{var"#13#threadsfor_fun#7#9"{StandardABM{Nothing, User, typeof(Agents.Schedulers.fastest), Dict{Symbol, Int64}, Random.TaskLocalRNG}, Base.ValueIterator{Dict{Int64, User}}}}, static::Bool)
   @ Base.Threads ./threadingconstructs.jl:38
 [3] macro expansion
   @ ./threadingconstructs.jl:89 [inlined]
 [4] model_step!
   @ ~/src/masiri/Initial model/demo_agents_thread.jl:49 [inlined]
 [5] step!(model::StandardABM{Nothing, User, typeof(Agents.Schedulers.fastest), Dict{Symbol, Int64}, Random.TaskLocalRNG}, agent_step!::Function, model_step!::typeof(model_step!), n::Int64, agents_first::Bool)
   @ Agents ~/.julia/packages/Agents/kEwy8/src/simulations/step.jl:54
 [6] run!(model::StandardABM{Nothing, User, typeof(Agents.Schedulers.fastest), Dict{Symbol, Int64}, Random.TaskLocalRNG}, agent_step!::typeof(dummystep), model_step!::typeof(model_step!), n::Int64; when::Bool, when_model::Bool, mdata::Vector{Symbol}, adata::Nothing, obtainer::Function, agents_first::Bool, showprogress::Bool)
   @ Agents ~/.julia/packages/Agents/kEwy8/src/simulations/collect.jl:153
 [7] top-level scope
   @ ~/src/masiri/Initial model/demo_agents_thread.jl:58

    nested task error: MethodError: no method matching firstindex(::Base.ValueIterator{Dict{Int64, User}})
    Closest candidates are:
      firstindex(::Any, ::Any) at abstractarray.jl:402
      firstindex(::Union{Tables.AbstractColumns, Tables.AbstractRow}) at ~/.julia/packages/Tables/AcRIE/src/Tables.jl:182
      firstindex(::LinRange) at range.jl:693
     [1] #13#threadsfor_fun#7
       @ ./threadingconstructs.jl:69 [inlined]
     [2] #13#threadsfor_fun
       @ ./threadingconstructs.jl:51 [inlined]
     [3] (::Base.Threads.var"#1#2"{var"#13#threadsfor_fun#8"{var"#13#threadsfor_fun#7#9"{StandardABM{Nothing, User, typeof(Agents.Schedulers.fastest), Dict{Symbol, Int64}, Random.TaskLocalRNG}, Base.ValueIterator{Dict{Int64, User}}}}, Int64})()
       @ Base.Threads ./threadingconstructs.jl:30

This is my code:

using Agents, Base.Threads

const FIB_NUM = 10

mutable struct User <: AbstractAgent

mutable struct MyModel 

function fibonacci(n::Int)
    if n <= 1
        return n
        return fibonacci(n - 1) + fibonacci(n - 2)

function user_step!(agent::User, model)
    agent.fibonacci_num += fibonacci(FIB_NUM)[:total_fib] +=  agent.fibonacci_num 

function create_my_model()
    n_users = 100
    users = [User(i, 0) for i in 1:n_users]
    agents = users

    properties = Dict(
        :total_fib => 0,
        :tick => 1

    model = ABM(User; properties=properties)
    for agent in agents
        add_agent!(agent, model)

    return model

function model_step!(model)
    model.tick += 1
    @threads for agent in allagents(model)
        user_step!(agent, model)

model = create_my_model()
n_steps = 356
run!(model, dummystep, model_step!, n_steps-1, mdata = [:total_fib])
println("total Fibonacci:", model.total_fib )

What goes on in this stack trace? … WHY? How can I make this run in parallel?
This must be a common request - how are you supposed to parallelize your agents’ computations?

Updating a model property inside the for-loop that iterates over all agents is not a good idea as concurrent access to the underlying data is likely to occur and that’s certainly not what you want. Instead you should use something like model.total_fib = sum(model[id].fibonacci_num for id in allids(model)) at the end of the model_step! function.

But that is not what the error is about, right? I am aware of the race condition - I had hoped Julia would magically (with multiple dispatches?) solve that, as the order of summation or access does not matter - and yes, I didn’t fully understand multiple dispatches yet. :slight_smile:

but you are of course right, your suggestion is very good to resolve the race condition. I will update the example with your change.

wtf, the error is gone! How can that stack trace point to that race condition?!

Parallelisation of agent iteration is not something I have dealt with so far because all my models run fast enough that I really didn’t have the need to do that. Parallel execution of multiple model replicates were totally enough for me. So I can’t really answer any of your questions about the stacktrace that the @threads macro has thrown at you. Glad that my quick look at the model_step! and user_step! functions has resolved your issue though. :slight_smile: