ABM performance tips

I have a mainly complete ABM built using Agents.jl (awesome package!) which functions as I want (albeit with placeholder parameters) now I am at the point I want to improve the performance of the model, but I am struggling a little.

I have looked at the Agents.jl performance tips page:

  • I haven’t gotten to parallelization, but I will use this once I am ready to start doing model runs.
  • I have 3 agent species, I am using a field type to differentiate the species.
  • No agents to represent spatial properties.
  • Use type-stable containers – I am using a mutable struct for model properties.
  • Type stability – this is something I am struggling with, I don’t think I have this correct.

Would anyone be willing to have a look at my model and provide some feedback/suggestions? Obviously I don’t expect line by line editing, but I anticipate I am making lots of very basic errors, I hope if these can be pointed out/corrected I can improve performance quite a bit.

1 Like

These:

            swimable_cells = []

should be typed (i. e. typeof(cell)[] or whatever). Fix all abstract containers like these.

Then you should profile the code to see where the bottlenecks are. If you have many fish, at some point finding the nearby fishes will be costly, and you may want to use CellListMap (there is an example in the Agents manual on how to integrate it).

ps: cool model!

4 Likes

Imiq is correct. swimable_cells is type Any. So that will incur some performance penalty. I don’t see any other major issues.

I don’t think you need a type annotation here

I’m not sure you need to use collect here

Those are very minor issues if at all.

You can also use @code_warntype my_function() to identify type instability.

2 Likes
# somewhere to store params - not sure if this is correct setup, but I read somewhere if I am using mixed
# types I should use mutable struct

You’re doing it properly :slight_smile:, all of the types T inside the mutable struct should return isconcretetype(T) == true.

koaro_prey = [
        x.pos for x in nearby_agents(smelt, model, model.vision_smelt) if
            x.type == :koaro && x.length < 10.0
            ]

This is really performance engineering at this point, but you don’t actually need to instantiate this as a vector until you call sample on it, which is done conditionally on !isempty(koaro_prey) && smelt.length > 10.0. Hence, you might see a small speedup if this condition happens relatively often:

# Generator as opposed to a vector
koaro_prey = (
        x.pos for x in nearby_agents(smelt, model, model.vision_smelt) if
            x.type == :koaro && x.length < 10.0
            ) 

     # if there are juvenile koaro prey nearby move to them - only if the smelt is an adult   
    if !isempty(koaro_prey) && smelt.length > 10.0
            # need to add weight - move to cell with the most prey
        move_agent!(smelt, sample(collect(koaro_prey)), model)
... rest of the code ...

A similar thing applies to swimmable_cells, but in this case I just think, because push!-ing is equivalent to filtering and filtering reads a little nicer, that you could save some space and write

swimmable_cells = filter(
    cell -> model.swim_walkmap[cell...] > 0 && model.basal_resource[cell...] > 5.0, 
    near_cells
)

Overall, this is good code. The type instability has been brought up by others already and I think this will make the largest difference.

2 Likes

All of the comments and tips above are very helpful and I also think that you’ve done a great job writing your code. Very clean, good structure, and a lot of comments here and there. :+1:

A few more very minor remarks, not related to performance per se, just ideas I had when skim reading your code:

  • Your model properties struct is pretty verbose and a lot of your fields overlap in what they actually do (vision_, mortality_random_, breed_prob_, etc). This makes the code a bit unwieldy to read, imho. Maybe consider creating a helper struct that holds all these fields BreedParameters and then instantiate an instance of this struct for each of the fish type and store those three in a breeds::NamedTuple or something similar inside your actual Parameters struct. You should then be able to access the data stored inside via model.breeds[:koaro].
    struct BreedParameters
        prob::Float64
        mortality::Float64
        mortality_random::Float64
        # and so on
    end
    
    Base.@kwdef mutable struct Parameters
        breeds::NamedTuple = (;
            :trout = BreedParameters(1.0,2.0,0.5),
            :koaro = BreedParameters(2.0,3.0,0.2),
            :smelt = BreedParameters(2.0,4.0,0.4)
        )
        # and so on
    end
    
    This should hopefully make the code a bit easier to read and reason about while at the same reducing redundancy.
    NB: Of course this approach is not very helpful if you want to change these parameters during simulation runs. Which I just assumed that you wouldn’t because why would size_mature or resource_pref change over time. If that’s necessary, you can always opt for a mutable struct and collection.
  • You might want to also use an explicit elseif here to make sure that you don’t possibly get unwanted behaviour at some point in the future when you add new fish types.
  • Lastly, I would like to suggest that you put your model structs and functions inside a module in a separate file from the actual script to test and run it. This way you can just using the module and have everything neatly organised and containerised.

Otherwise this looks pretty good as is. Don’t fret too much about potential performance issues. Your model seems unlikely to actually hit the computational ceiling of a regular modern laptop and should run just fast enough. If you really find that your model is unfathomably slow and you feel like that shouldn’t be the case, first check for the regular stuff (unnecessary iteration without early termination or unnecessary allocations during runtime) before diving deeper into the fine-tuning. Have fun! :slight_smile:

2 Likes

Thank you everyone, lots of great feedback! I am really enjoying how supportive the Julia community is to new comers!

1 Like