Using CellListMap.jl within Agents.jl framework

Hi, I am using CellListMap in one of my ABM simulations (using Agents.jl). It suits very well what I want to do: There some active agents and some static obstacles. CellListMap finds collisions between agents and obstacles and calculates desired forces and torques. Since positions of obstacles are fixed, it makes sense to not build the CellList every step, but update it using UpdateCellList!. It gives 2x boost in the speed of the simulation.

However, there is a problem that I couldn’t resolve yet: if the number of obstacles is smaller than the number of agents, the cell list doesn’t get updated (fresh construction of the cell list work)! I was wondering if somebody has already used CellListMap and UpdateCellList! withing Ageints.jl framework and knows something about this problem.

My setup is straightforward: I create the cell list and give it as property to the Agents.jl model in a initialize_model function


    box_sides = [extent...]
    box = CellListMap.Box(box_sides, 2rcut)
    if n_obs > 0 # checking number of obstacles
        # rods_pos : vector of agents positions
        # obs_pos: vector of obstacles positions
        cl = CellListMap.CellList(rods_pos, obs_pos, box)
    else
        cl = CellListMap.CellList(rods_pos, box)
    end
    aux = CellListMap.AuxThreaded(cl)

    # model parameters as a mutable struct.
    properties = Parameters(dt, va, λ⁰, γ, l, d, ζ,
                            n_obs, obs_pos, obs_radius,
                            [forces, torque],
                            rcut, rcut^2,
                            # here are the cell-list related stuff
                            box, cl, aux)
    # define the model
    model = ABM(ActiveRod,
        space2d,
        rng=rng,
        properties = properties
    )

Then, in model_step! function, the time evolution of the model is implemented and the cell list should be updated:


function model_step!(model::ABM)
    # get the vector of rods positions
    rods_pos = [[model[id].pos...] for id in 1:model.agents.count]
    obs_pos = [[model.obs_pos[id]...] for id in 1:model.n_obs]
    # reset forces and torques
    for id in 1:model.agents.count
        model.force_torque[1][id] = [0.0, 0.0]
        model.force_torque[2][id] = 0.0
    end
    if model.n_obs > 0
        # update the cell list
        model.cell_list = CellListMap.UpdateCellList!(rods_pos, model.obs_pos, 
                                                      model.box, 
                                                      model.cell_list,
                                                      model.cl_aux)

        # calculate pairwise forces and torques
        CellListMap.map_pairwise!(
            (x, y, i, j, d2, force_torque) -> calc_forces_torque!(x, y, i, j, d2, model, force_torque),
            model.force_torque, model.box, model.cell_list)
    end
end

As soon as the number of obstacles are larger than the number of agents, updating the cell-list works perfectly.

2 Likes

Hello,

I had no idea about the existence of CellListMap.jl. Sounds like this could be wonderfully integrated with Agents.jl and accelerate existing functions like interacting_pairs we have there… I wish the authors of the package had contacted us about this!

In the meantime, @ehsani63 are you perhaps interested in contributing an integration example in our documentation, similar to the integration examples with e.g., DifferentialEquations.jl or other packages?

Now to your question:

I don’t think this is a good assumption to make! E.g., the code you provide is not runnable, nor do you provide any error stacktraces about what the actual error is. I’d advise you to make a Minimal Working Example or look here for more info on helping us help you: Please read: make it easier to help you

Nevertheless it seems to me that the error probably comes from the map_pairwise! function.

2 Likes

Hello, I’m the author of CellListMap. Its sunday, so I wont be able to really test things now. But what may be happening is that, by default, the cell lists are constructed for the smallest set of points, thus you may be experiencing some issue associated to that automatic switching, particularly if the number of agents change during the simulations.

You can disable that using

cl = CellListMap.CellList(rods_pos, obs_pos, box, autoswap = false)

maybe that solves already your problem.

Otherwise tomorrow I will take a closer look to your example. And check how I can help to integrate it better with Agents.jl.

3 Likes

Thanks @Datseris. I will prepare a minimal working example tonight and post it here.

The problem is not the map_pairwise! function. That function correctly maps a given function on pairs of points within the given distance, generated by CellList. The problem is that in some situations of my simulations the list does not get updated. So the problem should be in UpdateCellList!. I will send the minimal example tonight.

1 Like

One thing: is there the possibility of starting with n_obs = 0, and later change that, in the same simulation? That update won’t work, the underlying data structures are different.

I will have to check how to best deal with that case.

(The calculation mapped in one case or the other are different, by the way, when two sets are provided the interaction between the sets is mapped, while the interactions within the set are mapped when only one set is given).

Kinda off topic, I started the discussion on connecting Agents.jl with CellListMap.jl here: Integration with CellListMap.jl · Issue #659 · JuliaDynamics/Agents.jl · GitHub

1 Like

@lmiq Thanks. I tried that before, with autoswap=false, CellList does not get updated at all. And number of particles and obstacles are fixed in the simulation. I will send the minimal code to reproduce the issue by tonight, hope it helps.

@Datseris I think CellListMap.jl would be really useful in the context of ABM and Agents.jl. I would be more than happy to contribute in making an example of such integration. This is an example movie of the simplified version of my code, simulating the transportation of bacteria in porous media:
test

3 Likes

No. n_obs is a system parameter, fixed from the beginning. Actually the cell list is generated correctly for the very first step (which is due to the constructor of CellListPair), the update function is the problematic one.

wow that’s amazing! Well you got the job nearly done! Writing an example that leads to this animation (provided we fix the errors you have at the moment) is most likely all that’s mecessary!

1 Like

Thanks. Code is ready and working fine, quite fast thanks to CellListMap.jl. The problem is using UpdateCellList!, that gives 2x performance boost, which is very tempting.

I think I found the bug. I will probably post a new version today, or at most tomorrow, with the fix. (will be 0.7.18).

1 Like

I think it is done (at least in my tests, I would be glad if you test it there as well). Version 0.7.18 should be released at any moment in the register. Thanks for reporting that, and let me know if you find any other issue, or have any other suggestion concerning the use of CellListMap.

I would also love that we create some guidelines on how to integrate it with Agents.jl. If you provide us a minimal working example, I will be glad to help (even by modifying something on CellListMap if needed).

1 Like

@lmiq Thanks for the fix. It works perfectly now. Only one minor thing: With the new release rods_pos and obs_pos (x and y in the CellListMap notation) apparantly can’t be Vector{Vector}. I changed their type to Vector of SVector, which is better from performance point of view, then everything is fine again.

Yes, Static arrays are better, but I’ll check that and probably post a fix.

@Datseris I discussed my suggestions for the integration example here: Integration with CellListMap.jl · Issue #659 · JuliaDynamics/Agents.jl · GitHub

One option would be the simulation of run & tumbling bacteria in porous media (check the video), another would be a similar system with disks instead of rods, which a bit simpler and similar to the SIR example with addition of fixed obstacles. Both scenarios are related to my current research projects, to be submitted soon.

test

1 Like

Fixed that on 0.7.19 (and profited the opportunity to add and organize a bunch of unit tests to avoid future regressions like this).

I will take a careful look at the Agents example now. Thanks for the feedback and for the cool usage example.

1 Like