[ANN] Ark.jl v0.5 - Archetype-based Entity Component System (ECS) for Julia

Hi all!

Happy to announce the release of Ark.jl 0.5. It was a slow burning effort this time, where we tried to make the package more robust in several aspects. It took time and effort, but I can say we made some good progress :slight_smile:

We haven’t introduced any big feature in this release, only small ones, a lot of performance improvements and some breaking changes to improve the API. Performance-wise maybe the most important improvement was due to some internal redesign which allowed us to remove some inference failures causing unnecessary allocations. I was somewhat surprised not to have catched this previously since we have a lot of benchmarks in CI, apparently applying the package to some big simulations helped!

So here are the major changes:

Breaking Changes

The relation API has been greatly improved. Instead of a keyword argument relations, which required to repeat the component information, now, relations are passed directly as pairs without the need of extra information, e.g. before one would had need to write new_entity!(world, (ChildOf(),); relations=(ChildOf => parent,)), while now one needs only new_entity!(world, (ChildOf() => parent,)).

Besides that, instead of requiring components to be defined as relations at struct definition time through a Relationship abstract type, now relation components can have any abstract type and are defined as relations when the World is initialized. This should help extensibility, since any struct can be used as relations, also ones which are naturally so but come from other packages.

Performance Improvements

While moving BeforeIT.jl (a big macroeconomics simulation model) to use Ark.jl, we found some inference issues with the API, but, thanks to some (more) @generated magic we were able to fix those little problematic aspects causing dynamic dispatch. Also, thanks to this, we also noticed some allocations in queries using relation components, and so, we fixed also that for the cause.

Apart from these issues which came from battle testing the package, we also found that many batch operations could have been made faster, achieving 3x faster results in some cases. And also some more caching could have helped in single entity operations, getting a small 10/20% improvement, still significant I would say since that part of the package is already very optimized (I looped through it trying to improve transition handling tens of times).

Features

We added support for sorting (and partitioning) at the level of tables, this way for each table one can sort the entities in it with a single sort_entities! call, which follows the sort! interface.


In general, I think that Ark.jl is getting very robust, the bugs we found were minor things (and so I didn’t described them here), and, also, its API is now in a state with which I don’t have any big complain with.

In any case, happy to receive any suggestions or discuss anything about it!

For a more extensive list of changes, see the CHANGELOG.

congratulations on the release!

I’m impressed and terrified :joy:

one thing I feel compelled to bring to your attention is that there are a great many sharp edges around generated functions, one of which being that it is illegal to call into inference during the code generation. and untyped list comprehensions do this!

e.g.

vec_types = [:(_gpuvectorview_type($t, $QB)) for t in types]

should become

vec_types = Expr[:(_gpuvectorview_type($t, $QB)) for t in types]

also I think access to .parameters property access of a type is neither a stable nor safe way to access type parameters, especially not inside @generated

arghhh, thank you a lot for bringing this to my attention!

Never knew untyped list comprehesions had that effect. Hopefully that + the access of type internals are the only issues. We took some care to follow the guidelines for generated functions not to do unsound stuff. Will try to fix those in 0.5.1 :wink:

I’m (…) terrified :joy:

Me too initially, but then I got used to it :laughing: I think some of the generated functions could be spared, by trusting a bit more Julia to optimize the code by itself. Though, some others (the majority I think) are instead really fundamental for performance.