[ANN] Ark.jl: archetype-based entity component system (ECS) for games and simulations

[mlange-42]

We are excited to announce the release of Ark.jl v0.1.0, an archetype-based entity component system (ECS) for Julia, ported from the Go ECS library Ark.

If you are unfamiliar with ECS, scroll down to learn why it is especially relevant for the Julia community.

Ark’s features

• Designed for performance and highly optimized.
• Well-documented, type-stable API.
• Blazing fast batch entity creation.
• No systems. Just queries. Use your own structure.
• Minimal dependencies, 100% test coverage.

Why ECS?

ECS was originally invented for game development, but there is an aspect that is probably more interesting for the Julia community:

ECS is exceptionally well suited for individual-based and agent-based models (IBMs/ABMS) and particle simulation models because it offer modularity, scalability, and performance, which aligns perfectly with the needs of simulating large numbers of autonomous agents.

An entity component system is a software architecture that separates data (components) from behavior (systems or queries), and organizes simulation elements (entities) as compositions of components. This design offers several key advantages for IBMs and ABMs:

1. Modularity and Flexibility

• Each agent is an entity composed of components like position, velocity, health, or behavior traits.
• You can easily add, remove, or modify components at any time in a running simulation.
• This makes it simple to represent heterogeneous agents with varying attributes and behaviors.

2. Scalability

• ECS is designed to handle millions of entities efficiently.
• Queries operate only on the required components, enabling high-performance, cache-friendly processing.

3. Separation of Concerns

• Behavior logic is encapsulated in systems or queries, which operate on specific component types.
• This clean separation allows for easier debugging, testing, and extension of simulation logic.
• For example, a “movement system” might update all entities with Position and Velocity components, regardless of other traits.

ECS provides a high-performance, modular, and extensible foundation for agent-based simulations, making it ideal for modeling complex systems composed of many interacting, heterogeneous individuals. Its ability to cleanly separate data and behavior, scale to large populations, and support dynamic agent lifecycles makes it a natural fit for modern simulation needs.

Why Ark.jl?

Ark is primarily designed for performance and ease of use, with great care for good documentation. It allows domain experts to leverage Julia’s performance and modern hardware for their models, without needing deep software engineering expertise.

Ark provies a minimal core ECS implementation, avoiding rigid frameworks and giving you full control over your simulation architecture

Performance comparison

Finally, to demonstrate that Ark does not just offer great flexibility and modularity but also superb performance, here is a chart that compares iteration speed abainst the frequently use Array of Structs (AoS) approach where “model entities” are structs stored in a vector. Lines of different width represent entity size (in bytes or number of state variables). The figure shows that AoS slows down as entity count or size increases, while Ark maintains blazing speed across scales.

See the resp. documentation section for details.

We highly appreciate your feedback and contributions!

Special thanks to @Tortar for his substantial contributions to Ark, and for helping me to get started with Julia, particularly its compile-time features. His support was crucial for the progress we made and for the outstanding performance we achieved.

[oschulz]

Just curious, how does Ark compare to ReactiveECS, design-wise and so on?

[mlange-42]

ReactiveECS.jl currently uses one huge “table” for all entities, which has the issue of potentially vasting a lot of memory. However, the author @Gesee still seems to be iterating on the architecture, as far as I can tell from what I read on the Julia and Flecs Discord servers. There is also some discussions about that in this forum thread.

Ark.jl uses a classical and proven archetype-based architecture, which uses a separate “table” for all entities with the same set of components. This avoids the mentioned waste of memory, as each table only has the required columns. At the same time, query iteration is way more efficient than in sparse-set ECS, like Overseer.jl. Adding and removing components to/from entities, however, is more costly in archetype-based ECS compared to sparse-set. At least in theory, I have not yet benchmarked Ark against Overseer.

[aplavin]

Oh, didn’t know General still accepts such short package names

[Tortar]

We had quite a long discussion about this, and as far as I can tell, this package will probably have the honor of being the last one granted such an exception (since no formal rule is in place preventing it but general registry mantainers are thinking to add one)

[oschulz]

Thanks for the explanation (of the architecture)!

[sdanisch]

This is really cool
While reading I just got a fun idea!
We’ve been discussing to use an entity system for Makie for quite a while to solve one annoying performance problem and make Makie more appealing for more game engine like use cases:

for arr in arrays 
     plot!(..., arr) 
end 
# the above much slower than
plot!(..., vcat(arrays))

This kind of batching is what most ECS systems solve, and quite desirable, since sometimes it’s just simpler to reason about small entities, instead of always concating them into one array.
Now, the idea is, why switch Makie to an entity system, if we can make a pretty efficient integration of plotting and updating all the batched entities, and let the entity system be the game engine like interface Makie users could use to solve this use case?
A few advantages of this:

• we dont have to make another 3d engine in Julia for Ark
• Makie has many backends and more are coming! It would immediately be able to run in the browser and we’re also planning to add a real time ray tracing backend and improve our 3d renderer
• I do plan to implement custom shaders for custom makie plots, so we could do some pretty cool new rendering types
• The ray tracing backend would allow to run custom julia callbacks and could even put them on the GPU

If you’re interested and i find some time, I could make a little prototype.

I just see that you seem to have tried using GLMakie:
Rework demo to use MiniFB instead of GLMakie

Any reason why it didn’t work out?
I would also like to have a fun 3D game demo, is there anything working as an example I could start from?

[Gesee]

Actually I solved the problem using FragmentArrays.jl which allocate only for entities having the given component… and even improved benchmarks…
But right now I’m taking a break from ECS to polish my game engine Cruise.jl so people can use it and extend it at will.
So you should use Ark.jl for now, ReactiveECS.jl is cool but still need some polish.

No worry when it will be released it will be a big shot

[Gesee]

Would be really happy to have Ark as a plugin for Cruise…
Just need to register it in the package manager..

[Gesee]

I would actually make it possible to use Makie in my game engine… just that I still need to see how I can make 2d/3d render with it and how it integrate in a game engine…
I also though that makie would lack some optimizations specific to game engines such as level of detail or impostors.
But I’m not too sure about this…

[mlange-42]

Thank you @sdanisch for these interesting suggestions! It would be amazing to see Ark.jl used as a backend of a high quality, frequently used library like Makie! Although I do not really understand the implications of your plot example yet.

Regarding your list of advantages, Ark.jl will never become a full game engine with integrated rendering etc. It is meant to be a pure ECS implementation that can be integrated in any framework, particulary as simulation models that mostly run headless on HPCs are the primary focus. Most probably, there will not even be classical ECS “systems” in the core library. Instead, Ark should be seen as kind of a blazing fast database-like library, where its queries can be used in any architecture that suits the host application.

Regarding our switch to MiniFB.jl, this had several reasons. First, I used a gayscale image so save memory, but that caused huge allocations each frame, presumably due to conversion before sending to the GPU. This in turn caused regular short halts of the demo, I guess due to GC pauses. When I tried to use PNGImages.jl instead of Images.jl for loading the logo, it became even much worse. Not sure what the issue was there, particularly as this was really used only once during initialization, and the image was not used for the actual graphics, only for creating the entities. Maybe some type piracy problem as @Tortar suggested?

After switching to RGB for the image, the demo ran fluently, but there were still allocations equal to the size of the image on every frame. Further, I got weird async double execution of some frames, but this was probably because I was not using the render tick callback correctly. You can see the implementation before the change in the PR you linked.

Finally, I switched to MiniFB.jl to get lighter on dependencies and to reduce the compile time for the curious user, and that also worked smoothly together with PNGImages.jl. But when there will be demos that require more sophisticated graphics, I will certainly use GLMakie again.

For an example to get started, there is currently only the demo and the short examples in the docs. The demo should give you an idea how to implement systems and a scheduler (which is dead simple). For the graphics, you probably have more expertise anyway. But happy to discuss details if you give it a try and need any advice.

Seeing your projects on GitHub, I would be very interested in discussing how GPU acceleration could be used by or integrated with Ark’s queries.

[sdanisch]

I see, I initially wasn’t aware that you’re rendering into an image, instead of actually using e.g. scatter to draw particle like entities.
It should be possible to update image plots without allocations, you’d just need to hit the correct type. I could look into this, but I guess efficiently displaying an image for something rendered somewhere else wasn’t really my idea

Compiling whole operations of the ECS to the GPU would be super cool and a pretty cool julia on the GPU use case But would certainly be quite a bit of work.

Regarding your list of advantages , Ark.jl will never become a full game engine

Ah, I didn’t really mean it in that sense, more like a Game engine interface could be created within a package which uses an ECS to animate/organize entities and Makie to render things - not wanting to change anything about Ark itself.

@geese, I was also thinking reversing the dependency (not making Makie use the ECS), would also open up the opportunity to not just support one blessed ECS, but any ECS could then just make it’s own “frontend” for Makie
About the level of detail etc, you’re right - but I’d actually would expect the ECS enabling such optimizations, since we dont have a good way in Makie to do calculations “on the whole world” and updating meshes/entities based on that (LOD pretty much does just that usually on the CPU, at least for not super optimized game engines). With Raycore.jl, which tries to be scene compatible to Makie in a way, one could add more operations than ray triangle intersections, which would enable efficient LOD or Physics calculations on the GPU…

Happy to discuss with anyone about creating some proof of concepts.
Easiest would be on discord: Makie

[mlange-42]

Ah, I didn’t really mean it in that sense, more like a Game engine interface could be created within a package which uses an ECS to animate/organize entities and Makie to render things - not wanting to change anything about Ark itself.

@sdanisch Yes, sure, I also did not understand it this way. This was primarily related to your first point:

we dont have to make another 3d engine in Julia for Ark

I.e., I fully agree with this.

[Gesee]

@sdanisch

Game engine interface could be created within a package which uses an ECS to animate/organize entities and Makie to render things - not wanting to change anything about Ark itself.

It’s actually pretty easy to do with Cruise.jl by making an ECS interface plugin.

Now actual implementations could just use it and overload it.

Now we can make some sort of makie plugin relying on the ECS interface plugin to optimize.

This way Cruise stay extensible, Makie and Ark stay clean as they are.

I think it’s the best solution

Also you should check the docs about plugin so you understand what is it

[Tortar]

Compiling whole operations of the ECS to the GPU would be super cool and a pretty cool julia on the GPU use case But would certainly be quite a bit of work.

Is this necessary to realize the idea to make Makie use Ark as a back-end? Currently Ark is CPU-only. Maybe I’m actually misunderstanding the meaning of this sentence though.

[Tortar]

A little bit I’d like to add to the features mentioned by @mlange-42.

One cool aspect about Ark infrastructure is that we really track performance at a very granular level of detail: we have a big set of automatic performance tests run at every commit which allows not to include any regression when implementing new functionalities, see Initial capacity for entities in world and archetypes by mlange-42 · Pull Request #288 · mlange-42/Ark.jl · GitHub as an example.

[mlange-42]

@Tortar This has nothing to do with Makie or graphics. The idea is to accelerate operations in queries by performing calculations on the GPU.

[Tortar]

Ah understood, thanks for the clarification.

[simsurace]

Just wanted to say thank you for starting this! I already tried the package for some simulations and I find it very usable already. The big thing missing for me would be entity relations. I think I can work around that for now though. Looking forward to further developments!

[mlange-42]

Thank you @simsurace, I am really pleased to read this!

Entity relationships are a key feature that we will definitely implement. They will be similar to those in the Go Ark, but Julia will allow for a more ergonomic API. To get an idea what you can expect to come feature-wise, see the chapter on entity relationships of the Go version.

I am optimistic that we can provide them in January, if not even by the end of this year.