Should we make a push into gaming industry?

It’s not easy, but it can be easy if you adjust your goals. I.e. if you want to do a good pure Julia 3D game engine, or good 3D game, then it’s decades of work to get competitive, probably. And millions of dollars yes, for a single game, also because mo-cap isn’t cheap (maybe 140 people working on such non-programming task).

But if you adjust your goals and you can reuse a game engine and/or do some of it in Rust with its or C++ community, then the goal gets realistic. You want to reuse as much of the software (and external tools) already used. It’s no longer pure Julia, shouldn’t be the goal IMHO, it’s just questionable if Julia provides an additional benefit by adding it into the mix of software development. Some game devs might that way only see Julia as the main language they use, and Rust/C++ and game engine hidden from them, but is Julia a good trade-off for what they get as a language in the engines already?

E.g. racing games, Forza, rely on neural networks (Bayesian in first version, or all versions? First version was rewritten), I saw a very intriguing video on it (and the rubber-banding concept). If you use e.g. such tech or NeRF e.g. something rather new, then Julia might help more, but Julia might also be behind there (NeRFs first elsewhere).

I’m not asking for a pure-Julia as in arbitrarily pure. Julia itself is also implemented using ccalls in multiple places. I’m asking for pure enough that it would not suffer from the 2 cultures problem, that is, all the game logic and likely rendering logic as well (in case one wants to modify it), though many parts of logic in other languages could also be okay if it acts on Julia object like how some linear algebra in Julia works. External system calls such as rendering the final 2d array of colors onto the screen/etc can be in C/Rust/etc… Though if we have to implement something new, I think we’d rather do it in Julia just because we like it. Yeah…

I don’t think the GC is a showstopper IF you structure your code correctly. How can I say that when it’s known the GC has been problematic (in unusual cases) in the past? I think the problem has been fixed in 1.9, and more importantly you want to avoid the GC.

Some claim (rightly) that languages without GC (i.e. with reference counting) are better for hard real-time. Games are only soft-realtime and can handle GC, likely most games already have GC. But the games are split: a part (i.e. the game engine) is written in most certainly C++ (no GO), and another part (for sure a different process?) for game logic in e.g. C# (with GC). Even Python (that is CPython) is in some ways better than C# and Julia with reference counting, as more predictable (GC is on top of that for cycles), why Mojo keeps that aspect and will not go to full tracing GC.

BUT what you can do is structure code into two or more processes, the game engine in one with NO GC, and C# or whatever language with GC in another with game logic. Julia could replace C# there, but then only for game logic. Likely C#'s GC is better, so I’m not sure Julia’s would be good enough, but I think so for game logic process, for the other part Julia’s GC would be a problem, unless you basically disable it by not allocating, and then the game logic (Julia) part NEEDS to run in a different process. [Can people confirm more than one process actually used? Someone in the know, or simply a gamer could run their favorite game and see in the task manager if only on process started.] It’s sometimes not better to split into processes, but since you have only one master GC per process then I’m not sure it’s good with Julia’s current GC, until its GC algorithm is improved.

There ARE actually real-time GC algorithms available (e.g. in Java), but in Julia only generational. If I recall mmtk already has real-time GC, but I don’t recall what that project added to Julia.

There’s also, seemingly redundant efforts to improve the GC in Julia’s github.

Many PRs with “GC” label have been merged (recently for 1.9 or 1.10/master and 6 still open), e.g. I think this is relevant so I would develop games on 1.10/master:

  • New option --gcthreads to set how many threads will be used by the Garbage Collector ([#48600]). The default is set to N/2 where N is the amount of worker threads (--threads) used by Julia.

This one had “merge me” label (and “embarrising-bugfix” label, but no “GC” label), is on 1.9.1 backports: “Use gc alloc instead of alloc typed in lowering #48642

This issue has been closed (so fixed I assume): v1.9 rc1 significant slowdown in garbage collection #49120

Total runtime for ´julia test.jl’ on my machine is

around 5 s for Julia 1.8
around 94 s for Julia 1.9

Other PRs may also be relevant to games, possibly this in 1.9:

Multi-threading changes

  • Threads.@spawn now accepts an optional first argument: :default or :interactive. An interactive task desires low latency and implicitly agrees to be short duration or to yield frequently. Interactive tasks will run on interactive threads, if any are specified when Julia is started (#42302).
  • Threads started outside the Julia runtime (e.g. from C or Java) can now become able to call into Julia code by calling jl_adopt_thread. This is done automatically when entering Julia code via cfunction or a @ccallable entry point. As a consequence, the number of threads can now change during execution (#46609).

Making “a push into gaming industry” seems easier when reframed as
making “a push with gaming industry”. There is a current push in the gaming industry toward geometric algebra for the same reason there is a current push in theoretical physics toward geometric algebra: unified concepts (e.g., in geometric algebra, translation and rotation are the same thing, force and torque are the same thing, Maxwell’s four equations can be written as one equation) result in more concise implementations. Perhaps Julia’s push into geometric algebra is already happening given that this year’s JuliaCon has two scheduled talks about geometric algebra:

  1. “Geometric Algebra at compile-time with SymbolicGA.jl” by Cédric Belmant
  2. “SimpleGA. A lightweight Geometric Algebra library” by Chris Doran

Replacing one gigantic, hard, and expensive implementation with another gigantic, hard, and expensive implementation is not compelling. What is compelling is replacing one gigantic, hard, and expensive implementation with an implementation that is less than half the size with less than half the bugs and technical debt. For example, this video shows Dr. Todd Ell, senior technical fellow at Collins Aerospace (with 68,000 employees, the world’s largest supplier of aerospace components) describing how he is pushing to convert the (highly regulated) Collins Aerospace design and development tool chain currently based upon linear algebra (as is the gaming industry) to being based instead upon geometric algebra.

2 Likes

I think Godot would be an interesting candidate to partner with because it has two entry points.

  1. The scripting language. I know Julia does not have an advantage over GDScript, but I understand the spirit of the OP to be advocating for an eventual goal of a One Language Solution ™. Julia as scripting language would likely need to be part of that. And making a mini language to script in would be a start.

  2. As I understand it, GDExtensions allow basically any language to add functionality to Godot. I do not know how the compilation/packaging works for this, so that could be a hurdle for us. However, this could be a place to plug-in solvers, AI, etc implemented in Julia. This is were we could work on experimental packages that could be building blocks of a future, Julia-managed engine.

4 Likes

I’ m wondering how hard it is to use julia and godot to implement something like I tried to make the “UltraHand” from 《The Legend of Zelda: Tears of the Kingdom》 using Godot Engine . Would it be a good project to explorer how julia can be used? Does multiple dispatch and JIT have an advantage for this kind?

It would be stellar to have someone explore the possibility. All my posts are mostly speculation based on some casual experimentation with Godot long before I used Julia.

Re: multiple dispatch: it would be awesome ergonomically. I’d have an update(::AbstractAgent) method style API that dispatches on Agent type.

Re: JIT, I think mostly disadvantage. JIT can cause latency that is bad for games. Though it was mentioned in this thread that shaders need to be JITed, so not all bad. Julia has a growing set of tools for AOT too.

Edit: spelling

1 Like

Just brainstorming, but maybe we could code small and fun games where the focus is on having a good AI? Or what about a game where each player trains and AI, and then we get to watch the AIs fight each other?

2 Likes

Oh my, I’m having to learn geometric algebra because of this. https://www.youtube.com/@sudgylacmoe/videos
Looks like Sudgy has a good video on it.

2 Likes

Hello. I’m actually actively working on a game engine myself: GitHub - Kyjor/JulGame.jl: JulGame is a game engine based on the Julia programming language with the help of SDL2.
to be honest, it’s my first engine, and I’m not really writing it in a julian way, but I’ve been having fun with it. It’s very rough, but I plan on getting as far as the engine having a gui and being able to make simple 2d games. Currently, I am working on a scene editor. I have been able to build games out to a .exe
After this I think I’ll try and expand to 3d using the lessons I learned from this engine.

Here’s a game I created for a game jam written with julgame:

I plan on entering another game jam when I “finish” the editor

23 Likes


A bit of video of what the editor currently looks like

7 Likes

Well, I pledged my allegiance to the goddess of gaming anyway hehe. I know that the current game industry has been stagnant except for graphics, which is, even then, hitting its point of diminishing return. I know that something has to change, and Julia might be the best tool, not that it is currently good, but because of its potential. It’s a greedy language after all.

1 Like

Feel free to try julgame out and let me know what you would like to see in it. Or raise a pr. I’m just learning as I go about julia and game engines.

Note, the package I had in mind (it wasn’t easily googlabe and is different from NeRF.jl), i.e. with the video demo:

Video games and VFX

This is another vertical that has already seen NeRFs begin to shake things up. As I mentioned earlier, there are now ways to get NeRFs into Unreal Engine and Blender, opening the possibilities. It’s possible that, by the end of 2023, NeRFs will have dramatically lowered the barrier and budget-to-entry of virtual production.

Updated 10 May 2023: NeRF: Neural Radiance Field in 3D Vision, Introduction and Review

2 Likes

@Kyjor that looks awesome! I just wanted to suggest adding a Project.toml for your package dependencies and getting it set up so people can install it using Pkg, even if not registered its nice to do

] add https://github.com/Kyjor/julgame

and it just installs everything. You can just copy a registered package to see how its set up.

You may also want to call it like SomeGameEngineName.jl so it looks like a package, but that’s less important I guess.

4 Likes

Thanks! I still have a lot of work to do, but I’m making decent progress. And I’ll definitely do that! I wasn’t aware that I could. Thank you!

5 Likes

It seems like we’re climbing the ladder here. We’re making progress in simple 2d dungeon crawlers/etc games. Of course, I believe we have the potential to rewrite the order of game development. We CAN surpass the current AAA standard. At least I believe so.

Why? Because the two cultures problem means that game engines are only well-developed in aspects that can be turned into visual tools or high-level code without performance loss. This means tools like physics simulation/etc, if existing, are hard-coded and not very flexible. Some systems might even be purely cosmetic and CAN’T interact with the game logic because… well… game logic code inefficient script…

MY OPINION ONLY.

Another domain is gaming AI. In this Chinese forum job post , the company looks for intern using Julia to develop game simulation environment, they might have started working on it here.

3 Likes

I think one of the potential difficulties with using Makie for game dev may be the Reactive design. I’m not sure whether that doesn’t become too uncontrollable for a big complicated system. Just wondering though, I have never tried using it to that extent.

What about Starlight or GameZero.jl?