Hardship lying ahead for the Julia community

Julia was created on greed. We wanted everything of a programming language.

Now, we have this cool language, a good foundation.
Now, we want Julia to get into applications. We want Julia to DO something.
Hardship lies ahead in every direction. To keep up its greed, each wish is to be followed by lots of hard work. Some of us may want to launch a server that runs simulation on millions of CPU, build a big database, quickly code lots of efficient microservices, make a triple-A game engine, make real-time control system, etc… We may even want to build a system with many of these requirements combined, for example, a real-time control system that uses machine learning and difeq modeling to control a robot to adapt to its environment. We may want a big climate simulation running on a big server.

Some, if not all, of these will probably be painstakingly coded up by extremely talented members, but as the ambition grows, so is the resistance.
Let’s pick some example applications. These are by no mean necessary for Julia to do but we’re using Julia to do something right?
AWS is comprised of hundreds of different services. To match even a quarter of that, so people with Disks, RAMs, CPUs, and accelerators can build their own servers?
Unity is a game engine company with several thousand employees. To make a game engine to match that?
Controlling robots and so on will also require lots of infrastructure. For example, let’s say you want to formally verify system behavior’s correctness, Julia’s reflection capability might allow good tools to be developed, but these are just theories right now.
Or even developing app GUI efficiently, well, you need a large variety of patterns from buttons, scrolling bar, etc…, lots of styling options, and more, that’s also lots of work. And you want small binaries so add that to a list of work to be done.

So much hardship lies ahead. Where are we heading as the Julia community?

2 Likes

38 Likes

^^ My thoughts exactly

1 Like

It’s a challenge that we are committed to tackling. We have plans and don’t intend to back down. Real-time control mixed with SciML and large-scale simulation is where Julia needs to go. It will take sweat and grit but the language will only become better by the end.

27 Likes

Can’t be any happier and more excited about the focus on Real-Time. Any specific plans on how to tackle this challenge? Are we going toward a real-time GC or language subset/new features?

Lots of ideas, more to be shared in the near future.

5 Likes

Too bad we don’t some kind of Conference about Julia. That would be a great time for announcements! :wink:

5 Likes

Given the current advancements in machine learning / artificial intelligence, I would say Julia definitely needs some institutional money and resources to bring Tensorflow/pytorch/jax capabilities into the ecosystem. If Julia can really compete with these libraries, it would accelerate the adoption. I really hope that big companies like Google and Nvidia are thinking about it.

5 Likes

And what would be the benefit for Julia? These libraries are usually highly abstracted DSLs to call some optimized engine under the hood, making the underlying language barely relevant.

I hope they are not. This is what would happen if a big and powerful company decided that Julia is their next favorite tool for whatever purpose:

  1. insane amounts of money pouring into Julia development. this sounds good, until you realize that it would also come with a lot of influence, direct and indirect.

  2. changes that are relevant for the big company would meet much less resistance. PRs would be pushed through because big company wants them, and with less QC because hey, it’s big company programmers, they have to be really smart. Whenever this does not happen, we would get implicit or explicit threats about big company taking its custom elsewhere.

  3. one fine morning big company would drop Julia basically overnight, for the next shiny toy, leaving all the technical debt of features they asked for. Many would consider Julia a second rate language from then on, because hey, it’s big company programmers who made this decision (and they are really smart).

If you keep an eye on these things, you have seen this happen multiple times.

24 Likes

Admittedly I could have lept the sentence below for myself but…

“I haven’t understand this topic” :neutral_face: