Foreword
One of the rare joys of running an open source project is that every few weeks you’ll get a bunch of messages that are variations of “you should really say something here” with just a link to some Twitter, Discourse, GitHub post etc. And with an uncanny degree of consistency, this invariable happens at the most annoying times. One of my most viewed discourse post “On Open Source” (On Open Source) was written on Christmas Eve, sitting in my parents’ living room by the fire place.
The past few weeks have been extremely busy trying to get compiler things in place for 1.12, so today happened to be the first day that I took off completely and decided to go dance Brazilian Zouk all weekend. Anyway, invariably, as I was resting by the side of the dance floor towards the end of the night today, I saw that I been sent that kind of mesage about this thread asking me to say something. I don’t think this particular thread is all that bad as these things go, but some themes that I’ve seen repeatedly for a few weeks (here and elsewhere), so I thought a somewhat extended post was appropriate.
Now, this is quite an extended intro (ahead of an even longer post) before getting to anything substantial and a significantly more personal voice than I usually use in my writing, but I do have a point to make first, which is the following: I’m posting here in my personal capacity as someone who’s been working to make this Julia thing happen for more than a decade now. There are real people behind all these organizations, with grand ambitions, way too few resources, and real feelings. I think brainstorming about ways to improve things, innovate ways to fund open source, etc, is all good, but (without prejudice to whether or not this happened in this particular thread), it is very exhausting for maintainers to keep hearing that they don’t care enough, aren’t working hard enough, aren’t spending enough money, etc. Open source software is a gift. It’s a gift we all give each other and a gift to the world. Yes, there can be politics, economics, and agendas involved, but it’s a gift first and foremost and I think it’s important to for all of us to keep reminding ourselves of that.
Alright, on to the substantive part, there’s a few different topics that keep getting intermixed, but let me try to make a few separate observations and hopefully between them, everything is covered. With respect to some of the governance gripes, I think what I wrote in the “On Open Source” post linked above are still good, so I won’t repeat myself there.
The economics of Julia
The first point I want to make is that there is no organization whose mission in life it is to improve julia for the average julia user. Not MIT or any of the other universities (whose mission is to do research). Not JuliaHub, Pumas, Relational, Beacon, or any of the other companies whose products are built in Julia and employ Julia (core and package) developers. And not even really NumFocus, whose mission is primarily educational and scientific. The second point I want to make is that even the current level of Julia development is expensive. Just counting the people whose primary job it is to work on Julia or core ecosystem packages (again for whatever purpose their organization may need such work), you’re probably looking at $10M a year, easily. And that’s not counting anybody contributing in their free time.
Since I’m sure not everyone knows how these kinds of organizations work, let me briefly expand on that in particular. The first is research organizations. Sometimes people think that there is significant research/grant dollars for work on Julia. However, that’s not the case. Generally you get research grants to solve some problem X and you convince the funding agency that the best way to do X is to spend 10-30% of the grant money making julia some part of Julia better, because then doing X will be so much easier. The nature of X where this work depends on project maturity. Early on it was more things like “make data science better” which obviously translates much more directly to Julia package development. However, as Julia has matured and become widely, doing those kinds of things in Julia is generally considered “not research” anymore. So now, you mostly get the applied science research grants, which translate somewhat less directly. But still, all the work there is in support of whatever science goal the funding is for, not general Julia development. There have in the past been occasional grants for Julia itself (e.g. there was a NASA grant which funded some of the work improving release quality and putting out patch releases), but none are currently active, and they are often small by comparison.
Similar story with the various companies that use Julia such as JuliaHub. I can’t speak for anyone else, and obviously this is not an official JuliaHub statement, but the reality is that Julia itself is not a JuliaHub product. That was a very deliberate choice when we started the company. You wouldn’t want it to be a product. If it was, someone would eventually try to make you pay for it. That’s where all the source available licenses, relicensing controversies, etc. come from. Things there can look great for a while, lots of development, beatiful documentation, developer relations staff, etc., but often when people try to turn on monetization, the project dies. We were very acutely aware of these dynamics and wanted to ensure that the Julia project is strong and independent of the company. But that also meant that we had to figure out something else that could be a product for JuliaHub. It took us a while to get there, but at this point in time, JuliaHub is a company that builds (mostly cloud-based) modeling and simulation products and workflows for various industries (which happen to be written in Julia). I saw a tweet recently where somebody was looking at the JuliaHub employee count and saying that they didn’t see anything intersting enough in Julia to justify that many employees, but that’s the misconception here. JuliaHub’s mission is not to develop Julia. JuliaHub’s mission is to develop its products and because we the founders care about Julia, we happened to set it up in such a way that the only way to do that is for JuliaHub to spend millions of dollars a year supporting Julia and its ecosystem, so the company does that.
Relatedly, sometimes people come to us asking for specific feature improvements. Ironically, over the years, I’ve learned that the more generally useful a feature is, the less people want to pay for it. The reasoning goes “this isn’t specific to me - everybody needs it”. There’s this idea out there that somehow there’s a magic pot of money for things that “the community” needs and that “the community” should pay for. Neither of those things really exist. In particular, if you ask those same pepople to pay for general development you get “but i might not need whatever gets worked on”.
Finally, there’s NumFocus, which is where the donations go. The primary economic activity of NumFocus (with respect to Julia) is the organization of JuliaCon. In some years, there’s some money left over from JuliaCon in addition to donations. These finances are open and you can see them here: Julia Language · Expenses - Open Collective (not fully sure that’s everything, since I’m not involved in any of that, but I believe it’s most of it at least). As you can see there covers various community infrastructure, services, etc. and funds the small development grants, travel grants, etc. But you can also see that the total amount of yearly funding available through NumFocus is less than what even just JuliaHub spends on Julia maintenance and development by about two orders of magnitude. In fact, if NumFocus was actually paying for all Julia community infrastrucutre (CI in particular, which, other than free AWS credits, is mostly paid for by MIT or JuliaHub), there’d probably be no money left at all.
Over the years, various people have asked about setting up a pot of money (either through NumFocus or otherwise) to fund Julia development, but basically everytime it’s been dicussed there wasn’t actually any long-term funding commitments available, people just wanted specific features done for a small amount of money. You can’t fund professional software development that way (you can maybe fund some grants, but even there, who pays for admin, management, review, etc).
On feature prioritization
Given the above realities, there was some discussion of how the people that do get paid to work on Julia prioritize work. The answer is pretty simple. Other than the ordinary bug fixing and release polishing, it’s “whatever the organization mission happens to require at this moment” plus personal annoyances of the people that are doing the day-to-day work. But that’s it.
The people who are getting paid to work on Julia generally work on large Julia code bases, so both of those tend to skew towards the issues that come with scaling julia to such code bases (native code caching, TTFX, juliac, binding invalidation, etc.). If you’re not working on code bases of these sizes, some of that prioritization can seem odd or unnecessary, making it seem that obvious issues are being ignored in favor of some niche thing, but often times issues that seem like corner cases in regular become absolutely debilitating at scale (to give a non-JuliaHub example - see Relational’s work on GC improvements) and thus require prioritization.
What I mean to say here is that there is no master plan for what Julia will look like in two years. There’s various things we know are important and want to do, but there aren’t enough people to do them all, so which ones of them become important you only find out once you’ve gotten through the current round. I can tell you with some confidence what the core Julia developers will be working on the next 3-4 months, have some idea of what might be being worked on for the next year but have absolutely no idea what will be the highest priority next December.
What can be done
Now, however pessimistic all of that sounded, I will actually say that often times funding is not actually the bottleneck - it’s scoping the work, finding the people who can do it, and making it happen. Various people in the community regularly help contributors who need funding for a particular project get that done. If there’s a concrete plan of 1) What needs to be done, 2) Who will do it and 3) What will it cost, it’s very possible that someone might be able to find money for it, whether at a company that uses Julia, finding a way to write it into a grant proposal, getting the person an internship or temporary academic position or finding some other pot of money some where. Not infrequently, there’s something creative that can be done. This is not like a formal offering or anything, but definitely let people (Viral and I do a lot of this, as do Avik and Chris R - or just post on discourse) - if you have a concrete project/person that you want to get funded (or have excess funding that you’d like to spend on getting something useful done in Julia - we’ll happily take referrals for excess grant money or left over research budgets ;). Obviously I can’t promise anything here, but certainly if nobody knows about it, the chance it’ll happen is 0.
Conclusion
I think all that to say. We’ve been doing this for a while now. I think compared to a lot of other projects, we’ve actually done a very good job at funding basic development work. It’s not easy to sustain this kind of investment for more than a decade and it’s taken a lot of (often very underappreciated) work by many, many, people to keep the lights on. But let’s please direct our energy towards actionable plans and discussions (NOTE: Complaints that other people don’t have actionable plans is not itself an actionable plan 
). And finally, and most importantly, in the words of a particularly funky Zouk remix that happened to be playing as I was reading this thread
I’ll be there for you
(Like I’ve been there before)
I’ll be there for you
('Cause you’re there for me too)
