Is PyJulia org, e.g. juliacall (or pyjulia) Python packages maintained?

Nobody disagrees that we need to continue making the interoperability tooling the best that it can be. It’s more of a question of who or how. Let me first dispel a few myths to put us on the right path and then describe what are the concrete steps that can be made:

JuliaHub is a cloud computing company which focuses on building and supporting software for modeling and simulation, such as Pumas (pharmacometrics, developed by a separate company Pumas-AI), JuliaSim (general industrial), and Cedar (EDA, circuit simulation). While many major contributors to many open source projects have worked or are affiliated with JuliaHub, it as a company has to earn revenue and the open source contributions are not the business model. There are some teams focused more on open source than others, and those require grant funds to be sustained (to be described below).

Another source of a lot of contributors is the MIT Julia Lab, which for example is what funded Takafumi during his creation and maintenance of PyJulia. Most of the grants are focused on scientific projects, these days many are scientific machine learning or HPC related, or larger scientific collaborations like CESMIX. In that sense, there really isn’t anything special about the Julia Lab other than the fact that it has cultivated a culture where open source contributions are valued, and those who are very active in the open source communities are prioritized over “traditional academics”.

I am adamant that this is the right thing for academia in general. I would argue that open source scientific software developers are some of the most impactful and influential scientists in our current system. However, this bent of the lab is simply a cultural choice generated by Alan’s leadership and any other academic lab could also hire contributors. Indeed, I think there’s a good case to be made for doing so: instead of hiring a good neuroscientist, hire a neuroscientist who also happens to be working on the compiler for the language you’re using and can fix any bugs your research program is running into. For reference, Taka (pyjulia’s maintaine, developer of Transducers.jl and many threading capabilities in Julia) was a neuroscientist, and so was Valentin (who builds/maintains CUDA.jl, Enzyme.jl, and a lot of the Julia compiler), so this is only half a hypothetical. It’s not the average way to do science, many labs build one-off tools with workarounds rather than contributing to widely used ecosystems to make their experiments work, but I think there’s a good history to show that the latter has a more cumulative effect and in the end is a better choice!

So what can you do? A lot, since all grants are “open source” grants!

So okay, we’ve established that there’s nothing special. There’s JuliaHub and Julia Lab that do have a good number of contributors, but they are companies and scientific labs like any other, but just happen to also put the work in to value the time and have some (not all, and not the majority) individuals do open source contributions.

With that in mind, how could you do the same thing if you wanted to? Well for starters, there is nothing special about most of the funds to do this. Most of the grants are not “open source development” grants, those are very few and far between and could never sustain a community. Most of the funds are simply normal grants, climate models, pharmacology, etc. whatever the lab is doing. People who are “funded for open source” are actually funded for projects like “improved performance of climate models”, where you hire (or have a PhD student) a HPC/GPU specialist who happens to be a person who maintains CUDA.jl or part of the LLVM stack. This means that every grant is an “open source development” grant if you look at it the right way, and with every grant you can think of adding “Aim 3: Make it Scale” and have someone dedicated to the software sustaintability and scaling aspect.

But if I don’t have “extra” cap space and want to start writing, what are good directions?

So if you don’t have an established lab with a lot of folks in there to just happen to have some hires that could be understood to part time maintain libraries like pyjulia, then you need to find grant money. The good thing is, it’s not impossible and pretty much anyone can start doing this. I started writing grant after grant in my first your after PhD, so basically anyone with a PhD and an academic position that grants PI privledges can do it (that may seem to exclude postdocs by default, but in most institutions you can petition for PI privledges on a per-grant basis, so you can use this option and it can be used to bolster your academic CV).

So then you want to start writing, what grant? Here’s a few that I think have been the most helpful.

  1. NSF CSSI. https://new.nsf.gov/funding/opportunities/cyberinfrastructure-sustained-scientific . NSF grants are quite a PITA but these last awhile (3 years IIRC) and can have a focus on cyberinfrastructure. These grants sustained many projects like JuMP, automatic differentiation work, and more.
  2. NumFOCUS periodically has the Small Development Grants (SDG) . Small Development Grants - NumFOCUS These are usually on the order of $10k, so it is usually only a side project unless you’re using it to help give someone early career or in a developing country some time for independent focus work.
  3. The Chan-Zuckerburg Initiative has grants to software development (Science, Education, Community, and Other Grants - CZI). These are much smaller, basically 1 person max for 2 years (Science, Education, Community, and Other Grants - CZI). Notably, the a lot of the precompilation improvements were funded in part through a CZI helping fund a lab manager for Tim Holy’s lab to free up his hands (SciML Receives Chan Zuckerberg Institute Funding: Spatial SSAs, Identifiability, and Compile Times).
  4. NASA has the ROSES program that has some open source funding. For example, if someone has a current ROSES grant, you can submit a supplement by March 29th as part of Amendment 33: Supplement for Open-Source Science Final Text - NASA Science for open source development! Larger opportunities are every 3 years NASA Open Science Funding Opportunities - NASA Science.
  5. The Simons Foundation periodically has grants for open source Funding Opportunities.

And there’s many more I haven’t listed here. But again, usually domain-based funding sources (climate modeling, ecology, pharmacology, etc.) have much more funding, and so finding a way to integrate open source work into a “normal” grant is usually much more productive than looking for “open source grants”.

Conclusion: There is no Silver Bullet

If there was someone who was “allocating resources” of the Julia community, then it would probably be me. But as I have hopefully showed, there’s no silver bullet or wand of magic that can be waved to get even two developers to suddenly show up. Open source is actually built by volunteers, and any funds that exist in the system are also generated by efforts on volunteer time!

But, I hope that is actually empowering. While it may seem completely daunting, with internal thoughts of “how do we get a better Python to Julia bridge, what company can get $10mil to make this work?”, hopefully this shows that the community has never had any resources remotely close to this. Everything is built by having small contributions add up. It’s hiring a postdoc who is active in the community and having them spend 1/3 of their time maintaining the compiler, it’s putting the time in to get a $30k grant and fully funding someone in a developing country for a year, it’s collaborating with another lab on a grant to get them a lab manager so that they can do more contributions over the next year. This community wasn’t made on multi-million dollar contributions but the collective work of many individuals helping with whatever they could.

Call To Action

If you want to get something in motion, do feel free to reach out. I through the Julia Lab or JuliaHub do a ton of these (for reference, I probably at least write a letter of collaboration every other week), but each takes a lot of time and effort and so it would be great to have more people in the community to take on leadership roles. In the immediate term, anyone with a NASA ROSES grant would be in a great position to lead an open source supplement as mentioned above. That’s only $50k ($25k if the academic institution takes half as overhead :sweat_smile:) but it can get someone started and is due end of March. Then the next NumFOCUS SDG CfP should open in about a week, and if someone is interested I could help you get this written (SciML’s SDGs are already known, so this would likely need to be something JuliaLang, but that of course makes more sense).

And if you have funds but don’t know who to hire, or want to add this information to the grant but don’t know who you can put as a potential hire, please get in touch. I could name more than one person off the top of my head who might be interested in such a role but don’t want to commit them publically (especially without knowing caveats like if they’d have to move :sweat_smile:)

Meme-based tl;dr:

(now seriously though, off to finishing the next grant :sweat_smile:)

18 Likes