I’m researching the potential of software ecosystems for process modelling in the life-sciences. Its currently a wasteland of siloed frameworks and ancient Fortran repos…
Coordination of component interfaces at the level of a package ecosystem is completely absent. It strikes me that Julias github organisations and XXBase packages are unique and productive responses to this problem.
Does anyone know of any published research into this, or theoretical frameworks to describe what is happening here? How do I convey the power of these solution to other life scientists?
The whole approach is pretty new and should merit some academic attention. I am unaware of any publications but the most effective way of demonstrating its power is to effectively build something with it that overpowers an older alproach/system.
I don’t think the solution per se is complex enough to warrant an academic paper. Defining interfaces separately from the implementation is a very standard practice in programming, this is just what emerged as a way of doing it in Julia.
Also, note that while the approach has benefits, it also carries some overhead (packages and their authors need to coordinate). So I think it is best to just let people know about it, and they can use it as a solution when the need emerges.
Essentially it describes the architecture of DifferentialEquations.jl (Base library + dispatch-based API) and what features are directly derived from this architecture, and finally what problems are solved via these features.
I wonder if the lack of good optional dependency supports in the language played some additional role here. That is to say, the lack of optional dependency support actually forces the community to evolve in such direction and not entirely the response to solving the problem at hand. Of course, having able to write generic implementation (among other great features) is a prerequisite for this.
Great, that’s what I was looking for. Package-free APIs!
That’s pretty much what I’m trying to describe - that we can make agreements at a larger scale and encode them in a shared interfaces so that everything is interoperable, but it can still be flexible for your own needs. Theres lot of other great stuff in there too, thanks for writing it up.
I’m also really interested in the social consequences of these new structures - changes to the scale of collaboration from single repo frameworks to open interfaces shared with whoever cares about them, and the shift in how we discuss and organise these things. I guess a study of the social consequences will probably have to wait another 10 years, until everyone doing it
Edit: I see you address some of the social consequences too, like the academic environment and how much easier it is to contribute small personal package than to huge centralised repos. Great paper.
Ha that might have actually been a good thing, maybe you should hold off on that fix for a few years… But in seriousness, I think the package-free api has a broader usefullness as a way of setting clear communally agreed standards., and would probably have still happened because multiple dispatch makes it so easy.
For context I’m mostly comparing julia to Fortran: no one has any dependencies at all!!
