ModelingToolkit V11 Library Split and Licensing, Community Feedback Requested

My understanding is that the AGPL libraries will be hosted on Github, but under the JuliaHub organization. One of the libraries is GitHub - JuliaComputing/StateSelection.jl: Common algorithmic core for state selection in MTK, DAECompiler, JuliaSimCompiler, which is already on Github.

But hopefully someone from JuliaHub can confirm this.

1 Like

Yes, it is solving ODEs, but with some nuances. Symboltz uses MTK for at least:

  • observed variables eliminated from explicit algebraic equations (e.g. solving [D(x) ~ x, y ~ x^2] and easily getting sol[y]),
  • dummy derivative expansion of derivatives (e.g. solving [D(x) ~ y, y ~ D(z), z ~ x^2] and easily getting sol[z]),
  • reduction of higher-order ODEs to 1st order (e.g. solving [D(D(x)) ~ x] and easily getting sol[D(x)]),
  • generation of ODE code with e.g. common subexpression elimination (CSE) for optimal performance,
  • generation of analytical Jacobians, its sparsity pattern and optimal code for it,
  • initialization of the ODE from equations relating observed and/or derivative variables (i.e. not only direct unknown mappings).

Will these features continue to be availabile in the base MTK v11, for example?

I agree with that. The faster loading sounds great! SymBoltz does one important custom/domain-specific symbolic pass in particular, so making model transformations more modular/composable sounds meaningful to me. I think observed variable elimination, dummy derivative expansion, order reduction and SymBoltz’ custom pass could all be independent transformations that are applied sequentially to a raw model.

Thank you for being transparent and straightforward. I really hope we can get SymBoltz working with MTKv11 without any major complications.

3 Likes

@isaacsas - Thank you for putting together this note on behalf of the SciML steering council member.

Thanks everyone for the discussion and questions. I’ll respond to the questions in @GeorgeGkountouras’ message which I feel is the most comprehensive in articulating all the concerns.

My understanding from detailed discussions with @ChrisRackauckas is that it is only some of the passes (higher index DAEs and such that are discussed in the OP) that will be AGPL. I believe he has already responded to this.

I understand that license changes always raise many questions and concerns and my hope is that all of this makes it clear that we wish to be transparent and that we have put a great amount of thought into it. I do agree that the concern is not just large companies, but our concern is about building a robust ecosystem. Whether large companies or small are building and distributing products around MTK, we need the ecosystem to be robust. Our hope is that the AGPL will make the open source more robust in this situation. If more contributions come back, that is a win. If companies (small or large) want to sell MTK based products (that include the AGPL parts), we will be happy to work out a commercial license, which then makes it more commercially viable for us to maintain the codebase.

I’d like to point out that MTK along with its AGPL parts is significantly more permisive than Mathematica Desktop which is commercial, and provides no redistribution rights. I do think you make a good point about small entities, and we would be open to work out a license that kicks in only after revenues build up.

In this case, it just means hosted in a separate JuliaHub github repo as opposed to a SciML repo. Everything is going to be in General just like now. There is absolutely no intention to make it SaaS only or any such thing. In fact we have put in a lot of work into things like JuliaC to make it all run in even more places.

This is something we still don’t have a clear answer and thought process around, but will clarify by around January. The modeling and simulation use cases around artifacts/binaries/libraries are far more complex than traditional binaries that are built from C compilers, and there are wide ranging practices around licensing of these. At the very least there should be no uncertainty.

MTK and its associated SciML ecosystem needs much more than 2 full time maintainers. As with Julia itself, our goal is to build a robust open source community that is sustainable in the long run. A vibrant business on top of this open source base would align the interests of potential future owners of JuliaHub to continue in this way. If there are future license changes (or proprietary forks), the existing codebase up to that point will naturally continue to be around - those licenses cannot be changed.

Every single grant that has led to all of JuliaHub contributions has required a plan for sustainability and commercialization beyond the grant period. Getting open source projects funded through the grants process is extremely challenging, because no funder wants the grantee to keep coming back for more grants, and they do not want the projects to disappear. Academia is full of projects that die at the end of a grant. It is only because @alanedelman having a strong belief in what we do that Julia and SciML have turned out the way they have.

It is not our business to “go after” any entities. We feel that most people will respect the license, especially if the communication around it is clear.

I am not a lawyer, but I’ll share my view here. If the software would be completely unusable without the AGPL parts, I personally do not see how separating it into different parts avoids the AGPL. These issues are not AGPL specific, and have been discussed adequately on the internet around the GPL itself.

MTK has grown over time into a rich, but also complex, ecosystem consisting of a compiler, libraries, solvers and external frontends. We took a look carefully and our expectation is that the broad user base and frontends like Catalyst (and almost all the commercial projects) are not going to be affected. In fact they will be able to use MTKBase and benefit from lesser code being loaded, much faster compilation times, and an overall better experience. Commercial users will benefit from a strong ecosystem (whether through more contributions to the AGPL parts or through licensing).

-viral

11 Likes

For those of us who are not experts on all the technical details, could Chris perhaps write a bit about what this whole discussion about “index-1 DAEs” is about and which kind of models structural_simplify would affect and how that will change after the split? It’s all still a bit abstract for me personally…

2 Likes

I believe this draft blog post explains that, and if it is still not clear, it would be good to update that blog post.

Thank you, I read that. This explains the infrastructure part, which is clear. But I do not see any explanation of “high index DAEs” or the kind of models we’re talking about? I am trying to judge whether this would affect the things I am working on. I think for many users of MTK the structural_simplify step has always been something that “Just Worked™”

1 Like

For us, it was sometimes very slow, and the speed could decrease significantly after an upgrade. So we are looking forward to an improved version of it, and as far as I can see, a different license has no impact on our work because our code is open source anyway.

3 Likes

Since GPL must be carried over to derivative works, this also means that if you decide to use the GPL part of MTK, then your users will also be bound by GPL. So it will affect you one way or another, unless you have no users.

1 Like

Perhaps I do not fully understand the implications of the AGPL license. If I write an open-source model using MTK, and a company calls functions of this model using HTTP to test their controller, I would assume that their controller can remain closed-source. Is this understanding correct?

And only if they improve my model would they have to make these improvements open source?

1 Like

I don’t think a controller that uses your model would be considered a derivative work. But how often can you just take the model without having to modify anything for your application? Certainly never been the case for me. But let’s wait for some expert answers on this topic, I don’t want to speculate :slight_smile:

Well, the A in AGPL is essentially for the case of network services: see Why the GNU Affero GPL - GNU Project - Free Software Foundation . We indeed did choose the AGPL over GPL for this reason.

(I am not a lawyer, though…)

Your MTK model is not a derivative work of MTK (since you could use the MIT compiler (in some cases), or even simulate it manually on pen and paper). If you use MTK to compile the model, the resulting compiled artifact may or may not be considered a derivative work under AGPL (we’re waiting for clarification).

The above assumes you developed your model by copying equations from a book or running manual experiments. If you used AGPL MTK for auto-tuning, it is a derivative work.

If the HTTP is hosted internally in their server, they may be in the clear (they’re users, not distributors). If you host the HTTP for them, they are violating the license (the A in AGPL).

Furthermore, the AGPL viral license cannot “jump” from software to a hardware controller (e.g. Ladder Logic). However, it can still affect software bundled with hardware (e.g. the controller is julia code running in a Raspberry Pi).

Standard disclaimer: not a lawyer, this may be entirely incorrect.

TL;DR: It depends.

I’m not sure i understand this answer but maybe I didn’t understand the question :sweat_smile:, so maybe I should ask my own: Will MTK be available for free for academic work?

A big thank you to the SciML team for providing MTK, and for all the support I’ve received over the past year. I think we all want JuliaHub to thrive!

If MTKBase is more pleasant to use (reduced TTFX, better error messages, but slow), then I feel like paying for a better compiler is a reasonable request at megacorp, especially if it’s AGPL and we can give it a try before buying. So that part sounds good to me.

Isn’t there / shouldn’t there be a similar Dyad-language offering (DyadBase.jl)? I’m still confused about the licensing situation. Can I use Dyad to create System objects, as an @mtkmodel replacement, without going through “contact us to get a sales quote”?

From my side, “We’re using MTK but I think we should pay X$ and spend 1-2 dev-months to move to Dyad” is a very hard conversation, but “We’re already using DyadBase; if we pay X$ it runs much faster and we get a GUI, basic support and some super-secret components” is straight-forward. For some value of X, anyway.

As someone who’s watched the MTK repo for a year, and would like to contribute more, information is lacking. Most PRs by core contributors have zero text, and no linked issues. New MTK versions seem to suddenly materialize out of thin air.

I’m not complaining! I think it’s a fair way to do software development, and I admire the vision for MTK/Dyad. But if you really want to make it easier to get contributions, then “reduce complexity” (which V11 seems to be working towards) and “move discussion to github” would help a lot IMHO.

6 Likes

So much is speculative.
Are we able to get a lawyer in here?
Preferably a lawyer involved in this?

1 Like

I have always felt conflicted about the long-term sustainability of “serious projects” being MIT licensed, so while I’m apprehensive about the potential ripple effects of this change on different parts of the Julia ecosystem, I’m also cautiously optimistic.

I have a few questions. Maybe these have straight-forward answers, or maybe these aren’t the best questions, but I was unable to figure it out by myself, so I though these might be worth discussing publicly:

  1. There is a risk that whatever process is set up to to help monetize one audience profile might be a bad fit for other situations due to a natural “impedance mismatch”. What makes most sense for a $BigCorp with substantial revenue might be very different compared with a small scrappy startup (and many diverse points on that landscape, like a small team running a consulting business, or somebody monetizing Julia education/training, etc). Assuming $BigCorp might be the primary target market for monetization (please correct me if I’m wrong) what is the path forward for all these other kinds of commercial entities? There needs to be a clear & public answer to this question, and not just “reach out to us and we’ll see [on a case by case basis]” because that would have a chilling effects on (commercial) Julia adoption at most places (and perversely, would basically cement that Julia is largely unused at any place besides $University or $BigCorp). Fixed per-seat pricing (agnostic of context) is overly simplistic since the ability to pay for commercial components + generate value evolves quite non-linearly and heavily depends on context.

  2. Perhaps a somewhat naive question (I’m excited about Dyad, but don’t yet understand the nuts and bolts of the MTK ecosystem). Suppose a new entity starts offering commercial Julia components. Let’s assume the most wholesome scenario (growing the pie instead of competing for fractions of the pie) – that these offerings augment the landscape with complementary pieces that would be beneficial to all. How does the proposed licensing change interplay with the composability of different commercial components? What should the dependency architecture look like, license wise? Are there answers to this question that have been borne out by experience in other ecosystems? Keeping this as a “design requirement” might help design policies that do not accidentally strangle the ecosystem.

  3. As a process improvement detail, I wonder whether it might be useful for Project.toml to support listing a project’s own license, and a list of acceptable licenses for dependencies, and for Pkg to flag any licensing conflicts when a user tries to install a new package. Automated verifiability would help prevent unintended infringement – both when installing packages, and also at distribution / downstream use. The community could then lean on the distribution infrastructure to help with license hygiene.


While we sweat the details, I also want to say that the thought process laid out in the blog post, along with the fact that we’re having this (challenging) conversation makes me feel positive about the stewardship; I hope this step in the ecosystem’s evolution proves to be a significant enabler from the longer-term perspective.

6 Likes

This is very clear: yes. First of all, the DAE passes in the v11 upgraded form are licensed as AGPL which is an OSI-approved open source license. So everything here is still open source. The only restriction of AGPL is that it is a copy-left license, which means that you must open source modifications under a similar license if distributed or used in a web service. So if you work on open source things (whether you share it online or not, this is a licensing thing) or don’t distribute binaries and don’t put things onto web services, then you’re totally fine.

Note that Julia itself already has GPL binaries (in SuiteSparse), so the same statement also applies to distributing Julia with sparse matrix support (only difference with AGPL vs GPL is the web services part, but how copy-left is written is the same). Julia itself has a build toggle for building without GPL (which effectively disables sparse matrix solving, and LinearSolve.jl has had preferences to support this for a long time).

So basically a tl;dr is: if you don’t already have to deal with building GPL-free Julia and you don’t build commercial web services, then you can ignore this change (or maybe you’re already violating GPL licenses :sweat_smile:). My guess is that the vast majority of users are in the camp where they already don’t worry about GPL dependencies and use the standard Julia binary.

Yes, if you open source your code to a GPL-compatible license (such as MIT) then a change of a dependency to GPL has no impact.

6 Likes

The tl;dr is basically only uses cases that use acausal modeling. Expanded connections have a tendency to easily create high-index DAEs, so acausal modeling and the state selection, index reduction, tearing, etc. all go together. But they don’t necessarily.

If you write a DAE directly it is possible get a high index DAE (Cartesian pendulum is an example) but it is somewhat rare. A way to eyeball it is that every algebraic equation must be solving for a unique algebraic variable: if you have an algebraic equation with only differential variables, then it is high index.

structural_simplify (which was renamed about a year ago into mtkcompile :sweat_smile: but I think most people have deprecation warnings turned off) will still “Just Work™” in ModelingToolkit v11. This is because ModelingToolkit v11 will simply have dependencies on external pass libraries and make the default compilation pipeline be the version that is the best based on open source tools available in General, and since StateSelection.jl, ModelingToolkitTearing.jl, and SymbolicCompilerPasses.jl are open source libraries in General, the top level MTK will simply be setup to make mtkcompile similar to before. In fact, SymbolicCompilerPasses.jl is something that didn’t even exist before, and it’s adding things like better array-optimized code generation and even looking into things like Reactant.jl integration, so it should “Just Work™” even more than before.

Additionally I will say that our state selection passes were previously underdeveloped and cases like Multibody.jl would expose this issue. The next thing happening after this change is to make some major improvements to StateSelection.jl so that on this larger set of models it has the ability to “Just Work™”

This is indeed another case being worked on. The MTK v11 release will have some immediate improvements just because the Symbolics underpinnings are faster and because precompilation now works on the whole system, but then we’re also doing things like specializing arrays better in the tearing passes and code generation to hopefully make array differential equations take O(1), so something like MethodOfLines.jl should go from O(n^4) scaling on 2D PDEs with n spatial discretization points in each axis to O(1). This will require some updates to MethodOfLines.jl after the tearing updates, so it’s not too soon, but the changes to the compilation pipeline are designed to now make this possible.

4 Likes

Yes, we are going to have to improve the messaging here, @GeorgeGkountouras is one case that comes to mind immediately. We will need to come back with a better answer to this.

Julia itself has GPL dependencies, so we can look at how that design has played out over time as an example. What Julia offers is a no GPL flag in the Makefile right here:

So if you’re distributing Julia in a way that would violate GPL licensing you need to flip this flag and build GPL-free binaries. This has some downstream effects, such as for example SparseArrays.jl will no longer load CHOLMOD, SPQR, and UMFPACK if the flag is flipped to false.

This means that if A is a sparse matrix, lu(A), cholesky(A), and qr(A) will all fail using the normal LinearAlgebra.jl / SparseArrays.jl standard libraries in this no-GPL mode. Some downstream libraries like LinearSolve.jl then also hook into this to change behavior depending on the Julia-level GPL flag:

In a similar vein, MTK vs MTKBase is designed to make MTKBase a GPL-free build, while MTK then adds (A)GPL dependencies and updates the defaults on things like mtkcompile to automatically use the extensions. I think this is a design that should be copied as much as possible: if people really want to avoid GPL dependencies, especially if they aren’t using the feature that is GPL, making it easy for people to opt-out is helpful, especially if the GPL dependency is a targeted portion of the library’s larger feature set.

I will also just note again that if this GPL-free building isn’t a process you have already been doing and you don’t build commercial web services, then this update to having some AGPL dependencies does not effect your workflow.

6 Likes

@ChrisRackauckas can you clarify here if your comments about AGPL use implications are statements on behalf of JuliaHub or your personal opinion? They do not reflect any consensus among the SciML steering council as far as I am aware.

In the absence of an explicit statement by JuliaHub about how they will / intend to apply the AGPL in different use cases, I would again advise everyone here to consult a lawyer on this topic to understand how it applies to them.

2 Likes