Yes, it is solving ODEs, but with some nuances. Symboltz uses MTK for at least:
observed variables eliminated from explicit algebraic equations (e.g. solving [D(x) ~ x, y ~ x^2] and easily getting sol[y]),
dummy derivative expansion of derivatives (e.g. solving [D(x) ~ y, y ~ D(z), z ~ x^2] and easily getting sol[z]),
reduction of higher-order ODEs to 1st order (e.g. solving [D(D(x)) ~ x] and easily getting sol[D(x)]),
generation of ODE code with e.g. common subexpression elimination (CSE) for optimal performance,
generation of analytical Jacobians, its sparsity pattern and optimal code for it,
initialization of the ODE from equations relating observed and/or derivative variables (i.e. not only direct unknown mappings).
Will these features continue to be availabile in the base MTK v11, for example?
I agree with that. The faster loading sounds great! SymBoltz does one important custom/domain-specific symbolic pass in particular, so making model transformations more modular/composable sounds meaningful to me. I think observed variable elimination, dummy derivative expansion, order reduction and SymBoltzâ custom pass could all be independent transformations that are applied sequentially to a raw model.
Thank you for being transparent and straightforward. I really hope we can get SymBoltz working with MTKv11 without any major complications.
@isaacsas - Thank you for putting together this note on behalf of the SciML steering council member.
Thanks everyone for the discussion and questions. Iâll respond to the questions in @GeorgeGkountourasâ message which I feel is the most comprehensive in articulating all the concerns.
My understanding from detailed discussions with @ChrisRackauckas is that it is only some of the passes (higher index DAEs and such that are discussed in the OP) that will be AGPL. I believe he has already responded to this.
I understand that license changes always raise many questions and concerns and my hope is that all of this makes it clear that we wish to be transparent and that we have put a great amount of thought into it. I do agree that the concern is not just large companies, but our concern is about building a robust ecosystem. Whether large companies or small are building and distributing products around MTK, we need the ecosystem to be robust. Our hope is that the AGPL will make the open source more robust in this situation. If more contributions come back, that is a win. If companies (small or large) want to sell MTK based products (that include the AGPL parts), we will be happy to work out a commercial license, which then makes it more commercially viable for us to maintain the codebase.
Iâd like to point out that MTK along with its AGPL parts is significantly more permisive than Mathematica Desktop which is commercial, and provides no redistribution rights. I do think you make a good point about small entities, and we would be open to work out a license that kicks in only after revenues build up.
In this case, it just means hosted in a separate JuliaHub github repo as opposed to a SciML repo. Everything is going to be in General just like now. There is absolutely no intention to make it SaaS only or any such thing. In fact we have put in a lot of work into things like JuliaC to make it all run in even more places.
This is something we still donât have a clear answer and thought process around, but will clarify by around January. The modeling and simulation use cases around artifacts/binaries/libraries are far more complex than traditional binaries that are built from C compilers, and there are wide ranging practices around licensing of these. At the very least there should be no uncertainty.
MTK and its associated SciML ecosystem needs much more than 2 full time maintainers. As with Julia itself, our goal is to build a robust open source community that is sustainable in the long run. A vibrant business on top of this open source base would align the interests of potential future owners of JuliaHub to continue in this way. If there are future license changes (or proprietary forks), the existing codebase up to that point will naturally continue to be around - those licenses cannot be changed.
Every single grant that has led to all of JuliaHub contributions has required a plan for sustainability and commercialization beyond the grant period. Getting open source projects funded through the grants process is extremely challenging, because no funder wants the grantee to keep coming back for more grants, and they do not want the projects to disappear. Academia is full of projects that die at the end of a grant. It is only because @alanedelman having a strong belief in what we do that Julia and SciML have turned out the way they have.
It is not our business to âgo afterâ any entities. We feel that most people will respect the license, especially if the communication around it is clear.
I am not a lawyer, but Iâll share my view here. If the software would be completely unusable without the AGPL parts, I personally do not see how separating it into different parts avoids the AGPL. These issues are not AGPL specific, and have been discussed adequately on the internet around the GPL itself.
MTK has grown over time into a rich, but also complex, ecosystem consisting of a compiler, libraries, solvers and external frontends. We took a look carefully and our expectation is that the broad user base and frontends like Catalyst (and almost all the commercial projects) are not going to be affected. In fact they will be able to use MTKBase and benefit from lesser code being loaded, much faster compilation times, and an overall better experience. Commercial users will benefit from a strong ecosystem (whether through more contributions to the AGPL parts or through licensing).
For those of us who are not experts on all the technical details, could Chris perhaps write a bit about what this whole discussion about âindex-1 DAEsâ is about and which kind of models structural_simplify would affect and how that will change after the split? Itâs all still a bit abstract for me personallyâŚ
Thank you, I read that. This explains the infrastructure part, which is clear. But I do not see any explanation of âhigh index DAEsâ or the kind of models weâre talking about? I am trying to judge whether this would affect the things I am working on. I think for many users of MTK the structural_simplify step has always been something that âJust Workedâ˘â
For us, it was sometimes very slow, and the speed could decrease significantly after an upgrade. So we are looking forward to an improved version of it, and as far as I can see, a different license has no impact on our work because our code is open source anyway.
Since GPL must be carried over to derivative works, this also means that if you decide to use the GPL part of MTK, then your users will also be bound by GPL. So it will affect you one way or another, unless you have no users.
Perhaps I do not fully understand the implications of the AGPL license. If I write an open-source model using MTK, and a company calls functions of this model using HTTP to test their controller, I would assume that their controller can remain closed-source. Is this understanding correct?
And only if they improve my model would they have to make these improvements open source?
I donât think a controller that uses your model would be considered a derivative work. But how often can you just take the model without having to modify anything for your application? Certainly never been the case for me. But letâs wait for some expert answers on this topic, I donât want to speculate
Your MTK model is not a derivative work of MTK (since you could use the MIT compiler (in some cases), or even simulate it manually on pen and paper). If you use MTK to compile the model, the resulting compiled artifact may or may not be considered a derivative work under AGPL (weâre waiting for clarification).
The above assumes you developed your model by copying equations from a book or running manual experiments. If you used AGPL MTK for auto-tuning, it is a derivative work.
If the HTTP is hosted internally in their server, they may be in the clear (theyâre users, not distributors). If you host the HTTP for them, they are violating the license (the A in AGPL).
Furthermore, the AGPL viral license cannot âjumpâ from software to a hardware controller (e.g. Ladder Logic). However, it can still affect software bundled with hardware (e.g. the controller is julia code running in a Raspberry Pi).
Standard disclaimer: not a lawyer, this may be entirely incorrect.
Iâm not sure i understand this answer but maybe I didnât understand the question , so maybe I should ask my own: Will MTK be available for free for academic work?
A big thank you to the SciML team for providing MTK, and for all the support Iâve received over the past year. I think we all want JuliaHub to thrive!
If MTKBase is more pleasant to use (reduced TTFX, better error messages, but slow), then I feel like paying for a better compiler is a reasonable request at megacorp, especially if itâs AGPL and we can give it a try before buying. So that part sounds good to me.
Isnât there / shouldnât there be a similar Dyad-language offering (DyadBase.jl)? Iâm still confused about the licensing situation. Can I use Dyad to create System objects, as an @mtkmodel replacement, without going through âcontact us to get a sales quoteâ?
From my side, âWeâre using MTK but I think we should pay X$ and spend 1-2 dev-months to move to Dyadâ is a very hard conversation, but âWeâre already using DyadBase; if we pay X$ it runs much faster and we get a GUI, basic support and some super-secret componentsâ is straight-forward. For some value of X, anyway.
As someone whoâs watched the MTK repo for a year, and would like to contribute more, information is lacking. Most PRs by core contributors have zero text, and no linked issues. New MTK versions seem to suddenly materialize out of thin air.
Iâm not complaining! I think itâs a fair way to do software development, and I admire the vision for MTK/Dyad. But if you really want to make it easier to get contributions, then âreduce complexityâ (which V11 seems to be working towards) and âmove discussion to githubâ would help a lot IMHO.
I have always felt conflicted about the long-term sustainability of âserious projectsâ being MIT licensed, so while Iâm apprehensive about the potential ripple effects of this change on different parts of the Julia ecosystem, Iâm also cautiously optimistic.
I have a few questions. Maybe these have straight-forward answers, or maybe these arenât the best questions, but I was unable to figure it out by myself, so I though these might be worth discussing publicly:
There is a risk that whatever process is set up to to help monetize one audience profile might be a bad fit for other situations due to a natural âimpedance mismatchâ. What makes most sense for a $BigCorp with substantial revenue might be very different compared with a small scrappy startup (and many diverse points on that landscape, like a small team running a consulting business, or somebody monetizing Julia education/training, etc). Assuming $BigCorp might be the primary target market for monetization (please correct me if Iâm wrong) what is the path forward for all these other kinds of commercial entities? There needs to be a clear & public answer to this question, and not just âreach out to us and weâll see [on a case by case basis]â because that would have a chilling effects on (commercial) Julia adoption at most places (and perversely, would basically cement that Julia is largely unused at any place besides $University or $BigCorp). Fixed per-seat pricing (agnostic of context) is overly simplistic since the ability to pay for commercial components + generate value evolves quite non-linearly and heavily depends on context.
Perhaps a somewhat naive question (Iâm excited about Dyad, but donât yet understand the nuts and bolts of the MTK ecosystem). Suppose a new entity starts offering commercial Julia components. Letâs assume the most wholesome scenario (growing the pie instead of competing for fractions of the pie) â that these offerings augment the landscape with complementary pieces that would be beneficial to all. How does the proposed licensing change interplay with the composability of different commercial components? What should the dependency architecture look like, license wise? Are there answers to this question that have been borne out by experience in other ecosystems? Keeping this as a âdesign requirementâ might help design policies that do not accidentally strangle the ecosystem.
As a process improvement detail, I wonder whether it might be useful for Project.toml to support listing a projectâs own license, and a list of acceptable licenses for dependencies, and for Pkg to flag any licensing conflicts when a user tries to install a new package. Automated verifiability would help prevent unintended infringement â both when installing packages, and also at distribution / downstream use. The community could then lean on the distribution infrastructure to help with license hygiene.
While we sweat the details, I also want to say that the thought process laid out in the blog post, along with the fact that weâre having this (challenging) conversation makes me feel positive about the stewardship; I hope this step in the ecosystemâs evolution proves to be a significant enabler from the longer-term perspective.
This is very clear: yes. First of all, the DAE passes in the v11 upgraded form are licensed as AGPL which is an OSI-approved open source license. So everything here is still open source. The only restriction of AGPL is that it is a copy-left license, which means that you must open source modifications under a similar license if distributed or used in a web service. So if you work on open source things (whether you share it online or not, this is a licensing thing) or donât distribute binaries and donât put things onto web services, then youâre totally fine.
Note that Julia itself already has GPL binaries (in SuiteSparse), so the same statement also applies to distributing Julia with sparse matrix support (only difference with AGPL vs GPL is the web services part, but how copy-left is written is the same). Julia itself has a build toggle for building without GPL (which effectively disables sparse matrix solving, and LinearSolve.jl has had preferences to support this for a long time).
So basically a tl;dr is: if you donât already have to deal with building GPL-free Julia and you donât build commercial web services, then you can ignore this change (or maybe youâre already violating GPL licenses ). My guess is that the vast majority of users are in the camp where they already donât worry about GPL dependencies and use the standard Julia binary.
Yes, if you open source your code to a GPL-compatible license (such as MIT) then a change of a dependency to GPL has no impact.
The tl;dr is basically only uses cases that use acausal modeling. Expanded connections have a tendency to easily create high-index DAEs, so acausal modeling and the state selection, index reduction, tearing, etc. all go together. But they donât necessarily.
If you write a DAE directly it is possible get a high index DAE (Cartesian pendulum is an example) but it is somewhat rare. A way to eyeball it is that every algebraic equation must be solving for a unique algebraic variable: if you have an algebraic equation with only differential variables, then it is high index.
structural_simplify (which was renamed about a year ago into mtkcompile but I think most people have deprecation warnings turned off) will still âJust Workâ˘â in ModelingToolkit v11. This is because ModelingToolkit v11 will simply have dependencies on external pass libraries and make the default compilation pipeline be the version that is the best based on open source tools available in General, and since StateSelection.jl, ModelingToolkitTearing.jl, and SymbolicCompilerPasses.jl are open source libraries in General, the top level MTK will simply be setup to make mtkcompile similar to before. In fact, SymbolicCompilerPasses.jl is something that didnât even exist before, and itâs adding things like better array-optimized code generation and even looking into things like Reactant.jl integration, so it should âJust Workâ˘â even more than before.
Additionally I will say that our state selection passes were previously underdeveloped and cases like Multibody.jl would expose this issue. The next thing happening after this change is to make some major improvements to StateSelection.jl so that on this larger set of models it has the ability to âJust Workâ˘â
This is indeed another case being worked on. The MTK v11 release will have some immediate improvements just because the Symbolics underpinnings are faster and because precompilation now works on the whole system, but then weâre also doing things like specializing arrays better in the tearing passes and code generation to hopefully make array differential equations take O(1), so something like MethodOfLines.jl should go from O(n^4) scaling on 2D PDEs with n spatial discretization points in each axis to O(1). This will require some updates to MethodOfLines.jl after the tearing updates, so itâs not too soon, but the changes to the compilation pipeline are designed to now make this possible.
Yes, we are going to have to improve the messaging here, @GeorgeGkountouras is one case that comes to mind immediately. We will need to come back with a better answer to this.
Julia itself has GPL dependencies, so we can look at how that design has played out over time as an example. What Julia offers is a no GPL flag in the Makefile right here:
So if youâre distributing Julia in a way that would violate GPL licensing you need to flip this flag and build GPL-free binaries. This has some downstream effects, such as for example SparseArrays.jl will no longer load CHOLMOD, SPQR, and UMFPACK if the flag is flipped to false.
This means that if A is a sparse matrix, lu(A), cholesky(A), and qr(A) will all fail using the normal LinearAlgebra.jl / SparseArrays.jl standard libraries in this no-GPL mode. Some downstream libraries like LinearSolve.jl then also hook into this to change behavior depending on the Julia-level GPL flag:
In a similar vein, MTK vs MTKBase is designed to make MTKBase a GPL-free build, while MTK then adds (A)GPL dependencies and updates the defaults on things like mtkcompile to automatically use the extensions. I think this is a design that should be copied as much as possible: if people really want to avoid GPL dependencies, especially if they arenât using the feature that is GPL, making it easy for people to opt-out is helpful, especially if the GPL dependency is a targeted portion of the libraryâs larger feature set.
I will also just note again that if this GPL-free building isnât a process you have already been doing and you donât build commercial web services, then this update to having some AGPL dependencies does not effect your workflow.
@ChrisRackauckas can you clarify here if your comments about AGPL use implications are statements on behalf of JuliaHub or your personal opinion? They do not reflect any consensus among the SciML steering council as far as I am aware.
In the absence of an explicit statement by JuliaHub about how they will / intend to apply the AGPL in different use cases, I would again advise everyone here to consult a lawyer on this topic to understand how it applies to them.