[ANN] Symbolics.jl: A Modern Computer Algebra System for a Modern Language

[ChrisRackauckas]

Hi @certik! Of course I know who you are , though I know you and @isuruf are probably sad to read about this.

@certik

I am the original author of SymPy and SymEngine. Python is indeed too slow, so it is hard for SymPy to compete in terms of speed, but it is still very useful to a lot of people and there is a large community around it. The SymEngine project is written in C++ and it is very fast (a few years ago it has been generally faster than any other computer algebra system that we tried, including Mathematica, Maple, GiNaC, Sage, SymPy, Maxima, etc.). @ChrisRackauckas have you run some benchmarks against SymEngine?

SymEngine is pretty fast. The release doesn’t mention performance qualms with SymEngine, only SymPy. The only performance qualms I have with SymEngine is the fact that specialized tools like FLINT can beat it in their specific regimes and we want to specialize the calls for that, but that’s relatively minor. It’s more features and the interaction with Julia’s expressions that came into play here.

@certik

Indeed, SymEngine is missing features compared to SymPy. We were hoping you would be interested in working with us to contribute them, especially since you were already using it. Because if you do, then not just Julia, but every language including Python and SymPy would benefit.

It is easier for the Julia community to contribute if the code is written in Julia, just like it is easier for the Python community to contribute to SymPy since the code is in Python. That is true and growing the developer community is important.

We did contribute to the SymEngine community for years (community more than code), and it was the preferred symbolic underbelly of SciML through ParameterizedFunctions.jl for that a very long time. But there were some issues. One is that we just kept growing, and as our projects kept growing they needed more customization. Our symbols need arbitrary metadata to store information about types, flowness, units, etc. for doing context pushing in ModelingToolkit (I’m sure adding a boolean flow to the symbol type in SymEngine’s C++ core would be a non-starter, but this kind of thing was crucial for us). I don’t know other CAS’s which allow arbitrary customization of carried metadata from Julia (which would also be an issue with Axiom, Maxima, etc.). We also need array symbolics, i.e. not arrays of symbolic variables but variables which represent arrays and act with non-commutative rules. So SymEngine is still probably a better CAS right now, but it isn’t a better CAS for ModelingToolkit.jl, which has been a big driver to this project.

I agree we could in theory contribute back to a C++ core, but the time cost of doing that in a lot of cases is rather high. It was easy to prototype some fixes in ModelingToolkit and then plan to add them back to SymEngine, but quickly we learned that no one ever got the energy. That’s to be expected though: there really isn’t a feature benefit to our projects to spend that time. So over time there was a drift until MTK essentially had a CAS, in which case @shashi stepped in and wrote a similar-in-style rewriting system with similar specializations, as a launching pad to start playing around with new ideas in the area. We integrated tools from AbstractAlgebra.jl (i.e. parts of the Nemo CAS), created a metadata system, parallelized the internal operations with task-based parallelism, created a very extravagant extension to lambdify known as build_function, and… it would take a huge amount of time to do some of this in C++!

But maybe what we learn can be helpful for SymEngine! I think there will be some healthy cross-pollination between the projects. I think a joint dev conference day where we just share ideas we found useful could be fun.

@certik

But I believe it is important to write libraries that can be used from any language. C++ is one of the options for such a common language. Last time I tried Julia’s ahead-of-time compilation to static executables was not yet as easy as with C++ or Fortran:

Right now our focus is just on making the experience for Julia users as good as possible. I agree that multi-language usage is a noble intention, but it’s not a priority for us. We instead need something that does not compromise to act seamlessly with Julia, and there were always a few little hiccups that were hard to solve by something that wasn’t made to exactly match Julia’s semantics. You’ll see why this “exactly match Julia” such a big deal for us with ModelingToolkit.jl when its paper comes out in 2 days (note the paper is not about CAS’s at all, but it’s built on Symbolics.jl and some important features).