[ANN] Trixi.jl v0.3: SciML integration and a new modular approach for easy extension

Trixi.jl is a tree-based numerical simulation framework for hyperbolic PDEs in Julia. We have rewritten Trixi.jl completely to make it more modular and Julian - we made the transition from “Trixi as a monolith” to “Trixi as a library” (search for PRs/commits/issues mentioning “Taal” in our repo). In particular, we got rid of TOML parameter files to set up simulations and replaced them by pure Julia code, an approach many people have asked for, e.g., @stillyslalom, @rveltz, and @antoine-levitt in the first announcement of Trixi and @PetrKryslUCSD on GitHub. You can find an example at examples/2d/elixir_advection_basic.jl.

On top of that, we made Trixi more modular to make better use of other Julia packages. In particular, you can now use all of the SciML ecosystem for time stepping. Thanks again to @ChrisRackauckas for reviewing our PRs to OrdinaryDiffEq.jl to make this work!

As a side effect, these changes allowed us to reduce the latency of Trixi significantly and to make further improvements, e.g., optimization of the AMR part and simplification of our multi-physics approach. Other changes include the addition of a 1D DGSEM solver, positivity-preserving limiters, and proper support for weakly-enforced boundary conditions.

Trixi’s features include (differences to v0.1/v0.2 highlighted):

  • Hierarchical quadtree/octree grid with adaptive mesh refinement
  • Native support for 1D, 2D, and 3D simulations
  • High-order accuracy in space in time
  • Nodal discontinuous Galerkin spectral element methods
    • Kinetic energy-preserving and entropy-stable split forms
    • Entropy-stable shock capturing
    • Positivity-preserving limiting
  • Compatible with the SciML ecosystem for ordinary differential equations
  • Square/cubic domains with periodic and weakly-enforced boundary conditions
  • Multiple governing equations:
    • Compressible Euler equations
    • Magnetohydrodynamics equations
    • Hyperbolic diffusion equations for elliptic problems
    • Scalar advection
  • Multi-physics simulations
  • Shared-memory parallelization via multithreading
  • Visualization of results with Julia-only tools (Trixi2Img)
    or ParaView/VisIt (Trixi2Vtk)

Xref: First Trixi.jl announcement

33 Likes

Great package! I plan to migrate a domain specific PDE solver from C++ to Julia. Trixi set a good example for doing it. I will learn it carefully.

For a library, it may be better to process the input in pure Julia. But for an application, it is not bad to use .toml or .yaml or .json or any specialized text files to set parameters. Users of an application may not even know how to code!

3 Likes

This is wonderful, thank you. I will try to write a small example using BifurcationKit.jl to see how the AMR behaves.

3 Likes

That sounds great! I’m looking forward to this. If you have any questions, please feel free to ping us here or open an issue in our repo.

Cool. I see your package has a DOI. How did you get that?

Thank you very much! If you have any questions on Trixi.jl during this process, please feel free to ask.

That might be the case. On the other hand, it’s not necessarily more difficult for people to set up a simulation defined in terms of Julia code than in a TOML file - if there are enough examples. For example, our new basic example is not really more verbose than it’s old equivalent and we think it’s at least as readable as the TOML file used previously. In addition, using Julia code forces people to combine similar parameters closer together to create corresponding structs while it’s more easy to create a mess in TOML files. And there are of course many more advantages of using a library approach and Julia code instead of TOML files (easier to extend, …).

We use https://zenodo.org/ for that.You can log in there with your GitHub account. Then, you can go to your account settings (the button with your email address in the top right after logging in), go to GitHub settings in the menu on the left-hand side, and activate Zenodo DOIs for your repositories on GitHub individually. You will get the first DOI after creating a new GitHub release.
If you have any questions on that process, please feel free to ask.

6 Likes

I fully second @ranocha’s answer above! As the one who is responsible for using TOML in Trixi in the first place, I was at first skeptical about switching to a library-like approach. However, while working with and developing for Trixi-as-a-library, I have became a convert and I now strongly favor the new style.

With this background, let me add this: If you ever plan to have other people extend or modify your implementation, rather than just using it, I’d unequivocally argue to not use text-based parameter files. Besides the obvious advantages of developers being able to directly add new functionality right in the Julia-based setup files (we call them “elixirs”, because they allow you to “mix” your “ingredients”), it also forces you to think more carefully about your software’s design (and data structures!), especially about how you can make its individual components composable and less tightly coupled. Yes, there are also limitations, but as of yet we have not encountered anything that I’d consider a major issue.

Looking back, using TOML files to drive Julia code now feels like receiving a PDF document via email, printing it, signing it by hand, scanning it again, and sending it back as an email attachment…

3 Likes

However, I saw two difficulties here:

  1. When the user wants to run the same simulation multiple times to sweep a particular parameter. For configuration file, it only needs to add a new keyword p_range = [start, stop, step]. However, using Julia code, it at least requires the user to learn how to code loops.

  2. User has to learn the API (exact name of a method) of a library. Using configuration file, user only has to know the meaning of the keyword.

To enable this using a TOML-based setup, you need to support these kind of ranges in your code. In that case, you can also allow users to input this parameter as p_range = range(start, stop, step=step).

Users also need to know the exact name of a keyword in a setup based on text files. All of this should be documented and there should be enough examples to make it easy for users to adapt an existing one. If the interface of the library is flexible enough, there’s usually some high-level interface where people can input parameters as keywords of some constructors.

For example, I don’t think that our new Julia code to create a mesh

# Create a uniformely refined mesh with periodic boundaries
coordinates_min = (-1, -1) # minimum coordinates (min(x), min(y))
coordinates_max = ( 1,  1) # maximum coordinates (max(x), max(y))
mesh = TreeMesh(coordinates_min, coordinates_max,
                initial_refinement_level=4,
                n_cells_max=30_000) # set maximum capacity of tree data structure

is less readable or less easy to modify/understand than the previous setup in a TOML file

# TreeMesh with uniform refinement
coordinates_min = [-1, -1]
coordinates_max =  [1, 1]
initial_refinement_level = 4
n_cells_max = 30000

If one supports default arguments when parsing a TOML file, one can also support the same default arguments (and keyword arguments) in a constructor.

Of course, some parts are also a matter of taste and I don’t want to convince anybody to do something they don’t want to do. It’s just my own experience and taste telling me that I prefer libraries and Julia code to set up simulations and you’re welcome to do it differently in your own projects.

3 Likes

Do you think it is possible to have a plot recipe? For example, how do you plot while simulating?

2 Likes

Absolutely. The best type to create a recipe for would probably be SemidiscretizationHyperbolic, since it contains all information about the solution and its discretization (node positions, mesh connectivity, approximation order etc.):

Right now, we don’t. Instead we write out solution data as HDF5 files and rely on our postprocessing tools Trixi2Img.jl (which relies on Plots.jl) and Trixi2Vtk.jl (which relies on ParaView/VisIt) to visualize it.

Since our focus at the moment is more on large-scale HPC simulations with MPI, in-situ visualization with a serial package such as Plots.jl is not a current priority. However, I think such a feature would be a great tool for local use and during code development! It could, e.g., be realized as a callback that lives in the Trixi2Img package, which already contains Plots.jl-related functionality. If you’re interested to look into it, we’d be happy to assist with questions regarding the data structures and implementation in Trixi.

1 Like

Just a minor addition to the nice answer of @sloede above: A useful plot recipe will need to make use of the ODE solution vector (u inside DiffEq and u_ode inside Trixi) in addition to the SemidiscretiationHyperbolic (which doesn’t store the current solution value, just intermediate caches and information on the mesh and the kind of semidiscretization).

1 Like

Oops, of course :man_facepalming: Sorry for the confusion, this is absolutely correct.

@sloede and I chatted in another channel and we really like your suggestion, @rveltz. Would you mind creating an issue at Trixi.jl to keep track of that?
This would be a really nice feature for local development and small-scale problems.

This looks wonderful. Looking forward to going through it more carefully as I develop my own codes…

Are there any plans to factor out any functionality (e.g., AMR) as a separate package?

2 Likes

Thanks!

Not in the near future. In our usual development workflow,

  • we get a working version of some feature at first,
  • implement tests,
  • improve the implementation a bit while using tests to make sure that we do not break things,
  • think again about the implementation/abstraction when a second version of some feature is added.

Since we have basically only a single version of our mesh with AMR, we will probably refactor some parts later in step 4. We might consider separating Trixi into more packages in the future when the interfaces stabilized more and we have more versions of the core ingredients. But that will take some time since we will add more features like MPI, another mesh type, …

By the way, we’re of course looking at your nice papers. For example, we’re working on implementing the entropy-based mortar methods of your paper in Support for EC mortars with flux correction à la Chan et al. by sloede · Pull Request #247 · trixi-framework/Trixi.jl · GitHub. You’re always welcome to join us there!

1 Like

Really nice! I will have a closer look on it in the next days
and also starting to run some simulations.

1 Like

Thanks! Feel free to ask any questions you might have.

By the way, we’re of course looking at your nice papers. For example, we’re working on implementing the entropy-based mortar methods of your paper in https://github.com/trixi-framework/Trixi.jl/pull/247 ! You’re always welcome to join us there!

That’s really cool to hear someone’s using that method, and I would love to contribute (especially since reading your code will probably improve my own Julia skills)!

This last week is busy wrapping up our semester, but I’m hoping to try out Trixi for an AMR-based project next week. If that works, I’d love to help with mortars.

Looking forward to corresponding more!