How to Maintain Consistent Julia Package Versions Across a Team - Julia & ModelingToolkit Versioning Issues – Best Practices?

New to Julia
,
1

Hello everyone,

Our team has been working with Julia and ModelingToolkit.jl for a project, and after updating to the latest versions, we are facing issues. We are seeking guidance on:

  1. How to enforce specific package versions across our team to maintain consistency.
  2. How to stay updated on changes in ModelingToolkit (such as function name changes and deprecated syntax) to avoid future compatibility issues.

##Background

We initially installed Julia 1.10.5 (September 2024) and cloned our repository. We activated the environment and instantiated dependencies with:

using Pkg
Pkg.activate(".")
Pkg.instantiate()
  • Our Project.toml contained:
[deps]
DifferentialEquations = "0c46a032-eb83-5123-abaf-570d42b7fbaa"
IJulia = "7073ff75-c697-5162-941a-fcdaad2a7d2a"
ModelingToolkit = "961ee093-0014-501f-94e3-6117800e7a78"
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
  • Our installed versions (from Manifest.toml) were:
    DifferentialEquations.jl** → 7.12.0
    IJulia.jl** → 1.24.2
    ModelingToolkit.jl** → 8.75.0
    Plots.jl** → 1.40.8

Everything was working fine with Julia 1.10.5 and these package versions.

The Problem

Recently, we:

  1. Upgraded to Julia 1.11.4
  2. Updated all packages to their latest versions

After doing this, many functions * After doing this, many functions *stopped working * because they no longer exist in the updated packages.

For example, functions like states(sys) (which worked in ModelingToolkit 8.75.0) no longer function correctly in the latest version.

:red_question_mark: Questions & Help Needed

:one: How to Enforce Specific Package Versions Across Our Team?

We want all team members to use the exact same package versions, no matter which Julia version they have.

How should we modify Project.toml or use Pkg commands to enforce this?
Is there a way to lock specific versions across all machines in our team?

:two: How to Stay Updated on ModelingToolkit Function Changes?

Many functions seem to have changed or been deprecated.
Is there an official changelog or an easy way to track function name changes and syntax updates?

What’s the best way to avoid running into unexpected issues when upgrading?

:light_bulb: Any insights, best practices, or recommendations would be highly appreciated!

Thanks in advance for your help. :blush:

1 Like
2
3 Likes
3

Note that Semantic Versioning signals that changes to the highest version number are indicators of a breaking change. Major.minor.patch, where updating a major is a signal of breaking changes. See https://semver.org/ for details.

In particular, MTK v9’s release notes mentioning what the breaking changes were is as Fredrik mentioned inhttps://github.com/SciML/ModelingToolkit.jl/blob/master/NEWS.md . In particular at the very top of that is:

The function states is renamed to unknowns .

which is one of the largest breaks we have done, and control engineers rejoiced because they really disliked states, yadayada, details details, but that should help you upgrade.

3 Likes
4

I’m sure you’ll get more detailed and interesting insight from people closer to the MTK ecosystem, but very briefly:

  1. How to enforce specific package versions - that’s what the manifest file is for, rather than the project file. However you further specify that you want people to use the exact same package versions no matter which Julia version they are on. This won’t be possible in general, as packages can have different Julia version compatibilities for different package versions (e.g. a package might release a new version and drop compat for an older Julia version). Since Julia 1.11 (?) manifests are version specific though, so you can at least enforce everyone on the same Julia version to have the same package versions.

  2. Not sure what kind of release notes MTK uses, but in general you can check GitHub for releases:

3 Likes
5

The endless fight against dependency hell is a lot easier with some restrictions in Project.toml’s [compat] field though; there’s a team standard that can be incrementally adjusted if updates improve or break anything. Could definitely stop people independently upgrading major versions of dependencies without knowing.

2 Likes
6

You could use the freeze(Pkg) function of the package GitHub - ufechner7/PkgHelpers.jl: Helper functions for the Julia package manager to freeze your package versions.

But this is not always the best approach.

I usually freeze the version only for packages that have breaking changes very often, for example i currently use

[compat]
LinearSolve = "=2.39.0"
ModelingToolkit = "~9.54.0"

in my Project.toml file.

In addition, when I have a working set of packages I create the files:

Manifest-v1.10.toml.default
Manifest-v1.11.toml.default

as copy of the last working Manifest.toml file and add them to git.

You can then have an install script that renames the correct file to Manifest-v1.10.toml or Manifest-v11.toml depending on the Julia version you are using and instantiates the manifest.

See for example: KiteModels.jl

But don’t expect that you can avoid any breakage completely. If you want new features you will always have to test new package versions and update the Project and possibly Manifest files in your repo if you think it makes sense.

3 Likes
7

I would have a git repo that contains just a Project.toml and Manifest.toml (i.e. an environment). Then all team members git clone that environment and work in it (with julia --project=.) Once in a while, some updates the environment with ]up, and everybody git pull the new environment.

Adding [compat] bounds to Project.toml isn’t a bad idea, but I personally try to stay up to date with the latest versions unless there’s a breakage.

Instead of a separate git repo, you can also put it in YourPackage/dev_env, and ]dev ..

1 Like
8

In 1.12 I would say to have a monorepo with all internal packages and have everything in a common workspace (10. Project.toml and Manifest.toml · Pkg.jl) so that there is only one manifest file for all packages.

4 Likes
9

Well, in my projects there are always a few packages where the newest versions either don’t work at all, or don’t work with my code. I don’t know which packages you use, but packages like ModellingToolkit.jl and Makie.jl have often breaking changes, every few months. So better restrict the versions of these packages in Project.toml to a known, working version.

10

A monorepo suggestion from @kristoffer.carlsson is a big surprise! Do you have any other (non-julia) reading to recommend? Is there any public Julia github monorepo we can look at for inspiration?