Error running highdim_pde script from 2019 on 2022 SciML versions

I am trying to run the “Universal Differential Equations” demos posted at the github repository:
GitHub - ChrisRackauckas/universal_differential_equations: Repository for the Universal Differential Equations for Scientific Machine Learning paper, describing a computational basis for high performance SciML, created by @ChrisRackauckas .
I am running with Julia 1.8.2. I updated all the packages and ran the code, and get an error.
I am running the smallest example: highdim_pde . Here is the error (the stack trace is so large that finding the error seems rather difficult. I am returning to Julia after a two year hiatus :slight_smile:

I am running on Ubuntu 22.04.

Here is the top portion of a very long trace:

 Activating project at `~/src/2022/universal_differential_equations/highdim_pde`
Program start
ERROR: MethodError: vcat(::TrackedArray{…,Vector{Float32}}, ::Float32) is ambiguous. Candidates:
  vcat(x::Union{TrackedArray, Tracker.TrackedReal}, xs::Union{Number, AbstractArray}...) in Tracker at /home/erlebach/.julia/packages/Tracker/e5fg8/src/lib/array.jl:174
  vcat(A::Union{Number, UniformScaling, AbstractVecOrMat}...) in LinearAlgebra at /usr/share/julia/stdlib/v1.8/LinearAlgebra/src/uniformscaling.jl:409
Possible fix, define
  vcat(::Union{Tracker.TrackedReal, TrackedMatrix{T, A} where {T, A<:AbstractMatrix{T}}, TrackedVector{T, A} where {T, A<:AbstractVector{T}}}, ::Vararg{Union{Number, AbstractVecOrMat}})
Stacktrace:

Has this demo been tested with July 1.8.2? The manifest is from an older standard and does not include the version of Julia that is meant to be used.

Thanks, Gordon.

The manifest in the root of the repo says julia 1.7.3, though at 1.7.3 we already had the new manifest format if I’m not remembering wrong so I’m not sure why the manifests inside each example does not have that. But could be worth checking if that works at least.

That manifest is probably for Julia v1.2 or v1.3. Instead of trying to maintain a separate repo like this, I’m working them into the docs of each package.

https://docs.sciml.ai/HighDimPDE/stable/

That’s for this, but indeed something weird was added to tracker/base and I need to track that down.

Hi @ChrisRackauckas ,

The process failed with Julia 1.7.3. So I might indeed have to go down to 1.2 or 1.3 as you suggest.
But you are saying that currently one probably has to rewrite the examples from scratch if one hopes to run them on a more recent version of Julia?

Given that most of the packages have unit tests, you should probably know which packages are failing ?

I am looking at your documentation for DifferentialEquations.jl (DifferentialEquations.jl: Scientific Machine Learning (SciML) Enabled Simulation and Estimation · DifferentialEquations.jl). Where does one find the Julia version compatible for each of the tutorials? Julia is advancing quite rapidly, and as the ecosystem becomes more complex, it is increasingly hard to guaranty that packages and demos keep working. This of course, increases the resistance to adoption.

Looking at DifferentialEquations.jl Project.toml, I see that it is compatible with Julia 1.6. I take this to mean that there is no guarantee the library will work with either past or future versions of Julia. How do you keep the different versions synchronized? Are all your tutorials tested everytime Julia is upgraded? Also, all the packages required for your demos are not necessarily compatible with the same versions of Julia. Is there a strategy to keep this magnificient SCiML collection of modules in sync so that they are quasi-guaranteed to work as Julia gets upgraded? Is this strategy described anywhere?

Why is the number of packages in Project.toml so large for the examples you provide? It is hard for me to believe that most of those packages are actually necessary to run the example. Perhaps I am missing some crucial element that would help me understand. Thanks!

I don’t know how you do it, Chris!

Gordon

Summary

  1. Look at package documentation for tested examples, tutorials, and information. The package documentation is the one true canonical source on the state of the package and its versioned.
  2. We are still in the middle of redoing some of our documentation (https://docs.sciml.ai/Overview/dev/). I know some of the examples from this repo are missing from those docs, that’s why I haven’t fully deprecated it, I’m just looking for the time to do it.
  3. The examples run if you use the version of Julia that matches the manifests. Julia manifests from v1.7 onward (and in later v1.6-LTS patches). This means that the very old manifests do not self-document their version. Sorry, but it’s also pre-LTS so the best thing is to just update those examples.
  4. That entire repo is in the process of being removed with the examples being part of tested packages. So it’s not the best source anymore. The package documentation is what will show you update to date documentation and tutorials on the packages.
  5. We know that repo is very old. In fact, with the highdim_pde PDE example in there not having been updated since 2019, that code actually pre-dates the existence of the SciML organization and our use of the term in general.
  6. In 2020 when the SciML organization was founded, we had a multi-year goal set to define an all encompassing differentiable simulation interface for modeling and simulation. By summer of 2022 we are now at the state you see in SciML: Open Source Software for Scientific Machine Learning with Julia · Overview of Julia's SciML. Importantly, this means that the big backlog of package renames, interface expansions, and compilation handling is completed. In other words, the interfaces have stabilized. One of our upcoming ecosystem blog posts will go into detail on this, so I’ll refer to SciML Scientific Machine Learning Blog as the source for more information.

In other words, yes, example scripts like highdim_pde that were written before SciML existed need an update. They are so old that Julia didn’t even version the manifests back then. But things are very different now. SciML has already completed its “v1.0” event and now things are stabilizing around a brand new organization wide documentation system. That said, not all tutorials have made it into the new documentation, these ones are definitely on the list, and more information on this process will come from the standard source SciML Scientific Machine Learning Blog shortly.

Point by Point

Yes, but those scripts are not in tests. I suggest you use the tutorials in the packages. For example:

https://docs.sciml.ai/SciMLSensitivity/stable/neural_ode/neural_ode_flux/

It shows the package version. And this is an up-to-date tutorial that is used within the unit tests so if it fails then it will alert the developers who then either need to ensure the tutorial gets updated or that any breaks are reversed. That kind of treatment does not happen on repos of random scripts from a few years ago: I highly recommend using the packaged tutorials.

That is a lower bound. See how the designations in Pkg.jl are read:

https://pkgdocs.julialang.org/dev/compatibility/

SciML supports the LTS onwards:

The examples in the package documentation are tested, synchronized, and updated with the packages. Hence, refer to the package tutorials for updated code. Anything outside of that are demos with a manifest that show how something could be done at one point in the past, but won’t necessarily be updated.

Just use the documentation in the packages? That’s literally just tested by adding the packages and running the code. There’s nothing else extra.

I don’t understand. There’s 3 packages in there. Let’s take a look at:

Most are standard libraries. The things which are not standard libraries are:

Flux = "587475ba-b771-5e3f-ad9e-33799f191a9c"
NeuralNetDiffEq = "8faf48c0-8b73-11e9-0e63-2155955bfa4d"
StochasticDiffEq = "789caeaf-c7a9-5a7d-9973-96adeb23e2a0"

3 packages. This example puts neural networks (Flux) inside of a stochastic differential equation (StochasticDiffEq) with the connection being the package NeuralNetDiffEq.jl back in 2020. In 2022 we have split NeuralNetDiffEq.jl into the Physics-Informed Neural Network (PINN) functionality (NeuralPDE.jl), and the high dimensional PDE solver functionality (HighDimPDE.jl), which is where we then refer to this stuff today. We are still in the middle of handling some of the last parts of HighDimPDE.jl, and when that’s done I’d just refer to there to its tested examples instead of the demo repo.

1 Like

Thank you for this detailed and helpful reply, @ChrisRackauckas .

Just one detail.

“Yes, but those scripts are not in tests. I suggest you use the tutorials in the packages. For example:
https://docs.sciml.ai/SciMLSensitivity/stable/neural_ode/neural_ode_flux/
It shows the package version.”

I did not see a package version at the link you pointed me towards. Could you point it out to me? Thanks.
.
Your work and dedication are amazing Chris! Thanks again.

Gordon

Every doc is versioned the same like that.

Though I guess what you might be wondering is what is the full manifest of the tutorial generator?

If that’s your question, then, good point.

https://docs.sciml.ai/SciMLTutorialsOutput/stable/models/01-classical_physics/

We do it in the “extended tutorials” (which we are slowly going to morph into the package docs now), and maybe we should add that to the others.

Thanks. The number you circled is the package number, which is what I asked for, but the version of Julia is also important, isn’t it? One cannot always assume it is the latest version. But as you said, the manifest provides that information. The last link was very informative. I appreciate the time you took to reply.

Best regards,

Gordon

Yes, on the first page of each doc site we should probably put the required Julia versions and link to the blog post explanation. And a way to see the manifest of the tutorials easily (maybe folded unless you clock on it?)

Capturing it here: Versioning in the docs · Issue #69 · SciML/SciMLDocs · GitHub

Yes, the manifest in folded form would be great. I am finding that when reexecuting the various demos, it is often useful to delete the manifest and instantiate within the package manager, after saving a copy of the manifest of course.

But isn’t the Project.toml more important the Manifest.toml since the Manifest is auto-generated? One question is this: in Project.toml, the package versions are not listed, but rather there is some kind of hashtag, which is not human-readable. Do you know why this choice was made?

Thanks,

Gordon

The Project.toml doesn’t list the version because that’s the Manifests job. The project file lists the packages you ] add and there is a UUID associated with the package which is the “real name” of the package (nothing prevents name collisions across multiple repositories, but UUIDs are unique)

Thanks. Therefore, to add a package with a specific version, that is part of the Pkg.add command?

gives the method