Question for anyone interested: What should be the default behavior of a @stable-ized function encountering type instability?
Error (current behavior)
Warning (can set with default_mode="warn")
Comparison:
Advantages of error: (i) absolutely refuses to run with any type instability. Itās sort of like a strict compiler such as Rust compiler in that it guarantees certain behavior; (ii) does not flood the REPL output with messages (particularly bad with unstable kernels in loops) as you just exit immediately, it only shows the first instability.
Advantages of warning: (i) does not quit on the first instability, displays a warning for each unstable function throughout the stacktrace, which makes it easier to track down the source directly from the warning; (ii) allows you to run code without needing the allow_unstable function (which new users might now know about)
You can always set the default_mode yourself as a library developer or user, or set a default via Preferences.jl. However, what should the default be?
I see this primarily as a development tool. Once I prove everything is stable, do I really want to keep proving stability on every userās machine?
In this case, I see it being used as follows. In development, I add this package and use it to establish everything is stable. Then I remove this package and the uses of @stable and ship it off. I would do this since I want the code to load as quickly as possible on the userās machine and not have further ramifications for the user.
Thereās a lost opportunity there though since @stable as an annotation is actually useful. It shows me that someone used this package to check stability, and more importantly it shows that if I wanted to re-examine the question I could easily do so.
Erroring during precompilation is an disaster for the user. Not only does that prevent my package from loading, but it prevents all other packages which depend on my package from loading. That means that if some small part of my package becomes unstable, I crash the userās entire environment.
I experienced this when trying to use SnoopCompile.jl during Julia 1.11 development. It depended on JET.jl and Cthulhu.jl, which are both very complicated packages. They both errored durring precompilation, meaning I could not use a completely unrelated part of SnoopCompile.jl. While the solution there is to turn those into weak dependencies, this was not a great user experience. It was one I only able to overcome as an experienced Julia developer. Nonetheless it made the path to doing what I wanted excruciatingly long.
This package would be most interesting to me if it became an inert annotation when deployed, so I do not need to disable or remove it. However, I would also appreciate if became fully functional and errored early and often in development.
The user is least able to change the default behavior. The user may not even be a primary user of my package or even of Julia. Expecting the user to change the default seems to be too much.
The trick then is how to detect the presence of a development environment and then activating errors by default only then. One method may be to recognize that Iām trying to load an environment that is the package environment of the package Iām trying to load. Another method may be the presence of packages like Revise or Test being loaded. These could be detected via the package extension mechanism.
Note that it does automatically turn off during precompilation. And as you might suspect it was for exactly the reasons you shared ā the precompilation errors are annoying So now all errors are disabled during that stage.
Regarding inertness, I would mostly agree. Although there are downstream applications where a user wants to ensure type stability like static compilation or using Enzyme (even if they are not developing a library).
So I really like your idea of adding it with Preferences, so any user can āturn onā stability checks for any package.
This is something you can actually push to users, because they need to explicitly turn on the stability checking for the errors to show up (which some might want to do, for the reasons above). Otherwise the macro turns off entirely and has no effect on code.
I think you would want to have it work out of the box locally too though, especially for beginners. Basically I think that @stable f() =Val(rand()); f() should just work and give an error, without any configuration needed. Advanced users can always set up their CI to configure the preferences, but beginners shouldnāt have any extra barriers. Thoughts?
errors that only show up in CI can be very hard to debug, especially if say some upstream package toggled on errors in CI and youāre using that functionality in a downstream package (so maybe you arenāt familiar with DispatchDocter etc). So IMO itās better to not have errors turn on automatically in CI (I think itās OK for errors to turn on in testing, e.g. Pkg.test uses --check-bounds=yes, but that shows up both locally and CI).
for error; 
for warning)
So now all errors are disabled during that stage.