These are some good examples of a fundamental challenge: that in Julia the burden of hitting the right semantics for the optimizer has been shifted largely from the type system (incl checker) to the user, vs static languages. As you point out, it’s even harder when considering compositionality, which could result in playing optimizer whack-a-mole. Tkf calls this aptly ‘programming in a optimizer defined sub-dialect’. Things grow more acute as the set of optimizations and transforms grows beyond what was initially co-designed with the type system / language, like AD, GPU codegen, escape analysis/memory elision and various combinations of the aforementioned. (see here for more of my thoughts on that for ML specifically).
I wonder if there is a general solution to this. Can there be domain specific opt in static typing or will improvement in tooling like JET help much? (Have you tried JET?)
The extensible static typing route could be particularly interesting for julia as we aren’t locked into one static system and all the tradeoffs that entails. Though maybe this is very wrong? I don’t know.
But perhaps the designs around user defined compiler passes should be paired with some system for composable user defined semantic invariants.
I know @cscherrer has similar concerns has he relies heavily on type based compile time programming in soss.jl with GitHub - JuliaStaging/GeneralizedGenerated.jl: A generalized version of Julia generated functions @generated to allow closures in generated functions and avoid the use of runtime eval or invokelatest. , but that apparently may be an antipattern. If so, what is the replacement?