I have been working with this repo, GitHub - cecimerelo/BraveNewAlgorithm.jl and running energy benchmarks, as well as performance benchmarks.
After upgrading to 1.12, and as measured by an external CLI (pinpoint), the speed has degraded by 10% and the energy consumption by even more, 20%.
I’m pretty sure this is not a problem with Julia itself. The issue is that I’m measuring startup time, compilation time, as well as the program itself. I would say that it’s the startup time that’s killing this: maybe this version includes more optimized JIT compiler or something, and the actual speed since the script has actually started to the end is actually improving.
However, this forces me to change methodology of measurement (trying to measure baseline, for instance), which might be major, so I’d like to hedge my bets here and ask before: does this hunch make sense, or actual speed might have always taken a hit for some reason?
edit With the version upgrade I had to upgrade a series of libraries, including Optimization; that shouldn’t impact much since I don’t use it directly.
I just want to point out that in this thread we also noticed a ~10% slowdown with Julia 1.12. In microbenchmarks the difference was even around 40%.
3 Likes
Is that intended? Or just teething pains?