No. The difference is only 5 seconds, so for me 368 s instead of 363 s. And you are right, the multithreading is actually only used when the packages are precompiled.
BUT, Julia 1.11.5 compiles the same module via PackageCompiler 63 % slower than Julia 1.10.9, see my detailed test here.