At least in my CI runs, the time reported by CI includes package instantiation time. And then you mainly measure the compile time and not the runtime.
And if you want to report a runtime regression, you need to:
- run the code locally preferably on a desktop computer, or if you have only a laptop, at least make sure it is connected to power and the CPU temperature is not at its upper limit
- make sure all other programs, including the browser, are closed
- share the output of
Pkg.status() - share the output of
versioninfo() - Share the code you executed, preferably in a form that can be copied and pasted
The reason is that it is very difficult to measure the performance in a reproducible way, so please make it easier for others to reproduce issues you encounter.