# 1.8 much slower than 1.6

**URL:** <https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254>\
**Category:** General Usage\
**Created:** [October 25, 2022, 3:33pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254 "2022-10-25T15:33:14Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Impressium](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/impressium/32/19575_2.png) [@Impressium](https://discourse.julialang.org/u/Impressium)\
**Post date:** [October 25, 2022, 3:33pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/1 "2022-10-25T15:33:14Z")

</div>

Hi all 🙂

I stumbled over a very strange thing. I’m updating from 1.6 to 1.8. The only problem all the scripts use much more CPU even to a point where everything is used. The scripts are CPU intensive and use lots of threads. Any ideas why that could happen?

---

<div class="post-metadata">

**Author:** ![Krastanov](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krastanov/32/6817_2.png) [@Krastanov](https://discourse.julialang.org/u/Krastanov)\
**Post date:** [October 25, 2022, 3:37pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/2 "2022-10-25T15:37:07Z")

</div>

Some more information would probably be needed to answer.

- Do you have a reproducible Project.toml file so we know what versions and what packages we are talking about? Just having a nice clean Project.toml file will avoid a ton of pitfalls.
- Do you see the slowdown in a brand new project environment or only in the old project environment inherited from 1.6?
- Do you have any small examples of code experiencing slowdown? It would be pretty difficult to track the issue without that. You can use `@time` and `BenchmarkTools.@benchmark` to see where the slowdown is.
- Is runtime slow? Or precompilation? Or installation itself? As in, is julia slow to compile your code or to execute the compiled code?

---

<div class="post-metadata">

**Author:** ![Impressium](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/impressium/32/19575_2.png) [@Impressium](https://discourse.julialang.org/u/Impressium)\
**Post date:** [October 26, 2022, 4:55pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/3 "2022-10-26T16:55:41Z")

</div>

It’s not so easy. The script run for the whole day transforming live data. There are multiple threads and communication between different services.

I tried to minimize everything that could effect it - clear install, deleting .julia folder etc. Didn’t work. After a while the 1.8 version just spins up

---

<div class="post-metadata">

**Author:** ![Impressium](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/impressium/32/19575_2.png) [@Impressium](https://discourse.julialang.org/u/Impressium)\
**Post date:** [October 28, 2022, 9:03am UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/4 "2022-10-28T09:03:31Z")

</div>

I don’t know if I started Julia and the scripts the right way. I wrote a c# API which starts a Julia process within a the build in Process class. Three I just use the absolute path to julia.exe and after that the script path. Maybe there was a change from 1.6 to 1.8?

---

<div class="post-metadata">

**Author:** ![lrnv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lrnv/32/19373_2.png) [@lrnv](https://discourse.julialang.org/u/lrnv)\
**Post date:** [October 28, 2022, 10:26am UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/5 "2022-10-28T10:26:08Z")

</div>

Julia is not designed to write scripts, but rather to stay with one session opened, as you probably already know.

Maybe you’ll enjoy this workflow: [GitHub - dmolina/DaemonMode.jl: Client-Daemon workflow to run faster scripts in Julia](https://github.com/dmolina/DaemonMode.jl) ?

Cannot say anything about the difference between 1.8 and 1.6 though without taking a deeper look at what you are doing.

---

<div class="post-metadata">

**Author:** ![Impressium](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/impressium/32/19575_2.png) [@Impressium](https://discourse.julialang.org/u/Impressium)\
**Post date:** [October 28, 2022, 10:47am UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/6 "2022-10-28T10:47:28Z")

</div>

Thank you 🙂 this looks nice for different use case. My scripts need to run for about 8h and compile time shouldn’t be a problem. They compile in the morning and should be good for the day. It was with way till I tried to update to 1.7/1.8.  
I use more that 10 threads. One steams data and deserialize it, second one cumulates, third one waites for events and so on. Real-time data analysis…

---

<div class="post-metadata">

**Author:** ![lrnv](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lrnv/32/19373_2.png) [@lrnv](https://discourse.julialang.org/u/lrnv)\
**Post date:** [October 28, 2022, 11:11am UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/7 "2022-10-28T11:11:16Z")

</div>

I’m sorry, your use case seems a little complicated and I might not be able to help you.

Are you saying that each of these threads runs in some kind of ‘server’ mode, runnig waiting for inputs to arrive all around the day ? Or do you re-launch julia and the scripts several time per day ?

In the second case, DeamonMode.jl might help. In the first case, I cannot help you without deeper and more precise information about what exactly did go slower than 1.6 : which packages, which operation, etc, e.g. with a MWE.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [October 28, 2022, 12:04pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/8 "2022-10-28T12:04:41Z")

</div>

@Impressium, you could also try the nightly 1.9-DEV. Of course we don’t want regressions, but if fixed there good to know. You might want to bisect or narrow down, e.g. test on 1.7 too (at least of not fixed on master).

> [@lrnv](#):
>
> Julia is not designed to write scripts

Yes, and no. More designed for long-running (HPC type) code, but as you point out you can do scripts with that package, or simply also an option:

```julia
julia -O0

```

Have you tried that and do you see if 1.8 is faster (than 1.6) that way? Of course we don’t want a regression for any code, I suppose 1.8 is faster for most code.

---

<div class="post-metadata">

**Author:** ![Impressium](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/impressium/32/19575_2.png) [@Impressium](https://discourse.julialang.org/u/Impressium)\
**Post date:** [October 28, 2022, 12:36pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/9 "2022-10-28T12:36:41Z")

</div>

It’s the first case. The scripts are running the whole day. No restarts

---

<div class="post-metadata">

**Author:** ![Impressium](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/impressium/32/19575_2.png) [@Impressium](https://discourse.julialang.org/u/Impressium)\
**Post date:** [October 28, 2022, 12:45pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/10 "2022-10-28T12:45:37Z")

</div>

`julia -O3`  
Means the fastest code and longest compile time?

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [October 28, 2022, 12:52pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/11 "2022-10-28T12:52:54Z")

</div>

Yes, `julia -O3` should have longest compilation time, i.e. higher number means more optimization, thus slower to compile (potentially faster at runtime). Of course worth to try that too, or lower -O1 or -O0, though I was answering for scripts (short running where you want to minimize optimization time), still good to try all possibilities for you. And:

```julia
 (@v1.8) pkg> status --outdated

```

and make sure you have same versions of your packages (and their dependencies). Julia tends to downgrade packages for me (or for people (like me) that misuse…).

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [October 28, 2022, 2:17pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/12 "2022-10-28T14:17:03Z")

</div>

> [@Impressium](#):
>
> much more CPU even to a point where everything is used. T

I assume you are multithreading? And also maybe calling Linear Algebra?

Possibly your BLAS is using more threads than before and this is interfering with your Julia threads? Try adjusting the number of blas threads with BLAS.set\_num\_threads

Try setting it to some number less than the number of physical cores.

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [October 28, 2022, 4:53pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/13 "2022-10-28T16:53:06Z")

</div>

It seems to me you need to use ProfileView.jl on a small chunk of your problem and see where the time is being spent. You could also compare that with 1.6. That should point you towards a fix.

Anything else is speculation as we cant read your code and the context seems quite complicated.

---

<div class="post-metadata">

**Author:** ![Impressium](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/impressium/32/19575_2.png) [@Impressium](https://discourse.julialang.org/u/Impressium)\
**Post date:** [October 28, 2022, 5:12pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/14 "2022-10-28T17:12:57Z")

</div>

I don’t use LinearAlgebra.jl

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [October 28, 2022, 5:59pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/15 "2022-10-28T17:59:09Z")

</div>

How much slower is it?

It might be related to/same cause and solution:

> [@Very different performance in different computers due to garbage collection?](https://discourse.julialang.org/t/very-different-performance-in-different-computers-due-to-garbage-collection/87563/6):
>
> Very strange indeed. If you want to try and reduce/eliminate most of the allocations (will speedup as 95% is just cleaning up memory), take a look at this talk - [Hunting down allocations with Julia 1.8's Allocation Profiler | JuliaCon 2022](https://live.juliacon.org/talk/YHYSEM). It’s for Julia 1.8 so you would have to upgrade, but there’s a source code view which tells you which lines of your code are allocating and how much. This can be very helpful when optimising your code. It seems worth it in this case if you have to run it ove…

I would try 1.9 as I mentioned (then you don’t need to compile our own 1.8.3) and 1.8.3 seems near, it’s getting very stable (1 regression), and might be close.

---

<div class="post-metadata">

**Author:** ![Impressium](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/impressium/32/19575_2.png) [@Impressium](https://discourse.julialang.org/u/Impressium)\
**Post date:** [November 1, 2022, 11:00am UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/16 "2022-11-01T11:00:33Z")

</div>

I’ve got one more observation.  
I’m running multiple scripts on this server. One script is just running fine (1% CPU load). I started a second one and both of them use 100% split 50/50. I stop the second script and the first one goes back to 1%. I can do it again and the same happens

---

<div class="post-metadata">

**Author:** ![Impressium](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/impressium/32/19575_2.png) [@Impressium](https://discourse.julialang.org/u/Impressium)\
**Post date:** [November 1, 2022, 11:25am UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/17 "2022-11-01T11:25:31Z")

</div>

I tried 1.9. No difference

---

<div class="post-metadata">

**Author:** ![Per](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/per/32/10387_2.png) [@Per](https://discourse.julialang.org/u/Per)\
**Post date:** [November 1, 2022, 12:14pm UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/18 "2022-11-01T12:14:05Z")

</div>

Maybe Julia 1.8 is somehow busy-waiting for some resource, while 1.6 was able to do it in a better way.

If it is the difference between 1 % and 100 %, then profiling should clearly show which function it is that is eating all the CPU cycles.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [November 2, 2022, 10:26am UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/19 "2022-11-02T10:26:16Z")

</div>

> [@Impressium](#):
>
> I’m running multiple scripts on this server. One script is just running fine (1% CPU load). I started a second one and both of them use 100% split 50/50.

Multiple scripts, of I assume this script (though would be bad either way).

I assume you do:

```julia
$ julia -t auto

julia> Threads.nthreads()
16

so try halving:

$ julia -t 8

```

Then run just two scripts like this, or 4 halving yet again etc. The auto is there to use the highest number of practically usable threads. Maybe going over the edge for one (or many scripts) it’s going to be expected to get bad performance.

It’s possible though that it shouldn’t fall of a cliff, if it did that. Do you get better (or same) overall performance with some such settings? Why are you running many scripts (of the same kind?) since you’re using threads anyway? This seems to be a regression in Julia (and if not to be expected) you could file an issue.

---

<div class="post-metadata">

**Author:** ![AdamWysokinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adamwysokinski/32/206422_2.png) [@AdamWysokinski](https://discourse.julialang.org/u/AdamWysokinski)\
**Post date:** [November 2, 2022, 10:47am UTC](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254/20 "2022-11-02T10:47:26Z")

</div>

I have recently encountered a significant performance hit when running my scripts at maximum threads count. Try reducing nthreads value as suggested above by Palli.

[Next page](https://discourse.julialang.org/t/1-8-much-slower-than-1-6/89254.md?page=2)