# Threads maxing out all cores, but no performance increase

**URL:** <https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590>\
**Category:** General Usage\
**Tags:** performance, threads\
**Created:** [April 5, 2021, 1:43am UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590 "2021-04-05T01:43:59Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![seulberg1](https://avatars.discourse-cdn.com/v4/letter/s/e480ec/32.png) [@seulberg1](https://discourse.julialang.org/u/seulberg1)\
**Post date:** [April 5, 2021, 1:43am UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/1 "2021-04-05T01:43:59Z")

</div>

Hello All

I started using Julia, because of its speed in an application, where the workload in pseudo-code looks like I added below (no MWE, but hopefully this is diagnostic enough - otherwise I will create a MWE, but it could be that this is an Julia-novice mistake that I wasn’t able to solve through googling):

There are 4000 csv files (4-7mb each), for each, the code does some data prep and then calls a function that iterates through each row and does a bunch of arithmetic. Unfortunately, the arithmetic is time-series data / state dependent and cannot be vectorized.

When single-threading it, the average iteration takes 7 seconds per csv file. When using Threads.@threads and 8 threads for the first loop `Threads.@threads for i in 1:4000`, the average iteration takes 7.8 seconds.

I changed the environment variable, so Julia always starts with 8 threads, check for it before running anything and most importantly, the CPU utilization in the multi-threaded case goes up to 100% on all 8 cores of my CPU, but the execution still is slower. Any advice on how to use multi-threading and actually speed up the code execution?

Any help is much appreciated

```julia
resArr = DataFrame()

for i in 1:4000
        df = DataFrame(CSV.File(file_i)))
        dummy = fun1(df)
        resArr[i] = dummy
end

function fun1(df)
      *someDataPrep*
      res = fun2(df)
      return res
end

function fun2(df)
      for (i, row) in enumerate(eachrow(df))
           *a lot of arithmetic*
      end
      return x, y, z
end

```

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [April 5, 2021, 3:03am UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/2 "2021-04-05T03:03:15Z")

</div>

Before you try parallelizing, have you profiled parts of your code using BenchmarkTools or Profile? Excessive allocations may be occurring, and those tools will help you find them and optimize.

---

<div class="post-metadata">

**Author:** ![andrewdinhobl](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/andrewdinhobl/32/23614_2.png) [@andrewdinhobl](https://discourse.julialang.org/u/andrewdinhobl)\
**Post date:** [April 5, 2021, 5:10am UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/3 "2021-04-05T05:10:57Z")

</div>

Is each file taking 7.8 seconds (rather than 7), but you are doing ~8 files at time? That would be an improvement. Or are you saying that your total throughout is now 1 file in 7.8 seconds?

---

<div class="post-metadata">

**Author:** ![seulberg1](https://avatars.discourse-cdn.com/v4/letter/s/e480ec/32.png) [@seulberg1](https://discourse.julialang.org/u/seulberg1)\
**Post date:** [April 5, 2021, 3:53pm UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/4 "2021-04-05T15:53:03Z")

</div>

Unfortunately its the later case - the total time until completion is longer when on 8 cores than single-threaded. I built the same code in Python (which on single-thread is 4 times slower) and used Dask to multi-thread it and got an approximate 7 times speed up (so right now Python is 1.75 times faster), but I hope to get the same speed up when utilizing multi-threading properly.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [April 5, 2021, 5:44pm UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/5 "2021-04-05T17:44:50Z")

</div>

> [@seulberg1](#):
>
> ` df = DataFrame(CSV.File(file_i)))`

Sorry if this is a stupid question, but where is `file_i` defined in this code? Shouldn’t be `file[i]` where `file` is a vector with the file names? (maybe you are mistakenly reading the same file 4000 times here?)

Independently of the above, try putting this in a function. I am not sure how much it will help, but certainly won’t hurt:

```julia
function readdata(files)
  resArr = zeros(length(files)) # ?? Not sure how to initialize it here
  Threads.@threads for i in 1:length(files)
          df = DataFrame(CSV.File(files[i]))) #???
          dummy = fun1(df)
          resArr[i] = dummy
  end
  return resArr
end

```

---

<div class="post-metadata">

**Author:** ![seulberg1](https://avatars.discourse-cdn.com/v4/letter/s/e480ec/32.png) [@seulberg1](https://discourse.julialang.org/u/seulberg1)\
**Post date:** [April 5, 2021, 5:54pm UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/6 "2021-04-05T17:54:44Z")

</div>

I called @benchmark with .samples set to 5, and based on the print statement it also ran it 5 times, but it reports somehow only 1 sample and 1 evals/sample. Could be the max time cancelling some runs prematurely. But the numbers are consistent with that @time reports and vary by ±5% at most. The memory estimate seems high?

Single Threaded: (ran the entire program for 20 csv files )

```julia
BenchmarkTools.Trial: 
  memory estimate: 35.15 GiB
  allocs estimate: 724012212
  --------------
  all times: 123.552 s (3.96% GC)
  --------------
  samples: 1
  evals/sample: 1

```

fun1 by itself: (For 1 csv file)

```julia
BenchmarkTools.Trial: 
  memory estimate: 1.99 GiB
  allocs estimate: 40912629
  --------------
  minimum time: 6.978 s (3.91% GC)
  median time: 6.986 s (3.89% GC)
  mean time: 6.993 s (3.90% GC)
  maximum time: 7.019 s (3.84% GC)
  --------------
  samples: 5
  evals/sample: 1

```

fun2 (For 1 csv file 1 call - this gets called 11 times per csv file in the original file)

```julia
BenchmarkTools.Trial: 
  memory estimate: 185.95 MiB
  allocs estimate: 3718819
  --------------
  minimum time: 620.072 ms (3.64% GC)
  median time: 631.499 ms (3.73% GC)
  mean time: 631.443 ms (3.87% GC)
  maximum time: 646.031 ms (3.80% GC)
  --------------
  samples: 5
  evals/sample: 1

```

185mb in the function 2 seems high - especially given that this is called 4000\*11 times.

Stepwise, I deleted all the arithmetic, then the iterator and then everything except for the following 2 lines, that cause 90% of the memory allocation:

```julia
function fun2_Barebone(df)
    df[!,"DateTime"] = floor.(Dates.DateTime.(df[!,:Date] , "yyy-mm-dd HH:MM:SS"),Dates.Hour)
    df = leftjoin(df, histP[!,["varP","Hour"]],on=[:DateTime => :Hour])
    return true
end

```

The df that goes into the function has a size of 3.4 mb (based on Base.summarysize) and comes from the csv file, the histP DataFrame I am merging has a size of 0.2mb and is defined as a global constant and contains one time series of prices and dates that has to be merged with each csv file separately as the csv files contain gaps.

Two questions - a) why do those 2 lines create 164MiB of memory allocation and is there an easy way of making this more efficient? b) Why does this not scale with multiple threads?

---

<div class="post-metadata">

**Author:** ![seulberg1](https://avatars.discourse-cdn.com/v4/letter/s/e480ec/32.png) [@seulberg1](https://discourse.julialang.org/u/seulberg1)\
**Post date:** [April 5, 2021, 5:59pm UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/7 "2021-04-05T17:59:22Z")

</div>

Putting it in a function has no perceptible effect on the speed or allocation.  
The file comes from iterating over the Directory with the cleaned .csv files (the directory string is stored in the variable called cleanDir).  
` for (row, file) in collect(enumerate(readdir(cleanDir)))`

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [April 5, 2021, 6:01pm UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/8 "2021-04-05T18:01:42Z")

</div>

Every time you do `[something]` you’re telling Julia to allocate an array on the heap.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [April 5, 2021, 6:01pm UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/9 "2021-04-05T18:01:46Z")

</div>

> [@seulberg1](#):
>
> `histP[!,["varP","Hour"]]`

As far as I understand DataFrames, this allocates a new array here, you could use `@view histP[!,["varP","Hour"]]` instead (or add `@views` to `@views function ... `.

Also `["varP","Hour"]` allocates that two-element small vector. If you can avoid that somehow, it would improve things as well.

---

<div class="post-metadata">

**Author:** ![seulberg1](https://avatars.discourse-cdn.com/v4/letter/s/e480ec/32.png) [@seulberg1](https://discourse.julialang.org/u/seulberg1)\
**Post date:** [April 5, 2021, 6:50pm UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/10 "2021-04-05T18:50:04Z")

</div>

Thank you everyone so far.

I dug a little bit deeper, worked with the view for histP, and got a 1MiB improvement (nice but not material). Out of the two lines, 7MiB come from the leftjoin, but 155 come from what I have now re-written as  
`df.DateTime = floor.(Dates.DateTime.(df.Date , "yyy-mm-dd HH:MM:SS"),Dates.Hour)`

I guess replacing notation does not change memory allocation.

1. Is there any faster way of converting a string column to DateTime and rounding it to hours?
2. With 32GB RAM on this machine which is barely used and the SSD mostly idling, why does the multi-threading not help speed up code execution, while still utilizing more cores?

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [April 5, 2021, 7:00pm UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/11 "2021-04-05T19:00:57Z")

</div>

> [@seulberg1](#):
>
> df.DateTime = floor.(Dates.DateTime.(df.Date , “yyy-mm-dd HH:MM:SS”),Dates.Hour)

Here, again if I am not mistaken, you are allocating a new array on the right. Try using a broadcasted assignment (`df.DateTime .= floor....`) with a dot in the `.=`.

If you provided a minimal working example helping would be much easier. I would not focus on the parallel execution until you get the best possible serial version, but allocations can hurt parallel performance a lot, in particularly if you have lots of them, as that means a lot of concurrent accesses to memory.

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [April 5, 2021, 11:50pm UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/12 "2021-04-05T23:50:50Z")

</div>

> [@lmiq](#):
>
> Here, again if I am not mistaken, you are allocating a new array on the right. Try using a broadcasted assignment ( `df.DateTime .= floor....` ) with a dot in the `.=` .

I don’t know if that matters, and might actually add unnecessary overhead. I don’t think you can avoid allocating an array on the right side at all? I think you’d have to just do a simple loop over the individual elements to avoid it. But that should just add one allocation per iteration, which shouldn’t be a big deal?

Hmm… I’m seeing that the type of the result here isn’t being inferred:

```julia
function g()
    o = "2001-04-05 12:34:25"
    floor(Dates.DateTime(o, "yyyy-mm-dd HH:MM:SS"), Dates.Hour)
end
@code_warntype g()

Variables
  #self#::Core.Const(g)
  o::String

Body::Any
1 ─ (o = "2001-04-05 12:34:25")
│ %2 = Dates.DateTime::Core.Const(DateTime)
│ %3 = o::Core.Const("2001-04-05 12:34:25")::Core.Const("2001-04-05 12:34:25")
│ %4 = (%2)(%3, "yyyy-mm-dd HH:MM:SS")::Any
│ %5 = Dates.Hour::Core.Const(Hour)
│ %6 = Main.floor(%4, %5)::Any
└── return %6

```

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [April 6, 2021, 12:10am UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/13 "2021-04-06T00:10:25Z")

</div>

Type inferrence failures are a big source of slow performance, because the compiler doesn’t emit optimized code for the types.

I see an old issue related to this: [Date(::String, ::DateFormat) isn’t type stable · Issue #13644 · JuliaLang/julia (github.com)](https://github.com/JuliaLang/julia/issues/13644)

I wonder if this is an edge case there? When I run the code in a function I see type instability, but not when I run it directly.

**All that said** , I found how to make it more performant: wrap the format string in `Dates.DateFormat()`.

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [April 6, 2021, 12:14am UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/14 "2021-04-06T00:14:28Z")

</div>

Ah, from the documentation:

```julia
DateTime(dt::AbstractString, format::AbstractString; locale="english") -> DateTime

  Construct a DateTime by parsing the dt date time string following the pattern given in the format string (see
  DateFormat for syntax).

  This method creates a DateFormat object each time it is called. If you are parsing many date time strings of the
  same format, consider creating a DateFormat object once and using that as the second argument instead.

```

---

<div class="post-metadata">

**Author:** ![andrewdinhobl](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/andrewdinhobl/32/23614_2.png) [@andrewdinhobl](https://discourse.julialang.org/u/andrewdinhobl)\
**Post date:** [April 6, 2021, 12:18am UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/15 "2021-04-06T00:18:42Z")

</div>

Oooh. I think I’ve run into this too, and wondered to myself why it was slow. Good to know.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [April 6, 2021, 12:36am UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/16 "2021-04-06T00:36:35Z")

</div>

> [@BioTurboNick](#):
>
> But that should just add one allocation per iteration, which shouldn’t be a big deal?

That depends on the size of array. Hard to know. Surely improper type inference doesn’t help.

---

<div class="post-metadata">

**Author:** ![seulberg1](https://avatars.discourse-cdn.com/v4/letter/s/e480ec/32.png) [@seulberg1](https://discourse.julialang.org/u/seulberg1)\
**Post date:** [April 6, 2021, 3:06pm UTC](https://discourse.julialang.org/t/threads-maxing-out-all-cores-but-no-performance-increase/58590/17 "2021-04-06T15:06:41Z")

</div>

Awesome, everyone - that did the trick. It is a bit counterintuitive to me that running the function over the column is implemented that way, but thank you for digging this up from the documentation.

These two lines

```julia
myDF = Dates.DateFormat("yyy-mm-dd HH:MM:SS")
df.DateTime = floor.(Dates.DateTime.(df.Date , myDF),Dates.Hour)

```

reduces allocation from 155mb down to 7.7mb:)

Broadcasting in this case increased memory allocation to 8.06, so I am going without broadcasting for this line.

The best news: Now the multi-threading works - processing 100 csv files takes:  
Single-thread: 45 seconds  
8-Threads, on 8 cores: 11 seconds.

Much appreciated, the proper Date parsing and the multi-threading speeds up the code factor 56!