# Julia slower in Windows

**URL:** <https://discourse.julialang.org/t/julia-slower-in-windows/21027>\
**Category:** Performance\
**Created:** [February 20, 2019, 10:52pm UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027 "2019-02-20T22:52:13Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![donquicote](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@donquicote](https://discourse.julialang.org/u/donquicote)\
**Post date:** [February 20, 2019, 10:52pm UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/1 "2019-02-20T22:52:13Z")

</div>

Hello,

I’m running this simple piece of code in two different computers:

```julia
using BenchmarkTools

function testing()
    rand(1000000)
end

@btime testing()

```

- HP EliteBook 840 G3. i5-6300U 2.4Ghz. 16GB DRR4. Windows 10.

```julia
4.048 ms (2 allocations: 7.63 MiB)

```

- MacBook Air, Mid 2013. i5-4250U 1.3Ghz. 8GB DDR. macOS Mojave 10.14.3

```julia
1.973 ms (2 allocations: 7.63 MiB)

```

I was expecting the HP to be much faster. Can anyone tell me how is this possible? Is there something wrong with Windows performance? How can I fix this?

Thank you,

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [February 20, 2019, 11:54pm UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/2 "2019-02-20T23:54:23Z")

</div>

Caching of memory? It’s much slower on Windows than Mac for me too, and it’s not related to `rand` but rather memory writes in general. On my Windows laptop, something seems to happen right around 1 MB:

```julia
julia> @btime zeros(130_000);
  62.768 μs (2 allocations: 1015.70 KiB)

julia> @btime zeros(131_000);
  341.468 μs (2 allocations: 1023.52 KiB)

```

On my Mac, I don’t see such sudden degradation of performance.

It’s quite an artificial test though. I normally wouldn’t expect Windows to be any slower than Mac. Do you have a less artificial test that also shows similar performance difference between Windows and Mac?

---

<div class="post-metadata">

**Author:** ![c42f](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/c42f/32/52842_2.png) [@c42f](https://discourse.julialang.org/u/c42f)\
**Post date:** [February 20, 2019, 11:59pm UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/3 "2019-02-20T23:59:37Z")

</div>

This does seem a bit odd at face value (very crudely, userbenchmark suggests [+30% difference in favor of the HP](https://cpu.userbenchmark.com/Compare/Intel-Core-i5-6300U-vs-Intel-Core-i5-4250U/m27864vsm1787). Are these the exact same versions of julia on each machine? I assume you have 64 bit windows?

Keep in mind here that you might be mostly benchmarking the random number library; digging in a bit using Cthulu.jl and `@descend testing()`, I see that Julia is currently using [DSFMT](https://github.com/MersenneTwister-Lab/dSFMT) as the Mersenne Twister implementation in this case. This means the speed will depend on the C compilation flags which were used to build DSFMT, not so much on the julia JIT. It might be that the DSFMT build is faster on your MacBook Air because the compiler flags were more specialized to the architecture there.

Having written the above, I just saw the reply by @bennedich so perhaps it’s not related to the build at all. That `@btime zeros` result is very odd though, I’m not sure what to make of it.

---

<div class="post-metadata">

**Author:** ![c42f](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/c42f/32/52842_2.png) [@c42f](https://discourse.julialang.org/u/c42f)\
**Post date:** [February 21, 2019, 12:11am UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/4 "2019-02-21T00:11:29Z")

</div>

Is this memory writes or allocation? I do wonder whether the windows allocator is just falling off a performance cliff above your threshold there. 300 μs seems quite excessive though.

PS: There’s definitely some performance cliffs in the Vista allocator, see [Memory Allocator Benchmarks](https://locklessinc.com/benchmarks_allocator.shtml) for example.

---

<div class="post-metadata">

**Author:** ![donquicote](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@donquicote](https://discourse.julialang.org/u/donquicote)\
**Post date:** [February 21, 2019, 12:15am UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/5 "2019-02-21T00:15:44Z")

</div>

Yes, I’m using 1.1.0 in both computers and 64 bit Windows.

@bennedich I observe the same jump!

HP:

```julia
julia> @btime zeros(130000)
  102.154 μs (2 allocations: 1015.70 KiB)

julia> @btime zeros(131000)
  457.025 μs (2 allocations: 1023.52 KiB)

```

MacBook Air:

```julia
julia> @btime zeros(130000)
  91.101 μs (2 allocations: 1015.70 KiB)

julia> @btime zeros(131000)
  93.157 μs (2 allocations: 1023.52 KiB)

```

What would be a less artificial test?

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [February 21, 2019, 12:18am UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/6 "2019-02-21T00:18:58Z")

</div>

I don’t quite see it for allocations on my Windows laptop:

```julia
julia> @btime Vector{Float64}(undef, 130_000);
  4.191 μs (2 allocations: 1015.70 KiB)

julia> @btime Vector{Float64}(undef, 131_000);
  4.671 μs (2 allocations: 1023.52 KiB)

```

System:

```julia
julia> versioninfo()
Julia Version 1.0.1
Commit 0d713926f8 (2018-09-29 19:05 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, broadwell)

```

---

<div class="post-metadata">

**Author:** ![Seif\_Shebl](https://avatars.discourse-cdn.com/v4/letter/s/eada6e/32.png) [@Seif\_Shebl](https://discourse.julialang.org/u/Seif_Shebl)\
**Post date:** [February 21, 2019, 12:27am UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/8 "2019-02-21T00:27:14Z")

</div>

And a bizarre huge jump:

```julia
julia> @btime Vector{Float64}(undef, 130_000);
  114.000 ns (2 allocations: 1015.70 KiB)

julia> @btime Vector{Float64}(undef, 131_000);
  3.721 μs (2 allocations: 1023.52 KiB)

```

System:

```julia
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, haswell)

```

---

<div class="post-metadata">

**Author:** ![c42f](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/c42f/32/52842_2.png) [@c42f](https://discourse.julialang.org/u/c42f)\
**Post date:** [February 21, 2019, 12:44am UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/9 "2019-02-21T00:44:12Z")

</div>

> [@donquicote](#):
>
> What would be a less artificial test?

A real computation you want to do in practice is always a good test 😉

Failing that I guess you could run the official julia [microbenchmarks](https://github.com/JuliaLang/Microbenchmarks).

---

<div class="post-metadata">

**Author:** ![donquicote](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@donquicote](https://discourse.julialang.org/u/donquicote)\
**Post date:** [February 21, 2019, 12:54am UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/10 "2019-02-21T00:54:59Z")

</div>

I’ve realized of this performance differences when running a set of much more elaborated nested functions. Then is when I decided to test something simpler like the `rand` example.

I’ll try [microbenchmarks](https://github.com/JuliaLang/Microbenchmarks). Thanks!

---

<div class="post-metadata">

**Author:** ![c42f](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/c42f/32/52842_2.png) [@c42f](https://discourse.julialang.org/u/c42f)\
**Post date:** [February 21, 2019, 1:23am UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/11 "2019-02-21T01:23:44Z")

</div>

> [@donquicote](#):
>
> I’ve realized of this performance differences when running a set of much more elaborated nested functions.

Oh right, that’s good to have some context. Then I’d suggest profiling the larger computation and pulling out the hot functions for individual attention. Perhaps this is what led you to `rand()` in the first place? In that case it’s time to investigate why `rand` is slow, which might very well be the DSFMT compile options, or something weird with memory access.

It’s worth noting that the supposedly weird numbers `130_000` and `131_000` are actually arrays of size straddling the `2^20` boundary (1MiB). Though your HP has 3MiB L3 cache and 512K L2 which suggests this isn’t processor related. In addition, bandwidth to main memory for the i5-6300U is [supposedly a max of 31 GiB/s](https://en.wikichip.org/wiki/intel/core_i5/i5-6300u) which should be ~ 30 μs for 1 MiB (is this symmetrical for read/write?). That suggests something really wrong is happening if it’s taking 300 μs just to allocate an array of zeros of size about 1 MiB, even forgetting about the cache.

If you want to keep digging into the weirdness with `zeros` I’d suggest you benchmark `fill!` separately from the allocation.

```julia
@btime fill!($(zeros(131_000)), 0.0);

```

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [February 21, 2019, 8:59am UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/12 "2019-02-21T08:59:04Z")

</div>

It seems OS related. Running the script below in Julia 1.1 on the same system booted in Windows 10 and Ubuntu illustrates a performance cliff on Windows right around 1 MB, not present on Ubuntu.

Of interest is that I don’t see a performance cliff when measuring _average_ times. Both the script below and `@btime` measures _minimum_ time of many invocations.

```julia
using Plots

benchmark_op(n) = zeros(n * 1024)

sizes = []
times = []
for n = round.(Int, 1.1 .^(24:73))'
    t = minimum(1:1000 .|>_-> @elapsed benchmark_op(n))
    append!(sizes, n*sizeof(Float64))
    append!(times, t * 1e6)
end

plot(sizes, times; title="Elapsed time for zero(n)",
    xlabel="vector size (KiB)", ylabel="microseconds", legend=:none)

```

Windows on the left, Ubuntu on the right:

![](https://global.discourse-cdn.com/julialang/original/3X/2/b/2b616cab141e474aff632d508b501fefffefef45.png)

&nbsp; ![](https://global.discourse-cdn.com/julialang/original/3X/4/1/416b4bb7613ee8d54d7f0ab865a430ffcbcea0f1.png)

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [February 21, 2019, 10:53am UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/13 "2019-02-21T10:53:46Z")

</div>

There are many times the OS doesn’t commit the allocation until the space is actually used so these type of artificial tests don’t say much of what would happen in a real situation.

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [February 21, 2019, 11:18am UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/14 "2019-02-21T11:18:48Z")

</div>

Agree with this when it comes to allocating (and that taking the minimum of 1000 invocations is a terrible estimator of actual memory IO performance), but wouldn’t calling `zeros` use the space?

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [February 21, 2019, 12:51pm UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/15 "2019-02-21T12:51:31Z")

</div>

Just ran the same test in Java on Windows, and I’m seeing a similar cliff at 1 MB. So thankfully it seems unrelated to Julia.

```java
for (int k = 24; k <= 73; ++k) {
    int n = (int)(Math.pow(1.1, k) + 0.5);
    long min = Long.MAX_VALUE;
    for (int i = 0; i < 1000; ++i) {
        long t = System.nanoTime();
        double[] A = new double[n * 1024];
        for (int e = 0; e < A.length; ++e)
            A[e] = 0.0;
        t = System.nanoTime() - t;
        if (t < min) min = t;
    }
    System.out.printf("%d, %f, ", n*8, min / 1000.0);
}

```

![](https://global.discourse-cdn.com/julialang/original/3X/9/8/9832f27bafc022dd61a20e2ab07a2f3b6fe15cba.png)

---

<div class="post-metadata">

**Author:** ![c42f](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/c42f/32/52842_2.png) [@c42f](https://discourse.julialang.org/u/c42f)\
**Post date:** [February 22, 2019, 2:39am UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/16 "2019-02-22T02:39:39Z")

</div>

> [@bennedich](#):
>
> Just ran the same test in Java on Windows, and I’m seeing a similar cliff at 1 MB. So thankfully it seems unrelated to Julia.

It’s great to see those various plots. As far as I can tell by looking at the julia GC runtime, all the large arrays are essentially allocated by a call to the standard system aligned version of `malloc`, so we might well be measuring the behavior of the default system `malloc`, in terms of how it ultimately uses syscalls to get memory and how aggressively it frees that back to the OS vs keeping it in a pool, etc.

I assume these performance oddities only happen for newly allocated memory?

> [@bennedich](#):
>
> Of interest is that I don’t see a performance cliff when measuring _average_ times. Both the script below and `@btime` measures _minimum_ time of many invocations.

That’s interesting I’ve always wondered why allocation introduces so much variance into timings. What does the distribution of times look like in (terms of min,max,median and quartiles for example)? The minimum time could be quite misleading for allocation heavy workloads.

Comparing the slopes of those two windows vs ubuntu graphs, it seems absurd that windows takes 0.18 ms/MiB whereas ubuntu takes 0.07 ms/MiB on the same machine.

Doing a bit more reading, I think we are measuring mostly the cost of OS page fault handling here. Allocating without touching the memory is fast because the OS doesn’t have to commit physical memory pages, but the first time a process iterates over a newly allocated array (to read or write it) a bunch of [major page faults](https://en.wikipedia.org/wiki/Page_fault#Major) will be generated. Whether the memory is truly new to the process is invisible to julia because it uses the system `malloc` which may choose to reuse recently freed blocks. Here’s an [interesting blog post](https://randomascii.wordpress.com/2014/12/10/hidden-costs-of-memory-allocation/) about the hidden costs of memory allocation. It’s interesting that they say 1MB is indeed the threshold at which the windows allocator decides to directly allocate from the OS via a syscall.

The page fault mechanism also explains why allocation introduces so much variance to timing measurements: it gives the OS an [excuse for a context switch](https://www.quora.com/Does-the-OS-dispatch-to-another-process-when-theres-a-page-fault-on-the-current-process).

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [February 22, 2019, 11:49pm UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/17 "2019-02-22T23:49:34Z")

</div>

> [@c42f](#):
>
> What does the distribution of times look like in (terms of min,max,median and quartiles for example)?

We can use the script below to plot creation speed histograms for `zeros` for a range of array sizes.

```julia
using Plots

benchmark_op(m) = zeros(m * 128)

anim = @animate for n = 1:1280
    t = 1e6(1:10000 .|>_-> @elapsed benchmark_op(n)) / n * 1024
    display(histogram(log10.(filter(k -> 88 < k < 640, [t;89;640])),
        nbins=120, xlims=(1.95,2.8), ylims=(0,2000), legend=:none,
        xformatter=x->round(Int, 10^x), title="array = $n KiB",
        xlabel="creation speed (us / MB)", ylabel="count"))
end

gif(anim, "allocations.gif", fps = 30)

```

Below are the results for a Windows 10 laptop. Note how creation speeds faster than ~300 μs/MiB completely disappear when the array is larger than 1024 KiB.

![allocations](https://global.discourse-cdn.com/julialang/original/3X/5/a/5ac117f58f07aeb9a55b860b86fa8dd0577f150b.gif)

---

<div class="post-metadata">

**Author:** ![bennedich](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bennedich/32/4894_2.png) [@bennedich](https://discourse.julialang.org/u/bennedich)\
**Post date:** [February 23, 2019, 8:56am UTC](https://discourse.julialang.org/t/julia-slower-in-windows/21027/18 "2019-02-23T08:56:10Z")

</div>

For completeness, below is the result of the same script run on Ubuntu. A bit more stable.

![allocations-ubuntu2](https://global.discourse-cdn.com/julialang/original/3X/6/b/6b5f80a0b21338813523945a6924c2dabe66f217.gif)