# With Missings, Julia is slower than R

**URL:** <https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838>\
**Category:** General Usage\
**Created:** [June 21, 2018, 3:17am UTC](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838 "2018-06-21T03:17:56Z")\
**Posts on this page:** 11\
**Page:** 2

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [June 27, 2018, 8:47pm UTC](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838/21 "2018-06-27T20:47:45Z")

</div>

Note that until the fixes are merged, a relatively straightforward solution is to use `sort!(collect(skipmissing(Y2)), alg=QuickSort)`. It’s about as fast as R and twice as fast as `sort(Y2)`. It’s also what R does by default (`na.last=NA`), i.e. skipping NAs (keeping NAs doesn’t affect performance).

We should probably define `sort(itr::SkipMissing)` to call `sort!(collect(itr), ...)`, since that’s a common need and it’s the most efficient possible definition.

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [February 25, 2021, 5:19pm UTC](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838/22 "2021-02-25T17:19:44Z")

</div>

Just want to check if missing values and sorting are faster in Julia than R now.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [February 25, 2021, 5:48pm UTC](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838/23 "2021-02-25T17:48:35Z")

</div>

It should be pretty easy to check this using `@btime` in Julia and the R library `rbenchmark`

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [February 25, 2021, 6:41pm UTC](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838/24 "2021-02-25T18:41:51Z")

</div>

> [@gaspardelanuit](#):
>
> `y = replace(x-> x<= 0.01, x, missing)`

This line of code no longer works.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [February 25, 2021, 7:13pm UTC](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838/25 "2021-02-25T19:13:07Z")

</div>

That is with Julia 0.7. You are resurrecting a very old thread.

If you do `?replace` you will see that the current syntax is

```julia
using Missings
x = allowmissing(rand(10_000))
y = replace(x-> x <= 0.01 ? x : missing, x)

```

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [February 25, 2021, 7:15pm UTC](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838/26 "2021-02-25T19:15:22Z")

</div>

It doesn’t look to me like things have changed:

```julia
julia> Y1 = rand(Float64, 10_000_000);

julia> Y2 = ifelse.(rand(length(Y1)) .< 0.9, Y1, missing);

julia> @btime sort($Y1);
  757.621 ms (2 allocations: 76.29 MiB)

julia> @btime sort($Y2);
  1.885 s (2 allocations: 85.83 MiB)

```

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [February 25, 2021, 11:46pm UTC](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838/27 "2021-02-25T23:46:11Z")

</div>

I find this very concerning. Missing values are very common when working with real-world data.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [February 26, 2021, 12:01am UTC](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838/28 "2021-02-26T00:01:04Z")

</div>

For sure. This should definitely be fixed. In an issue linked above it seems people had a strategy of improving this.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 26, 2021, 12:31am UTC](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838/29 "2021-02-26T00:31:43Z")

</div>

oh wow this is so hacky I love it…  
[https://github.com/JuliaLang/julia/issues/27781#issuecomment-401110632](https://github.com/JuliaLang/julia/issues/27781#issuecomment-401110632)

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [February 26, 2021, 2:59am UTC](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838/30 "2021-02-26T02:59:12Z")

</div>

To be honest, I think it is better to let new users understand that Julia is not always faster than R and Python. I keep seeing people getting frustrated about Julia when they find these cases.

I personally find that this community oversells Julia’s performance but undersells its syntax.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [February 26, 2021, 4:10am UTC](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838/31 "2021-02-26T04:10:59Z")

</div>

> [@Yifan\_Liu](#):
>
> Julia is not always faster than R and Python.

there are always multiple levels of understanding to this story. In this case, a relevant point is:

> [@nalimilan](#):
>
> R’s `sum` is implemented in C, so being only twice slower than it in pure Julia is already quite good. If you write a sum in pure R, it will be much slower than that! More fundamentally, the fact that writing the loop manually gives a better performance means that we should be able to match R relatively easily, and probably beat it at least in some cases.

More importantly, in almost all cases, Julia can fix the problem by writing idiomatic Julia code, instead of MUST relying on C/C++/Fortran subroutine like you see in Python/R.

If you compare the time R/Python people spent in writing C/C++ code to the time someone in Julia community casually writing PRs to make a specific use faster, hopefully you’d see the qualitative difference and the long-term implication of this.

[Previous page](https://discourse.julialang.org/t/with-missings-julia-is-slower-than-r/11838.md?page=1)