# How would I check for unique values across many arrays without for loops?

**URL:** https://discourse.julialang.org/t/how-would-i-check-for-unique-values-across-many-arrays-without-for-loops/40341
**Category:** General Usage
**Created:** [May 28, 2020, 2:14pm UTC](https://discourse.julialang.org/t/how-would-i-check-for-unique-values-across-many-arrays-without-for-loops/40341 "2020-05-28T14:14:29Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Julia1](https://avatars.discourse-cdn.com/v4/letter/j/db5fbb/32.png) [@Julia1](https://discourse.julialang.org/u/Julia1)
#### Post date: [May 28, 2020, 2:14pm UTC](https://discourse.julialang.org/t/how-would-i-check-for-unique-values-across-many-arrays-without-for-loops/40341/1 "2020-05-28T14:14:30Z")

</div>

Hi how’s it going?

Say I have 3 arrays:

X = [1,2,3,4,5]  
Y = [1,2,3,4]  
Z = [1,2,3,4]

I want to return that of the 5 unique values across X, Y and Z, only the number 5 is truly unique, because it only occurs in 1 array.

Now obviously with 3 lists there are many ways to do this, with the simplest and most naive being “For val in x, if val not in y and z…” etc.

But say I have hundreds of arrays and I want to check this. What would be the best way to implement?

---

<div class="post-metadata">

### Author: ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)
#### Post date: [May 28, 2020, 2:20pm UTC](https://discourse.julialang.org/t/how-would-i-check-for-unique-values-across-many-arrays-without-for-loops/40341/3 "2020-05-28T14:20:02Z")

</div>

> without for loops?

By the way, there is absolutely _no reason_ to avoid loops in Julia unless doing so helps you write clearer or easier to understand code. Loops in Julia are fast, and a well-written loop is usually the _fastest_ way to solve a particular problem.

---

<div class="post-metadata">

### Author: ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [May 28, 2020, 2:32pm UTC](https://discourse.julialang.org/t/how-would-i-check-for-unique-values-across-many-arrays-without-for-loops/40341/4 "2020-05-28T14:32:52Z")

</div>

You want to use `countmap`. Unfortunately it doesn’t seem to accept generic iterators, so you can’t use `Iterators.flatten`.

You can do

```julia
array_of_arrays = [X, Y, Z]
countmap(reduce(vcat, array_of_arrays))

```

Then you can loop through the `Dict` and keep just the things with the value `1`.

---

<div class="post-metadata">

### Author: ![Julia1](https://avatars.discourse-cdn.com/v4/letter/j/db5fbb/32.png) [@Julia1](https://discourse.julialang.org/u/Julia1)
#### Post date: [June 2, 2020, 4:10pm UTC](https://discourse.julialang.org/t/how-would-i-check-for-unique-values-across-many-arrays-without-for-loops/40341/5 "2020-06-02T16:10:08Z")

</div>

thank you very much!

---

<div class="post-metadata">

### Author: ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)
#### Post date: [June 2, 2020, 4:29pm UTC](https://discourse.julialang.org/t/how-would-i-check-for-unique-values-across-many-arrays-without-for-loops/40341/6 "2020-06-02T16:29:51Z")

</div>

> [@rdeits](#):
>
> there is absolutely _no reason_ to avoid loops in Julia unless doing so helps you write clearer or easier to understand code

Is this not a bit of an exaggeration? I think I saw several comments/issues where people where getting better performance with vectorized code, mostly due to how particular packages are implemented. I think one case was with Distributions.jl, maybe in relation with Turing.jl, but I can’t find it again. Another came up recently here: [Speed of vectorized vs for-loops using Zygote - #3 by ChrisRackauckas](https://discourse.julialang.org/t/speed-of-vectorized-vs-for-loops-using-zygote/40556/3) .

---

<div class="post-metadata">

### Author: ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)
#### Post date: [June 2, 2020, 4:32pm UTC](https://discourse.julialang.org/t/how-would-i-check-for-unique-values-across-many-arrays-without-for-loops/40341/7 "2020-06-02T16:32:53Z")

</div>

> [@sijo](#):
>
> Is this not a bit of an exaggeration? I think I saw several comments/issues where people where getting better performance with vectorized code, mostly due to how particular packages are implemented. I think one case was with Distributions.jl, maybe in relation with Turing.jl, but I can’t find it again. Another came up recently here: [Speed of vectorized vs for-loops using Zygote](https://discourse.julialang.org/t/speed-of-vectorized-vs-for-loops-using-zygote/40556/3) .

If Zygote is slow on loops but not broadcast then that seems like an issue particular to Zygote and not something that should be applied as a guideline to general Julia code (unless you are writing it specifically to be ADed by Zygote).

---

<div class="post-metadata">

### Author: ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)
#### Post date: [June 2, 2020, 5:12pm UTC](https://discourse.julialang.org/t/how-would-i-check-for-unique-values-across-many-arrays-without-for-loops/40341/8 "2020-06-02T17:12:28Z")

</div>

Agreed, I just thought “absolutely _no reason_” might be a bit strong. People might feel misled if we say that, and then when working on a particular problem they are told that a big slowdown is to be expected with for loops for a well-known package. After all, most serious code will use third-party packages…

---

<div class="post-metadata">

### Author: ![harven](https://avatars.discourse-cdn.com/v4/letter/h/3da27b/32.png) [@harven](https://discourse.julialang.org/u/harven)
#### Post date: [June 2, 2020, 5:21pm UTC](https://discourse.julialang.org/t/how-would-i-check-for-unique-values-across-many-arrays-without-for-loops/40341/9 "2020-06-02T17:21:16Z")

</div>

`counter` from the `DataStructures` library accepts generators so you can bypass the creation of an intermediate array as follows.

```
 julia> import DataStructures: counter

 julia> sole(v...) = [first(p) for p in counter(x for a in v for x in a) if last(p) == 1]

 julia> sole([1:5;], [1:4;], [0:6;])
 2-element Array{Int64,1}:
  0
  6

```

Arguably such an algorithm keeps counting after encountering an element twice, which may not be optimal.

Julia used to have an hist function for computing frequencies but it was removed for some reason.
