How can write a function to find unique elements in array without any allocation?

Hi,

I am using unique function to find unique elements in an array. But the memory allocation to use this function is increasing with array size. In my case array size is nearly 10000, and it causes 36 allocation.

a = rand(10000)
@btime unique(a)
335.624 μs (36 allocations: 450.50 KiB)

Moreover, I have to use this function multiple times, more than 10M times. So, is there any other methods to reduce the allocations in unique function? or Can you suggest some ideas to unique function with less allocation with similar execution time?
Thanks in Advance !
Manu

1 Like

from the docs, try:

a = rand(10000)
sort!(a)
unique!(a)

rand(10000) is a bad example as is probably unique anyway, but it’s a step.
Other option is using Set(a)

3 Likes

Also, interpolate a into the benchmark to avoid a measurement of the global lookup: @btime unique($a).

1 Like

@longemen3000 @carstenbauer Thanks for your reply. When I applied your suggestion, allocations reduced as follows:

@btime unique!($a)
 286.947 μs (23 allocations: 193.88 KiB)

Thank You,
Manu

if you don’t mind the result being sorted, try @btime unique!(sort!($a))

julia> a=rand(1:1_000,10_000); b=copy(a);

julia> @btime unique!($a);
  22.201 μs (18 allocations: 49.58 KiB)

julia> @btime (unique!(sort!($b)));
  7.000 μs (0 allocations: 0 bytes)
3 Likes