# How to sample data without using external packages?

**URL:** <https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240>\
**Category:** General Usage\
**Tags:** question, memory-allocation, random\
**Created:** [March 1, 2022, 1:52pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240 "2022-03-01T13:52:24Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dharik\_Arsath](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dharik_arsath/32/27461_2.png) [@Dharik\_Arsath](https://discourse.julialang.org/u/Dharik_Arsath)\
**Post date:** [March 1, 2022, 1:52pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/1 "2022-03-01T13:52:24Z")

</div>

I am trying to sample data from population within loop but that produces way too much allocations and making my algorithms lot slower. I need a way to sample data with no more than 1 allocation. How should I proceed??? Is there any inplace random number generator in julia.

---

<div class="post-metadata">

**Author:** ![lawless-m](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lawless-m/32/30869_2.png) [@lawless-m](https://discourse.julialang.org/u/lawless-m)\
**Post date:** [March 1, 2022, 2:14pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/2 "2022-03-01T14:14:46Z")

</div>

Uniform: [rand!](https://docs.julialang.org/en/v1/stdlib/Random/#Random.rand)

Normal: [randn!](https://docs.julialang.org/en/v1/stdlib/Random/#Random.randn!)

Exponential: [randexp!](https://docs.julialang.org/en/v1/stdlib/Random/#Random.randexp!)

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [March 1, 2022, 2:18pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/3 "2022-03-01T14:18:54Z")

</div>

Note that calls to `rand()` do not allocate. It will only allocate if you construct a vector, ie. `rand(10)`.

---

<div class="post-metadata">

**Author:** ![Dharik\_Arsath](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dharik_arsath/32/27461_2.png) [@Dharik\_Arsath](https://discourse.julialang.org/u/Dharik_Arsath)\
**Post date:** [March 1, 2022, 2:33pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/4 "2022-03-01T14:33:52Z")

</div>

I need to generate random numbers within particular range, like 1: length(X).

---

<div class="post-metadata">

**Author:** ![Dharik\_Arsath](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dharik_arsath/32/27461_2.png) [@Dharik\_Arsath](https://discourse.julialang.org/u/Dharik_Arsath)\
**Post date:** [March 1, 2022, 2:34pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/5 "2022-03-01T14:34:39Z")

</div>

I am trying mini batch gradient descent where I need small small batches of data, is it possible to sample without constructing a vector?

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [March 1, 2022, 2:36pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/6 "2022-03-01T14:36:48Z")

</div>

Something like

```julia
julia> v = Vector{Int}(undef, 3)
3-element Vector{Int64}:
 235792464
 235792496
 293017040

julia> v .= rand(1:100, 3)
3-element Vector{Int64}:
 78
 12
 50

```

?

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [March 1, 2022, 2:40pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/7 "2022-03-01T14:40:46Z")

</div>

```julia
julia> v = Vector{Int}(undef, 3)
3-element Vector{Int64}:
 0
 0
 0

julia> v .= rand.(Ref(1:100))
3-element Vector{Int64}:
  8
 71
 39

```

is much better as it doesn’t allocate yet another array

---

<div class="post-metadata">

**Author:** ![Seif\_Shebl](https://avatars.discourse-cdn.com/v4/letter/s/eada6e/32.png) [@Seif\_Shebl](https://discourse.julialang.org/u/Seif_Shebl)\
**Post date:** [March 1, 2022, 3:16pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/8 "2022-03-01T15:16:20Z")

</div>

Like this? Notice the 0-allocations.

```julia
function take_sample!(data, sample)
    N = length(data)
    for i = 1:100
        for j in eachindex(sample)
            sample[j] = data[rand(1:N)]
        end
    end 
end

N = 100
data = rand(-5:5, N)
sample = fill(0,5)
@btime take_sample!($data, $sample)
  3.750 μs (0 allocations: 0 bytes)

```

Or a faster version using rand():

```julia
function take_sample!(data, sample)
    N = length(data)
    for i = 1:100
        for j in eachindex(sample)
            sample[j] = data[trunc(Int,N*rand()+1)]
        end
    end 
end
@btime take_sample!($data, $sample)
  1.050 μs (0 allocations: 0 bytes)

```

You can get it even a bit faster if you use a vector for rand() since it could be SIMD’ed.

---

<div class="post-metadata">

**Author:** ![Dharik\_Arsath](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dharik_arsath/32/27461_2.png) [@Dharik\_Arsath](https://discourse.julialang.org/u/Dharik_Arsath)\
**Post date:** [March 1, 2022, 5:26pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/9 "2022-03-01T17:26:16Z")

</div>

This works good but I am unsure why Ref is used here?

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [March 1, 2022, 5:34pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/10 "2022-03-01T17:34:12Z")

</div>

Because otherwise you’d broadcast also `1:100`. Instead you want to say “for each iteration call `rand(1:100)`”

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [March 1, 2022, 5:36pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/11 "2022-03-01T17:36:51Z")

</div>

> [@giordano](#):
>
> `> v .= rand.(Ref(1:100))`

However this is a sample with replacement

---

<div class="post-metadata">

**Author:** ![goerch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerch/32/29122_2.png) [@goerch](https://discourse.julialang.org/u/goerch)\
**Post date:** [March 1, 2022, 5:41pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/12 "2022-03-01T17:41:48Z")

</div>

> [@dlakelan](#):
>
> However this is a sample with replacement

Hm. Did a quick check., would have expected a difference:

```julia
using Random, BenchmarkTools

Random.seed!(42)

v1 = Vector{Int}(undef, 100000)
v1 .= rand(1:2, length(v1))
@show v1

Random.seed!(42)

v2 = Vector{Int}(undef, 100000)
v2 .= rand.(Ref(1:2))
@show v2
@assert v1 == v2

```

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [March 1, 2022, 9:05pm UTC](https://discourse.julialang.org/t/how-to-sample-data-without-using-external-packages/77240/13 "2022-03-01T21:05:41Z")

</div>

> [@goerch](#):
>
> would have expected a difference:

nah they’re both samples with replacement.

Imagine you have 1M data points and you want to take a sample of 1000 of them… the question is will any of those 1M data points be included 2 or 3 or 4 etc times? Or is each data point included at most once?

To sample without replacement, you can just shuffle the indices and create a view:

```julia
mydata = rand(1_000_000);

indices = collect(1:1_000_000)
while ! done
   shuffle!(indices)
   subsample = @view(mydata[indices[1:100]])
   ... work with the subsample
end

```