# What is the equivalent of numpy.random.choice for generating a non-uniform random sample based on a user supplied probability?

**URL:** <https://discourse.julialang.org/t/what-is-the-equivalent-of-numpy-random-choice-for-generating-a-non-uniform-random-sample-based-on-a-user-supplied-probability/34461>\
**Category:** General Usage\
**Tags:** question\
**Created:** [February 11, 2020, 10:23am UTC](https://discourse.julialang.org/t/what-is-the-equivalent-of-numpy-random-choice-for-generating-a-non-uniform-random-sample-based-on-a-user-supplied-probability/34461 "2020-02-11T10:23:59Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![PyDataBlog](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pydatablog/32/12327_2.png) [@PyDataBlog](https://discourse.julialang.org/u/PyDataBlog)\
**Post date:** [February 11, 2020, 10:23am UTC](https://discourse.julialang.org/t/what-is-the-equivalent-of-numpy-random-choice-for-generating-a-non-uniform-random-sample-based-on-a-user-supplied-probability/34461/1 "2020-02-11T10:23:59Z")

</div>

Hi,

I’m on the lookout for the equivalent of `[numpy.random.choice()`]([numpy.random.choice — NumPy v1.15 Manual](https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.random.choice.html)).

I want to generate random indices based on non-uniform random sampling.

For example, I can do this with Numpy by passing a list of the associated probability of each entry as:

`rand_idx = numpy.random.choice(300, size=1, p=probability_list)`

I would like to do this in Julia like:  
`rand_idx = rand(1:300, 1, #supply_probability_list# )`

How can I do this in Julia?

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [February 11, 2020, 10:38am UTC](https://discourse.julialang.org/t/what-is-the-equivalent-of-numpy-random-choice-for-generating-a-non-uniform-random-sample-based-on-a-user-supplied-probability/34461/2 "2020-02-11T10:38:03Z")

</div>

You might be looking for [`StatsBase.sample`](https://juliastats.org/StatsBase.jl/stable/sampling/#StatsBase.sample)

---

<div class="post-metadata">

**Author:** ![PyDataBlog](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pydatablog/32/12327_2.png) [@PyDataBlog](https://discourse.julialang.org/u/PyDataBlog)\
**Post date:** [February 11, 2020, 2:58pm UTC](https://discourse.julialang.org/t/what-is-the-equivalent-of-numpy-random-choice-for-generating-a-non-uniform-random-sample-based-on-a-user-supplied-probability/34461/3 "2020-02-11T14:58:48Z")

</div>

I have tried `sample(1:300, probability_list)` to no avail. It’s not accepting the the probability\_list.

---

<div class="post-metadata">

**Author:** ![PyDataBlog](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pydatablog/32/12327_2.png) [@PyDataBlog](https://discourse.julialang.org/u/PyDataBlog)\
**Post date:** [February 11, 2020, 3:12pm UTC](https://discourse.julialang.org/t/what-is-the-equivalent-of-numpy-random-choice-for-generating-a-non-uniform-random-sample-based-on-a-user-supplied-probability/34461/4 "2020-02-11T15:12:35Z")

</div>

Finally got it. I have to convert the list into a probability weight abstract array like:

`sample(1:300, ProbabilityWeights(probability_list))`

---

<div class="post-metadata">

**Author:** ![affans](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/affans/32/11911_2.png) [@affans](https://discourse.julialang.org/u/affans)\
**Post date:** [February 11, 2020, 4:17pm UTC](https://discourse.julialang.org/t/what-is-the-equivalent-of-numpy-random-choice-for-generating-a-non-uniform-random-sample-based-on-a-user-supplied-probability/34461/5 "2020-02-11T16:17:37Z")

</div>

You can also use `Distributions` and `rand`

```julia
julia> using Distributions

julia> a = Categorical([0.2, 0.2, 0.2, 0.2, 0.2])
DiscreteNonParametric{Int64,Float64,Base.OneTo{Int64},Array{Float64,1}}(
support: Base.OneTo(5)
p: [0.2, 0.2, 0.2, 0.2, 0.2]
)

julia> rand(a)
3

```

`rand(a`) returns the index of the array with that probability.

If it matters,

```julia
julia> @time sample(1:5, ProbabilityWeights([0.2, 0.2, 0.2, 0.2, 0.2]))
  0.000016 seconds (7 allocations: 352 bytes)

julia> @time rand(a)
  0.000005 seconds (4 allocations: 160 bytes)

```

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [February 11, 2020, 6:18pm UTC](https://discourse.julialang.org/t/what-is-the-equivalent-of-numpy-random-choice-for-generating-a-non-uniform-random-sample-based-on-a-user-supplied-probability/34461/6 "2020-02-11T18:18:35Z")

</div>

For micro benchmarks like this you really should use BenchmarkTools. Also, you only included the construction of the weights in one of the benchmarks.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [February 12, 2020, 8:32am UTC](https://discourse.julialang.org/t/what-is-the-equivalent-of-numpy-random-choice-for-generating-a-non-uniform-random-sample-based-on-a-user-supplied-probability/34461/7 "2020-02-12T08:32:37Z")

</div>

Results will probably differ depending on your real world use case (Do you need to construct the weights once or more often? Do you sample once or many times? How large is the array you want to sample from etc) but as @DNF says reasonable benchmarking will help.

FWIW one attempt (including constructin the `Categorical`/`ProbabilityWeights` objects) at what I think the problem in the OP was:

```julia
julia> using Distributions, StatsBase

julia> vals = collect(1:300);
julia> weights = vals ./ sum(vals);

julia> sample_cat(v, w) = v[rand(Categorical(w))];
julia> sample_sb(v, w) = sample(v, ProbabilityWeights(w));

julia> using BenchmarkTools

julia> @btime sample_cat($vals, $weights)
774.490 ns (2 allocations: 2.53 KiB)

julia> @btime sample_sb($vals, $weights)
250.393 ns (1 allocation: 32 bytes)

```

---

<div class="post-metadata">

**Author:** ![Marco\_Antoniotti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marco_antoniotti/32/18697_2.png) [@Marco\_Antoniotti](https://discourse.julialang.org/u/Marco_Antoniotti)\
**Post date:** [November 2, 2023, 10:35pm UTC](https://discourse.julialang.org/t/what-is-the-equivalent-of-numpy-random-choice-for-generating-a-non-uniform-random-sample-based-on-a-user-supplied-probability/34461/8 "2023-11-02T22:35:14Z")

</div>

Hi

just a couple of nagging questions I have regarding this old topic. I guess that my ‘statistics-fu’ is very low.

1. Do the weights built by `pweights` need to sum up to 1?
2. Is the item returned by `sample` built from a cumulative sum of said `pweights`?

Just trying to understand without delving too much in the code. 😁

Thanks

MA