# Reworking Distributions.jl

**URL:** <https://discourse.julialang.org/t/reworking-distributions-jl/32339>\
**Category:** Statistics\
**Tags:** proposal\
**Created:** [December 16, 2019, 4:08pm UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339 "2019-12-16T16:08:04Z")\
**Posts on this page:** 12\
**Page:** 3

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [January 27, 2020, 9:04am UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339/41 "2020-01-27T09:04:13Z")

</div>

> [@mschauer](#):
>
> In total
> 
> ```julia
> abstract type Sampleable{X} end
> 
> ```
> 
> should be more than enough and even

What if I have a type that is Sampleable, but must participate in a completely different part of the type tree? I’m thinking about `Particles <: Real` or `VAE <: AbstractGenerativeModel` etc. These must be subtypes of their own type trees to work well, but can certainly be sampled from. The same with many probabilistic models etc., they often participate in their own type trees while acting as probability distributions at the same time. With this view point, it would be great if `IsSampleable` is a trait rather than an abstract type. Sorry for being late to the discussion, I didn’t realize this was potentially important to my application until now 😕

A type-based workaround would be to implement wrapper types for everything that can also act like a probability distribution

> [@cscherrer](#):
>
> Critically, there are no constraints on either of these at the type level. If you want `X` to be something other than `<: Real` or `<: Array` , go for it. No problem.

This is great, then one could stick `Particles` in there, or any other form of prior distribution etc.

> [@Amin\_Yahyaabadi](#):
>
> julia\> Uniform(1,2) Uniform{Float64}(a=1, b=2)

I would parameterize even harder to allow different types for mean and variance, consider e.g., that one wants to stick other distributions in there, such as `Uniform(Normal(...), Gamma(...))`

> [@cscherrer](#):
>
> function Uniform(a::A, b::B) where {A \<: Real, B \<: Real} T = promote\_type(A, B) return Uniform{Tuple{T,T}, T}(T(a), T(b)) end

This suggestion would solve that problem, but may result in `T=Any` if one of the types is a `Distribution` whereas the other one is a `VariationalAutoencoder`, but maybe this is a pie-in-the-sky kind of application for which some dynamic dispatch would be acceptable?

> [@cscherrer](#):
>
> it’s easy to add a `rand(SomeDistribution{P,X::CuArray})` method that specializes for GPU

That absolutely sounds worth considering

---

<div class="post-metadata">

**Author:** ![mschauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mschauer/32/13946_2.png) [@mschauer](https://discourse.julialang.org/u/mschauer)\
**Post date:** [January 27, 2020, 9:53am UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339/42 "2020-01-27T09:53:12Z")

</div>

> > In total
> > 
> > ```julia
> > abstract type Sampleable{X} end
> > 
> > ```
> > 
> > should be more than enough and even _Edit: that is too much_

> What if I have a type that is Sampleable, but must participate in a completely different part of the type tree?

You are absolutely right! I made these comments in situations where even more structure such as dimension, type of samples, type of probability, discreteness, uni- or multivariate was supposed to be baked into the abstract type. Having no abstract type, but a trait is very much preferable (or even no trait at all, just a set of functions with agreed behaviour, e.g. like we do it for iterators).

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [January 27, 2020, 3:53pm UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339/43 "2020-01-27T15:53:22Z")

</div>

> [@baggepinnen](#):
>
> With this view point, it would be great if `IsSampleable` is a trait rather than an abstract type.

The current implementation is trait-based:

```julia
@trait IsMeasure{M,X} >: HasRand{M,X} where {X = eltype(M)} begin
    rand :: [M] => eltype(M)
end

```

> [@baggepinnen](#):
>
> A type-based workaround would be to implement wrapper types for everything that can also act like a probability distribution

I’ve actually started to wonder if we might need this anyway. Signed measures form a vector space over the reals, where addition is superposition. But if `Distribution`s are also `Measure`s, we’re getting into type piracy. So maybe we need a wrapper for working with external structures that happen to also be `Measure`s?

> [@baggepinnen](#):
>
> I would parameterize even harder to allow different types for mean and variance, consider e.g., that one wants to stick other distributions in there, such as `Uniform(Normal(...), Gamma(...))`

I see the appeal of this, but going in this direction can cause a lot of problems. It’s important to distinguish a _type_ from a _measure on that type_. On the other hand, some measures like a Dirichlet process really do take another measure as a parameter.

But you could build something like you describe as a mixture of uniforms over a product measure.

> [@baggepinnen](#):
>
> This suggestion would solve that problem, but may result in `T=Any` if one of the types is a `Distribution` whereas the other one is a `VariationalAutoencoder` , but maybe this is a pie-in-the-sky kind of application for which some dynamic dispatch would be acceptable?

We definitely want to avoid dynamic dispatch if possible, ideally without preventing users from extending in this direction.

> [@mschauer](#):
>
> You are absolutely right! I made these comments in situations where even more structure such as dimension, type of samples, type of probability, discreteness, uni- or multivariate was supposed to be baked into the abstract type. Having no abstract type, but a trait is very much preferable (or even no trait at all, just a set of functions with agreed behaviour, e.g. like we do it for iterators).

Right, I was looking at this last option a bit last night. Overall, we have

- Hierarchy of abstract types, with wrappers for external interface
- CanonicalTraits
- Holy traits
- Duck typing

I expect we’ll end up with some mixture of maybe two of these, still not entirely clear

---

<div class="post-metadata">

**Author:** ![mschauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mschauer/32/13946_2.png) [@mschauer](https://discourse.julialang.org/u/mschauer)\
**Post date:** [January 27, 2020, 4:47pm UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339/44 "2020-01-27T16:47:00Z")

</div>

Yeah, for the sampling part, a variation of the iterator protocol would be very natural,  
querying new iterators from an iterator and new samples from a sampler is almost the same, the main differences is that there are two ingredients, the sampler and the random number generator. @rfourquet has some thoughts on this as well.

---

<div class="post-metadata">

**Author:** ![rfourquet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rfourquet/32/3610_2.png) [@rfourquet](https://discourse.julialang.org/u/rfourquet)\
**Post date:** [March 8, 2020, 11:52am UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339/45 "2020-03-08T11:52:38Z")

</div>

Thanks for the ping `@mschauer`. I’ve followed this discussion not closely as I would like, in big part because I’m not familiar with the `Distribution` package nor do I need work with advanced distributions mathematically.

I wish to have time to understand better the needs for `Distributions2`, but one thing which would be great (IMHO) is that the design allows to put a basic `Distribution` type in `Base` and define there simple distributions like `Normal` or `Uniform`, which is also useful for the package for more advanced distributions. I tried to include such a type at one point, but one of the [feedback](https://github.com/JuliaLang/julia/pull/24912#issuecomment-354355919) was that it can’t really be done independently of the `Distributions` package.

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [March 8, 2020, 12:18pm UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339/46 "2020-03-08T12:18:18Z")

</div>

adding constraints?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [March 8, 2020, 12:43pm UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339/47 "2020-03-08T12:43:48Z")

</div>

> [@rfourquet](#):
>
> put a basic `Distribution` type in `Base` and define there simple distributions like `Normal` or `Uniform`

At the moment Distributions.jl has reached the point in its lifecycle that some people have issues with the interface/implementation, and are experimenting with various approaches and interface conventions (eg see this topic, and many others). It is unclear what the common ground will be, and whether there will be a consensus that leads to a refactoring of Distributions.jl, a fork, or an alternative package. There is no consensus about what a `Distribution` is, how to define the interface to be extensible (eg what if I want a distribution over arbitrary objects? is it beneficial to consider something I can only sample from using MCMC a distribution for the purposes of this type?), about implementation choices (type hierarchies? traits?).

Because of this, I disagree with putting anything to do with distributions this into `Base`, or even `Random`. I think it is better to decouple all development from Julia releases whenever possible, so that interfaces can evolve freely.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 8, 2020, 12:57pm UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339/48 "2020-03-08T12:57:53Z")

</div>

Distributions 2: Redistributed

Would be a good movie title.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [March 8, 2020, 8:16pm UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339/49 "2020-03-08T20:16:09Z")

</div>

see [Home · Kaleido.jl](https://tkf.github.io/Kaleido.jl/dev/#Kaleido.constraining)

---

<div class="post-metadata">

**Author:** ![rfourquet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rfourquet/32/3610_2.png) [@rfourquet](https://discourse.julialang.org/u/rfourquet)\
**Post date:** [March 23, 2020, 12:27pm UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339/50 "2020-03-23T12:27:57Z")

</div>

Sorry for the naive question: is it critical to have a two-level hierarchy with `Distribution <: Sampleable` ? Is the definition of `Sampleable` roughly “has `rand`” and `Distribution` “also has probability distribution” like in `Distributions.jl` ? Again I almost never used this package, but is it necessary to dispatch on these two types, or just erroring when a method is not defined for a `Sampleable` would be enough?

---

<div class="post-metadata">

**Author:** ![danmackinlay](https://avatars.discourse-cdn.com/v4/letter/d/85f322/32.png) [@danmackinlay](https://discourse.julialang.org/u/danmackinlay)\
**Post date:** [April 26, 2020, 9:56pm UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339/51 "2020-04-26T21:56:46Z")

</div>

FWIW @cscherrer’s API design looks good to my mind (nice work!). But I’m not clear if this mean that we are gathering momentum to develop his `Measures.jl`, or on a new revision of `Distributions.jl`, or something else again. It seems, if I may summarise, that we have consensus that `Distributions.jl` is approaching end of life for the current API at least, but have not yet converged on the step of replacing it…? Is simply submitting pull requests the way forward, and if so, where?

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [April 27, 2020, 2:33am UTC](https://discourse.julialang.org/t/reworking-distributions-jl/32339/52 "2020-04-27T02:33:35Z")

</div>

> [@danmackinlay](#):
>
> FWIW @cscherrer’s API design looks good to my mind (nice work!).

Thanks @danmackinlay! I haven’t looked at this in a while, and you’ve inspired me to have another look over it 🙂

I haven’t been involved lately with discussions about the future of Distributions.jl. But in general, libraries become harder to change as they get wider adoption. I think the best path forward is to focus on wrapping all of Distributions.jl. Then there’s no debate needed, people can use whichever library suits them.

I think I’m going to simplify this quite a bit. I had been using @thautwarm’s `CanonicalTraits.jl` library, which really has some cool ideas. But I think it doesn’t lean too much on traits anyway, and a more standard route will make it easier for others to contribute.

[Previous page](https://discourse.julialang.org/t/reworking-distributions-jl/32339.md?page=2)