# Simplex transforms in Bijectors.jl (sorry)

**URL:** <https://discourse.julialang.org/t/simplex-transforms-in-bijectors-jl-sorry/56372>\
**Category:** Probabilistic Programming\
**Created:** [March 2, 2021, 11:12pm UTC](https://discourse.julialang.org/t/simplex-transforms-in-bijectors-jl-sorry/56372 "2021-03-02T23:12:52Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [March 2, 2021, 11:12pm UTC](https://discourse.julialang.org/t/simplex-transforms-in-bijectors-jl-sorry/56372/1 "2021-03-02T23:12:52Z")

</div>

I’d like to understand the ideas behind the way Bijectors handles simplices.

First, something I think I understand, for comparison:

# TransformVariables

```julia
julia> using TransformVariables

julia> t = UnitSimplex(4)
UnitSimplex(4)

julia> t(ones(3))
4-element Vector{Float64}:
 0.4753668864186717
 0.3022499950414722
 0.16257508655105501
 0.05980803198880098

```

A space with four numbers that add to one has three degrees of freedom, so the input here takes `ones(3)`. Giving it more throws an error:

```julia
julia> t(ones(4))
ERROR: ArgumentError: dimension(t) == length(x) must hold. Got
dimension(t) => 3
length(x) => 4
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/ArgCheck/5xEDR/src/checks.jl:243 [inlined]
 [2] transform
   @ ~/.julia/packages/TransformVariables/N5VYB/src/generic.jl:228 [inlined]
 [3] (::UnitSimplex)(x::Vector{Float64})
   @ TransformVariables ~/.julia/packages/TransformVariables/N5VYB/src/generic.jl:123
 [4] top-level scope
   @ REPL[14]:1 

```

This is all as I would expect.

# Bijectors

The corresponding code in Bijectors seems to be

```julia
julia> using Bijectors

julia> t = Bijectors.SimplexBijector()
Bijectors.SimplexBijector{1, true}()

julia> inv(t)(ones(4))
4-element Vector{Float64}:
 0.4753668864186717
 0.30224999504147226
 0.16257508655105507
 0.059808031988801025

```

It’s the same result (up to what looks like maybe rounding error). But it requires one more dimension than is needed! To confirm this,

```julia
julia> inv(t)(ones(4)) |> t
4-element Vector{Float64}:
 1.0000000000000002
 1.0
 1.0
 0.0

```

This shows that the `t` is not a bijection. Given that it’s possible to set up so it _is_ a bijection, what’s behind this design choice?

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [March 3, 2021, 3:27am UTC](https://discourse.julialang.org/t/simplex-transforms-in-bijectors-jl-sorry/56372/2 "2021-03-03T03:27:43Z")

</div>

I think it’s mostly historical. In Turing, we assume the transformed and untransformed domains have the same dimension (for simplicity I believe). Bijectors was born out of Turing’s code base and it’s used primarily there so it had to be made compatible. I am starting to appreciate the conceptual simplicity of separating the dimension of the input from that of the output more though and so it seems the other developers of Bijectors as well. For example, see the comments in [https://github.com/TuringLang/Bijectors.jl/pull/168](https://github.com/TuringLang/Bijectors.jl/pull/168) and [https://github.com/TuringLang/Bijectors.jl/issues/58](https://github.com/TuringLang/Bijectors.jl/issues/58). However, perhaps the biggest obstacle to this change would be making Turing/DynamicPPL compatible with it. This would require a more significant effort.

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [March 3, 2021, 3:58am UTC](https://discourse.julialang.org/t/simplex-transforms-in-bijectors-jl-sorry/56372/3 "2021-03-03T03:58:51Z")

</div>

That helps a lot, thanks @mohamed82008

---

<div class="post-metadata">

**Author:** ![mrVeng](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mrveng/32/8836_2.png) [@mrVeng](https://discourse.julialang.org/u/mrVeng)\
**Post date:** [April 4, 2021, 11:59am UTC](https://discourse.julialang.org/t/simplex-transforms-in-bijectors-jl-sorry/56372/4 "2021-04-04T11:59:10Z")

</div>

I am using Bijectors for almost all my work in Bayesian settings, it is a really nice and useful package!

But I also have questions regarding possible dimensionality reductions. If you work with larger Covariance Matrices, or many Simplices, the number of parameters in the transformed dimension could be quite considerably lower if the dimension would not be kept constant.

I have not looked at the internals of Turing, but do you keep the dimension the same also for your MCMC sampler? If yes, do you experience performance degradation in this case? I.e., the posterior covariance adaption could be done in a lower dimension, and the turn statistics for NUTS are based on the momentum, which depend on the parameter dimension.

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [April 4, 2021, 12:14pm UTC](https://discourse.julialang.org/t/simplex-transforms-in-bijectors-jl-sorry/56372/5 "2021-04-04T12:14:09Z")

</div>

I think for more general use (especially outside Turing) we need something like Bijectors but without the dimensionality issues you describe. Maybe some of the Bijectors should be ported to TransformVariables?

Orthogonally, I appreciate that TV is less entwined with Distributions.jl; that makes it easier for me to use from MeasureTheory.jl.

---

<div class="post-metadata">

**Author:** ![mrVeng](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mrveng/32/8836_2.png) [@mrVeng](https://discourse.julialang.org/u/mrVeng)\
**Post date:** [April 4, 2021, 12:31pm UTC](https://discourse.julialang.org/t/simplex-transforms-in-bijectors-jl-sorry/56372/6 "2021-04-04T12:31:13Z")

</div>

In my case, I really appreciate it that Bijectors is so connected to Distributions. If a distribution is contained in the Distributions package, then most likely I can transform/inverse transform it via Bijectors, and I can take gradients via mutliple AD packages thanks to DistributionsAD. I actually would find it amazing if this would all be one package, but that would probably be too big, and supporting many AD backends seems like a lot of work as well.

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [April 4, 2021, 12:41pm UTC](https://discourse.julialang.org/t/simplex-transforms-in-bijectors-jl-sorry/56372/7 "2021-04-04T12:41:51Z")

</div>

It could be fine if it were “available but not required”. Bijectors re-exports so many names from Distributions, which collide with MeasureTheory. A lot of this is on me, but I don’t see a good way for a Distributions alternative to have no names in common with it. Also, I haven’t brought this up to @torfjelde and @mohamed82008 before, because the dimensionality problem made it a no-go for me anyway.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [April 4, 2021, 12:56pm UTC](https://discourse.julialang.org/t/simplex-transforms-in-bijectors-jl-sorry/56372/8 "2021-04-04T12:56:44Z")

</div>

> [@mrVeng](#):
>
> If a distribution is contained in the Distributions package, then most likely I can transform/inverse transform it via Bijectors, and I can take gradients via mutliple AD packages thanks to DistributionsAD.

I think these features are pretty orthogonal though — you can only transform (log) pdfs for multivariate distributions in any case, and the ADability of the transformation and the original pdf need not be coupled at all.

---

<div class="post-metadata">

**Author:** ![torfjelde](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/torfjelde/32/206542_2.png) [@torfjelde](https://discourse.julialang.org/u/torfjelde)\
**Post date:** [April 4, 2021, 6:32pm UTC](https://discourse.julialang.org/t/simplex-transforms-in-bijectors-jl-sorry/56372/9 "2021-04-04T18:32:01Z")

</div>

Weell, one might also say that the implementations you demonstrate above isn’t bijective since the output isn’t constrained in anyway, e.g. I could pass the inverse of the transform any length 4 vector, not just those in the subspace.

But I also want to point out that there’s really nothing that fundamentally stops us from making `SimplexBijector` transform to space with `length(input) + 1`. We could just add a type-parameter that specifies whether or not this should be done, and implement the transforms.

And regarding the re-export, that is def something we could have a discussion about doing. I’m not attached to that at all (not a big fan even) 🙂

Also, IMO Bijectors.jl is ripe for a re-design. I’m not a big fan of the nasty dimensionality that is encoded in the bijectors (as I mention in one of the issues raised above), etc. So very open to ideas/suggestions/etc. about the design.

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [April 4, 2021, 6:57pm UTC](https://discourse.julialang.org/t/simplex-transforms-in-bijectors-jl-sorry/56372/10 "2021-04-04T18:57:17Z")

</div>

> [@torfjelde](#):
>
> Weell, one might also say that the implementations you demonstrate above isn’t bijective since the output isn’t constrained in anyway, e.g. I could pass the inverse of the transform any length 4 vector, not just those in the subspace.

That’s true, at some point it becomes a contract with the user. Maybe the biggest difference is that this is a contract _not to_ do something (“Please don’t pass values outside [0,1]”), which can be easier to deal with that a contract _to_ do something (“Please add extra code to ignore the final element in this particular case”).

Thanks for the suggestions. I’d really love to be able to use Bijectors from MeasureTheory. I have some code for [expressing pushforwards and pullbacks using TransformVariables](https://github.com/cscherrer/MeasureTheory.jl/blob/master/src/combinators/transforms.jl), and it would be great to have this for Bijectors as well. Maybe at some point these can merge or interoperate more easily, but for now neither has the complete functionality, so we need both.