# \[ANN\] MonteCarloMeasurements.jl

**URL:** <https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021>\
**Category:** Package Announcements\
**Tags:** announcement, monte-carlo, measurements\
**Created:** [April 11, 2019, 7:21am UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021 "2019-04-11T07:21:14Z")\
**Posts on this page:** 15\
**Page:** 2

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [April 25, 2019, 5:53am UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/21 "2019-04-25T05:53:40Z")

</div>

Thanks for the tip! I created a [WIP PR](https://github.com/JuliaDiffEq/DiffEqBase.jl/pull/212), WIP since it’s not quite working yet… 😕

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [May 8, 2019, 9:18am UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/22 "2019-05-08T09:18:22Z")

</div>

Some updates:

We now support

- Unscented transform, by initializing particles as [sigma points](https://en.wikipedia.org/wiki/Unscented_transform#Sigma_points)
- [latin hypercube sampling](https://en.wikipedia.org/wiki/Latin_hypercube_sampling)
- Easier way to overload new functions `register_primitive(yourfunc)`

Further, the documentation in the [readme](https://github.com/baggepinnen/MonteCarloMeasurements.jl) is extended quite a lot

# New examples

## [Robust probabilistic optimization](https://github.com/baggepinnen/MonteCarloMeasurements.jl/blob/master/examples/robust_controller_opt.jl)

Here, we use MonteCarloMeasurements to perform [robust optimization](https://en.wikipedia.org/wiki/Robust_optimization). With robust and probabilistic, we mean that we place some kind of bound on a quantile of an uncertain value, or otherwise make use of the probability distribution of some value that depend on the optimized parameters.

The application we consider is optimization of a PID controller. Normally, we are interested in controller performance and robustness against uncertainty. The robustness is often introduced by placing an upper bound on the, so called, sensitivity function. When the system to be controlled is parameterized by `Particles`, we can penalize both variance in the performance measure as well as the 90:th quantile of the maximum of the sensitivity function. This example illustrates how easy it is to incorporate probabilistic constrains or cost functions in an optimization problem using `Particles`.

## [Autodiff and Robust optimization](https://github.com/baggepinnen/MonteCarloMeasurements.jl/blob/master/examples/autodiff_robust_opt.jl)

Another example using MonteCarloMeasurements to perform [robust optimization](https://en.wikipedia.org/wiki/Robust_optimization), this time with automatic differentiation. We use Optim.jl to solve a linear program with probabilistic constraints using 4 different methods, two gradient free, one first-order and one second-order method.

## [Control systems](https://github.com/baggepinnen/MonteCarloMeasurements.jl/blob/master/examples/controlsystems.jl)

This example shows how to simulate control systems (using [ControlSystems.jl](https://github.com/JuliaControl/ControlSystems.jl)  
) with uncertain parameters. We calculate and display Bode diagrams, Nyquist diagrams and time-domain responses. We also illustrate how the package [ControlSystemIdentification.jl](https://github.com/baggepinnen/ControlSystemIdentification.jl) interacts with MonteCarloMeasurements to facilitate the creation and analysis of uncertain systems.

We also perform some limited benchmarks.

## [Lantin Hypercube Sampling](https://github.com/baggepinnen/MonteCarloMeasurements.jl/blob/master/examples/lhs.jl)

We show how to initialize particles with LHS and how to make sure the sample gets the desired moments. We also visualize the statistics of the sample.

## [How MC uncertainty propagation works](https://github.com/baggepinnen/MonteCarloMeasurements.jl/blob/master/examples/transformed_densities.jl)

We produce the first figure in this readme and explain in visual detail how different forms of uncertainty propagation propagates a probability distribution through a nonlinear function.

# New benchmark

The benchmark results below come from [`examples/controlsystems.jl`](https://github.com/baggepinnen/MonteCarloMeasurements.jl/blob/master/examples/controlsystems.jl) The benchmark consists of calculating the Bode curves for a linear system with uncertain parameters

```julia
w = exp10.(LinRange(-1,1,200)) # Frequency vector
p = 1 ± 0.1
ζ = 0.3 ± 0.1
ω = 1 ± 0.1
G = tf([p*ω], [1, 2ζ*ω, ω^2])
t1 = @belapsed bode($G,$w)
   ⋮

```

| Benchmark | Results |
| --- | --- |
| Time with 500 particles | 11.5937ms |
| Time with regular floating point | 0.3456ms |
| Time with Measurements | 0.5794ms |
| Time with 100 static particles | 1.8212ms |
| Time with static sigmapoints | 0.7252ms |
| 500×floating point time | 172.7780ms |
| Speedup particles vs. Manual | 14.9x |
| Slowdown particles vs. Measurements | 20.0x |
| Slowdown static vs. Measurements | 3.1x |
| Slowdown sigma vs. Measurements | 1.3x |

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [May 16, 2019, 10:33pm UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/23 "2019-05-16T22:33:57Z")

</div>

Very cool package! Are there any plans to implement multi-threading as an alternative to `pmap`?

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [May 17, 2019, 4:33am UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/24 "2019-05-17T04:33:48Z")

</div>

Thanks!

I have been considering it, but I think most functions would not benefit very much from naive threading. The speedup you get by using SIMD on almost every internal operation together with not redoing computation on deterministic input arguments is quite hard to beat. Possibly a blocking strategy where the computations are divided into exactly `nthreads` blocks could make use of both threads and SIMD. I’ll think about it 🙂

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 17, 2019, 11:52am UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/25 "2019-05-17T11:52:50Z")

</div>

Instead of just doing multithreading, can we try and get this finished up for automated GPU goodies? 🙂 [WIP: Automatic GPU-parallelism of swarms by ChrisRackauckas · Pull Request #6 · baggepinnen/MonteCarloMeasurements.jl · GitHub](https://github.com/baggepinnen/MonteCarloMeasurements.jl/pull/6) . Is there another place in the code which is calling maps?

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [May 17, 2019, 11:56am UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/26 "2019-05-17T11:56:58Z")

</div>

Yeah, I can’t really help out with that, can never become friends with the gpu drivers… 😕  
I would be nice to have though.

`map` is used in the macro `@bymap`, which acts as a fallback for a top-level function `f` that fails when called with particles `f(p1, p2, ...)`.

`@bymap f(p1, p2, ...)` just transforms the call to a map over `f` and assembles the result into a `Particles` in the end.

I don’t think `@bymap/@bypmap` needs GPU support though.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [December 19, 2019, 4:56am UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/27 "2019-12-19T04:56:50Z")

</div>

New version of Julia, new version of [MonteCarloMeasurements](https://github.com/baggepinnen/MonteCarloMeasurements.jl) 🙂

Since I last updated this post, there has been quite a lot of usability changes, some highlights

- Documentation has been improved and moved from the README to a proper website. [Home · MonteCarloMeasurements Documentation](https://baggepinnen.github.io/MonteCarloMeasurements.jl/dev/)
- We have a number of new [contributers](https://github.com/baggepinnen/MonteCarloMeasurements.jl/graphs/contributors), thanks to all of you 🙏
- Plotting is now more robust
- Discrete distributions (i.e., distributions where a sample is an `Int` or a `Bool` etc.) work much better, thanks to @simeonschaub
- `Particles` and `missing` should now play nicely together, thanks @simeonschaub.
- Performance has increased, in the updated benchmark below by about 30% (not counting improvements performance improvements to julia), details [here](https://baggepinnen.github.io/MonteCarloMeasurements.jl/dev/#Benchmark-1).
- Conversion from particles and arrays of particles to regular arrays is much smoother, thanks to @sethaxen
- MonteCarloMeasurements have been given support in the visualization library [ArviZ.jl](https://github.com/arviz-devs/ArviZ.jl), thanks to @sethaxen
- @cscherrer has written a nice blog post on how to make use of `Particles` for variational inference. MCM has also been incorporated into [Soss.jl](https://github.com/cscherrer/Soss.jl)
- The slack channel `#monte-carlo-measurements` have been created for discussions related to the package.
- Both Zygote and ForwardDiff can differentiate stuff that contains Particles, [robust optimization example](https://github.com/baggepinnen/MonteCarloMeasurements.jl/blob/master/examples/autodiff_robust_opt.jl).
- Better handling of `NamedTuple` with particle fields.

#### Benchmark

The new results below are obtained with a faster computer and newer version of Julia. The relative timings in the bottom are however indicative of some nice performance improvements:

| Benchmark | Old Results | New Results |
| --- | --- | --- |
| Time with 500 particles | 11.5937ms | 1.3632ms |
| Time with regular floating point | 0.3456ms | 0.0821ms |
| Time with Measurements.jl | 0.5794ms | 0.1132ms |
| Time with 100 static particles | 1.8212ms | 0.2375ms |
| Time with static sigmapoints | 0.7252ms | 0.0991ms |
| 500×floating point time | 172.7780ms | 41.0530ms |
| Speedup particles vs. Manual | 14.9x | 30.1x |
| Slowdown particles vs. Measurements.jl | 20.0x | 12.0x |
| Slowdown static vs. Measurements.jl | 3.1x | 2.1x |
| Slowdown sigma vs. Measurements.jl | 1.3x | 0.9x |

---

<div class="post-metadata">

**Author:** ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Post date:** [December 19, 2019, 12:02pm UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/28 "2019-12-19T12:02:46Z")

</div>

I’d like to add support for something like this to BAT.jl, and I was wondering: what’s the relationship (if any) between MonteCarloMeasurements.jl and @carstenbauer’s MonteCarloObservable.jl]([https://github.com/crstnbr/MonteCarloObservable.jl](https://github.com/crstnbr/MonteCarloObservable.jl))? If they’re similar approaches to the problem, I may try to add support for both.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [December 19, 2019, 12:28pm UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/29 "2019-12-19T12:28:28Z")

</div>

Based on the Readme of MonteCarloObservable it seems we are solving different problems.

I’m not sure how you envision support from MCM, it seems BAT is somewhat similar to Soss with some functionality overlapping with Arviz as well, so the support present in those packages may perhaos serve as inspiration for your package?

---

<div class="post-metadata">

**Author:** ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Post date:** [December 19, 2019, 1:04pm UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/30 "2019-12-19T13:04:02Z")

</div>

Ah, yes - I took a closer look at MonteCarloObservable, I think it goes in a different direction.

> BAT is somewhat similar to Soss with some functionality overlapping with Arviz as well, so the support present in those packages may perhaos serve as inspiration for your package

Thanks, I’ll definitely have a look! BAT different from Soss in that it’s about use cases where the user has a custom likelihood distribution, it doesn’t provide a probabilistic programming DSL, but it also doesn’t require a forward model. MonteCarloMeasurements should fit in very well, though.

---

<div class="post-metadata">

**Author:** ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Post date:** [December 19, 2019, 1:07pm UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/31 "2019-12-19T13:07:20Z")

</div>

I also want to see if I can mesh MonteCarloMeasurements with [ValueShapes.jl](https://github.com/oschulz/ValueShapes.jl).

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [December 19, 2019, 1:16pm UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/32 "2019-12-19T13:16:05Z")

</div>

I’m looking forward to see what you have in mind 🙂

---

<div class="post-metadata">

**Author:** ![oschulz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oschulz/32/2998_2.png) [@oschulz](https://discourse.julialang.org/u/oschulz)\
**Post date:** [December 19, 2019, 1:22pm UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/33 "2019-12-19T13:22:33Z")

</div>

> [@baggepinnen](#):
>
> I’m not sure how you envision support from MCM

Ah, what I mean is, I want to make BAT support MCM - as in, being able to convert the BAT MCMC output to MCM Particles.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [December 19, 2019, 1:28pm UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/34 "2019-12-19T13:28:19Z")

</div>

That should be quite straightforward I think, this is what Soss does 😃

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [January 22, 2020, 8:05am UTC](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021/35 "2020-01-22T08:05:44Z")

</div>

There is now a paper outlining MonteCarloMeasurements

> **[MonteCarloMeasurements.jl: Nonlinear Propagation of Arbitrary Multivariate...](https://arxiv.org/abs/2001.07625)**
>
> This manuscript outlines a software package that facilitates working with probability distributions by means of Monte-Carlo methods, in a way that allows for propagation of multivariate probability distributions through arbitrary functions. We...

There is also a [new branch](https://github.com/baggepinnen/MonteCarloMeasurements.jl/tree/gpu) which has support for GPU particles that are effective for sample sizes north of 100k and currently show a speedup of approximately 5-7 times.

[Previous page](https://discourse.julialang.org/t/ann-montecarlomeasurements-jl/23021.md?page=1)
