# The cognitive bias of low-probability events

**URL:** <https://discourse.julialang.org/t/the-cognitive-bias-of-low-probability-events/43867>\
**Category:** Offtopic\
**Created:** [July 29, 2020, 10:57am UTC](https://discourse.julialang.org/t/the-cognitive-bias-of-low-probability-events/43867 "2020-07-29T10:57:55Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [July 29, 2020, 10:57am UTC](https://discourse.julialang.org/t/the-cognitive-bias-of-low-probability-events/43867/1 "2020-07-29T10:57:55Z")

</div>

Just a tough reading the news about Madonna endorsing hydroxychloroquine.

The problem is when one has to judge the difference between distributions that differ in the tail, like a Bernoulli with p=0.95 and p=0.99.

People, individually, don’t have experience for that, only “big trials” can distinguish between a treatment for coronavirus that leads 95% to recover and a placebo where 90% recovers.  
The same for homeopathy: flu/colds are largely benign, it is really hard to judge the progress of a “real” medicine.

In all these cases fake treatments that exploit these cognitive bias can emerge.

---

<div class="post-metadata">

**Author:** ![anon37204545](https://avatars.discourse-cdn.com/v4/letter/a/439d5e/32.png) [@anon37204545](https://discourse.julialang.org/u/anon37204545)\
**Post date:** [July 29, 2020, 11:09am UTC](https://discourse.julialang.org/t/the-cognitive-bias-of-low-probability-events/43867/2 "2020-07-29T11:09:02Z")

</div>

Just like the (in)famous machine learning example. ML model that identifies tumors has an accuracy of 85%. But 90% of samples have tumor, so a simpler model which just returns true has an accuracy of 90%.

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [July 29, 2020, 11:19am UTC](https://discourse.julialang.org/t/the-cognitive-bias-of-low-probability-events/43867/3 "2020-07-29T11:19:08Z")

</div>

The severest problem is the decreasing trust in science, which is fueled by well known leaders these days.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [July 29, 2020, 11:39am UTC](https://discourse.julialang.org/t/the-cognitive-bias-of-low-probability-events/43867/4 "2020-07-29T11:39:22Z")

</div>

While I agree that in general you should not take medical advice from pop stars (doh), it is interesting to just plot the numbers. Specifically, let’s take a model of n Bernioulli(\alpha) draws with a flat prior, which ends up as a Beta posterior.

The graph below shows the log pdf of the success rate under a “true” \alpha of 0.95 (red) and 0.99 (blue) using dashed lines for n = 300, while the solid lines are the log posteriors. With only a few hundred draws, you can discriminate between these two pretty well under general circumstances. Of course in real life you have covariates, confounding variables, etc.

 ![x](https://global.discourse-cdn.com/julialang/original/3X/c/0/c0821dcdfef3f5e0c28ea8e65775d08342b82b7c.png)

> **code for plot**
>
> ```julia
> using PGFPlotsX, Distributions
> function plot_beta(color, α, n)
> x = range(0.9, 1; length = 200)
> @pgf (Plot({ color = color, no_marks },
> Table(x, logpdf.(Ref(Beta(α * n, (1 - α) * n)), x))),
> VLine({ color = color, dotted }, α),
> Plot({ color = color, dashed },
> Table(x, logpdf.(Ref(Binomial(n, α)), round.(Int, n .* x)), x)))
> 
> end
> @pgf Axis({ xlabel = "α", ylabel = "log posterior", ymajorgrids, ymin = -20 },
> plot_beta("red", 0.95, 300)...,
> plot_beta("blue", 0.99, 300)...)
> 
> ```

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [July 29, 2020, 12:40pm UTC](https://discourse.julialang.org/t/the-cognitive-bias-of-low-probability-events/43867/5 "2020-07-29T12:40:57Z")

</div>

…I think 300 draws of whatever medical condition are behind the experience of most individuals 🙂 🙂

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [July 29, 2020, 1:12pm UTC](https://discourse.julialang.org/t/the-cognitive-bias-of-low-probability-events/43867/6 "2020-07-29T13:12:57Z")

</div>

Sure, but inference about most medical conditions based on one’s personal experience is pretty nonsensical as the sample size is very small (except for recurring conditions like seasonal colds etc). I don’t think that’s the relevant baseline here.

Even if they have no formal training in statistics, I guess most people who are otherwise functionally literate would do OK with simple tabulations when sample sizes are comparable to a typical RCT, even if the probabilities are near 0.99.

IMO the problematic range is near 10^{-3} or 10^{-4}; for example this is the annual probability for a lot of eminently preventable household accidents involving children or adults. But these are low enough so that a typical person may not personally know of a case in their own social network.

Of course as @oheil pointed out, this whole thing may have little to do with the pseudoscience surrounding COVID-19. My point was simply that I disagree with your claim that

> [@sylvaticus](#):
>
> only “big trials” can distinguish between a treatment for coronavirus that leads 95% to recover and a placebo where 90% recovers