# Gentler transformation from real to positive

**URL:** <https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695>\
**Category:** Optimization (Mathematical)\
**Tags:** question\
**Created:** [January 23, 2020, 11:13am UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695 "2020-01-23T11:13:00Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 23, 2020, 11:13am UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/1 "2020-01-23T11:13:00Z")

</div>

I am solving an equilibrium of an economic model for prices p \> 0. I need to do this robustly for “wacky” parameter values when I am fitting the model to the data in an outer loop.

I have coded up a market clearing residual function `market_residuals(model, p)`.

I found that I can do this by, roughly,

```julia
NLsolve.nlsolve(x0; autodiff = :forward, iterations = 200,
                method = :newton, ftol = residual_tol) do x
    p = exp.(x)
    market_residuals(model, p)
end

```

The problem is that the Newton solver can very easily lead to arguments which are invalid when transformed, as

```julia
julia> exp(750)
Inf

julia> exp(-750)
0.0

```

Scaling `x` does not help as the Newton method is scale invariant.

Is there a “gentler” transformation from \Bbb{R} to \Bbb{R}\_+ that one could recommend?

---

<div class="post-metadata">

**Author:** ![Jakub\_Wronowski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jakub_wronowski/32/204030_2.png) [@Jakub\_Wronowski](https://discourse.julialang.org/u/Jakub_Wronowski)\
**Post date:** [January 23, 2020, 11:37am UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/2 "2020-01-23T11:37:02Z")

</div>

Maybe I don’t understand something, but can’t your function operate on logarithm of the price, so then simply skip call to exp?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 23, 2020, 11:38am UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/3 "2020-01-23T11:38:12Z")

</div>

That just puts the numerical problem somewhere else.

---

<div class="post-metadata">

**Author:** ![piever](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/piever/32/1815_2.png) [@piever](https://discourse.julialang.org/u/piever)\
**Post date:** [January 23, 2020, 12:05pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/4 "2020-01-23T12:05:35Z")

</div>

Maybe a trick is to use the identity (plus some offset) on positive numbers, and on negative numbers continue with something that plateaus, in such a way that the “junction” is still differentiable. For example the following:

f(x) = \begin{cases} x+\frac{\pi}{2} &\text{ if } x \>0\\ \mathrm{arctan}(x) + \frac{\pi}{2} &\text{ otherwise.} \end{cases}

It should still be continuously differentiable, because the derivative of \mathrm{arctan} in 0 is 1.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 23, 2020, 12:33pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/5 "2020-01-23T12:33:07Z")

</div>

> [@Tamas\_Papp](#):
>
> Is there a “gentler” transformation from ℝ to ℝ⁺ that one could recommend?

p²?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 23, 2020, 12:44pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/6 "2020-01-23T12:44:29Z")

</div>

I should have specified, but I need a bijection (continuous, monotone; does not need to be continuously differentiable but that would be nice, too).

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [January 23, 2020, 12:44pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/7 "2020-01-23T12:44:57Z")

</div>

Sigmoid takes you from R to in the interval between 0 and 1. Perhaps useful?

---

<div class="post-metadata">

**Author:** ![Jakub\_Wronowski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jakub_wronowski/32/204030_2.png) [@Jakub\_Wronowski](https://discourse.julialang.org/u/Jakub_Wronowski)\
**Post date:** [January 23, 2020, 12:53pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/8 "2020-01-23T12:53:38Z")

</div>

How to calculate sigmoid without exp(x)?

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [January 23, 2020, 12:57pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/9 "2020-01-23T12:57:02Z")

</div>

You can calculate it using either exp(x) or exp(-x) so you can choose one of them that works. But if one is too big I guess the other one would be too small…

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [January 23, 2020, 1:02pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/10 "2020-01-23T13:02:56Z")

</div>

How about a hyperbola:

```julia
f(x, slope, abcissa) = slope/2 * x + sqrt(slope^2/4 * x^2 + abcissa^2)

```

increasing `abcissa` should move the `x` where `f(x)==0` to smaller values.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 23, 2020, 1:07pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/11 "2020-01-23T13:07:47Z")

</div>

> [@Tamas\_Papp](#):
>
> I should have specified, but I need a bijection (continuous, monotone; does not need to be continuously differentiable but that would be nice, too).

Why? Finding a root of f(p²) means that every root is doubled, but if Newton converges to a negative root you can just flip the sign.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [January 23, 2020, 1:53pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/12 "2020-01-23T13:53:35Z")

</div>

What about using log(1+exp(x)). Is monotonic, smooth, and never blows up

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 23, 2020, 2:21pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/13 "2020-01-23T14:21:55Z")

</div>

> [@Oscar\_Smith](#):
>
> What about using log(1+exp(x)). Is monotonic, smooth, and never blows up

It still overflows because of the intermediate `exp(x)` calculation: `log(1+exp(710)) === Inf`. It’s also subject to underflow: `log(1+exp(-37)) == 0.0`. It would be better to use `log1p(exp(x))`, but even that eventually underflows: `log1p(exp(-746)) == 0.0`. You could define your own special-function implementation for `f(x) = log1p(exp(x))`, rewritten to avoid spurious overflow — for example, `f(x) = x > 0 ? x + log1p(exp(-x)) : log1p(exp(x))` suffices and is equivalent to `log(1+exp(x))` in exact arithmetic — but the underflow is unavoidable since `f(-746) ≈ 1.03e-324` is not representable as a `Float64`.

I still don’t understand why you need a bijection in a root-finding context, e.g. why `x -> x^2` is not acceptable, since you can flip a negative solution to a positive one _a posteriori_ as I commented [above](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/11).

---

<div class="post-metadata">

**Author:** ![cshen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cshen/32/217287_2.png) [@cshen](https://discourse.julialang.org/u/cshen)\
**Post date:** [January 23, 2020, 2:33pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/14 "2020-01-23T14:33:12Z")

</div>

> [@Tamas\_Papp](#):
>
> I should have specified, but I need a bijection (continuous, monotone; does not need to be continuously differentiable but that would be nice, too).

> [@Tamas\_Papp](#):
>
> Scaling `x` does not help as the Newton method is scale invariant.

Translating by the minimum element of the set?

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [January 23, 2020, 2:49pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/15 "2020-01-23T14:49:27Z")

</div>

If the error is only happening inside the Newton solver (ie. it is not the actual market clearing price) could you do a small check beforehand to bound the admissible set of `x` and then use a bounded solver instead?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 23, 2020, 7:54pm UTC](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/16 "2020-01-23T19:54:59Z")

</div>

> [@stevengj](#):
>
> I still don’t understand why you need a bijection in a root-finding context, e.g. why `x -> x^2` is not acceptable, since you can flip a negative solution to a positive one _a posteriori_ as I commented [above](https://discourse.julialang.org/t/gentler-transformation-from-real-to-positive/33695/11).

I usually try to do these domain transformations with bijections in Bayesian inference, but I realized that here there is no compelling reason for that, so I will experiment with your suggestion. Thanks for bringing it up again.