# Python lmfit takes weights into account, but Julia LsqFit package seems not to

**URL:** <https://discourse.julialang.org/t/python-lmfit-takes-weights-into-account-but-julia-lsqfit-package-seems-not-to/90679>\
**Category:** General Usage\
**Tags:** fit, curve-fitting, lsqfit\
**Created:** [November 23, 2022, 3:20am UTC](https://discourse.julialang.org/t/python-lmfit-takes-weights-into-account-but-julia-lsqfit-package-seems-not-to/90679 "2022-11-23T03:20:16Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![humbleclaw](https://avatars.discourse-cdn.com/v4/letter/h/b3f665/32.png) [@humbleclaw](https://discourse.julialang.org/u/humbleclaw)\
**Post date:** [November 23, 2022, 3:20am UTC](https://discourse.julialang.org/t/python-lmfit-takes-weights-into-account-but-julia-lsqfit-package-seems-not-to/90679/1 "2022-11-23T03:20:16Z")

</div>

Hello everyone.

Using Python I have been able to do non-linear fits with lmfit creating a residual function for a potential mathematical function to fit, just as follows

```julia

def pow_res(pars, E, data, sig):
    values = pars.valuesdict()
    a = values["a"]
    C = values["C"]   
    model = C/E**a
    
    residual = model-data
    chi_sqr = np.divide(residual **2,sig** 2)
    
    return chi_sqr

params = Parameters()
params.add("a", value = a0)
params.add("C", value = C0)

```

I can minimize this function and obtain the pair of parameters `a` and `C`, minimizing the chi\_sqr value where I add the weights of the data . I do this with two kind of weights. In the first case I set weights to have value one to all the data and I obtain the next fit

![image](https://global.discourse-cdn.com/julialang/original/3X/a/e/ae307f09f8bee4bd71750f4f7722b0245d9946f4.png)

which has sense because the weight of the big values is more important than the weight of the lower values, making the fit to go crazy over low values. Then I do the second case: set the weights as half the magnitude of each data value, giving it now the same importance to every value in the data, obtaining the next fit

![image](https://global.discourse-cdn.com/julialang/original/3X/d/9/d95d1ed9724b3d005c312f12f98262d76cd71a59.png)

This was all made with Python. Now, trying the same with Julia I am trying to use the equivalent library to make nonlinear fits, which would be [LsqFit.jl](https://julianlsolvers.github.io/LsqFit.jl/latest/tutorial/). With this library I am doing the same process that I made with python, working with the two cases I mentioned above. At the beginning I obtain the same fitting result with Julia than with Python when I set the weights to one, as you can see in the figure below

 ![image](https://global.discourse-cdn.com/julialang/original/3X/c/8/c8144a70f0f6bfb0bfb4370a0b2f381ee192533e.png)

However, when I test the second case, where I set the weights to half the value of the data, then I obtain the same fitting result than when I have the weights set to one, which, as you saw in the Python result, has no sense

 ![image](https://global.discourse-cdn.com/julialang/original/3X/f/a/fac72311689fe8a851b03a79bd9fb213a7c49f03.png)

The code I am running to do this is

```julia
a0 = (log(absortion[1])-log(last(absortion)))/(log(last(energy))-log(energy[1]))
C0 = absortion[1]*energy[1]^a0
p0 = [C0, a0]

model(E, p) = p[1]./(E.^p[2])

synth_wt = 0.5 .*absortion

fit = lsq.curve_fit(model, energy, absortion, synth_wt, p0)
params = msr.measurement.(fit.param, lsq.stderror(fit))

```

where `synth_wt` are the weights I sinthetically give to the data, `absortion` are the values in the _y_ axis, `energy` the values of the _x_ axis, `model` is the potential function I am trying to fit, and `p0` are the values of initialization of the curve fitting.

I don’t understand why this is happening if I am following the same example of giving the weights to the fitting function that is shown in the [LsqFit.jl](https://julianlsolvers.github.io/LsqFit.jl/latest/tutorial/) documentation. If someone knows better, it would be a great help.

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 23, 2022, 3:48am UTC](https://discourse.julialang.org/t/python-lmfit-takes-weights-into-account-but-julia-lsqfit-package-seems-not-to/90679/2 "2022-11-23T03:48:05Z")

</div>

Does it always give the same answer? The fitting procedure is Levenberg Marquardt which is iterative and may only find a local minimum. Perhaps try a different starting point several times just to rule out this issue.

---

<div class="post-metadata">

**Author:** ![humbleclaw](https://avatars.discourse-cdn.com/v4/letter/h/b3f665/32.png) [@humbleclaw](https://discourse.julialang.org/u/humbleclaw)\
**Post date:** [November 23, 2022, 3:58am UTC](https://discourse.julialang.org/t/python-lmfit-takes-weights-into-account-but-julia-lsqfit-package-seems-not-to/90679/3 "2022-11-23T03:58:25Z")

</div>

Actually, my initialization values give a curve more or less close to the one I should obtain with half-value weights.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/e/b/ebe5fefd7146ca76f7cd9e1739532829e5b318c2.png)

However I tried changing the initialization values to half the ones I used and I obtained the same result again.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/e/6/e607733aaad4e4bbaaa929102010c30f0244c306.png)

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 23, 2022, 4:20am UTC](https://discourse.julialang.org/t/python-lmfit-takes-weights-into-account-but-julia-lsqfit-package-seems-not-to/90679/4 "2022-11-23T04:20:37Z")

</div>

Yes seems like the weights are not being taken into account. What version of LsqFit.jl are you using?

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [November 23, 2022, 3:12pm UTC](https://discourse.julialang.org/t/python-lmfit-takes-weights-into-account-but-julia-lsqfit-package-seems-not-to/90679/5 "2022-11-23T15:12:22Z")

</div>

Just in case, the weights provided to `LsqFit.jl` should be inverse variances. See this [other thread](https://discourse.julialang.org/t/nonlinear-curve-fitting-with-weights-with-lsqfit/78478/11).

---

<div class="post-metadata">

**Author:** ![humbleclaw](https://avatars.discourse-cdn.com/v4/letter/h/b3f665/32.png) [@humbleclaw](https://discourse.julialang.org/u/humbleclaw)\
**Post date:** [November 23, 2022, 4:59pm UTC](https://discourse.julialang.org/t/python-lmfit-takes-weights-into-account-but-julia-lsqfit-package-seems-not-to/90679/6 "2022-11-23T16:59:04Z")

</div>

Thank you all. It was just as rafael.guerra said, the algorithm receives the weights as the inverse variances. Thank you all for your help!

 ![image](https://global.discourse-cdn.com/julialang/original/3X/2/f/2f113e375295bf83789e7939d3c9083262faf45f.png)  
l

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 23, 2022, 9:19pm UTC](https://discourse.julialang.org/t/python-lmfit-takes-weights-into-account-but-julia-lsqfit-package-seems-not-to/90679/7 "2022-11-23T21:19:56Z")

</div>

The documentation says it’s minimizing sum(w\_i\*err\_i) if you want it to take errors more seriously on the larger values you’d think that larger w would do it. So is there a problem with the documentation?

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [November 23, 2022, 9:24pm UTC](https://discourse.julialang.org/t/python-lmfit-takes-weights-into-account-but-julia-lsqfit-package-seems-not-to/90679/8 "2022-11-23T21:24:06Z")

</div>

The [Weighted Least Squares section](https://julianlsolvers.github.io/LsqFit.jl/latest/tutorial/#Weighted-Least-Squares-1) of the documentation tutorial says: “_We know the error variance and we set the weight as the inverse of the variance_…”.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [November 23, 2022, 11:13pm UTC](https://discourse.julialang.org/t/python-lmfit-takes-weights-into-account-but-julia-lsqfit-package-seems-not-to/90679/9 "2022-11-23T23:13:02Z")

</div>

So is the tutorial at odds with the mathematical expressions in the docs? I’m on my phone it’s hard to figure out what’s what.

Intuitively if it’s minimizing the sum of w\*err and we make w bigger this means that error is more important. The OP tries to weight each error by 10% of the value of the measurement. So bigger measurements get more weight. It should force the fit to go closer to the large measurements… But it doesn’t. What is the actual math being carried out?

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [November 23, 2022, 11:26pm UTC](https://discourse.julialang.org/t/python-lmfit-takes-weights-into-account-but-julia-lsqfit-package-seems-not-to/90679/10 "2022-11-23T23:26:35Z")

</div>

> [@dlakelan](#):
>
> So is the tutorial at odds with the mathematical expressions in the docs?

No, it doesn’t look like that.

From dimensional analyses, you can see that the weights `wi` in the following expression in the docs should have the same units as the inverse of a variance:

 ![LsqFit](https://global.discourse-cdn.com/julialang/original/3X/2/0/20111f6512d4f88f8e3b8cfd146e2ef91e5d516b.png)

And intuitively, regardless of the magnitude of the fitting errors at some data points, if their uncertainties are large then their respective weights should be small.