# Repo for Scientific Machine Learning - UDE paper does not seem to give consistent results on rerunning

**URL:** <https://discourse.julialang.org/t/repo-for-scientific-machine-learning-ude-paper-does-not-seem-to-give-consistent-results-on-rerunning/98157>\
**Category:** Machine Learning\
**Tags:** question\
**Created:** [May 1, 2023, 11:43am UTC](https://discourse.julialang.org/t/repo-for-scientific-machine-learning-ude-paper-does-not-seem-to-give-consistent-results-on-rerunning/98157 "2023-05-01T11:43:58Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Daniel\_Doyle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/daniel_doyle/32/48125_2.png) [@Daniel\_Doyle](https://discourse.julialang.org/u/Daniel_Doyle)\
**Post date:** [May 1, 2023, 11:43am UTC](https://discourse.julialang.org/t/repo-for-scientific-machine-learning-ude-paper-does-not-seem-to-give-consistent-results-on-rerunning/98157/1 "2023-05-01T11:43:58Z")

</div>

Hello,

I was going through the code behind the paper for Scientific machine learning here: [universal\_differential\_equations/scenario\_1.jl at master · ChrisRackauckas/universal\_differential\_equations · GitHub](https://github.com/ChrisRackauckas/universal_differential_equations/blob/master/LotkaVolterra/scenario_1.jl)

and I am not able to duplicate the sparse regression. Instead of getting the two parameter model I instead get a 22 parameter model. I have not changed any code (aside from not allowing f\_increases for BFGS because the model was unable to finish training if I did not include that) or dependencies and am using Julia 1.7.3 (which I think from the top level Manifest file is the right version). Does anyone know what might be going on here? I would have thought a consistent manifest file wouldn’t have issues with reproducibility. Happy to provide more details if needed

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 2, 2023, 12:15pm UTC](https://discourse.julialang.org/t/repo-for-scientific-machine-learning-ude-paper-does-not-seem-to-give-consistent-results-on-rerunning/98157/2 "2023-05-02T12:15:22Z")

</div>

I would recommend following the tutorial which is tested and displayed here:

[https://docs.sciml.ai/Overview/stable/showcase/missing\_physics/](https://docs.sciml.ai/Overview/stable/showcase/missing_physics/)

@Julius_Martensen can we work to get the Hudson Bay examples into the SciMLSensitivity docs? It would be much easier to maintain these pieces if they were all tested tutorials.

---

<div class="post-metadata">

**Author:** ![Daniel\_Doyle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/daniel_doyle/32/48125_2.png) [@Daniel\_Doyle](https://discourse.julialang.org/u/Daniel_Doyle)\
**Post date:** [May 2, 2023, 6:11pm UTC](https://discourse.julialang.org/t/repo-for-scientific-machine-learning-ude-paper-does-not-seem-to-give-consistent-results-on-rerunning/98157/3 "2023-05-02T18:11:19Z")

</div>

Thanks very much!

It seems like the main difference is that the ADMM optimizer is used instead of STLSQ. Do you know why STLSQ no longer performs well but used to?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 2, 2023, 7:24pm UTC](https://discourse.julialang.org/t/repo-for-scientific-machine-learning-ude-paper-does-not-seem-to-give-consistent-results-on-rerunning/98157/4 "2023-05-02T19:24:22Z")

</div>

STLSQ just needs a lot of tuning. You need to choose lambda correctly for it to lop off extra terms. How high the lambdas need to be can be dependent on the fit. That’s one of the major reasons why I don’t prefer STLSQ these days.