# Packages for Optimization

**URL:** <https://discourse.julialang.org/t/packages-for-optimization/50926>\
**Category:** General Usage\
**Tags:** question, package, jump, optimization\
**Created:** [November 29, 2020, 4:03am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926 "2020-11-29T04:03:15Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 29, 2020, 4:03am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/1 "2020-11-29T04:03:15Z")

</div>

What are some good packages for optimization. I’ve been using Roots.jl and ModelingToolkit.jl to do symbolic derivatives and find the zero roots, but I’m considering packages that actually look for minimums and maximums. I know of ForwardDiff.jl, but also Optim.jl, and JuMP.jl . What packages would anyone recommend? Are there any good options that I have missed?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 29, 2020, 4:05am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/2 "2020-11-29T04:05:13Z")

</div>

GalacticOptim.jl is a bit underdocumented right now but that’s changing rapidly. It pulls together what’s in the ecosystem into one interface. It might not be the answer right now, but it’ll at least be in like 3 months.

[https://github.com/SciML/GalacticOptim.jl](https://github.com/SciML/GalacticOptim.jl)

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 29, 2020, 4:08am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/3 "2020-11-29T04:08:48Z")

</div>

Cool, I just found another page that was using Roots with ForwardDiff, and was going to play with that for a while. I check this out too!

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 29, 2020, 4:14am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/4 "2020-11-29T04:14:10Z")

</div>

Roots.jl isn’t for optimization.

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 29, 2020, 4:14am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/5 "2020-11-29T04:14:46Z")

</div>

I know that, I was just using it to find inflection points.

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 29, 2020, 4:25am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/6 "2020-11-29T04:25:20Z")

</div>

What about ForwardDiff?

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [November 29, 2020, 4:29am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/7 "2020-11-29T04:29:11Z")

</div>

ForwardDiff is for autodiff. Optim will use ForwardDiff if you use the `autodiff=true` option.  
You could also look at [GitHub - JuliaNLSolvers/NLSolvers.jl: No bells and whistles foundation of Optim.jl](https://github.com/JuliaNLSolvers/NLSolvers.jl)  
If you care about performance, you may also be interested in  
[https://github.com/JuliaComputing/NonlinearSolve.jl](https://github.com/JuliaComputing/NonlinearSolve.jl)  
for root finding.

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 29, 2020, 4:45am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/8 "2020-11-29T04:45:37Z")

</div>

When I try to install NonlinearSolve, I’m told it doesn’t exist

```julia
Pkg.add("NonlinearSolve")
ERROR: The following package names could not be resolved:
 * NonlinearSolve (not found in project, manifest or registry)
Stacktrace:
 [1] pkgerror(::String) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Types.jl:52    
 [2] ensure_resolved(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; registry::Bool) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Types.jl:837
 [3] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; preserve::Pkg.Types.PreserveLevel, platform::Pkg.BinaryPlatforms.Windows, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:177
 [4] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:139
 [5] #add#21 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:67 [inlined]       
 [6] add at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:67 [inlined]
 [7] #add#20 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:66 [inlined]       
 [8] add at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:66 [inlined]
 [9] add(::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:65
 [10] add(::String) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:65
 [11] top-level scope at REPL[19]:1

```

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 29, 2020, 5:00am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/9 "2020-11-29T05:00:27Z")

</div>

It won’t load, not sure if it’s a problem at my end, or if I should report an issue.

```julia
WARNING: could not import DiffEqBase.OptimizationFunction into GalacticOptim
WARNING: could not import DiffEqBase.AbstractADType into GalacticOptim
ERROR: LoadError: LoadError: UndefVarError: AbstractADType not defined
Stacktrace:
 [1] top-level scope at C:\Users\Brett\.julia\packages\GalacticOptim\TzUrT\src\function.jl:1
 [2] include(::Function, ::Module, ::String) at .\Base.jl:380
 [3] include at .\Base.jl:368 [inlined]
 [4] include(::String) at C:\Users\Brett\.julia\packages\GalacticOptim\TzUrT\src\GalacticOptim.jl:1
 [5] top-level scope at C:\Users\Brett\.julia\packages\GalacticOptim\TzUrT\src\GalacticOptim.jl:15
 [6] include(::Function, ::Module, ::String) at .\Base.jl:380
 [7] include(::Module, ::String) at .\Base.jl:368
 [8] top-level scope at none:2
 [9] eval at .\boot.jl:331 [inlined]
 [10] eval(::Expr) at .\client.jl:467
 [11] top-level scope at .\none:3
in expression starting at C:\Users\Brett\.julia\packages\GalacticOptim\TzUrT\src\function.jl:1
in expression starting at C:\Users\Brett\.julia\packages\GalacticOptim\TzUrT\src\GalacticOptim.jl:15
ERROR: Failed to precompile GalacticOptim [a75be94c-b780-496d-a8a9-0878b188d577] to C:\Users\Brett\.julia\compiled\v1.5\GalacticOptim\fP6Iz_nUQ8j.ji.

```

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 29, 2020, 5:20am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/10 "2020-11-29T05:20:20Z")

</div>

You’re on an old version of ModelingToolkit and GalacticOptim (pre-release).

> [@brett\_knoss](#):
>
> When I try to install NonlinearSolve, I’m told it doesn’t exist

You have to add from URL since it’s not registered yet.

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 29, 2020, 5:25am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/11 "2020-11-29T05:25:11Z")

</div>

Ok, I’m not sure how to add from the URL. Is that the url under source code in GitHub?

Also, I updated the packages several times

```julia
v(1.5)> up

```

do I need to do more to update specific packages?

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [November 29, 2020, 5:51am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/12 "2020-11-29T05:51:53Z")

</div>

MTK or GalacticOptim might be constrained by another package’s requirements in the default package environment. You can try activating a new package environment with

```julia
] activate "path/to/test_env"

```

to create a blank slate where you can add only the necessary packages for what you’re trying to do at the moment. If you need a certain version of a package, you can add it with, e.g.,

```julia
(test_env) pkg> add Measurements#v2.2.0

```

You can return to the default environment with

```julia
] activate

```

Note that Julia maintains a cache of the different versions of packages you might have installed, so you don’t have to re-download anything when switching environments.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 29, 2020, 8:53am UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/13 "2020-11-29T08:53:55Z")

</div>

> [@brett\_knoss](#):
>
> What are some good packages for optimization.

The question is not well-specified in this form — it depends on properties of your problem:

1. dimension (small or large),
2. domain (integers, real numbers, boxes, other linear or nonlinear constraints?)
3. continuity and differentiability,
4. linear, non-linear, or some special form,
5. black box or implemented in Julia.

and probably other things I missed.

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 29, 2020, 6:43pm UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/14 "2020-11-29T18:43:20Z")

</div>

I can see why there is a whole section on this. Right now I’m doing basic stuff, and studying the maths as much as anything.

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 29, 2020, 6:46pm UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/15 "2020-11-29T18:46:52Z")

</div>

Is

```julia
 "path/to/test_env" 

```

an actual command, or is it replaced with a file path?

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 29, 2020, 6:58pm UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/16 "2020-11-29T18:58:58Z")

</div>

> [@stillyslalom](#):
>
> `activate "path/to/test_env"`

How do I install the latest version of MTK?

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [November 29, 2020, 7:03pm UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/17 "2020-11-29T19:03:09Z")

</div>

It’s just a file path to a directory of your choice.

```julia
(@v1.6) pkg> ?activate
  activate
  activate [--shared|--temp] [path]

  Activate the environment at the given path, or the home project environment if no path is specified. The active
  environment is the environment that is modified by executing package commands. When the option --shared is given,
  path will be assumed to be a directory name and searched for in the environments folders of the depots in the depot
  stack. In case no such environment exists in any of the depots, it will be placed in the first depot of the stack.
  Use the temp option to create temporary environments. This should be useful for experimenting with packages.

```

```julia
(@v1.6) pkg> activate --temp
  Activating new environment at `/tmp/jl_DinAU8/Project.toml`

(jl_DinAU8) pkg> add ModelingToolkit#master
    Updating git-repo `https://github.com/SciML/ModelingToolkit.jl.git`
   Resolving package versions...
Updating `/tmp/jl_DinAU8/Project.toml`
  [961ee093] + ModelingToolkit v4.0.7 `https://github.com/SciML/ModelingToolkit.jl.git#master`
Updating `/tmp/jl_DinAU8/Manifest.toml`
...

```

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 29, 2020, 7:52pm UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/18 "2020-11-29T19:52:50Z")

</div>

> [@stillyslalom](#):
>
> `activate --temp`

I’m still on v3.20.1, and the latest on GitHub is 4.7.0

```julia
(@v1.5) pkg> status "ModelingToolkit"
Status `C:\Users\Brett\.julia\environments\v1.5\Project.toml`
  [961ee093] ModelingToolkit v3.20.1

(@v1.5) pkg> activate --temp
 Activating new environment at `C:\Users\Brett\AppData\Local\Temp\jl_g1PN9o\Project.toml`

(jl_g1PN9o) pkg> add "ModelingToolkit"
  Resolving package versions...
Updating `C:\Users\Brett\AppData\Local\Temp\jl_g1PN9o\Project.toml`
  [961ee093] + ModelingToolkit v4.0.6
Updating `C:\Users\Brett\AppData\Local\Temp\jl_g1PN9o\Manifest.toml`
  [c3fe647b] + AbstractAlgebra v0.11.1
  [79e6a3ab] + Adapt v2.3.0
  [ec485272] + ArnoldiMethod v0.0.4
  [4fba245c] + ArrayInterface v2.13.7
  [56f22d72] + Artifacts v1.3.0
  [d360d2e6] + ChainRulesCore v0.9.19
  [861a8166] + Combinatorics v1.0.2
  [34da2185] + Compat v3.23.0
  [e66e0078] + CompilerSupportLibraries_jll v0.3.4+0
  [187b0558] + ConstructionBase v1.0.0
  [adafc99b] + CpuId v0.2.2
  [9a962f9c] + DataAPI v1.4.0
  [864edb3b] + DataStructures v0.18.8
  [e2d170a0] + DataValueInterfaces v1.0.0
  [2b5f629d] + DiffEqBase v6.48.2
  [c894b116] + DiffEqJump v6.10.1
  [b552c78f] + DiffRules v1.0.2
  [ffbed154] + DocStringExtensions v0.8.3
  [e2ba6199] + ExprTools v0.1.3
  [59287772] + Formatting v0.4.1
  [069b7b12] + FunctionWrappers v1.1.1
  [615f187c] + IfElse v0.1.0
  [d25df0c9] + Inflate v0.1.2
  [42fd0dbc] + IterativeSolvers v0.8.4
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [692b3bcd] + JLLWrappers v1.1.3
  [b964fa9f] + LaTeXStrings v1.2.0
  [2ee39098] + LabelledArrays v1.3.0
  [23fbe1c1] + Latexify v0.14.5
  [093fc24a] + LightGraphs v1.3.0
  [bdcacae8] + LoopVectorization v0.8.26
  [1914dd2f] + MacroTools v0.5.6
  [961ee093] + ModelingToolkit v4.0.6
  [46d2c3a1] + MuladdMacro v0.2.2
  [77ba4419] + NaNMath v0.3.5
  [6fe1bfb0] + OffsetArrays v1.4.0
  [efe28fd5] + OpenSpecFun_jll v0.5.3+4
  [bac558e1] + OrderedCollections v1.3.2
  [d96e819e] + Parameters v0.12.1
  [e409e4f3] + PoissonRandom v0.4.0
  [fb686558] + RandomExtensions v0.4.3
  [e6cf234a] + RandomNumbers v1.4.0
  [3cdcf5f2] + RecipesBase v1.1.1
  [731186ca] + RecursiveArrayTools v2.7.2
  [f2c3362d] + RecursiveFactorization v0.1.4
  [ae029012] + Requires v1.1.1
  [f2b01f46] + Roots v1.0.5
  [7e49a35a] + RuntimeGeneratedFunctions v0.4.2
  [21efa798] + SIMDPirates v0.8.26
  [476501e8] + SLEEFPirates v0.5.5
  [1bc83da4] + SafeTestsets v0.0.1
  [699a6c99] + SimpleTraits v0.9.3
  [276daf66] + SpecialFunctions v0.10.3
  [90137ffa] + StaticArrays v0.12.5
  [d1185830] + SymbolicUtils v0.6.0
  [3783bdb8] + TableTraits v1.0.0
  [bd369af6] + Tables v1.2.1
  [a759f4b9] + TimerOutputs v0.5.7
  [a2a6695c] + TreeViews v0.3.0
  [3a884ed6] + UnPack v1.0.2
  [1986cc42] + Unitful v1.5.0
  [3d5dd08c] + VectorizationBase v0.12.33
  [700de1a5] + ZygoteRules v0.2.0
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [b77e0a4c] + InteractiveUtils
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [4607b0f0] + SuiteSparse
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode

(jl_g1PN9o) pkg> activate
 Activating environment at `C:\Users\Brett\.julia\environments\v1.5\Project.toml`
(@v1.5) pkg> status "ModelingToolkit"
Status `C:\Users\Brett\.julia\environments\v1.5\Project.toml`
  [961ee093] ModelingToolkit v3.20.1

```

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [November 29, 2020, 8:13pm UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/19 "2020-11-29T20:13:48Z")

</div>

I’m seeing the latest release as 4.0.7, not 4.7.0. You successfully installed v4.0.6 in a temporary environment, then switched back to the default environment, which still has v3.20.1 because of a constraint imposed by some other package. If you want the latest version, you’ll need to switch to a non-default environment and stay there (`activate "path/to/test_env"` if you want to use it again, or `activate --temp`, as described earlier). You can also try `add ModelingToolkit#v4.0.7` in the default environment – the error messages may help you figure out which package is constraining MTK (see [here](https://julialang.github.io/Pkg.jl/v1/managing-packages/#conflicts) for more info on package conflicts).

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 29, 2020, 8:18pm UTC](https://discourse.julialang.org/t/packages-for-optimization/50926/20 "2020-11-29T20:18:23Z")

</div>

> [@stillyslalom](#):
>
> add ModelingToolkit#v4.0

```julia
NM, it has to have all 3 numbers.
```

[Next page](https://discourse.julialang.org/t/packages-for-optimization/50926.md?page=2)