What are some good packages for optimization. I’ve been using Roots.jl and ModelingToolkit.jl to do symbolic derivatives and find the zero roots, but I’m considering packages that actually look for minimums and maximums. I know of ForwardDiff.jl, but also Optim.jl, and JuMP.jl . What packages would anyone recommend? Are there any good options that I have missed?
GalacticOptim.jl is a bit underdocumented right now but that’s changing rapidly. It pulls together what’s in the ecosystem into one interface. It might not be the answer right now, but it’ll at least be in like 3 months.
2 Likes
Cool, I just found another page that was using Roots with ForwardDiff, and was going to play with that for a while. I check this out too!
Roots.jl isn’t for optimization.
I know that, I was just using it to find inflection points.
What about ForwardDiff?
ForwardDiff is for autodiff. Optim will use ForwardDiff if you use the autodiff=true option.
You could also look at GitHub - JuliaNLSolvers/NLSolvers.jl: No bells and whistles foundation of Optim.jl
If you care about performance, you may also be interested in
https://github.com/JuliaComputing/NonlinearSolve.jl
for root finding.
2 Likes
When I try to install NonlinearSolve, I’m told it doesn’t exist
Pkg.add("NonlinearSolve")
ERROR: The following package names could not be resolved:
* NonlinearSolve (not found in project, manifest or registry)
Stacktrace:
[1] pkgerror(::String) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Types.jl:52
[2] ensure_resolved(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; registry::Bool) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Types.jl:837
[3] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; preserve::Pkg.Types.PreserveLevel, platform::Pkg.BinaryPlatforms.Windows, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:177
[4] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:139
[5] #add#21 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:67 [inlined]
[6] add at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:67 [inlined]
[7] #add#20 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:66 [inlined]
[8] add at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:66 [inlined]
[9] add(::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:65
[10] add(::String) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:65
[11] top-level scope at REPL[19]:1
It won’t load, not sure if it’s a problem at my end, or if I should report an issue.
WARNING: could not import DiffEqBase.OptimizationFunction into GalacticOptim
WARNING: could not import DiffEqBase.AbstractADType into GalacticOptim
ERROR: LoadError: LoadError: UndefVarError: AbstractADType not defined
Stacktrace:
[1] top-level scope at C:\Users\Brett\.julia\packages\GalacticOptim\TzUrT\src\function.jl:1
[2] include(::Function, ::Module, ::String) at .\Base.jl:380
[3] include at .\Base.jl:368 [inlined]
[4] include(::String) at C:\Users\Brett\.julia\packages\GalacticOptim\TzUrT\src\GalacticOptim.jl:1
[5] top-level scope at C:\Users\Brett\.julia\packages\GalacticOptim\TzUrT\src\GalacticOptim.jl:15
[6] include(::Function, ::Module, ::String) at .\Base.jl:380
[7] include(::Module, ::String) at .\Base.jl:368
[8] top-level scope at none:2
[9] eval at .\boot.jl:331 [inlined]
[10] eval(::Expr) at .\client.jl:467
[11] top-level scope at .\none:3
in expression starting at C:\Users\Brett\.julia\packages\GalacticOptim\TzUrT\src\function.jl:1
in expression starting at C:\Users\Brett\.julia\packages\GalacticOptim\TzUrT\src\GalacticOptim.jl:15
ERROR: Failed to precompile GalacticOptim [a75be94c-b780-496d-a8a9-0878b188d577] to C:\Users\Brett\.julia\compiled\v1.5\GalacticOptim\fP6Iz_nUQ8j.ji.
You’re on an old version of ModelingToolkit and GalacticOptim (pre-release).
You have to add from URL since it’s not registered yet.
Ok, I’m not sure how to add from the URL. Is that the url under source code in GitHub?
Also, I updated the packages several times
v(1.5)> up
do I need to do more to update specific packages?
MTK or GalacticOptim might be constrained by another package’s requirements in the default package environment. You can try activating a new package environment with
] activate "path/to/test_env"
to create a blank slate where you can add only the necessary packages for what you’re trying to do at the moment. If you need a certain version of a package, you can add it with, e.g.,
(test_env) pkg> add Measurements#v2.2.0
You can return to the default environment with
] activate
Note that Julia maintains a cache of the different versions of packages you might have installed, so you don’t have to re-download anything when switching environments.
1 Like
The question is not well-specified in this form — it depends on properties of your problem:
- dimension (small or large),
- domain (integers, real numbers, boxes, other linear or nonlinear constraints?)
- continuity and differentiability,
- linear, non-linear, or some special form,
- black box or implemented in Julia.
and probably other things I missed.
4 Likes
I can see why there is a whole section on this. Right now I’m doing basic stuff, and studying the maths as much as anything.
Is
"path/to/test_env"
an actual command, or is it replaced with a file path?
How do I install the latest version of MTK?
It’s just a file path to a directory of your choice.
(@v1.6) pkg> ?activate
activate
activate [--shared|--temp] [path]
Activate the environment at the given path, or the home project environment if no path is specified. The active
environment is the environment that is modified by executing package commands. When the option --shared is given,
path will be assumed to be a directory name and searched for in the environments folders of the depots in the depot
stack. In case no such environment exists in any of the depots, it will be placed in the first depot of the stack.
Use the temp option to create temporary environments. This should be useful for experimenting with packages.
(@v1.6) pkg> activate --temp
Activating new environment at `/tmp/jl_DinAU8/Project.toml`
(jl_DinAU8) pkg> add ModelingToolkit#master
Updating git-repo `https://github.com/SciML/ModelingToolkit.jl.git`
Resolving package versions...
Updating `/tmp/jl_DinAU8/Project.toml`
[961ee093] + ModelingToolkit v4.0.7 `https://github.com/SciML/ModelingToolkit.jl.git#master`
Updating `/tmp/jl_DinAU8/Manifest.toml`
...
I’m still on v3.20.1, and the latest on GitHub is 4.7.0
(@v1.5) pkg> status "ModelingToolkit"
Status `C:\Users\Brett\.julia\environments\v1.5\Project.toml`
[961ee093] ModelingToolkit v3.20.1
(@v1.5) pkg> activate --temp
Activating new environment at `C:\Users\Brett\AppData\Local\Temp\jl_g1PN9o\Project.toml`
(jl_g1PN9o) pkg> add "ModelingToolkit"
Resolving package versions...
Updating `C:\Users\Brett\AppData\Local\Temp\jl_g1PN9o\Project.toml`
[961ee093] + ModelingToolkit v4.0.6
Updating `C:\Users\Brett\AppData\Local\Temp\jl_g1PN9o\Manifest.toml`
[c3fe647b] + AbstractAlgebra v0.11.1
[79e6a3ab] + Adapt v2.3.0
[ec485272] + ArnoldiMethod v0.0.4
[4fba245c] + ArrayInterface v2.13.7
[56f22d72] + Artifacts v1.3.0
[d360d2e6] + ChainRulesCore v0.9.19
[861a8166] + Combinatorics v1.0.2
[34da2185] + Compat v3.23.0
[e66e0078] + CompilerSupportLibraries_jll v0.3.4+0
[187b0558] + ConstructionBase v1.0.0
[adafc99b] + CpuId v0.2.2
[9a962f9c] + DataAPI v1.4.0
[864edb3b] + DataStructures v0.18.8
[e2d170a0] + DataValueInterfaces v1.0.0
[2b5f629d] + DiffEqBase v6.48.2
[c894b116] + DiffEqJump v6.10.1
[b552c78f] + DiffRules v1.0.2
[ffbed154] + DocStringExtensions v0.8.3
[e2ba6199] + ExprTools v0.1.3
[59287772] + Formatting v0.4.1
[069b7b12] + FunctionWrappers v1.1.1
[615f187c] + IfElse v0.1.0
[d25df0c9] + Inflate v0.1.2
[42fd0dbc] + IterativeSolvers v0.8.4
[82899510] + IteratorInterfaceExtensions v1.0.0
[692b3bcd] + JLLWrappers v1.1.3
[b964fa9f] + LaTeXStrings v1.2.0
[2ee39098] + LabelledArrays v1.3.0
[23fbe1c1] + Latexify v0.14.5
[093fc24a] + LightGraphs v1.3.0
[bdcacae8] + LoopVectorization v0.8.26
[1914dd2f] + MacroTools v0.5.6
[961ee093] + ModelingToolkit v4.0.6
[46d2c3a1] + MuladdMacro v0.2.2
[77ba4419] + NaNMath v0.3.5
[6fe1bfb0] + OffsetArrays v1.4.0
[efe28fd5] + OpenSpecFun_jll v0.5.3+4
[bac558e1] + OrderedCollections v1.3.2
[d96e819e] + Parameters v0.12.1
[e409e4f3] + PoissonRandom v0.4.0
[fb686558] + RandomExtensions v0.4.3
[e6cf234a] + RandomNumbers v1.4.0
[3cdcf5f2] + RecipesBase v1.1.1
[731186ca] + RecursiveArrayTools v2.7.2
[f2c3362d] + RecursiveFactorization v0.1.4
[ae029012] + Requires v1.1.1
[f2b01f46] + Roots v1.0.5
[7e49a35a] + RuntimeGeneratedFunctions v0.4.2
[21efa798] + SIMDPirates v0.8.26
[476501e8] + SLEEFPirates v0.5.5
[1bc83da4] + SafeTestsets v0.0.1
[699a6c99] + SimpleTraits v0.9.3
[276daf66] + SpecialFunctions v0.10.3
[90137ffa] + StaticArrays v0.12.5
[d1185830] + SymbolicUtils v0.6.0
[3783bdb8] + TableTraits v1.0.0
[bd369af6] + Tables v1.2.1
[a759f4b9] + TimerOutputs v0.5.7
[a2a6695c] + TreeViews v0.3.0
[3a884ed6] + UnPack v1.0.2
[1986cc42] + Unitful v1.5.0
[3d5dd08c] + VectorizationBase v0.12.33
[700de1a5] + ZygoteRules v0.2.0
[2a0f44e3] + Base64
[ade2ca70] + Dates
[8bb1440f] + DelimitedFiles
[8ba89e20] + Distributed
[b77e0a4c] + InteractiveUtils
[76f85450] + LibGit2
[8f399da3] + Libdl
[37e2e46d] + LinearAlgebra
[56ddb016] + Logging
[d6f4376e] + Markdown
[a63ad114] + Mmap
[44cfe95a] + Pkg
[de0858da] + Printf
[3fa0cd96] + REPL
[9a3f8284] + Random
[ea8e919c] + SHA
[9e88b42a] + Serialization
[1a1011a3] + SharedArrays
[6462fe0b] + Sockets
[2f01184e] + SparseArrays
[10745b16] + Statistics
[4607b0f0] + SuiteSparse
[8dfed614] + Test
[cf7118a7] + UUIDs
[4ec0a83e] + Unicode
(jl_g1PN9o) pkg> activate
Activating environment at `C:\Users\Brett\.julia\environments\v1.5\Project.toml`
(@v1.5) pkg> status "ModelingToolkit"
Status `C:\Users\Brett\.julia\environments\v1.5\Project.toml`
[961ee093] ModelingToolkit v3.20.1
I’m seeing the latest release as 4.0.7, not 4.7.0. You successfully installed v4.0.6 in a temporary environment, then switched back to the default environment, which still has v3.20.1 because of a constraint imposed by some other package. If you want the latest version, you’ll need to switch to a non-default environment and stay there (activate "path/to/test_env" if you want to use it again, or activate --temp, as described earlier). You can also try add ModelingToolkit#v4.0.7 in the default environment – the error messages may help you figure out which package is constraining MTK (see here for more info on package conflicts).
NM, it has to have all 3 numbers.