Dear all,
I’m running a maximum likelihood estimation of 1000+ parameters and thought to use ReverseDiff. I found a video online (1:47) that said to run something similar to the following if interested in using ReverseDiff:
using Optim
using ReverseDiff
M(par) = -likelihood(par, some_other_args)
params = rand(1000)
lb = repeat([0], 1000)
ub = repeat([Inf], 1000)
tape = ReverseDiff.GradientTape(M, params)
g!(G, x) = ReverseDiff.gradient!(G, tape, x)
opti = optimize(M, g!, lb, ub, params, Fminbox(LBFGS()))
Issue is: After computing the gradient tape, the optimizer begins and immediately stops; it just runs 1 iteration (not converged).
After some reading, I thought to use GalacticOptim (first time user), but it’s been giving me a headache. The interface is great, but I experienced an error that led to a chain of other errors. I’ve actually never seen this error before. The error is straight forward to understand, but I can’t seem to solve it. First, I run this:
using GalacticOptim
using ReverseDiff
using Optim
params = rand(4)
lb = repeat([0], 4)
ub = repeat([Inf], 4)
M(par) = -likelihood(par, some_other_args)
f = OptimizationFunction(M, GalacticOptim.AutoReverseDiff())
prob = OptimizationProblem(f, params, lb, ub)
sol = solve(prob, Fminbox(LBFGS()). maxiters=100)
On the last line, I get this output:
ERROR: Optimization algorithm not found. Either the chosen algorithm is not a valid solver
choice for the `OptimizationProblem`, or the Optimization solver library is not loaded.
Make sure that you have loaded an appropriate Optimization.jl solver library, for example,
`solve(prob,Optim.BFGS())` requires `using OptimizationOptimJL` and
`solve(prob,Adam())` requires `using OptimizationOptimisers`.
For more information, see the Optimization.jl documentation: optimization.sciml.ai/dev.
Chosen Optimizer: BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}(LineSearches.InitialStatic{Float64}
alpha: Float64 1.0
scaled: Bool false
, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}
delta: Float64 0.1
sigma: Float64 0.9
alphamax: Float64 Inf
rho: Float64 5.0
epsilon: Float64 1.0e-6
gamma: Float64 0.66
linesearchmax: Int64 50
psi3: Float64 0.1
display: Int64 0
mayterminate: Base.RefValue{Bool}
, nothing, nothing, Flat())
Stacktrace:
[1] __solve(::OptimizationProblem{true, OptimizationFunction{true, Optimization.AutoReverseDiff, typeof(rosenbrock), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}, Vector{Float64}, Nothing, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ SciMLBase ~/.julia/packages/SciMLBase/mOGJz/src/solve.jl:95
[2] __solve(::OptimizationProblem{true, OptimizationFunction{true, Optimization.AutoReverseDiff, typeof(rosenbrock), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}, Vector{Float64}, Nothing, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat})
@ SciMLBase ~/.julia/packages/SciMLBase/mOGJz/src/solve.jl:95
[3] solve(::OptimizationProblem{true, OptimizationFunction{true, Optimization.AutoReverseDiff, typeof(rosenbrock), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}, Vector{Float64}, Nothing, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ SciMLBase ~/.julia/packages/SciMLBase/mOGJz/src/solve.jl:71
[4] solve(::OptimizationProblem{true, OptimizationFunction{true, Optimization.AutoReverseDiff, typeof(rosenbrock), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}, Vector{Float64}, Nothing, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, ::BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat})
@ SciMLBase ~/.julia/packages/SciMLBase/mOGJz/src/solve.jl:71
[5] top-level scope
@ ~/Dropbox/Distributional_Dynamics/5_Code/Master.jl:155
BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Nothing, Flat}(LineSearches.InitialStatic{Float64}
alpha: Float64 1.0
scaled: Bool false
, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}
delta: Float64 0.1
sigma: Float64 0.9
alphamax: Float64 Inf
rho: Float64 5.0
epsilon: Float64 1.0e-6
gamma: Float64 0.66
linesearchmax: Int64 50
psi3: Float64 0.1
display: Int64 0
mayterminate: Base.RefValue{Bool}
, nothing, nothing, Flat())```
So then I try to install OptimizationOptimJL, but I get:
Unsatisfiable requirements detected for package DataFrames [a93c6f00]:
DataFrames [a93c6f00] log:
├─possible versions are: 0.11.7-1.3.6 or uninstalled
├─restricted to versions * by an explicit requirement, leaving only versions 0.11.7-1.3.6
├─restricted by compatibility requirements with DataFramesMeta [1313f7d8] to versions: 0.13.0-1.3.6
│ └─DataFramesMeta [1313f7d8] log:
│ ├─possible versions are: 0.4.0-0.12.0 or uninstalled
│ ├─restricted to versions * by an explicit requirement, leaving only versions 0.4.0-0.12.0
│ └─restricted by compatibility requirements with Reexport [189a3867] to versions: [0.4.0-0.5.1, 0.6.1-0.12.0] or uninstalled, leaving only versions: [0.4.0-0.5.1, 0.6.1-0.12.0]
│ └─Reexport [189a3867] log:
│ ├─possible versions are: 0.2.0-1.2.2 or uninstalled
│ ├─restricted by compatibility requirements with StatsPlots [f3b207a7] to versions: 0.2.0-1.2.2
│ │ └─StatsPlots [f3b207a7] log:
│ │ ├─possible versions are: 0.10.0-0.15.3 or uninstalled
│ │ └─restricted to versions * by an explicit requirement, leaving only versions 0.10.0-0.15.3
│ └─restricted by compatibility requirements with OptimizationOptimJL [36348300] to versions: 1.2.0-1.2.2
│ └─OptimizationOptimJL [36348300] log:
│ ├─possible versions are: 0.1.1-0.1.2 or uninstalled
│ └─restricted to versions * by an explicit requirement, leaving only versions 0.1.1-0.1.2
├─restricted by compatibility requirements with LowRankModels [15d4e49f] to versions: [0.11.7-0.19.4, 0.21.0-0.21.8], leaving only versions: [0.13.0-0.19.4, 0.21.0-0.21.8]
│ └─LowRankModels [15d4e49f] log:
│ ├─possible versions are: 1.0.0-1.1.1 or uninstalled
│ ├─restricted to versions * by an explicit requirement, leaving only versions 1.0.0-1.1.1
│ └─restricted by compatibility requirements with Optim [429524aa] to versions: 1.1.0-1.1.1 or uninstalled, leaving only versions: 1.1.0-1.1.1
│ └─Optim [429524aa] log:
│ ├─possible versions are: 0.15.3-1.7.3 or uninstalled
│ ├─restricted to versions * by an explicit requirement, leaving only versions 0.15.3-1.7.3
│ ├─restricted by compatibility requirements with OptimizationOptimJL [36348300] to versions: 1.0.0-1.7.3
│ │ └─OptimizationOptimJL [36348300] log: see above
│ └─restricted by compatibility requirements with StateSpaceModels [99342f36] to versions: [0.15.3-0.22.0, 1.2.0-1.7.3], leaving only versions: 1.2.0-1.7.3
│ └─StateSpaceModels [99342f36] log:
│ ├─possible versions are: 0.1.0-0.6.5 or uninstalled
│ ├─restricted to versions * by an explicit requirement, leaving only versions 0.1.0-0.6.5
│ └─restricted by compatibility requirements with Optim [429524aa] to versions: 0.5.4-0.6.5 or uninstalled, leaving only versions: 0.5.4-0.6.5
│ └─Optim [429524aa] log: see above
└─restricted by compatibility requirements with Reexport [189a3867] to versions: 0.22.3-1.3.6 or uninstalled — no versions left
└─Reexport [189a3867] log: see above
Stacktrace:
[1] propagate_constraints!(graph::Pkg.Resolve.Graph, sources::Set{Int64}; log_events::Bool)
@ Pkg.Resolve /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/Resolve/graphtype.jl:1063
[2] propagate_constraints! (repeats 2 times)
@ /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/Resolve/graphtype.jl:1000 [inlined]
[3] simplify_graph!(graph::Pkg.Resolve.Graph, sources::Set{Int64}; clean_graph::Bool)
@ Pkg.Resolve /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/Resolve/graphtype.jl:1519
[4] simplify_graph! (repeats 2 times)
@ /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/Resolve/graphtype.jl:1519 [inlined]
[5] resolve_versions!(env::Pkg.Types.EnvCache, registries::Vector{Pkg.Registry.RegistryInstance}, pkgs::Vector{Pkg.Types.PackageSpec}, julia_version::VersionNumber)
@ Pkg.Operations /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:335
[6] targeted_resolve(env::Pkg.Types.EnvCache, registries::Vector{Pkg.Registry.RegistryInstance}, pkgs::Vector{Pkg.Types.PackageSpec}, preserve::Pkg.Types.PreserveLevel, julia_version::VersionNumber)
@ Pkg.Operations /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:1154
[7] tiered_resolve(env::Pkg.Types.EnvCache, registries::Vector{Pkg.Registry.RegistryInstance}, pkgs::Vector{Pkg.Types.PackageSpec}, julia_version::VersionNumber)
@ Pkg.Operations /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:1139
[8] _resolve(io::Base.TTY, env::Pkg.Types.EnvCache, registries::Vector{Pkg.Registry.RegistryInstance}, pkgs::Vector{Pkg.Types.PackageSpec}, preserve::Pkg.Types.PreserveLevel, julia_version::VersionNumber)
@ Pkg.Operations /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:1160
[9] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, new_git::Set{Base.UUID}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform)
@ Pkg.Operations /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:1176
[10] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform, kwargs::Base.Pairs{Symbol, Base.TTY, Tuple{Symbol}, NamedTuple{(:io,), Tuple{Base.TTY}}})
@ Pkg.API /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/API.jl:268
[11] add(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Pkg.API /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/API.jl:149
[12] add(pkgs::Vector{Pkg.Types.PackageSpec})
@ Pkg.API /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/API.jl:144
[13] #add#27
@ /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/API.jl:142 [inlined]
[14] add
@ /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/API.jl:142 [inlined]
[15] try_prompt_pkg_add(pkgs::Vector{Symbol})
@ Pkg.REPLMode /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/Pkg/src/REPLMode/REPLMode.jl:710
[16] #invokelatest#2
@ ./essentials.jl:716 [inlined]
[17] invokelatest
@ ./essentials.jl:714 [inlined]
[18] check_for_missing_packages_and_run_hooks(ast::Any)
@ REPL /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:175
[19] eval_user_input(ast::Any, backend::REPL.REPLBackend)
@ REPL /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:145
[20] repl_backend_loop(backend::REPL.REPLBackend)
@ REPL /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:246
[21] start_repl_backend(backend::REPL.REPLBackend, consumer::Any)
@ REPL /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:231
[22] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool)
@ REPL /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:364
[23] run_repl(repl::REPL.AbstractREPL, consumer::Any)
@ REPL /Applications/Julia-1.7.app/Contents/Resources/julia/share/julia/stdlib/v1.7/REPL/src/REPL.jl:351
[24] (::Base.var"#930#932"{Bool, Bool, Bool})(REPL::Module)
@ Base ./client.jl:394
[25] #invokelatest#2
@ ./essentials.jl:716 [inlined]
[26] invokelatest
@ ./essentials.jl:714 [inlined]
[27] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
@ Base ./client.jl:379
[28] exec_options(opts::Base.JLOptions)
@ Base ./client.jl:309
[29] _start()
@ Base ./client.jl:495
After updating, removing and reinstalling packages, I have made zero progress on this. Any suggestions?