# Trying to use the new optimization algorithm: Sophia

**URL:** https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259
**Category:** Optimization (Mathematical)
**Tags:** optimization
**Created:** [February 15, 2024, 3:44pm UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259 "2024-02-15T15:44:30Z")
**Posts on this page:** 13
**Page:** 1

<div class="post-metadata">

### Author: ![KianH](https://avatars.discourse-cdn.com/v4/letter/k/c6cbf5/32.png) [@KianH](https://discourse.julialang.org/u/KianH)
#### Post date: [February 15, 2024, 3:44pm UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/1 "2024-02-15T15:44:30Z")

</div>

I am trying to use the new optimization algorithm in Optimization.jl called Sophia, which is the first algorithm that shows up in the link below.

> **[Optimisers.jl · Optimization.jl](https://docs.sciml.ai/Optimization/stable/optimization_packages/optimisers/)**
>
> Documentation for Optimization.jl.

However, I seem to get an error message that Sophia is an undefined variable. Shouldn’t it be enough to just call the algorithm as I have done below?

```julia
odeoutmodel = Optimization.solve(optprob,
                                        Sophia(),
                                        progress = true,
                                        maxiters = 1000)

```

---

<div class="post-metadata">

### Author: ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)
#### Post date: [February 15, 2024, 4:47pm UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/2 "2024-02-15T16:47:44Z")

</div>

`Sophia` is apparently not exported. Try `OptimizationOptimisers.Sophia`

---

<div class="post-metadata">

### Author: ![KianH](https://avatars.discourse-cdn.com/v4/letter/k/c6cbf5/32.png) [@KianH](https://discourse.julialang.org/u/KianH)
#### Post date: [February 16, 2024, 1:16pm UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/3 "2024-02-16T13:16:05Z")

</div>

I get a similar error when using `OptimizationOptimisers.Sophia`. I suppose we have to wait until it is exported.

---

<div class="post-metadata">

### Author: ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)
#### Post date: [February 16, 2024, 1:20pm UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/4 "2024-02-16T13:20:35Z")

</div>

Strange. What I get is

```julia
julia> using Optimization, OptimizationOptimisers
julia> OptimizationOptimisers.Sophia()
OptimizationOptimisers.Sophia(0.001, (0.9, 0.999), 1.0e-8, 0.1, 10, 0.04)

Status `/private/var/folders/sh/h134ydrd1gzbtd4qflbl50km0000gn/T/jl_fXjMwL/Project.toml`
  [3bd65402] Optimisers v0.3.2
  [7f7a1694] Optimization v3.22.0
  [42dfb2eb] OptimizationOptimisers v0.2.1

```

---

<div class="post-metadata">

### Author: ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [February 16, 2024, 1:34pm UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/5 "2024-02-16T13:34:53Z")

</div>

are you on an old version of the package?

---

<div class="post-metadata">

### Author: ![e3c6](https://avatars.discourse-cdn.com/v4/letter/e/e79b87/32.png) [@e3c6](https://discourse.julialang.org/u/e3c6)
#### Post date: [February 16, 2024, 1:39pm UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/6 "2024-02-16T13:39:54Z")

</div>

Seems weird to define a new algorithm within `OptimizationOptimisers.jl`, since that’s supposed to be just a wrapper of `Optimisers.jl` …

---

<div class="post-metadata">

### Author: ![KianH](https://avatars.discourse-cdn.com/v4/letter/k/c6cbf5/32.png) [@KianH](https://discourse.julialang.org/u/KianH)
#### Post date: [February 19, 2024, 10:57am UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/7 "2024-02-19T10:57:56Z")

</div>

Seems like I am on an old version. Despite updating `OptimizationOptimisers` as I have shown below, the version does not change. Do you have any advice for how to get the correct version installed?

```julia

julia> Pkg.status("OptimizationOptimisers")
⌃ [42dfb2eb] OptimizationOptimisers v0.1.2
Info Packages marked with ⌃ have new versions available and may be upgradable.

julia> Pkg.update("OptimizationOptimisers")
    Updating registry at `C:\Users\.julia\registries\General.toml`
  No Changes to `C:\Users\.julia\environments\v1.10\Project.toml`
  No Changes to `C:\Users\.julia\environments\v1.10\Manifest.toml`
Precompiling project...
  ? DomainSets
  ? DiffEqSensitivity
  ? SciMLSensitivity
  ? Symbolics
  ? ModelingToolkit
  ? DataInterpolations → DataInterpolationsSymbolicsExt
  ? DataDrivenDiffEq
  ? DiffEqFlux
  ? DataDrivenSparse

julia> Pkg.status("OptimizationOptimisers")
⌃ [42dfb2eb] OptimizationOptimisers v0.1.2
Info Packages marked with ⌃ have new versions available and may be upgradable.

```

---

<div class="post-metadata">

### Author: ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)
#### Post date: [February 19, 2024, 1:24pm UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/8 "2024-02-19T13:24:01Z")

</div>

If there are new versions available after Pkg.update, they should be indicated with a `⌅`.  
To find out which dependencies prevent others from updating, try `status --outdated`. That produces a report like the following:

```julia
(MismatchMM) pkg> status --outdated
Project MismatchMM v2.2.0
Status `~/Documents/projects/p2022/mismatch/MismatchMM/Project.toml`
⌃ [13f3f980] CairoMakie v0.11.6 (<v0.11.8)
⌅ [ffbed154] DocStringExtensions v0.8.6 (<v0.9.3): MultiStartLH
⌅ [e30172f5] Documenter v0.27.25 (<v1.2.1): MultiStartLH, StructLH, UtilityFunctionsLH, ValueHistoriesLH
⌃ [5903a43b] Infiltrator v1.6.4 (<v1.7.0)
⌃ [033835bb] JLD2 v0.4.45 (<v0.4.46)
⌅ [86f7a689] NamedArrays v0.9.8 (<v0.10.0) [compat]
⌃ [f2b01f46] Roots v2.1.1 (<v2.1.2)

```

In this example, `DocStringExtensions` cannot update because of a restriction in `MultiStartLH`’s `compat` entries.

---

<div class="post-metadata">

### Author: ![KianH](https://avatars.discourse-cdn.com/v4/letter/k/c6cbf5/32.png) [@KianH](https://discourse.julialang.org/u/KianH)
#### Post date: [February 19, 2024, 4:36pm UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/9 "2024-02-19T16:36:16Z")

</div>

Alright thanks! I tried what you suggested and tried to for instance update `Flux` as I have shown below however, even here it does not update despite its dependency `OptimizationFlux` is not in need of an update, therefore `Flux` should be able to update, right?

```julia
(@v1.10) pkg> update Flux
    Updating registry at `C:\Users\.julia\registries\General.toml`
  No Changes to `C:\Users\.julia\environments\v1.10\Project.toml`
  No Changes to `C:\Users\environments\v1.10\Manifest.toml`
Precompiling project...
  ? DomainSets
  ? SciMLSensitivity
  ? Symbolics
  ? DiffEqSensitivity
  ? ModelingToolkit
  ? DataInterpolations → DataInterpolationsSymbolicsExt
  ? DiffEqFlux
  ? DataDrivenDiffEq
  ? DataDrivenSparse

(@v1.10) pkg> status --outdated
Status `C:\Users\.julia\environments\v1.10\Project.toml`
⌃ [b0b7db55] ComponentArrays v0.13.7 (<v0.15.9)
⌃ [2445eb08] DataDrivenDiffEq v1.0.2 (<v1.3.0)
⌃ [a93c6f00] DataFrames v1.3.6 (<v1.6.1)
⌅ [2b5f629d] DiffEqBase v6.108.0 (<v6.147.0): OrdinaryDiffEq
⌃ [aae7a2af] DiffEqFlux v2.4.0 (<v3.3.1)
⌃ [0c46a032] DifferentialEquations v7.6.0 (<v7.12.0)
⌅ [587475ba] Flux v0.13.17 (<v0.14.11): OptimizationFlux
⌃ [b2108857] Lux v0.5.5 (<v0.5.17)
⌅ [f1d291b0] MLUtils v0.3.1 (<v0.4.4): DataDrivenDiffEq
⌃ [961ee093] ModelingToolkit v8.36.0 (<v8.75.0)
⌃ [50b5cf9e] NLLSsolver v3.1.2 (<v4.0.0)
⌅ [3bd65402] Optimisers v0.2.20 (<v0.3.2): Flux, OptimizationOptimisers
⌃ [7f7a1694] Optimization v3.14.0 (<v3.22.0)
⌃ [253f991c] OptimizationFlux v0.1.4 (<v0.2.1)
⌃ [4e6fcdb7] OptimizationNLopt v0.1.5 (<v0.2.0)
⌃ [36348300] OptimizationOptimJL v0.1.8 (<v0.2.2)
⌃ [42dfb2eb] OptimizationOptimisers v0.1.2 (<v0.2.1)
⌃ [1dea7af3] OrdinaryDiffEq v6.33.3 (<v6.71.0)
⌃ [1ed8b502] SciMLSensitivity v7.16.0 (<v7.55.0)
⌃ [102930c3] SmoothingSplines v0.3.1 (<v0.3.2)
⌅ [2913bbd2] StatsBase v0.33.21 (<v0.34.2): DataDrivenDiffEq, Forecast, LearnBase, MLDataPattern, MLDataUtils, MLLabelUtils, MLUtils, SmoothingSplines

```

---

<div class="post-metadata">

### Author: ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)
#### Post date: [February 19, 2024, 6:32pm UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/10 "2024-02-19T18:32:07Z")

</div>

That report basically says:

Optimization, OptimizationOptimisers can be updated (try that)

Optimisers cannot be updated b/c of Flux  
Flux cannot be updated b/c of OptimizationFlux

So one option is to update OptimizationFlux and keep your fingers crossed.

But then: why don’t you update all the packages?  
You want to use the latest optimizers, but not the latest versions of other packages? Any specific reasons to pin dependencies at outdated versions?

---

<div class="post-metadata">

### Author: ![KianH](https://avatars.discourse-cdn.com/v4/letter/k/c6cbf5/32.png) [@KianH](https://discourse.julialang.org/u/KianH)
#### Post date: [February 19, 2024, 8:44pm UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/11 "2024-02-19T20:44:58Z")

</div>

I have tried to update `OptimizationOptimisers` and `Optimization` however as the snippet below shows, the version of `OptimizationOptimisers` remains unchanged. As I mentioned earlier, this is where I am currently stuck. Theoretically this should work since there are not restrictions to updating `OptimizationOptimisers`, right ?

```julia
(@v1.9) pkg> update OptimizationOptimisers
    Updating registry at `C:\Users\.julia\registries\General.toml`
  No Changes to `C:\Users\.julia\environments\v1.9\Project.toml`
  No Changes to `C:\Users\.julia\environments\v1.9\Manifest.toml`

(@v1.9) pkg> status OptimizationOptimisers
Status `C:\Users\.julia\environments\v1.9\Project.toml`
⌃ [42dfb2eb] OptimizationOptimisers v0.1.2
Info Packages marked with ⌃ have new versions available and may be upgradable.

```

If possible I would like to update all packages but thought of starting with these for now.

---

<div class="post-metadata">

### Author: ![hendri54](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hendri54/32/9621_2.png) [@hendri54](https://discourse.julialang.org/u/hendri54)
#### Post date: [February 20, 2024, 1:26pm UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/12 "2024-02-20T13:26:06Z")

</div>

Strange … what do your `compat` entries look like?

What happens if you just update everything?

---

<div class="post-metadata">

### Author: ![KianH](https://avatars.discourse-cdn.com/v4/letter/k/c6cbf5/32.png) [@KianH](https://discourse.julialang.org/u/KianH)
#### Post date: [March 28, 2024, 9:47am UTC](https://discourse.julialang.org/t/trying-to-use-the-new-optimization-algorithm-sophia/110259/13 "2024-03-28T09:47:13Z")

</div>

Sorry for the late reply, I hadn’t seen your response. So when I run `compat` I get the following output:

```julia
(@v1.10) pkg> compat
  Select an entry to edit:
 > julia none
   [fbb218c0] BSON none
   [336ed68f] CSV none
   [b0b7db55] ComponentArrays none
   [717857b8] DSP none
   [2445eb08] DataDrivenDiffEq none
   [5b588203] DataDrivenSparse none
   [a93c6f00] DataFrames none
   [2b5f629d] DiffEqBase none
   [aae7a2af] DiffEqFlux none
   [41bf760c] DiffEqSensitivity none
   [0c46a032] DifferentialEquations none
   [31c24e10] Distributions none
   [5789e2e9] FileIO none
   [587475ba] Flux none
   [6efb3b9f] Forecast none
   [f6369f11] ForwardDiff none
   [a75be94c] GalacticOptim none
   [7073ff75] IJulia none
   [6a3955dd] ImageFiltering none
   [a98d9a8b] Interpolations none
   [033835bb] JLD2 none
   [7f56f5a3] LSODA none
   [0fc2ff8b] LeastSquaresOptim none
   [2fda8390] LsqFit none
   [b2108857] Lux none
   [10e44e05] MATLAB none
   [f0e99cf1] MLBase none
   [cc2ba9b6] MLDataUtils none
   [f1d291b0] MLUtils none
   [961ee093] ModelingToolkit none
   [50b5cf9e] NLLSsolver none
   [2774e3e8] NLsolve none
   [a15396b6] OnlineStats none
   [429524aa] Optim none
   [3bd65402] Optimisers none
   [7f7a1694] Optimization none
   [253f991c] OptimizationFlux none
   [4e6fcdb7] OptimizationNLopt none
   [36348300] OptimizationOptimJL none
   [42dfb2eb] OptimizationOptimisers none
   [1dea7af3] OrdinaryDiffEq none
   [f0f68f2c] PlotlyJS none
   [91a5bcdd] Plots none
   [f27b6e38] Polynomials none
   [49802e3a] ProgressBars none
   [438e738f] PyCall none
   [d330b81b] PyPlot none
   [1ed8b502] SciMLSensitivity none
   [012d725f] Smoothers none
   [102930c3] SmoothingSplines none
   [860ef19b] StableRNGs none
   [10745b16] Statistics none
   [2913bbd2] StatsBase none
   [9ff05d80] TickTock none
   [770da0de] UpdateJulia none
   [fdbf4ff8] XLSX none
   [e88e6eb3] Zygote none
   [0518478a] deSolveDiffEq none

```

When trying to update all packages using `up`, I get the following message:

```julia
(@v1.10) pkg> up
    Updating registry at `C:\Users\user\.julia\registries\General.toml`
    Updating git-repo `https://github.com/SciML/deSolveDiffEq.jl`
  No Changes to `C:\Users\user\.julia\environments\v1.10\Project.toml`
  No Changes to `C:\Users\user\.julia\environments\v1.10\Manifest.toml`
Precompiling project...
  ? DomainSets
  ? SciMLSensitivity
  ? DiffEqSensitivity
  ? Symbolics
  ? ModelingToolkit
  ? DataInterpolations → DataInterpolationsSymbolicsExt
  ? DiffEqFlux
  ? DataDrivenDiffEq
  ? DataDrivenSparse

```

So it seems like nothing really happens.

I tried to update some of my other packages such as `FileIO` and it was successful. However when I try to update the other packages nothing changes.