Problems with Ipopt in Big Sur

Hello, I am using JuMP and Ipopt. It was all working but it failed after my Mac upgrade to Big Sur. I cannot even get the basic examples to work. The code runs, but the solver fails to find a solution due to numerical issues.

I suspect it is due to some problems with the libraries, but after a lot of debugging, reinstalling packages, brew, xcode, etc. I am at a loss. I am no expert, so any help from someone who has run into this issue would be greatly appreciated!

Sample code that runs into EXIT: Restoration Failed!

using JuMP, Ipopt
model = Model(with_optimizer(Ipopt.Optimizer))
@variable(model, x, start = 0.0)
@variable(model, y, start = 0.0)

@NLobjective(model, Min, (1 - x) ^ 2 + 100 * (y - x ^ 2) ^ 2)


Here is the test summary running test/nlp_solver.jl in JuMP:

Test Summary:                                        | Pass  Fail  Total
NLP solver tests                                     |   28   131    159
  HS071                                              |    1     3      4
  HS071 (no macros)                                  |    2     3      5
  HS071 (epigraph)                                   |          3      3
  HS109                                              |    1     3      4
  HS110                                              |    2            2
  HS111                                              |    1     3      4
  HS112                                              |    1     3      4
  HS114                                              |    1     3      4
  HS116                                              |    1     3      4
  HS118                                              |    1     4      5
  Two-sided constraints                              |    2     6      8
  Two-sided constraints (no macros)                  |    2     6      8
  Duals                                              |    6    46     52
  Quadratic inequality constraints, linear objective |    1     4      5
  Quadratic inequality constraints, NL objective     |    1     4      5
  Quadratic equality constraints                     |    1     4      5
  Fixed variables                                    |          4      4
  ifelse                                             |          3      3
  infeasible problem                                 |          1      1
  unbounded problem                                  |          1      1
  Derivatives of x^4, x < 0                          |    1     2      3
  Entropy maximization                               |          4      4
  Changing objectives                                |          6      6
  Setting NLobjective then objective                 |          6      6
  User-defined functions                             |          3      3
  Univariate user-defined functions                  |          3      3
  Issue #927                                         |    3            3
ERROR: Some tests did not pass: 28 passed, 131 failed, 0 errored, 0 broken.

What is the output of ] st -m?

I do not seem to be able to run the command that you suggest… (sorry, I only use Julia for mathematical programming, so I am not very familiar with it otherwise)

I have been exploring, and it seems to be a numerical problem with the derivatives that is affecting how the solver is deciding the next steps (or sth similar). For example, the above program works if I start it at the optimum. But almost exclusively when I do, otherwise it derails in different ways.

This works:

@variable(model, x, start = 1.0)
@variable(model, y, start = 1.0)

This start leads to: EXIT: Restoration Failed!

@variable(model, x, start = 0.0)
@variable(model, y, start = 0.0)

This start leads to a different error: EXIT: Iterates diverging; problem might be unbounded.

@variable(model, x, start = 0.9)
@variable(model, y, start = 1.0)

This is a simple example, but the same seems to happen to my code that was working before upgrading to Big Sur…

] st -m needs to be run from the REPL. If you can’t get it to work, then run:

import Pkg; Pkg.status(mode = Pkg.PKGMODE_MANIFEST)

Sorry, you probably were hoping for the manifest… It worked now:

Status `~/.julia/environments/v1.5/Manifest.toml`
  [ae81ac8f] ASL_jll v0.1.1+4
  [621f4979] AbstractFFTs v0.5.0
  [79e6a3ab] Adapt v2.3.0
  [b19378d9] ArrayFire v1.0.7
  [56f22d72] Artifacts v1.3.0
  [6e4b80f9] BenchmarkTools v0.5.0
  [b99e7846] BinaryProvider v0.5.10
  [6e34b625] Bzip2_jll v1.0.6+5
  [336ed68f] CSV v0.8.2
  [49dc2e85] Calculus v0.5.1
  [324d7699] CategoricalArrays v0.9.0
  [523fee87] CodecBzip2 v0.7.2
  [944b1d66] CodecZlib v0.7.0
  [bbf7d656] CommonSubexpressions v0.3.0
  [34da2185] Compat v3.23.0
  [e66e0078] CompilerSupportLibraries_jll v0.3.4+0
  [8f4d0f93] Conda v1.5.0
  [a8cc5b0e] Crayons v4.0.4
  [717857b8] DSP v0.6.9
  [9a962f9c] DataAPI v1.4.0
  [a93c6f00] DataFrames v0.22.1
  [864edb3b] DataStructures v0.18.8
  [e2d170a0] DataValueInterfaces v1.0.0
  [163ba53b] DiffResults v1.0.3
  [b552c78f] DiffRules v1.0.2
  [e2ba6199] ExprTools v0.1.3
  [8f5d6c58] EzXML v1.1.0
  [7a1cc6ca] FFTW v1.2.4
  [f5851436] FFTW_jll v3.3.9+6
  [59287772] Formatting v0.4.2
  [f6369f11] ForwardDiff v0.10.14
  [14197337] GenericLinearAlgebra v0.2.4
  [2e9cd046] Gurobi v0.8.1
  [cd3eb016] HTTP v0.9.1
  [83e8ac13] IniFile v0.5.0
  [1d5cc7b8] IntelOpenMP_jll v2018.0.3+0
  [d8418881] Intervals v1.5.0
  [41ab1584] InvertedIndices v1.0.0
  [b6b21f68] Ipopt v0.6.3
  [9cc047cb] Ipopt_jll v3.13.1+1
  [c8e1da08] IterTools v1.3.0
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.1.3
  [682c06a0] JSON v0.21.1
  [7d188eb4] JSONSchema v0.3.3
  [4076af6c] JuMP v0.21.5
  [2ddba703] Juniper v0.6.5
  [94ce4f54] Libiconv_jll v1.16.0+7
  [d00139f3] METIS_jll v5.1.0+5
  [856f044c] MKL_jll v2020.2.254+0
  [d7ed1dd3] MUMPS_seq_jll v5.3.5+0
  [1914dd2f] MacroTools v0.5.6
  [b8f27783] MathOptInterface v0.9.19
  [fdba3010] MathProgBase v0.7.8
  [739be429] MbedTLS v1.0.3
  [c8ffd9c3] MbedTLS_jll v2.16.8+1
  [e1d29d7a] Missings v0.4.4
  [78c3b35d] Mocking v0.7.1
  [d8a4904e] MutableArithmetics v0.2.12
  [76087f3c] NLopt v0.6.1
  [079eb43e] NLopt_jll v2.7.0+0
  [77ba4419] NaNMath v0.3.5
  [6fe1bfb0] OffsetArrays v1.4.0
  [656ef2d0] OpenBLAS32_jll v0.3.9+4
  [efe28fd5] OpenSpecFun_jll v0.5.3+4
  [bac558e1] OrderedCollections v1.3.2
  [69de0a69] Parsers v1.0.12
  [0f51c51e] PiecewiseLinearOpt v0.3.0
  [f27b6e38] Polynomials v1.1.13
  [2dfb63ee] PooledArrays v0.5.3
  [08abe8d2] PrettyTables v0.10.1
  [438e738f] PyCall v1.92.1
  [3cdcf5f2] RecipesBase v1.1.1
  [189a3867] Reexport v0.2.0
  [82193955] SCIP v0.9.6
  [91c51154] SentinelArrays v1.2.16
  [a2af1166] SortingAlgorithms v0.3.1
  [276daf66] SpecialFunctions v0.10.3
  [90137ffa] StaticArrays v1.0.0
  [856f2bd8] StructTypes v1.1.0
  [3783bdb8] TableTraits v1.0.0
  [bd369af6] Tables v1.2.1
  [f269a46b] TimeZones v1.5.2
  [3bb67fe8] TranscodingStreams v0.9.5
  [5c2747f8] URIs v1.1.0
  [81def892] VersionParsing v1.2.0
  [02c8fc9c] XML2_jll v2.9.10+3
  [a5390f91] ZipFile v0.9.3
  [83775a58] Zlib_jll v1.2.11+18
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8bb1440f] DelimitedFiles
  [8ba89e20] Distributed
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [44cfe95a] Pkg
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA
  [9e88b42a] Serialization
  [1a1011a3] SharedArrays
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode

Update your packages with import Pkg; Pkg.update(). You need to see [b6b21f68] Ipopt v0.6.5. This was a known issue we fixed.

Still same error. To be clear, it was perfectly running before upgrading to Big Sur, not sure if the previous known bug was Mac related. Now:

  [b6b21f68] Ipopt v0.6.5
  [9cc047cb] Ipopt_jll v3.13.2+1

Ok, never mind. I restarted everything and it finally works. Not sure why the update of Mac threw everything off, thank you for your patience!!! After all day re-installing packages, Julia, and everything else from scratch, not sure why it didn’t pick up the latest version!!!

1 Like