LazyBridgeOptimizer not working with copy_to

Hi, I’m writing a script in order to convert files from CBF to NL format. See example below:

src = MOI.FileFormats.Model(format = MOI.FileFormats.FORMAT_CBF);
MOI.read_from_file(src, "path_to_file");
dest = MOI.FileFormats.Model(format = MOI.FileFormats.FORMAT_NL);
bridged = MOI.Bridges.full_bridge_optimizer(dest, Float64);
MOI.copy_to(bridged, src);
MOI.write_to_file(dest, replace("path_to_file", ".cbf.gz" => ".nl"))

I had to add the bridge due to there being constraints in the CBF files which were not accepted in NL format.
However, I now get the error: “LoadError: Model MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.FileFormats.NL.Model} does not support copy_to.”

Does anyone have an idea of a workaround ?

Huge thanks in advance !

Hi @thibault-vignon, welcome to the forum :smile:

The easiest approach is to use JuMP’s read_from_file and write_to_file. It will automate the bridging for you.

using JuMP
model = Model()
@variable(model, x >= 0)
write_to_file(model, "out.cbf")

function transform_file_format_jump(filename; from, to)
    model = read_from_file(filename)
    output_filename = replace(filename, from => to)
    write_to_file(model, output_filename)
    return output_filename
end
transform_file_format_jump("out.cbf"; from = ".cbf", to = ".nl")

But if you really want to use MOI, you need:

function transform_file_format_moi(filename; from, to)
    src = MOI.FileFormats.Model(; filename)
    MOI.read_from_file(src, filename)
    output_filename = replace(filename, from => to)
    inner = MOI.FileFormats.Model(; filename = output_filename)
    dest = MOI.instantiate(() -> inner; with_bridge_type = Float64)
    MOI.copy_to(dest, src)
    MOI.write_to_file(inner, output_filename)
    return output_filename
end
transform_file_format_moi("out.cbf"; from = ".cbf", to = ".nl")

Note that there’s little point in rewriting CBF to NL, because JuMP does not reformulate conic constraints into their scalar nonlinear equivalents. You’ll get an error like:

julia> using JuMP

julia> begin
           model = Model()
           @variable(model, x[1:3])
           @constraint(model, x in MOI.ExponentialCone())
           write_to_file(model, "out.cbf")
           transform_file_format_jump("out.cbf"; from = ".cbf", to = ".nl")
       end
ERROR: Unable to write problem to file because the chosen file format doesn't support constraints of the type MathOptInterface.VectorOfVariables-in-MathOptInterface.ExponentialCone.

Thanks for the answer @odow !
I was clearly barking up the wrong tree, as reformulating the conic constraints into their scalar nonlinear equivalents was exactly what I was hoping to use JuMP for.
Might you perhaps know of an other way (not necessarily in Julia) to do this ?

1 Like

What cones do you want? We could add this to JuMP

I’m trying to do this for the entire CBLIB benchmark, which I admit is potentially a lot of different kind of cones. You pointed out that exponential cones aren’t yet handled by JuMP, but when I tried your first suggestion I actually got errors indicating that it wasn’t able to handle constraints involving rotated second-order cones. This suprised me, because looking at the list of bridges in the docs I saw several meant to handle precisely rotated second-order cones ?

Some bridges are not added by default. You’d need:

using JuMP
model = Model()
@variable(model, x[1:3])
@constraint(model, x in RotatedSecondOrderCone())
write_to_file(model, "out.cbf")

function transform_file_format_moi(filename; from, to)
    src = MOI.FileFormats.Model(; filename)
    MOI.read_from_file(src, filename)
    output_filename = replace(filename, from => to)
    inner = MOI.FileFormats.Model(; filename = output_filename)
    dest = MOI.instantiate(() -> inner; with_bridge_type = Float64)
    MOI.Bridges.add_bridge(
        dest,
        MOI.Bridges.Constraint.SOCtoNonConvexQuadBridge{Float64},
    )
    MOI.copy_to(dest, src)
    MOI.write_to_file(inner, output_filename)
    return output_filename
end
transform_file_format_moi("out.cbf"; from = ".cbf", to = ".nl")

How do you expect PSDCone to be written to .nl files?

Using the above code snippet results in the .nl file for an empty problem (1 objective but no variables or constraints), whichever cbf file I feed into it.

To answer your question on PSD cones, I’m by no means an expert in conic optimization, but I’d naturally want to rewrite a PSD cone variable as M.transpose multiplied by M, where M is a general square matrix variable.

I guess you need:

julia> function transform_file_format_moi(filename; from, to)
           src = MOI.FileFormats.Model(; filename)
           MOI.read_from_file(src, filename)
           output_filename = replace(filename, from => to)
           inner = MOI.FileFormats.Model(; filename = output_filename)
           dest = MOI.instantiate(() -> inner; with_bridge_type = Float64)
           MOI.Bridges.add_bridge(
               dest,
               MOI.Bridges.Constraint.SOCtoNonConvexQuadBridge{Float64},
           )
           MOI.copy_to(dest, src)
           MOI.Utilities.attach_optimizer(dest.model)  # <-- new
           MOI.write_to_file(inner, output_filename)
           return output_filename
       end

What’s the motivation to rewrite CBLIB. into nonlinear? Typically, people have taken nonlinear models and reformulated then into CBF. Going the other way naively isn’t obviously better, and we can’t easily recover structure that may have been present in the original model.

For example, if the original model had log(sum(exp.(x))), that’s a neat and tidy NLP expression that we can handle. But the conic reformulation introduces a bunch of extra variables and constraints. Going to that NLP formulation might be worse, but that doesn’t imply much.

I’m attempting to benchmark a general non-linear solver which is suited to SOCP problems but not specialized for them, and which doesn’t accept CBF as an input format.
By adding the ‘attach_optimizer’ call I was finally able to convert some instances to NL format, thanks for that I wouldn’t have been able to find that trick on my own. It ended up failing on an instance due to a PSD cone constraint, which you had foreseen. I see that you’ve already added a new bridge for exponential cones, are PSD cones on the menu soon ? Thanks for being so reactive !

are PSD cones on the menu soon

I don’t think so. But we have our monthly developer call on Thursday 2:30pm Eastern that I will discuss this at. (DM me if you want an invite)