Dear JuMP Community,
I have previously asked a question where Huber regression gave slow progress error in MOSEK. I am now facing similar (and even bigger) issues in logistic regression and I would like to understand if I am doing any mistakes. I followed the JuMP exponential cone modeling tricks, and I did several tests. I cannot see where I am making a mistake, or if there is a bug somewhere in the code. Any support will be extremely helpful.
Problem 1: SLOW Progress
I followed exactly the logistic regression modeling tricks on the JuMP Conic optimization tutorials page. Unfortunately, I get Slow progress error. Here are the relevant functions I am using:
"Takes a model and adds a softplus constraint. See https://jump.dev/JuMP.jl/stable/tutorials/conic/logistic_regression"
function softplus(model, t_aux, linear_transform) #exponential cone constraint
# models constraints of form: log(1 + exp(linear_transform)) <= t
z = @variable(model, [1:2], lower_bound = 0.0) #anonymized variable
#add the exp-cone constraints
@constraint(model, sum(z) <= 1.0)
@constraint(model, [linear_transform - t_aux, 1, z[1]] in MOI.ExponentialCone())
@constraint(model, [-t_aux, 1, z[2]] in MOI.ExponentialCone())
end
function build_logit_model(X, y)
N, n = size(X) #N rows, n predictors
model = Model(MosekTools.Optimizer) #start the model
@variable(model, beta[1:n]) #beta coefficients
@variable(model, beta_0) #intercept
@variable(model, t[1:N]) #auxiliary variables
for i in 1:N
linear_transform = -y[i]*(X[i, :]' * beta + beta_0)
softplus(model, t[i], linear_transform)
end
# Define objective
@objective(model, Min, sum(t)) #this gives slow progress error
return model
end
Problem 2: Wrong solutions
Recall that the above exponential conic constraints were used to model constraints in the following form:
t_i \geq \log(1 + \exp( - y^i \cdot (\mathbf{\beta}^\top \mathbf{x} + \beta_0))), \ i = 1,\ldots,N
and we minimize \sum_{i=1}^N t_i.
Now, I am adding a new variable, u \geq 0, and updating the above constraints as
t_i \geq \log(1 + \exp( - y^i \cdot (\mathbf{\beta}^\top \mathbf{x} + \beta_0) + u)), \ i = 1,\ldots,N. I achieved this by replacing the previous for loop (that added softplus constraints) with:
@variable(model, u >= 0.0)
for i in 1:N
linear_transform = (-y[i]*(X[i, :]' * beta + beta_0)) + u
softplus(model, t[i], linear_transform)
end
As \log and \exp are increasing functions, at optimality we need to have u=0. However, I get results like u = 0.5.
I am attaching my MWE codes – main_test is the 10-line main file, logistic_regression_tests includes the functions. Thank you for your time.
logistic_regression_test.jl (2.5 KB)
main_test.jl (610 Bytes)