DiffOpt - differentiating through a QP/different from manual

Specific Domains Optimization (Mathematical)
4

Well, it’s difficult to argue with finite differences… The signs matter, especially when using the function dual from JuMP. For the future:

Code for Amos and Kolter (2017) 
######This uses duals from JuMP which can be negative. 
######However, Amos and Kolter (2017) assume λ≥0, so
######we need to adjust the signs
######Duals from JuMP are non-positive for ≤ constraints
q = q
G = G
h = h
Q = Q

LHM = [
    Q -transpose(G)
    -Diagonal(dual.(cons))*G -Diagonal(G*value.(x)-h) 
]
LHMinv = inv(LHM)
k1 = length(LHM[:,1])
k2 = length(Diagonal(dual.(cons))[1,:])
RHMb = [zeros(k1-k2,k2)
-Diagonal(dual.(cons))] #Matrix(1.0I, size(h)[1])  
RHMc = [-Matrix(1.0I, 2,2)
zeros(k2,2)]

RHMc = [-Matrix(1.0I, 2,2)
zeros(k2,2)]

dallb = LHMinv*RHMb    ####both w and λ
dallc = LHMinv*RHMc    ####both w and λ

dwh = dallb[1:2,:] ####picking only w
dwq = dallc[1:2,:] ####picking only w

dAm = zeros(size(G))
dwA = []
for i = 1:2
        dAm = zeros(size(G))
        dAm[i] = 1.0 ######Do for every element of G
        RHMA = [
            transpose(dAm).* dual.(cons)
            Diagonal(dual.(cons)).* dAm* value.(x)
        ]
        dallAtemp = LHMinv * RHMA
        dwAtemp=dallAtemp[1:2,1:end] ####picking only w
        push!(dwA, dwAtemp)
end
dwG = reduce(hcat,dwA)

@show dwh,dwq,dwG
Code for Boot (1963) 
x_s = Q^(-1)*(-q)-Q^(-1)*G'*(G*Q^(-1)*G')^(-1)*(G*Q^(-1)*(-q)-h)

dxih=Q^(-1)*G'*(G*Q^(-1)*G')^(-1)
dxiq=(Q^(-1)-dxih*G*Q^(-1))
vh = (G*Q^(-1)*G')^(-1)*(G*Q^(-1)*(-q)-h)
dxiA1=-x_s[1]*dxih-vh[1]*dxiq[:,1] ####We use the expressions as they are derived by the authors
dxiA2=-x_s[2]*dxih-vh[1]*dxiq[:,2]
dxiG = hcat(dxiA1,dxiA2)

dxiq = -dxiq #####But we know that a=-q so we need to switch signs

@show dxih,dxiq,dxiG;
1 Like