I would like to implement an Input Convex Neural Networks ICNN with the awesome convex.jl tool. The purpose to minimize a cost function. However, in order to get convexity with Icnn, weighting matrices are nonnegative and activation function is convex and nonnegative. So within convex, there is the pos(x) and sound like relu activation function.

I did the following evaluation:

using Flux
using Convex
W = NNlib.relu.(rand(3,3))
b = rand(3,1)
x = Variable(3,1)
y = Variable(3,1)
expr = Convex.pos(W * x[:,1] + b)

Then I tried :

y[:,1] == Convex.pos(W * x[:,1] + b)

And Iβve got the warning

β Warning: Expression not DCP compliant. Trying to solve non-DCP compliant problems can lead to unexpected behavior.

Iβm not a professional with DCP subject. However I do not understand why added the convex.jl constraint lead to βwarmingβ as βyβ is a variable. Maybe a clever person can help me.

Hi!
I donβt know anything about ICNNs but I think I see where your problem is.
When doing convex optimization, you are allowed two types of constraints: affine equality constraints and convex inequality constraints. This ensures that the feasible region you define is a convex set.
The function f(x) -> pos(Wx+b) is convex but not affine, so I guess that is why the equality constraint y == f(x) is not accepted by Convex.jl? Could you try with an inequality just to check?

Yeah, just as @gdalle said, expressions of the form convex_function_of_variables >= 0 are not allowed (and if zero isnβt on one side, we subtract terms over to the other so it is), because that region is not a convex set. Think about a disk: the equation for (x,y) to be in a circular disk of radius r is x^2 + y^2 <= r^2: