Take positive part of weights in loss function

Specific Domains Machine Learning
1

Hi! I’m trying to constrain my weights matrix to contain only positive values.

I was confused by answers to similar questions, and I thought it might make sense to implement my loss function such that, when it takes my weights in, it evaluates only their positive part.

Could someone explain (to a complete flux beginner) how to constrain a weights matrix to contain only non-negative elements, or write up a quick loss function that works with the positive part of weights?

2

filter!(>(0), x)

3

Where in the code should something like that go? In the loss function? When I try to put it in my loss function, I’m getting an error (the weights vector is a “NamedTuple{(:weight,), Tuple{Matrix{Float32}}}” if that helps).

I have been able to take the positive part of the weights only in my loss function, using

 theta = max.(ComponentVector{Float64}(theta), 0)

where theta is the weights vector, but the final weights end up being negative (because this doesn’t actually change the weights, it just makes it so that we only consider their positive parts in the loss function). Is there a way to constrain the weights themselves so that they cannot be made negative?

4

I resolved this question by making the weights the exponentiated versions of the values I solve for – thus, the weights must always be positive.

1 Like