# How to force flux parameters to be within a certain compact set? (projection)

**URL:** <https://discourse.julialang.org/t/how-to-force-flux-parameters-to-be-within-a-certain-compact-set-projection/52561>\
**Category:** General Usage\
**Tags:** flux\
**Created:** [December 29, 2020, 8:34am UTC](https://discourse.julialang.org/t/how-to-force-flux-parameters-to-be-within-a-certain-compact-set-projection/52561 "2020-12-29T08:34:47Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [December 29, 2020, 8:34am UTC](https://discourse.julialang.org/t/how-to-force-flux-parameters-to-be-within-a-certain-compact-set-projection/52561/1 "2020-12-29T08:34:47Z")

</div>

Hi, I’m trying to realise projected gradient descent-like methods.

[As Flux said](https://fluxml.ai/Flux.jl/stable/training/training/#Custom-Training-loops-1), I constructed my custom training loop as follows,

```julia
function train!(m::ICNN, loss, data, opt)
    ps = Flux.params(ICNN)
    for d in data
        gs = gradient(ps) do
            training_loss = loss(d...)
            return training_loss
        end
        Flux.update!(opt, ps, gs)
        # projection
        for layer in m.layers
            layer.Wz = project_nonnegative(layer.Wz)
        end
    end
end

```

However, the network are defined as immutable struct, and that’s why the projection procedure occurs the following errors:

```nohighlight
ERROR: LoadError: setfield! immutable struct of type ICNN_Layer cannot be changed
Stacktrace:
 [1] setproperty!(::ICNN_Layer, ::Symbol, ::Array{Float64,2}) at ./Base.jl:34
 [2] train!(::ICNN, ::Function, ::DataLoader{Tuple{LinearAlgebra.Adjoint{Float64,Array{Float64,2}},LinearAlgebra.Adjoint{Float64,Array{Float64,2}},LinearAlgebra.Adjoint{Float64,Array{Float64,2}}}}, ::Flux.Optimise.Optimiser) at /home/jinrae/.julia/dev/GliderPathPlanning/src/InputConvexNeuralNetworks.jl:168
 [3] top-level scope at /home/jinrae/.julia/dev/GliderPathPlanning/test/test.jl:81
 [4] include(::String) at ./client.jl:457
 [5] top-level scope at REPL[1]:1
in expression starting at /home/jinrae/.julia/dev/GliderPathPlanning/test/test.jl:81

```

What’s the best practical way to project network parameters?

EDIT: the network is constructed in a way similar to [Dense function](https://github.com/FluxML/Flux.jl/blob/8fb94be4a682fe6879368d55c26e50359c049703/src/layers/basic.jl#L85-L105).

---

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [December 29, 2020, 9:15am UTC](https://discourse.julialang.org/t/how-to-force-flux-parameters-to-be-within-a-certain-compact-set-projection/52561/2 "2020-12-29T09:15:40Z")

</div>

I used such an alternative way but it seems not be easily generalised for other cases:

```julia
function train!(m::ICNN, loss, data, opt)
    ps = Flux.params(ICNN)
    for d in data
        gs = gradient(ps) do
            training_loss = loss(d...)
            return training_loss
        end
        Flux.update!(opt, ps, gs)
        # projection
        for layer in m.layers
            if isdefined(layer, :Wz) # some layer may not have Wz for convenience
                project_nonnegative!(layer.Wz)
                if !all(layer.Wz .>= 0.0)
                    error("Projection seems not work.")
                end
            end
        end
    end
end

function project_nonnegative!(ps)
    ps .-= ps .* (ps .< 0.0)
end

```