# Individual ctivation function for each network output

**URL:** <https://discourse.julialang.org/t/individual-ctivation-function-for-each-network-output/102676>\
**Category:** Machine Learning\
**Created:** [August 10, 2023, 11:53am UTC](https://discourse.julialang.org/t/individual-ctivation-function-for-each-network-output/102676 "2023-08-10T11:53:24Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![tommy\_J](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tommy_j/32/51703_2.png) [@tommy\_J](https://discourse.julialang.org/u/tommy_J)\
**Post date:** [August 10, 2023, 11:53am UTC](https://discourse.julialang.org/t/individual-ctivation-function-for-each-network-output/102676/1 "2023-08-10T11:53:24Z")

</div>

Hi,

Can I use a different activation function for each output of my network?

```julia
using Lux
net = Chain(Dense(3, 10, tanh), Dense(10, 3, tanh))

```

I know that output `2` can only be positive, but `1` and `3` can be negative.

Is it possible to define a particular activation function for each output of the network?  
`1` and `3` → tanh  
`2` → relu

If so, how can I do that?

---

<div class="post-metadata">

**Author:** ![contradict](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@contradict](https://discourse.julialang.org/u/contradict)\
**Post date:** [August 11, 2023, 8:38pm UTC](https://discourse.julialang.org/t/individual-ctivation-function-for-each-network-output/102676/2 "2023-08-11T20:38:19Z")

</div>

You can probably do this with a custom layer.

```julia
struct ChannelActivations{T}
  activations::T
end

ChannelActivations(args...) = ChannelActivations(args)

function (a::ChannelActivations)(x::AbstractMatrix, ps, st::NamedTuple)
    y = reduce(vcat, [f.(v) for (f, v) in zip(a.activations, eachrow(x))])
    return y, st
end

Lux.parameterlength(::ChannelActivations) = 0

Lux.statelength(::ChannelActivations) = 0

# no specified activation on the second Dense makes it a linear layer.
net = Chain(Dense(3, 10, tanh), Dense(10, 3), ChannelActivations(tanh, relu, tanh))

```