# How to apply an activation function to a subset of output units?

**URL:** <https://discourse.julialang.org/t/how-to-apply-an-activation-function-to-a-subset-of-output-units/33202>\
**Category:** Machine Learning\
**Tags:** flux\
**Created:** [January 10, 2020, 3:34pm UTC](https://discourse.julialang.org/t/how-to-apply-an-activation-function-to-a-subset-of-output-units/33202 "2020-01-10T15:34:37Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![holylorenzo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/holylorenzo/32/34169_2.png) [@holylorenzo](https://discourse.julialang.org/u/holylorenzo)\
**Post date:** [January 11, 2020, 4:30pm UTC](https://discourse.julialang.org/t/how-to-apply-an-activation-function-to-a-subset-of-output-units/33202/2 "2020-01-11T16:30:47Z")

</div>

Since in Flux, all layers and activation functions are just functions, this is straightforward to implement.

Let’s say we’re working with a `Dense` layer with a `relu` activation. Now usually you would construct your layer with `Dense(nin, nout, relu)` for `relu` to be applied to every output of the `Dense` layer.

We can write a custom activation layer that applies a regular activation function to all but the first output as follows:

```julia
struct PartialActivation
    activationfn
end
(pa::PartialActivation)(xs) = map((i, x) -> i == 1 ? x : pa.activationfn(x), eachindex(xs), xs)

```

This will apply the `activationfn` to all but the first element.

To use it in a model, you will have to switch from something that probably looks like

```julia
Chain(.., Dense(10, 10, relu), ...)

```

to

```julia
Chain(.., Dense(10, 10), PartialActivation(relu) ...)

```

Hope this helps and feel free to ask questions!

---

_[View the full topic](https://discourse.julialang.org/t/how-to-apply-an-activation-function-to-a-subset-of-output-units/33202)._