# Dense Layers, softmax, relu

**URL:** <https://discourse.julialang.org/t/dense-layers-softmax-relu/35520>\
**Category:** New to Julia\
**Created:** [March 4, 2020, 2:04pm UTC](https://discourse.julialang.org/t/dense-layers-softmax-relu/35520 "2020-03-04T14:04:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![samantp](https://avatars.discourse-cdn.com/v4/letter/s/f475e1/32.png) [@samantp](https://discourse.julialang.org/u/samantp)\
**Post date:** [March 4, 2020, 2:04pm UTC](https://discourse.julialang.org/t/dense-layers-softmax-relu/35520/1 "2020-03-04T14:04:40Z")

</div>

I am running through the 60 minute blitz in flux’s model zoo and was looking for some clarity on functions that I havn’t seen before. I am on line 205 and I am struggling to understand what the Dense function does, as well as what the relu option does. I have not been able to find documentation of functions in flux (?Dense returns that no documentation was found).

I know that this function somehow creates a dense layer in a neural net, and presumably the numbers are the number of columns and rows of the multiplication matrix in this layer? But I have so far had to learn just based on inference regarding what was going on. Can someone briefly explain either where I can find the documentation for functions in packages or alternately what `Dense(10,5, relu)` (model zoo, 60 minute blitz, line 205) does?

Thanks

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [March 4, 2020, 2:07pm UTC](https://discourse.julialang.org/t/dense-layers-softmax-relu/35520/2 "2020-03-04T14:07:07Z")

</div>

[https://fluxml.ai/Flux.jl/stable/models/layers/#Flux.Dense](https://fluxml.ai/Flux.jl/stable/models/layers/#Flux.Dense)

---

<div class="post-metadata">

**Author:** ![samantp](https://avatars.discourse-cdn.com/v4/letter/s/f475e1/32.png) [@samantp](https://discourse.julialang.org/u/samantp)\
**Post date:** [March 4, 2020, 2:14pm UTC](https://discourse.julialang.org/t/dense-layers-softmax-relu/35520/4 "2020-03-04T14:14:26Z")

</div>

Thanks, I think that makes some sense to me, and certainly I think I understand most of the line at this point, my only question remaining is what does the relu option do here?

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [March 4, 2020, 2:17pm UTC](https://discourse.julialang.org/t/dense-layers-softmax-relu/35520/5 "2020-03-04T14:17:17Z")

</div>

It’s the activation function:

[https://fluxml.ai/Flux.jl/stable/models/nnlib/#Activation-Functions-1](https://fluxml.ai/Flux.jl/stable/models/nnlib/#Activation-Functions-1)

---

<div class="post-metadata">

**Author:** ![samantp](https://avatars.discourse-cdn.com/v4/letter/s/f475e1/32.png) [@samantp](https://discourse.julialang.org/u/samantp)\
**Post date:** [March 4, 2020, 2:19pm UTC](https://discourse.julialang.org/t/dense-layers-softmax-relu/35520/6 "2020-03-04T14:19:30Z")

</div>

Thank you!!!