# Basics steps to build an ANN?

**URL:** <https://discourse.julialang.org/t/basics-steps-to-build-an-ann/21366>\
**Category:** Machine Learning\
**Created:** [March 1, 2019, 9:02pm UTC](https://discourse.julialang.org/t/basics-steps-to-build-an-ann/21366 "2019-03-01T21:02:16Z")\
**Posts on this page:** 5\
**Page:** 2

<div class="post-metadata">

**Author:** ![sairus7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sairus7/32/10816_2.png) [@sairus7](https://discourse.julialang.org/u/sairus7)\
**Post date:** [March 13, 2019, 3:57pm UTC](https://discourse.julialang.org/t/basics-steps-to-build-an-ann/21366/21 "2019-03-13T15:57:01Z")

</div>

There is a pure manual NN example on [macOS Python with numpy faster than Julia in training neural network - Stack Overflow](https://stackoverflow.com/questions/49719076/macos-python-with-numpy-faster-than-julia-in-training-neural-network)  
Also, l’ve recently tweaked that example further with matrix multiplications (got about x10 speed increase), l will make it public soon.

---

<div class="post-metadata">

**Author:** ![dellison](https://avatars.discourse-cdn.com/v4/letter/d/ce73a5/32.png) [@dellison](https://discourse.julialang.org/u/dellison)\
**Post date:** [March 13, 2019, 5:44pm UTC](https://discourse.julialang.org/t/basics-steps-to-build-an-ann/21366/22 "2019-03-13T17:44:14Z")

</div>

> [@BLI](#):
>
> What’s the advantage of using, say, `@Flux.epochs 3000 Flux.train!(...)` vs.
> 
> ```julia
> for i in 1:3000
> Flux.train!(...)
> end
> 
> ```

`Flux.@epochs` is just a simple loop that prints the epoch each time through, so you’re not losing anything by writing that loop yourself.

> <https://github.com/FluxML/Flux.jl/blob/master/src/optimise/train.jl#L103>

---

<div class="post-metadata">

**Author:** ![BLI](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bli/32/37206_2.png) [@BLI](https://discourse.julialang.org/u/BLI)\
**Post date:** [March 14, 2019, 9:51pm UTC](https://discourse.julialang.org/t/basics-steps-to-build-an-ann/21366/23 "2019-03-14T21:51:32Z")

</div>

Thanks for clarification!

---

<div class="post-metadata">

**Author:** ![BLI](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bli/32/37206_2.png) [@BLI](https://discourse.julialang.org/u/BLI)\
**Post date:** [March 18, 2019, 7:55am UTC](https://discourse.julialang.org/t/basics-steps-to-build-an-ann/21366/24 "2019-03-18T07:55:17Z")

</div>

Slight follow-up question. From the REPL, I get:

```julia
julia> using Flux

help?> Dense
search: Dense DenseArray DenseVector DenseMatrix DenseVecOrMat DimensionMismatch codeunits ncodeunits

  Dense(in::Integer, out::Integer, σ = identity)

  Creates a traditional Dense layer with parameters W and b.

  y = σ.(W * x .+ b)

  The input x must be a vector of length in, or a batch of vectors represented as an in × N matrix. The out y will be
  a vector or batch of length out.

```

It seems to me that the calling:

```julia
julia> d = Dense(param(randn(Float64, 2, 4)), param(zeros(Float64, 2)), σ)
Dense(4, 2, NNlib.σ)

```

uses different/undocumented arguments? Or does `param(randn(Float64,2,4)))` insert the `W` matrix into the model and respond with the `in` dimension?

---

<div class="post-metadata">

**Author:** ![BLI](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bli/32/37206_2.png) [@BLI](https://discourse.julialang.org/u/BLI)\
**Post date:** [March 18, 2019, 10:04am UTC](https://discourse.julialang.org/t/basics-steps-to-build-an-ann/21366/25 "2019-03-18T10:04:38Z")

</div>

OK… a little bit more testing. Seems like I can choose the initial choice of parameters by:

```julia
julia> using Flux
julia> W = [1 2 3 4;1 -1 2 0]
julia> b = [1,2]
julia> Dense(param(W),param(b));

```

… if I want to — I’m not saying that this is very useful, but it is good to know of the possibility.

I tried to do the same for an RNN, but in that case, this doesn’t work…

- Is that because the RNN doesn’t support this setting of initial parameters, or
- Is it because the RNN has feedback from the output of the activation function, and needs an additional weight matrix W\_f for this feedback?

I also tried to do:

```julia
julia> params(Dense(2,3))
Params([Float32[1.06203 0.2185; -0.234419 0.557728; -1.00303 -0.15665] (tracked), Float32[0.0, 0.0, 0.0] (tracked)])

```

which to me looks like a (tracked) array `[W, b]`.

On the other hand, doing:

```julia
julia> params(RNN(2,3))

```

leads to what looks like a (tracked) array `[W,W_f,b,b_f]` — my guess stems from `W` being a 2\times 3 matrix, `W_f` being a 3\times 3 matrix, while the two additional elements are 3 element vectors.

- The second vector is a null vector, i.e., `b_f = [0,0,0]`.
- I’m surprised there appears to be two bias vectors (`b`, `b_f` in my syntax). I wouldn’t think that the feedback signal around the activation function would need a bias — isn’t `b + b_f` the bias into the activation function?

[Previous page](https://discourse.julialang.org/t/basics-steps-to-build-an-ann/21366.md?page=1)