Input to Neural Network

Specific Domains Machine Learning
1

Hi,
I am new to both Julia and deep learning.

Suppose we define a following network:

using Flux


nn_width = 10
m = Chain(Dense(1,nn_width,tanh),
            Dense(nn_width,nn_width,tanh),
            Dense(nn_width,1))

input1 =Vector(-2:0.01:2)'
input2 =Vector(-2:0.01:2)



m(input1) # this works
m(input2) # this doesn't work

Q-1 What’s the reason for this behavior? My initial prior was both won’t work.

Q-2 How can we do something like m.(input) ? map(m, input) doesn’t work.

1 Like
2

The model m as defined takes a one-dimensional vector as input and outputs a one-dimensional vector:

julia> m([1.2])
1-element Vector{Float64}:
 -0.7267242113929453

Now,

  1. why does m(input2) not work?
    You are passing a vector of 401 dimensions instead of one.
  2. why does m.(input2) or map(m, input2) not work?
    This calls the model on each element, i.e., a scalar, of input. Yet, a Flux model requires a vector as input. The following will work:
    map(x -> m([x]), input2)
m.(eachrow(input2))
    Passing multiple inputs in this fashion is inconvenient though and the output is a nested vector of vectors. Thus, Flux allows passing a matrix of shape input dimension \times batch size to compute the outputs on a whole batch of inputs.
  3. why does m(input1) work?
    input1 has shape (1, 401) and is therefore interpreted as a batch of 401 one-dimensional inputs. Accordingly, it does work, computes the outputs on all 401 inputs and collects them into a matrix of shape output dimension \times batch size.
3 Likes