# Multiple linear regression model using Flux.jl

Hello everyone,

I’m new to machine learning and I was trying to extend the basic example from Flux documentation to a multiple linear regression model where the input `S` and the output `V` are both vectors with 3 elements each, x, y and z.

As long as the relation between `S` and `V` is identity, I get great predictions using my model. Simply changing the function between `S` and `V` as in the following example so that V_y = 2S_y, leads to wrong results.

What could be the cause of this problem?

``````## Random Data set
F = 3
D = 1800

S_train, S_test = rand(1:100, F, D), rand(1:100, F, D)
``````
``````## V = ƒ(S), both V & S are vectors with 3 elements
function ƒ(S::Matrix)
V = zero(S)
D = size(S, 1)

D > 1 && for (x, y, z) in Iterators.partition(eachindex(S), D)
V[x] = S[x]
V[y] = 2S[y]
V[z] = S[z]
end

isone(D) && (V .= map(actual, S))
return V
end
V_train, V_test = ƒ(S_train), ƒ(S_test)
``````
``````# default activation function: identity
using Flux: train!

opt = Descent(1e-7) ## optimizer: classic gradient descent strategy
data = [(V_train, S_train)]
model = Dense(F => F)
parameters = Flux.params(model)
loss(x, y) = Flux.Losses.mse(model(x), y)

for _ in Base.OneTo(100_000)
train!(loss, parameters, data, opt)
end
``````
``````model(S_test)
3×1800 Matrix{Float32}:
79.9905  51.0251  83.9732   1.19865  …   7.04939  99.9106  69.8579  21.1866
12.4992  36.5154  16.4895  20.1102      41.5276   20.9554  21.4264  43.6019
80.0067  62.9638  77.0303  81.7296      25.9406   68.1154  14.1989  97.7402

V_test
3×1800 Matrix{Int64}:
80   51  84   1  40  86   97   77  61  …  82  49  28  75    7  100  70   21
50  146  66  80  44  42  154  194   4     30   8  62  54  166   84  86  174
80   63  77  82   3  68   94   35   9     78  77  81  32   26   68  14   98
``````

Train calls the loss function as `loss(d...)` if each datum `d` is a tuple, i.e., with your definition of `data` it is called as `loss( (V_train, S_train)... ) = loss(V_train, S_train)`. On the other hand, your loss function is defined as taking input and target as arguments (in that order). Thus, your model learns to map `V` to `S` instead of the intended mapping. Flipping either the order of targets and inputs in your data set or in the arguments of the loss function will fix it.

2 Likes