# Flux vs Keras time series prediction performance

**URL:** <https://discourse.julialang.org/t/flux-vs-keras-time-series-prediction-performance/38421>\
**Category:** Machine Learning\
**Tags:** question\
**Created:** [April 29, 2020, 1:30pm UTC](https://discourse.julialang.org/t/flux-vs-keras-time-series-prediction-performance/38421 "2020-04-29T13:30:38Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Volker](https://avatars.discourse-cdn.com/v4/letter/v/77aa72/32.png) [@Volker](https://discourse.julialang.org/u/Volker)\
**Post date:** [April 29, 2020, 1:30pm UTC](https://discourse.julialang.org/t/flux-vs-keras-time-series-prediction-performance/38421/1 "2020-04-29T13:30:38Z")

</div>

Hi,

I compare the prediction quality of two neural networks on a time series. One trained by Keras and one trained by Flux. Unfortunately I´m not able to achieve the same quality (see figure) and I don´t know why.

![grafik](https://global.discourse-cdn.com/julialang/original/3X/7/e/7e0acf3043f9e894be290d4d184e2436d9688ba4.png)

I used the same data and tried to use the same settings (e.g. batch size, optimizer, layers, activation functions, neurons).

Can anyone see what I´ve forgotten? Or explain why the performance is so different?

```julia
batch_size = 32
train_loader = Flux.Data.DataLoader(X', y_train', batchsize=batch_size)

num_epochs = 1000

opt = ADAM(0.001, (0.9, 0.999))
m = Chain(Dense(size(X, 2), 30), Dense(30, 30), Dense(30, 1))
loss(x, y) = Flux.mse(m(x), y)
ps = Flux.params(m)

@time Flux.train!(loss, ps, ncycle(train_loader, num_epochs), opt)

@time MyKerasWrapper.train_nn((X, y_train), (X, y_train), [30, 30, 1],["sigmoid", "sigmoid", "sigmoid"], num_epochs,[], 1000, String(@ __DIR__ ) * raw"\keras_test_model_4_comparision_with_flux",
                                        1, "adam", 0.001, batch_size)

![grafik|600x400](upload://qJcEuUChNWJOc7k5izp08OmWU58.png) 

y_model_train_keras = Float64.(MyKerasWrapper.eval_nn(X, String(@ __DIR__ ) * raw"\keras_test_model_4_comparision_with_flux"))

y_model_train_flux = Float64.(vec(m(X')))

time_vec = collect(0.1:0.1:length(y_train) * 0.1)

plot(time_vec, [y_train, y_model_train_flux, y_model_train_keras], xlabel = "t in s", ylabel = "y", label=["Measurement" "Flux-Model" "Keras-Model"])

```

---

<div class="post-metadata">

**Author:** ![DrChainsaw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/drchainsaw/32/8497_2.png) [@DrChainsaw](https://discourse.julialang.org/u/DrChainsaw)\
**Post date:** [April 29, 2020, 2:08pm UTC](https://discourse.julialang.org/t/flux-vs-keras-time-series-prediction-performance/38421/2 "2020-04-29T14:08:20Z")

</div>

Syncing hyperparameters between frameworks to get the same performance is generally painful to do imo. Sometimes it is faster to just retune for the wanted framework.

Check weight initialization and whether the mse is averaged or summed as these things are typically not the same between frameworks.

---

<div class="post-metadata">

**Author:** ![CarloLucibello](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carlolucibello/32/3278_2.png) [@CarloLucibello](https://discourse.julialang.org/u/CarloLucibello)\
**Post date:** [April 29, 2020, 4:30pm UTC](https://discourse.julialang.org/t/flux-vs-keras-time-series-prediction-performance/38421/3 "2020-04-29T16:30:35Z")

</div>

It looks like you are not using any non-linearity: `Dense(30,30)` is equivalent to `Dense(30, 30, identity)`, you want `sigmoid` instead

---

<div class="post-metadata">

**Author:** ![Volker](https://avatars.discourse-cdn.com/v4/letter/v/77aa72/32.png) [@Volker](https://discourse.julialang.org/u/Volker)\
**Post date:** [April 30, 2020, 7:45am UTC](https://discourse.julialang.org/t/flux-vs-keras-time-series-prediction-performance/38421/4 "2020-04-30T07:45:45Z")

</div>

Thanks, that was the problem. I thought, that sigmoid was the default activation function. Now it works 🙂

![grafik](https://global.discourse-cdn.com/julialang/original/3X/c/f/cf0ef7ecff1bc7eeed3159ad2f4a1a799a8fbe6b.png)