# Flux is lagging far beyond tensorflow with a pretty basic use case

**URL:** <https://discourse.julialang.org/t/flux-is-lagging-far-beyond-tensorflow-with-a-pretty-basic-use-case/67942>\
**Category:** Machine Learning\
**Tags:** tensorflow, flux\
**Created:** [September 9, 2021, 3:26pm UTC](https://discourse.julialang.org/t/flux-is-lagging-far-beyond-tensorflow-with-a-pretty-basic-use-case/67942 "2021-09-09T15:26:29Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![egolep](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/egolep/32/23498_2.png) [@egolep](https://discourse.julialang.org/u/egolep)\
**Post date:** [September 9, 2021, 3:26pm UTC](https://discourse.julialang.org/t/flux-is-lagging-far-beyond-tensorflow-with-a-pretty-basic-use-case/67942/1 "2021-09-09T15:26:29Z")

</div>

I was trying to move from a Python ML/DL stack to Julia (so from something like sklearn+pytorch/tensorflow to MLJ+Flux).  
So I decided to rewrite the Kaggle courses in Julia, switching from the Python libraries to the Julia ones too. In particular, I was implementing the notebook regarding underfitting and overfitting, using also early stopping (you can find it opening the relative exercise here: [Learn Intro to Deep Learning Tutorials](https://www.kaggle.com/learn/intro-to-deep-learning)).  
I reached the second neural network, the first “deep” neural network without early stopping and I stumbled upon a huge difference in predictive performance: in fact, in just 50 epochs tensorflow is able to achieve a loss of 0.1992 and start from something like 0.29, while the same architecture, with the same loss and the same optimizer in flux starts from something like 90.000 and reaches a loss ranging from 3.43 to 0.736 (depending on rngs, I suppose).

Which could be the problem? I followed the same steps as much as possible and followed the models in the Flux’s model zoo for the specific implementations

---

<div class="post-metadata">

**Author:** ![CarloLucibello](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carlolucibello/32/3278_2.png) [@CarloLucibello](https://discourse.julialang.org/u/CarloLucibello)\
**Post date:** [September 9, 2021, 4:04pm UTC](https://discourse.julialang.org/t/flux-is-lagging-far-beyond-tensorflow-with-a-pretty-basic-use-case/67942/2 "2021-09-09T16:04:00Z")

</div>

Such difference in starting loss is highly suspicious, there is likely a mistake in your port. You should post the two scripts for specific help

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 9, 2021, 4:21pm UTC](https://discourse.julialang.org/t/flux-is-lagging-far-beyond-tensorflow-with-a-pretty-basic-use-case/67942/3 "2021-09-09T16:21:32Z")

</div>

I agree - if they fundamentally differ in such a way (even with trying to use the same methods) there’s likely some discrepancy left between your tensorflow code and your Flux code. I wouldn’t primarily place the blame on RNGs here - such a big difference for the “same architecture” shouldn’t happen just based on RNG alone.

Do you mind posting your two versions (tensorflow/Flux) so we can take a look?

---

<div class="post-metadata">

**Author:** ![egolep](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/egolep/32/23498_2.png) [@egolep](https://discourse.julialang.org/u/egolep)\
**Post date:** [September 9, 2021, 4:36pm UTC](https://discourse.julialang.org/t/flux-is-lagging-far-beyond-tensorflow-with-a-pretty-basic-use-case/67942/4 "2021-09-09T16:36:24Z")

</div>

I (partially) unveiled the mistery: using MLJ to preprocess the data. If I use a version of a MinMaxScaler implemented by me for MLJModels and OneHotEncoder from MLJ, I get the behavior described above.  
Then I implemented both parts “by hands” (using onehotbatch from Flux and manually scaling the fatures for the MinMaxScaler) I achieved performances comparable to Tensorflow.  
It is strange to me that MLJ cooperate that bad with Flux, so I’ll investigate further on why this actually happens.  
Meanwhile, I would call the problem closed since it’s obvious that Flux is not to be blamed.

Many thanks for your replies!

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [September 9, 2021, 4:42pm UTC](https://discourse.julialang.org/t/flux-is-lagging-far-beyond-tensorflow-with-a-pretty-basic-use-case/67942/5 "2021-09-09T16:42:05Z")

</div>

Rather than MLJ and Flux not cooperating (if they didn’t, [https://github.com/FluxML/MLJFlux.jl](https://github.com/FluxML/MLJFlux.jl) wouldn’t exist!), it seems like your by hand implementation may be different than what MLJ is doing. Worth comparing both to verify that’s the case.

---

<div class="post-metadata">

**Author:** ![egolep](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/egolep/32/23498_2.png) [@egolep](https://discourse.julialang.org/u/egolep)\
**Post date:** [September 9, 2021, 4:43pm UTC](https://discourse.julialang.org/t/flux-is-lagging-far-beyond-tensorflow-with-a-pretty-basic-use-case/67942/6 "2021-09-09T16:43:36Z")

</div>

That’s why I said that it looks quite strange. Now I’m looking at the output of the preprocessing that I did with MLJ and the one done by hand in order to understand what’s different.