Hi dear,
I did the same comparison using tensorflow, and the GPU performed better, so maybe Flux is not optimized on that type of GPU.
Anyways, thanks a lot for your support and guidance for this issue 
How much better?
If you feel up to it, you can profile the run, using e.g.
nvprof path/to/julia myfile.jl
and see what is taking time for the Flux model.
If not, it would make a valuable issue now that you have side-by-side Flux/TF implementations.
1 Like
Hi,
I ran MNIST training for 45 epochs, with different batch sizes and got below results:
Flux:
CPU:
batch size 100: 479.395901 seconds (52.17 M allocations: 303.536 GiB, 4.77% gc time)
batch size 512: 160.653196 seconds (6.71 M allocations: 184.139 GiB, 10.34% gc time)
batch size1024: 256.346342 seconds (3.39 M allocations: 169.667 GiB, 53.28% gc time)
batch size 2048: 250.305340 seconds (1.73 M allocations: 162.432 GiB, 55.05% gc time)
GPU:
batch size 100 483.669281 seconds (33.77 M allocations: 302.615 GiB, 4.81% gc time)
batch size 512: 159.605954 seconds (6.78 M allocations: 184.142 GiB, 10.33% gc time)
batch size1024: 255.784214 seconds (3.45 M allocations: 169.670 GiB, 53.21% gc time)
batch size 2048: 246.802858 seconds (1.80 M allocations: 162.434 GiB, 55.49% gc time)
Tensorflow:
GPU:
batch size 100: 368 seconds
batch size 512: 125 seconds
batch size1024: 111 seconds
batch size 2048: 97 seconds
@kristoffer, actually Iām using notebook, not a file.jl so anything you need me to do by this command ?
nvprof path/to/julia myfile.jl