Here are the run times for 5 epochs on the same machine:
Flux.jl: 29.8 seconds (smallest run-time out of 3 tests)
Tensorflow (python): 19.7 seconds
Unfortunately, the run time difference is too large for me, to migrate to Flux.jl. However, if I manage to optimize the Flux.jl model, I will post my updates here.
Flux: 0.9; julia 1.2.0; Tensorflow: 1.12; CuArrays 8e00af6