It’s the same advantage that most Julia packages have over most Python packages: rather than having to use specialized objects coming from some huge, complicated C or C++ code base that tend not to be compatible with anything else you are just using the normal objects of the language. It’s sort of as if you could have written TensorFlow using only Python lists and dicts (that’s a little bit of a false equivalency for a few reasons, but it does a good job expressing what I mean). Just look at how much easier it is to implement custom layers in Flux compared to TensorFlow! In Flux a custom layer is just perfectly ordinary code. You have to read through tons of TensorFlow documentation even to know where to start with it.
Also, I don’t know about PyTorch, but last I checked TensorFlow only worked with fully static computational graphs.
FYI,
If you use Windows, you can run CUDAnative.jl and CuArrays.jl with binary distribution of Julia v0.6.2 through ad-hoc patch to LLVM.jl.
You can use GPU without building Julia from source.
Thanks @yatra9! This worked for me and I’m now training a model on the GPU. I did run into an error with the crossentropy function that’s reported here:
So, it’s been one year since the last comment in this thread - how would y’all answer this question if asked again? Is Flux ready for a beginner deep learning project?
For example, one of the concerns was that using GPU with Flux wasn’t quite ready, but now it seems to be. I’m new to Julia, so it’s all new to me, but are there any recent developments that have contributed to Flux being more stable, suitable for beginners, and a better option than Pytorch/Tensorflow?
This is a provocative question. And the answer highly depends on your own previous experience with aforementioned libraries. What level are you at? Are you just starting out with this whole deep learning thing? Then Flux probably as good as it gets. Are you trying to do complex and high-performance experimentation? Then you already know the answer (hint: not Flux)
I probably have a little more deep learning experience than the poster when he originally asked the question, but I’m by no means an expert. But in general, I think the question as posed is useful to have a general answer to, for example to an audience of mixed skill levels. Generally, “is Flux a great tool for deep learning right now?”
When you say Flux is as good as it gets, are you saying it’s generally a better option than Tensorflow/Pytorch (taking into account multiple factors like stability, speed, documentation, etc.)?
Also, could you elaborate why Flux is not appropriate for complex and high-performance experimentation - is there a different Julia library that is better? I had read about some Flux slowness but wasn’t sure how much of an issue it still is.
As one of the maintainers of TensorFlow.jl,
I would recommend using Flux.
It is more intuitive, and more powerful.
On my benchmarks (CPU only), it is also marginally faster.
(Though do make sure to read the performance tips. It is much easier to write slow Flux code than it is to write slow TensorFlow.jl code. Because you simply can’t do a lot of slow things in TensorFlow very easily.)
I can’t imaging GPU being much slower, practically most of the operations are very standard and are handled by CUDA or CUDNN libraries. So it all ends up the same no matter the library.
Flux is very intuitive and tightly integrated with Julia. So it is easy to learn and get started with deep-learning using Flux. On the other hand, the other dominant deep learning libraries, PyTorch and TF, are “languages within languages”. Essentially you have to learn 2 separate languages and the internals are all hidden in C. Flux is all pure Julia, I think.
This is just my personal experience, to each their own. I think the main problem is that there hasn’t been enough critical mass of people who are working on optimizing and squeezing all possible juice from Flux+GPU. There are many GPU kernels that aren’t implemented or are too slow. It is just a matter of having enough developers throw their might at optimizing every kernel in Flux. After porting all my code from Flux into Pytorch I have never looked back, everything just worked right out of the gate, at blazingly fast speeds. It’s not a fault of Flux, it is just a matter of manpower.
Unfortunately Google decided to take TF into Swyft, instead of Julia. It could also be worthwhile looking into Swyft at this point, since they are likely to allocate a lot of resources to TF for Swyft.