Is it a good time for a PyTorch developer to move to Julia? If so, Flux? Knet?

Hi! I’m not qualified to comment on the ML frameworks per se, but I can tell you about the IDE/debugger situation and you can decide for yourself if Julia’s current capabilities satisfy you. Tl;dr yes, there are sufficiently good debuggers and IDEs (yes, debuggers, you read that right), but there are important differences between Julia and Python which necessitate a somewhat different mindset.

First, an answer to your question about Debugger/IDE

There are two main IDEs in Julia, which are really plugins for Atom and VS Code. The plugin for Atom is called Juno; the plugin for VS Code is called Julia for VS Code. They’re both really good and have nice autocomplete options, but if you want debugger support in the IDE, you want Juno. If you’re like me, and you end up using two windows, one with your text editor and one with your Julia REPL, as your “IDE,” any text editor with autocomplete will do.

There are also two different (main) debuggers that you’ll want to learn about, because each one has different use cases. Debugger.jl is your best bet if you want to step through functions, but it runs Julia code through the interpreter instead of compiling it, and if you need to figure out why your ML model starts diverging after training for 30 epochs on 10 million data points, that’s going to be too slow for you. If you need to debug but also compile your code for speed, you’ll need Infiltrator.jl, which will allow you to observe everything that’s going on at the breakpoints you set, but will only enter debug mode at the breakpoints that you set before running your code. I believe the integrated debugger in Juno is based on Debugger.jl, but I’m not 100% sure.

Second, some differences between Julia and Python

I want to point out something else that may be difficult to deal with if you’re coming from Python (it was for me!). Julia code is 100% compiled before execution, unless using the aforementioned interpreter, which you almost never do unless you’re debugging. Because of various issues that you may learn more about later, it’s difficult to store compiled code between Julia sessions (possible, but difficult).

The major upshot of this is that your basic workflow is probably going to be very different than it is in Python. If you have a script you want to run, say with julia myscript.jl, and you want it to analyze some data and make a plot, which is a pretty common thing to do in Python, Julia is going to have to compile your entire script plus all the functions you called from the plotting package and other packages, every time you run your script. This is a pathological case for running Julia code quickly. It may not make much difference if your ML models are big enough (i.e. they more than several seconds to run), but it will be very noticeable when trying to do quick data-analysis-type stuff.

The main way to deal with this is a Julia package called Revise which essentially hot-reloads changes you save to your Julia code. Essentially everybody here uses Revise. Learn to love it. :smiley: (I can comment more on how to use Revise if you’re interested, although everybody has their own way to do it.)

Another major difference (some might say, the major difference) between Julia and Python, is that Julia doesn’t have classes. Forget typing object.method(other_object); In Julia, it’s always method(object, other_object). This is because of something called multiple dispatch, which turns out to be really, really, powerful. I think it’s not an exaggeration to say that many of the really dedicated Julia users come for performance and stay for multiple dispatch.

Hopefully that’s helpful, and hopefully you find Julia to your liking!

50 Likes