Why is Python, not Julia, still used for most state-of-the-art AI research?

[Palli]

@ToucheSir

Distributed and multi-gpu training: […] CliMA is the only public project I know of that has distributed GPU support, but it’s not DL

FYI: I see multi-GPU done (while I guess not Distributed, then) in 2018, with Flux:

Multiple GPUs with Julia

Yes, I use Flux with multiple GPUs. But my approach is not elegant. In CUDAnative (commit f60c4754356225151c866da0ca512b434aa03abd) in src/CUDAnative.jl on line 75 I replaced default_device[] = CuDevice(0) with dev = haskey(ENV, "CUDADEV") ? parse(ENV["CUDADEV"]) : 0; default_device[] = CuDevice(dev), after which I can (in bash) export CUDADEV=1 && julia to open a julia session with CUDAnative using GPU 1.

Also interesting:

Flux vs Tensorflow training performance comparison?

The most recent benchmarks I’ve found have shown Flux a bit faster: though I would expect that not to be the case when deploying to large multi-GPU sessions, which is where TensorFlow shines. Though I do also know of http://denizyuret.github.io/Knet.jl/latest/tutorial/#Benchmarks-1 By all accounts, KNet is one of the fastest out there (and notice the Knet benchmarks were not written or ran by the Knet authors. They weren’t even written/ran by regular Julia users!), and Flux lags a bit b…

Estadistika

On the Nvidia blog in 2017:

On average, the CUDAnative.jl ports perform identical to statically compiled CUDA C++ (the difference is ~2% in favor of CUDAnative.jl, excluding nn).