# Why is Python, not Julia, still used for most state-of-the-art AI research?

**URL:** <https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896>\
**Category:** Offtopic\
**Tags:** knet, flux, machine-learning, mlj, sciml\
**Created:** [September 1, 2020, 1:49pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896 "2020-09-01T13:49:21Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [September 1, 2020, 4:47pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/21 "2020-09-01T16:47:31Z")

</div>

> [@Palli](#):
>
> Your comment and my response seems off-topic (still interesting, and maybe relevant in general about collaboration), regarding AI, but I really like to know if, with me just seeing Hyperbolic Deep Learning idea/thread today.

It’s not off topic, at the most recent conference where I presented, there were a whole bunch of presenters using geometric algebra for neural networks. They weren’t using the Julia language though, I believe.

Geometric algebra is generalized multi-linear algebra, so of course it can be applied to anything that uses linear algebra, including AI or whatever.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [September 1, 2020, 5:09pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/22 "2020-09-01T17:09:12Z")

</div>

Your whole post and this part is the best answer I’ve gotten so far:

> [@ToucheSir](#):
>
> Flux is consistently slower than both PyTorch and TensorFlow on training

In theory Julia and its packages will be as fast (I at least see no reason Julia should be slower than other languages, and I’ve seen it benchmark faster, it may have been with Knet; just not sure how realistic ones, at least weren’t done at scale, multi-GPU).

I’m not sure if Julia should be faster than alternatives for most the work I see. Yes, SciML can be 100x faster than alternatives, where it applies, I’m just not sure where that applies, if at all to mainstream ML.

As I said, I’m pinning my hopes on new algorithms/architecture, but the alternative is already down to 1-bit, so at that level Julia can probably not beat the speed others have with more clever datatypes. The algorithm complexity translates, but if done first on Python/C++ people may just stay there.

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [September 1, 2020, 5:52pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/23 "2020-09-01T17:52:07Z")

</div>

I downloaded yesterday some scripts of how taxes are computed that the French Tax ministry released (it is so complicated in France that you can’t compute or check the tax by yourself…)

It ended up it was a bunch of COBOL scripts (2020)…

---

<div class="post-metadata">

**Author:** ![dmolina](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dmolina/32/5246_2.png) [@dmolina](https://discourse.julialang.org/u/dmolina)\
**Post date:** [September 1, 2020, 6:13pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/24 "2020-09-01T18:13:16Z")

</div>

I think the answer is very simple:

Python is a very simple language, with a lot of consolidated libraries in the area (Scikit-learn, Numpy, PyTorch, Keras, OpenCV .API …). The majority of pre-processing algorithms are also available in Python (as sci. The Julia equivalents are, obviously, not so complete yet. Also, many researchers in the AI area are reluctant to learn another programming language (Python is the easier).

Also, the performance topic is not important for many (when the training/evaluation takes a lot the performance of the rest of the system is not so important).

Another important is the spread of the language. In research you want to collaborate, and make your algorithm/technique available. Python is a lot more known than Julia, and you will have more influence making available your proposal in Python.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [September 1, 2020, 9:06pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/25 "2020-09-01T21:06:06Z")

</div>

> [@ToucheSir](#):
>
> Distributed and multi-gpu training: […] CliMA is the only public project I know of that has distributed GPU support, but it’s not DL

FYI: I see multi-GPU done (while I guess not Distributed, then) in 2018, with Flux:

> [@Multiple GPUs with Julia](https://discourse.julialang.org/t/multiple-gpus-with-julia/11248/2):
>
> Yes, I use Flux with multiple GPUs. But my approach is not elegant. In CUDAnative (commit f60c4754356225151c866da0ca512b434aa03abd) in src/CUDAnative.jl on line 75 I replaced default\_device[] = CuDevice(0) with dev = haskey(ENV, "CUDADEV") ? parse(ENV["CUDADEV"]) : 0; default\_device[] = CuDevice(dev), after which I can (in bash) export CUDADEV=1 && julia to open a julia session with CUDAnative using GPU 1.

Also interesting:

> [@Flux vs Tensorflow training performance comparison?](https://discourse.julialang.org/t/flux-vs-tensorflow-training-performance-comparison/32926/2):
>
> The most recent benchmarks I’ve found have shown Flux a bit faster: though I would expect that not to be the case when deploying to large multi-GPU sessions, which is where TensorFlow shines. Though I do also know of [http://denizyuret.github.io/Knet.jl/latest/tutorial/#Benchmarks-1](http://denizyuret.github.io/Knet.jl/latest/tutorial/#Benchmarks-1) By all accounts, KNet is one of the fastest out there (and notice the Knet benchmarks were not written or ran by the Knet authors. They weren’t even written/ran by regular Julia users!), and Flux lags a bit b…

> **[Estadistika](https://estadistika.github.io//julia/python/packages/knet/flux/tensorflow/machine-learning/deep-learning/2019/06/20/Deep-Learning-Exploring-High-Level-APIs-of-Knet.jl-and-Flux.jl-in-comparison-to-Tensorflow-Keras.html)**
>
> statistics, data science, machine/deep learning

On the Nvidia blog in 2017:

> On average, the CUDAnative.jl ports perform identical to statically compiled CUDA C++ (the difference is ~2% in favor of CUDAnative.jl, excluding nn).

---

<div class="post-metadata">

**Author:** ![anon37204545](https://avatars.discourse-cdn.com/v4/letter/a/439d5e/32.png) [@anon37204545](https://discourse.julialang.org/u/anon37204545)\
**Post date:** [September 1, 2020, 9:32pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/26 "2020-09-01T21:32:48Z")

</div>

> [@lmiq](#):
>
> There are countries that still do not use the metric system, you figure it out.

Incidentally, one of those countries is also a birthplace of Julia…

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [September 1, 2020, 9:48pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/27 "2020-09-01T21:48:12Z")

</div>

Nice finds!

> [@](#):
>
> FYI: I see multi-GPU done (while I guess not Distributed, then) in 2018, with Flux…

Yup, I think this is possible on current CUDA.jl as well. However, multi-gpu training of the same model (whether on the same machine or across machines) requires functionality that isn’t implemented anywhere in the Flux ecosystem yet.

> [@](#):
>
> Also interesting…

Seems like all of these benchmarks are prior to the Flux Zygote transition (I see lots of Tracker). Knet likely performs even better now (and should probably receive more love), but Tracker → Zygote was a noticeable performance regression for certain workflows. ref. [https://fluxml.ai/2020/06/29/acclerating-flux-torch.html](https://fluxml.ai/2020/06/29/acclerating-flux-torch.html), [Flux vs pytorch cpu performance - #15 by d1cker](https://discourse.julialang.org/t/flux-vs-pytorch-cpu-performance/42667/15), [Poor performance relative to PyTorch · Issue #886 · FluxML/Flux.jl · GitHub](https://github.com/FluxML/Flux.jl/issues/886).

For more holistic comparisons, see also  
[Is it a good time for a PyTorch developer to move to Julia? If so, Flux? Knet? - #18 by dfdx](https://discourse.julialang.org/t/is-it-a-good-time-for-a-pytorch-developer-to-move-to-julia-if-so-flux-knet/38453/18) and [https://discourse.julialang.org/t/where-does-julia-provide-the-biggest-benefits-over-other-ml-frameworks-for-research](https://discourse.julialang.org/t/where-does-julia-provide-the-biggest-benefits-over-other-ml-frameworks-for-research) (the latter was started by a PyTorch contributor).

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [September 1, 2020, 10:05pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/28 "2020-09-01T22:05:07Z")

</div>

From your link, some research using Julia (and I also edited my top post, with state-of-the-art research from one of the main Knet/Julia guy):

> [@Where does Julia provide the biggest benefits over other ML frameworks for research?](https://discourse.julialang.org/t/where-does-julia-provide-the-biggest-benefits-over-other-ml-frameworks-for-research/28481/4):
>
> Personal Examples:  
> Although this paper I’m going to show you isn’t cutting edge amazing machine learning I did do all of the work for it in Julia: [https://arxiv.org/pdf/1907.11129.pdf](https://arxiv.org/pdf/1907.11129.pdf)

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [September 2, 2020, 2:03am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/29 "2020-09-02T02:03:56Z")

</div>

Will Julia be as fast as optimized cpp + python in machine learning? Most machine learning core parts in cpp are written by experts so maybe they do not have the two-language problem?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [September 2, 2020, 2:20am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/30 "2020-09-02T02:20:25Z")

</div>

If well written julia, then yes. In some cases it can be faster (If python-c interop creates bottlenecks). Also, even just using tensorflow or most other python deep learning libraries has a 2 language problem even if you’re just using python. python+tensorflow is 1000% uglier than pure python.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 2, 2020, 2:23am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/31 "2020-09-02T02:23:25Z")

</div>

PyTorch has had more development and resources. Flux.jl is still quite early days.

Any opinions on Knet vs Pytorch?

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [September 2, 2020, 2:43am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/32 "2020-09-02T02:43:19Z")

</div>

Are there any companies using Flux and Knet?

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 2, 2020, 2:49am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/33 "2020-09-02T02:49:47Z")

</div>

> [@Yifan\_Liu](#):
>
> any companies using Flux and Knet

i wonder too. must be, but I am not sure at what scale though.

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [September 2, 2020, 4:20am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/34 "2020-09-02T04:20:26Z")

</div>

I kind of feel like the two language problem is getting less important over time. Lower level languages are getting easier to use. Rust and Cpp with modules are very appealing.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [September 2, 2020, 6:09am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/35 "2020-09-02T06:09:44Z")

</div>

As far as I know Invenia uses Flux in production.

---

<div class="post-metadata">

**Author:** ![jgreener64](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jgreener64/32/2483_2.png) [@jgreener64](https://discourse.julialang.org/u/jgreener64)\
**Post date:** [September 2, 2020, 10:12am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/36 "2020-09-02T10:12:41Z")

</div>

> [@xiaodai](#):
>
> PyTorch has had more development and resources. Flux.jl is still quite early days.

This is the elephant in the room. PyTorch is _really_ good for many (not all) AI research tasks because it is mature, stable, fast, flexible, has lots of online help, and has cool features like mixed-precision training and GPU parallelism.

I am backing the Julia horse but it’s a bet I expect to pay off over a few years once the Julia ML ecosystem matures. The answer to the question in the title is that the Python ML ecosystem is better _right now_ for most use cases, plus some degree of inertia as others have mentioned. It doesn’t hurt our chances, or dismiss the amazing contributions thus far, to admit that Python has its charms.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [September 2, 2020, 10:41am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/37 "2020-09-02T10:41:47Z")

</div>

I don’t know, maybe you can get all the important (speed) benefits of PyTorch with (without giving up any Julia benefits?):

> **[GitHub - cambridge-mlg/ThArrays.jl: A Julia interface for PyTorch's C++...](https://github.com/cambridge-mlg/ThArrays.jl)**
>
> A Julia interface for PyTorch's C++ backend, focusing on Tensor, AD, and JIT - GitHub - cambridge-mlg/ThArrays.jl: A Julia interface for PyTorch's C++ backend, focusing on Tensor, AD, and JIT

EDIT: There’s already (available since at least April):

> **[GitHub - FluxML/Torch.jl: Sensible extensions for exposing torch in Julia.](https://github.com/FluxML/Torch.jl)**
>
> Sensible extensions for exposing torch in Julia. Contribute to FluxML/Torch.jl development by creating an account on GitHub.

The short answer to the question below was “Yes and Flux”, I also link directly to a more detailed answer:

> [@Is it a good time for a PyTorch developer to move to Julia? If so, Flux? Knet?](https://discourse.julialang.org/t/is-it-a-good-time-for-a-pytorch-developer-to-move-to-julia-if-so-flux-knet/38453/8):
>
> I have also switched from Pytorch. Within a few years I think the strengths of Julia will place it far ahead of Pytorch and others: Pytorch requires underlying code to be written in c++/cuda to get the needed performance, 10x as much code to write. With Flux in particular, native data types can be used. This means that you can potentially take the gradient through some existing code (say a statistics routine) that was never intended for use with Flux. To do this with Pytorch would requir…

I’m not sure how good [GitHub - boathit/JuliaTorch: Using PyTorch in Julia Language](https://github.com/boathit/JuliaTorch)  
is. It’s a wrapper, but I tried to install in, and I see now it’s not yet a proper package (so, neither registered), so you have to git clone or download.

I guess it is/would be nice to have it (with easy installation), while I’m not so sure you would use Julia to its full potential (nor sure you could mix with Julia’s frameworks), so I think a migration to a Julia-only solution (e.g. maybe with other registered package above?) should be on people’s radar.

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [September 2, 2020, 5:39pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/38 "2020-09-02T17:39:14Z")

</div>

> [@jgreener64](#):
>
> > [@xiaodai](#):
> >
> > PyTorch has had more development and resources. Flux.jl is still quite early days.
> 
> This is the elephant in the room. PyTorch is _really_ good for many (not all) AI research tasks because it is mature, stable, fast, flexible, has lots of online help, and has cool features like mixed-precision training and GPU parallelism.

I think this is the easy answer yet not necessarily the right one or at least not the whole story.  
[PyTorch started at September 2016](https://en.wikipedia.org/wiki/PyTorch). I remember using version `0.3.x` which was released less than 18 after the start.  
[Flux has started around ~May 2016 according to the first commit](https://github.com/FluxML/Flux.jl/commit/a1a2103bb64630750ebcd8fa90545f05c063b42e). Do you find it as usable as versions 0.3 or 0.4 of PyTorch?  
My point isn’t the exact dates but that the resources spent on PyTorch in 18 month are probably something achievable in 4 years of Flux. So I’d expect the maturity and capabilities to be similar. Especially if, as claimed, Julia gives the developers the ability to be more effective due its features. Flux is also developed by very capable Julians, so one would expect they can take full advantage of its capabilities.

I think it has to do with the goals set to Flux. PyTorch seems to me as a pragmatic approach. Each step getting better. Flux seems to try making many steps forward at once. I might be wrong on the analysis but looking from the side it looked just like the [Marshmallo Challenge](https://www.youtube.com/watch?v=H0_yKBitO8M).  
I found the early `Knet` to be much more intuitive and “PyTorch” like. Unfortunately it doesn’t get enough of the spotlight. Moreover later it tried to be more Flux like. I wish it was just a PyTorch clone. Simple and straight to the point.

For my subjective idea I would also say Python, as someone mentioned here, is much easier language than Julia. Coming from MATLAB looking at packages of Julia seems to me as an act of wizard. IT has to do with the Julia community being built with highly capable programmers. Sometimes, to me, things seems too much elegant yet not simple.

I have never encountered MATLAB code I couldn’t understand (MATLAB, to me, the easiest language out there). I can handle some Python code though having less Python hours than Julia. Yet most code in Julia’s packages seems like black magic to me. It might be like to many other engineers being very good at what they do yet only as good as they need in programming. They find Python to be a more welcoming tool, not only more popular (It is popular due its simplicity).

---

<div class="post-metadata">

**Author:** ![non-Jedi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-jedi/32/3645_2.png) [@non-Jedi](https://discourse.julialang.org/u/non-Jedi)\
**Post date:** [September 2, 2020, 6:22pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/39 "2020-09-02T18:22:43Z")

</div>

> [@RoyiAvital](#):
>
> I think this is the easy answer yet not necessarily the right one or at least not the whole story.  
> [PyTorch started at September 2016](https://en.wikipedia.org/wiki/PyTorch). I remember using version `0.3.x` which was released less than 18 after the start.  
> [Flux has started around ~May 2016 according to the first commit](https://github.com/FluxML/Flux.jl/commit/a1a2103bb64630750ebcd8fa90545f05c063b42e). Do you find it as usable as versions 0.3 or 0.4 of PyTorch?

PyTorch is largely porting the existing lua [Torch](https://en.wikipedia.org/wiki/Torch_(machine_learning)) package to python as I understand it. Torch development started in **2002** , 10 years before even Julia was around much less Flux.

> [@RoyiAvital](#):
>
> For my subjective idea I would also say Python, as someone mentioned here, is much easier language than Julia. Coming from MATLAB looking at packages of Julia seems to me as an act of wizard.

I’d be interested to see some examples of code that you find confusing in Julia packages. Coming from programming in python (and matlab before that), I found in general that my mental model of what a given piece of code is doing is simpler in julia than python which is itself more straightforward than matlab.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [September 2, 2020, 6:39pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/40 "2020-09-02T18:39:24Z")

</div>

> [@RoyiAvital](#):
>
> Coming from MATLAB looking at packages of Julia seems to me as an act of wizard.

Yes, this is more so for package developers.

There is a distinction between users and developers.

For users, Julia should be the easiest language, easier than python or matlab, because hopefully the API works like magic.

For contributors, it’s a different situation because the contributor needs to have more advanced knowledge, the underlying API might use metaprogramming.

So in Julia there is a wider spectrum of difficulty. It is easy to get started as a user, but as a contributor it is necessary to have deeper more advanced knowledge.

[Previous page](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896.md?page=1)

[Next page](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896.md?page=3)