# Why is Python, not Julia, still used for most state-of-the-art AI research?

**URL:** <https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896>\
**Category:** Offtopic\
**Tags:** knet, flux, machine-learning, mlj, sciml\
**Created:** [September 1, 2020, 1:49pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896 "2020-09-01T13:49:21Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [September 2, 2020, 6:40pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/41 "2020-09-02T18:40:53Z")

</div>

> [@chakravala](#):
>
> > [@RoyiAvital](#):
> >
> > Coming from MATLAB looking at packages of Julia seems to me as an act of wizard.
> 
> Yes, this is more so for package developers.
> 
> There is a distinction between users and developers.
> 
> For users, Julia should be the easiest language, easier than python or matlab.
> 
> For contributors, it’s a different situation because the contributor needs to have more advanced knowledge.
> 
> So in Julia there is a wider spectrum of difficulty. It is easy to get started as a user, but as a contributor it is necessart to have deeper more advanced knowledge.

I totally agree. You can write Julia like MATLAB and probably they will be as readable.  
The problem is when you want people to engage. The bar is high. It is not a shortcoming of Julia. It is a property. The dynamic range of the complexity of Julia is very high. Package developers who choose the upper part of the capabilities of language means they target smaller part of the Julia users.  
The whole question was why does Python get more engagement. So it could be a factor.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [September 2, 2020, 6:45pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/42 "2020-09-02T18:45:41Z")

</div>

> [@non-Jedi](#):
>
> PyTorch is largely porting the existing lua [Torch](https://en.wikipedia.org/wiki/Torch_(machine_learning)) package to python as I understand it. Torch development started in **2002** , 10 years before even Julia was around much less Flux.

Don’t forget Chainer! PyTorch has a rich heritage and very much started off of an evolution of those two libraries.

> [@RoyiAvital](#):
>
> I think it has to do with the goals set to Flux. PyTorch seems to me as a pragmatic approach. Each step getting better. Flux seems to try making many steps forward at once. I might be wrong on the analysis but looking from the side it looked just like the [Marshmallo Challenge](https://www.youtube.com/watch?v=H0_yKBitO8M).  
> I found the early `Knet` to be much more intuitive and “PyTorch” like. Unfortunately it doesn’t get enough of the spotlight.

I would go so far as to claim (controversially) that Flux’s continued emphasis on experimentation and “big ideas” _to the detriment of_ stability and polish negatively impacted adoption for the entire Julia DL ecosystem after the initial hype subsided. Thankfully, Flux has found more of a middle ground now and Knet is gradually gaining name recognition, so I’m hopeful that some of the stereotypical criticisms around stability, completeness, docs, etc. can be assuaged.

> [@RoyiAvital](#):
>
> …Coming from MATLAB looking at packages of Julia seems to me as an act of wizard. IT has to do with the Julia community being built with highly capable programmers. Sometimes, to me, things seems too much elegant yet not simple.
> 
> I have never encountered MATLAB code I couldn’t understand (MATLAB, to me, the easiest language out there)…

If anything, this is testament to how subjective “easy” is. As someone who has no “hard science”/engineering background outside of CS, I found MATLAB bizarre, unintuitive and extremely frustrating to use. For example, array indexing, slicing and function/class definitions are completely different from what you’d see in other programming languages, whereas I’d be hard pressed to find some part of Julia syntax that doesn’t exist in some mainstream programming language. Again, past experience matters a lot here.

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [September 2, 2020, 6:53pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/43 "2020-09-02T18:53:55Z")

</div>

> [@RoyiAvital](#):
>
> I have never encountered MATLAB code I couldn’t understand (MATLAB, to me, the easiest language out there).

Then I suspect you haven’t run into too much code that has been vectorized in absurdum to get the last bit of performance without resorting to mex files or attempts to implement algorithms for arbitrary dimensionality. And you never know what some crazy PhD student might come up with. This is maybe somewhat challenging to understand: [https://github.com/GunnarFarneback/spatial\_domain\_toolbox/blob/master/polyexp.m](https://github.com/GunnarFarneback/spatial_domain_toolbox/blob/master/polyexp.m)

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [September 2, 2020, 6:56pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/44 "2020-09-02T18:56:10Z")

</div>

> [@RoyiAvital](#):
>
> Coming from MATLAB looking at packages of Julia seems to me as an act of wizard.

Coming from Matlab I have the opposite opinion. The hoops you have to jump through to make Matlab ‘packages’ performant tend to make them very hard to read. _Everything_ must be contorted into maximal vectorization, while Julia library code can simply and straightforwardly implement the concepts in a natural and expressive way.

I have read my own ‘performant’ Matlab libraries, and they are like looking into nightmares.

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [September 2, 2020, 7:28pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/45 "2020-09-02T19:28:33Z")

</div>

> [@ToucheSir](#):
>
> …Thankfully, Flux has found more of a middle ground now and Knet is gradually gaining name recognition, so I’m hopeful that some of the stereotypical criticisms around stability, completeness, docs, etc. can be assuaged.  
> …

So far Flux gave me **the best experience** I have ever had doing DL barring one issue - stability. Has that gotten better or does everyone still lock pkg versions? If so I would be willing to take another crack at it.

---

<div class="post-metadata">

**Author:** ![SamBrand](https://avatars.discourse-cdn.com/v4/letter/s/c37758/32.png) [@SamBrand](https://discourse.julialang.org/u/SamBrand)\
**Post date:** [September 3, 2020, 7:06am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/46 "2020-09-03T07:06:47Z")

</div>

Same for me. My PhD supervisor was an absolute wizard at MATLAB, every time I got stuck just waiting for MATLAB to solve something I’d pop into his office and he would do some arcane fix to speed it up x1000. With about 200 more lines of code (slight exaggeration).

I switched to C++ because it was easier to get working in realistic time than optimized MATLAB.

I can sort of see why a MATLAB refugee might like the under-the-hood python, your eyes get used to skimming over reams of vectorisation and performance tweaks and lighting on the good stuff.

---

<div class="post-metadata">

**Author:** ![Yifan\_Liu](https://avatars.discourse-cdn.com/v4/letter/y/4da419/32.png) [@Yifan\_Liu](https://discourse.julialang.org/u/Yifan_Liu)\
**Post date:** [September 4, 2020, 3:26am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/47 "2020-09-04T03:26:31Z")

</div>

To be honest, even though people keep saying there is no need to avoid for loops in Julia, I still prefer to vectorize my code as much as possible. Vectorization feels more natural to me.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [September 4, 2020, 3:30am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/48 "2020-09-04T03:30:45Z")

</div>

IMO it depends a lot on the problem. Some problems are really obviously vectorized problems. For those, vectorization is great. The problem is that lots of problems are technically expressible in a vectorized form, but doing so isn’t obvious. The nice thing about Julia is that it gives you fast versions either way, whereas low level languages like C only have loops, and older high level languages only have vectorization (if you care about speed).

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 4, 2020, 7:30am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/49 "2020-09-04T07:30:07Z")

</div>

> [@Yifan\_Liu](#):
>
> there is no need to avoid for loops in Julia, I still prefer to vectorize my code

There is no contradiction here: since “vectorized” code is not special, you are free to do what you like.

Generally this point comes up when comparing to languages which provide “fast” vectorized kernels, usually written in C, compared to “slow” control flow of any kind (eg R). So someone who would prefer `for` loops in some context is still forced to go for vectorized code if they want performance.

Julia simply gives you the choice back between the two alternatives. Use what you prefer.

---

<div class="post-metadata">

**Author:** ![SamBrand](https://avatars.discourse-cdn.com/v4/letter/s/c37758/32.png) [@SamBrand](https://discourse.julialang.org/u/SamBrand)\
**Post date:** [September 4, 2020, 8:13am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/50 "2020-09-04T08:13:47Z")

</div>

Absolutely this.

I’m still glad for my MATLAB days because nothing makes you so absolutely rock solid on the basic operations of linear algebra as using MATLAB for anything!

With Julia I tend to vectorise if the code object represents a mathematical object that you’d naturally write as a vector. Not if it doesn’t.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [September 4, 2020, 11:06am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/51 "2020-09-04T11:06:07Z")

</div>

> [@Tamas\_Papp](#):
>
> Julia simply gives you the choice back between the two alternatives. Use what you prefer.

It’s a choice, but the alternatives aren’t always equal. In some (maybe many) cases there is no efficient way to express your computation in terms of a set of prepackaged vectorized kernels. Then, you either write loops, or the code _will_ be slow.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 4, 2020, 12:34pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/52 "2020-09-04T12:34:23Z")

</div>

That’s of course true, but the type system of Julia puts an extra wrinkle into this. Specifically, using a `for` loop, usually you need to allocate a container for the result, which requires an element type. In a lot of cases, it is trivially known, but generally it may be tricky to obtain. Using a functional style (`map`, `(map)reduce`, `(map)foldX`) can sidestep this issue.

A lot of exciting experiments are happening on this front, in packages like

[https://github.com/JuliaFolds/Transducers.jl](https://github.com/JuliaFolds/Transducers.jl)

`for` loops of course remain a valuable tool in the Julia programmer’s toolbox.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [May 20, 2021, 6:36pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/53 "2021-05-20T18:36:04Z")

</div>

Microsoft seems to already be using Julia for neural networks, at least one of their employee does in “slimgroup” collaboration work (and a paper on work building on this names him and Microsoft, and Julia):

> **[GitHub - slimgroup/InvertibleNetworks.jl: A Julia framework for invertible...](https://github.com/slimgroup/InvertibleNetworks.jl)**
>
> A Julia framework for invertible neural networks. Contribute to slimgroup/InvertibleNetworks.jl development by creating an account on GitHub.

> ## Authors
> 
> - Philipp Witte, Microsoft Corporation ([pwitte@microsoft.com](mailto:pwitte@microsoft.com))
> - […]

[IBM also has lots of ML Julia packages, latest I’ve seen: [GitHub - IBM/Lale.jl: a Julia wrapper of Python's lale automl package](https://github.com/IBM/Lale.jl) “a Julia wrapper of Python’s lale automl package”.]

I decided to look into this after seeing new package (that is not yet registered, while Microsoft’s older OptimSim.jl package is, and is pure Julia): [GitHub - microsoft/AzureClusterlessHPC.jl: A Julia package for clusterless distributed computing on Azure](https://github.com/microsoft/AzureClusterlessHPC.jl)

> **AzureClusterlessHPC.jl** is a package for simplified parallel computing on Azure.  
> […]
> 
> ## Applications
> 
> […]
> 
> - Generic batch, map-reduce and iterative map-reduce examples
> - Deep learning with AzureClusterlessHPC.jl and [Flux.jl](https://github.com/FluxML)
> - Seismic imaging and inversion with [COFII.jl](https://github.com/ChevronETC/Examples) and [JUDI.jl](https://github.com/slimgroup/JUDI.jl)

[https://arxiv.org/pdf/2101.03709.pdf](https://arxiv.org/pdf/2101.03709.pdf)

> Our implementation relies on [InvertibleNetworks.jl](https://github.com/slimgroup/InvertibleNetworks.jl) (P. Witte et al., 2020), a recently-developed memory-efficient framework for training invertible networks in the Julia programming language.

> **[GitHub - slimgroup/FastApproximateInference.jl: Code to reproduce results in...](https://github.com/slimgroup/FastApproximateInference.jl)**
>
> Code to reproduce results in "Preconditioned training of normalizing flows for variational inference in inverse problems" - GitHub - slimgroup/FastApproximateInference.jl: Code to reprodu...

> Here, we heavily rely on [InvertibleNetworks.jl](https://github.com/slimgroup/InvertibleNetworks.jl), a recently-developed, memory-efficient framework for training invertible networks in Julia.

> **[GitHub - slimgroup/XConv: Memory efficient convolution networks](https://github.com/slimgroup/XConv)**
>
> Memory efficient convolution networks. Contribute to slimgroup/XConv development by creating an account on GitHub.

> # Memory efficient convolution layer via matrix sketching
> 
> […] This package contains two implementation:
> 
> - A `julia` implementation that overloads [NNlib](https://github.com/FluxML/NNlib.jl) for the computation of `∇conv_filter` .
> - A [PyTorch](https://pytorch.org/) implementation that defines a new convolution layer `Xconv2D, Xconv3D` .

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [May 23, 2021, 3:36pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/54 "2021-05-23T15:36:54Z")

</div>

> [@Palli](#):
>
> Microsoft seems to already be using Julia  
> […]  
> IBM also has lots of ML Julia packages

At this stage in Julia’s life cycle, _not_ having any suppor and/or experiments from large players that make money from HPC & ML would be surprising, not the opposite.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [May 23, 2021, 9:41pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/55 "2021-05-23T21:41:34Z")

</div>

When I started this thread in Sept. it was puzzling why Julia not mentioned much ML/AI papers.

You see at [JuliaComputing.com](http://JuliaComputing.com) under “JULIA USERS AND JULIA COMPUTING CUSTOMERS” the logos from all the major (software) companies, even Apple, but I don’t know in most cases what that means. I was curious if it relates to ML, and I thought people might be intrigued. There’s also some Julia here:

> **[Google Research](https://github.com/google-research?q=&type=&language=julia&sort=)**
>
> Google Research has 292 repositories available. Follow their code on GitHub.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [May 23, 2021, 10:31pm UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/56 "2021-05-23T22:31:11Z")

</div>

I think academic and industry use may be different beasts here. Certainly for ML research, most of the operational pain points in real world implementation do not apply because of e.g. clean benchmark datasets, while there’s arguably more focus on getting people up to speed quickly and using as much existing code (regardless of quality or performance) as possible to save time. This is pure speculation, but I think lack of mature distributed training functionality is also a blocker for larger research groups (which drive much of what gets adopted) to get involved with the ecosystem.

---

<div class="post-metadata">

**Author:** ![Juan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juan/32/7657_2.png) [@Juan](https://discourse.julialang.org/u/Juan)\
**Post date:** [May 24, 2021, 12:51am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/57 "2021-05-24T00:51:51Z")

</div>

People spend a lot of time and effort learning a programming language in depth, and then they stick with it. It’s hard to recognize that what you’ve done isn’t as good as it used to be or as you thought. And it’s hard to start from scratch again.  
People prefer to use what the majority uses, you can find most things you need already done.

---

<div class="post-metadata">

**Author:** ![StatisticalMouse](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/statisticalmouse/32/43370_2.png) [@StatisticalMouse](https://discourse.julialang.org/u/StatisticalMouse)\
**Post date:** [May 24, 2021, 6:41am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/58 "2021-05-24T06:41:51Z")

</div>

Programming languages together with the ecosystem are more or less isolated silos, it’s rather hard to switch.

Consider a single professional, having worked in language A for five years, learning the ecosystem, getting work experience. You have bills to pay, children to feed, whatever.

The professional sees language B that is a little better in some ways. Will she spend the time to learn it, get a new job in that language with zero work experience? Or maybe convince the whole company to switch?

---

<div class="post-metadata">

**Author:** ![EvoArt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evoart/32/25357_2.png) [@EvoArt](https://discourse.julialang.org/u/EvoArt)\
**Post date:** [May 24, 2021, 7:15am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/59 "2021-05-24T07:15:14Z")

</div>

Especially when the majority of job opportunities will be based on experience in language P… ython

---

<div class="post-metadata">

**Author:** ![Wei\_Yang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/wei_yang/32/23951_2.png) [@Wei\_Yang](https://discourse.julialang.org/u/Wei_Yang)\
**Post date:** [May 24, 2021, 8:32am UTC](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896/60 "2021-05-24T08:32:53Z")

</div>

Well, and there is also a very popular J… … …ava language on the job market.

[Previous page](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896.md?page=2)

[Next page](https://discourse.julialang.org/t/why-is-python-not-julia-still-used-for-most-state-of-the-art-ai-research/45896.md?page=4)