# On Machine Learning and Programming Languages

**URL:** <https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574>\
**Category:** Machine Learning\
**Created:** [December 6, 2017, 5:17pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574 "2017-12-06T17:17:51Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [December 6, 2017, 5:17pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/1 "2017-12-06T17:17:51Z")

</div>

[New blog post on the Julia website](https://julialang.org/blog/2017/12/ml&pl)

---

<div class="post-metadata">

**Author:** ![viralbshah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viralbshah/32/54_2.png) [@viralbshah](https://discourse.julialang.org/u/viralbshah)\
**Post date:** [December 7, 2017, 8:49pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/2 "2017-12-07T20:49:24Z")

</div>

A lot of effort went into this post to make sure its not just about Julia, but really reviewing the developments in machine learning and asking the question in an unbiased way.

I personally think that many ML frameworks are slowly becoming more like programming languages, and something like Julia + native AD + Flux/KNet + GPU / other hardware codegen makes for a really compelling ML platform.

Would be great if folks could try out what we have, and see if it does in fact address the issues raised in this blog. Assuming all goes well, and we can complete the work, it would be good to develop this and submit it to an appropriate conference/journal.

(I also changed the tag of this post to Machine Learning)

-viral

---

<div class="post-metadata">

**Author:** ![viralbshah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viralbshah/32/54_2.png) [@viralbshah](https://discourse.julialang.org/u/viralbshah)\
**Post date:** [December 7, 2017, 8:51pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/3 "2017-12-07T20:51:39Z")

</div>

We saw many hits from China on this post. I wonder if we should do a Chinese translation and host it on [julialang.org](http://julialang.org) as well as send it to some Chinese tech outlets. If someone is up for this, I’d be happy to help.

-viral

---

<div class="post-metadata">

**Author:** ![davidbp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidbp/32/463_2.png) [@davidbp](https://discourse.julialang.org/u/davidbp)\
**Post date:** [December 7, 2017, 9:10pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/4 "2017-12-07T21:10:21Z")

</div>

> [@viralbshah](#):
>
> I personally think that many ML frameworks are slowly becoming more like programming languages, and something like Julia + native AD + Flux/KNet + GPU / other hardware codegen makes for a really compelling ML platform.

Nice post. The main problem is the lack of users and Big companies really pushing libraries for the language. Python has a lot of great engineers from Google and Facebook helping to build Tensorflow and Pytorch. Julia is not in the same situation.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [December 7, 2017, 9:23pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/5 "2017-12-07T21:23:48Z")

</div>

> [@davidbp](#):
>
> . The main problem is the lack of users and Big companies really pushing libraries for the language. Python has a lot of great engineers from Google and Facebook helping to build Tensorflow and Pytorch. Julia is not in the same situation.

And yet when I look at the domains where Julia development has focused, it really blows my mind how well Julia holds up against python’s libraries. Comparing DifferentialEquations.jl to offerings in Scipy is a night and day difference, yet DifferentialEquations.jl is mostly the work of _one_ developer.

Thats insane! It really seems to be that the work of a handful of highly productive people in Julia over a couple years is a fair match for the work of entire industries in Python over a decade. This isn’t because Julia devs are gods of programming (though it seems that some are close), its because the language is cooperating with them instead of fighting at every turn.

---

<div class="post-metadata">

**Author:** ![viralbshah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viralbshah/32/54_2.png) [@viralbshah](https://discourse.julialang.org/u/viralbshah)\
**Post date:** [December 7, 2017, 9:31pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/6 "2017-12-07T21:31:47Z")

</div>

You mean Julia Computing does not count as big as Google and Facebook? Just  
kidding. 😉 At one level, we can’t stop doing what we are doing just  
because we don’t have big companies behind us. But we definitely need to be  
smart and leverage our advantages. Also, I think that Google is invested in  
TensorFlow and Facebook in Torch - not necessarily in the wrapping  
language. Both of these are compilers in their own rights.

At some level, if all of us thought that big companies are necessary - we  
wouldn’t have started Julia or even got this far as a community.

-viral

---

<div class="post-metadata">

**Author:** ![davidbp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidbp/32/463_2.png) [@davidbp](https://discourse.julialang.org/u/davidbp)\
**Post date:** [December 7, 2017, 9:36pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/7 "2017-12-07T21:36:23Z")

</div>

I totally agree with you observation. There is actually a lot of space to explore. For example, there is no deep learning framework that allow us to use AMD GPUs. If Julia is the first language to solve this problem it might motivate a lot of people to try the language (HBM2 AMD GPUS for 600$ vs 10000$ of the competition).

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [December 7, 2017, 9:39pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/8 "2017-12-07T21:39:52Z")

</div>

> [@davidbp](#):
>
> For example, there is no deep learning framework that allow us to use AMD GPUs. If Julia is the first language to solve this problem it might motivate a lot of people to try the language (HBM2 AMD GPUS for 600$ vs 10000$ of the competition).

Flux.jl with CLArrays.jl?

---

<div class="post-metadata">

**Author:** ![viralbshah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viralbshah/32/54_2.png) [@viralbshah](https://discourse.julialang.org/u/viralbshah)\
**Post date:** [December 7, 2017, 9:44pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/9 "2017-12-07T21:44:34Z")

</div>

Well work is certainly afoot:

[https://github.com/JuliaLang/julia/pull/24142](https://github.com/JuliaLang/julia/pull/24142)

---

<div class="post-metadata">

**Author:** ![davidbp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidbp/32/463_2.png) [@davidbp](https://discourse.julialang.org/u/davidbp)\
**Post date:** [December 7, 2017, 9:44pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/10 "2017-12-07T21:44:44Z")

</div>

It might be a solution but the best thing would be to something similar to cudanative as an alternative to write opencl like code. Without the need of `Transpiler.jl`.

---

<div class="post-metadata">

**Author:** ![davidbp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidbp/32/463_2.png) [@davidbp](https://discourse.julialang.org/u/davidbp)\
**Post date:** [December 7, 2017, 9:46pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/11 "2017-12-07T21:46:30Z")

</div>

You posted it seconds before I wrote my comment! I didn’t know about this! AMD/Intel where are you? Put a couple of engineers working on this!

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [December 7, 2017, 9:46pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/12 "2017-12-07T21:46:32Z")

</div>

> [@davidbp](#):
>
> It might be a solution but the best thing would be to something similar to cudanative as an alternative to write opencl like code. Without the need of Transpiler.jl.

That’s in the works.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [December 7, 2017, 9:47pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/13 "2017-12-07T21:47:57Z")

</div>

> [@Mason](#):
>
> yet DifferentialEquations.jl is mostly the work of one developer.

My suspicious is that Chris cloned himself a few times. Even with Julia productive as it is, that is the only possible explanation!

---

<div class="post-metadata">

**Author:** ![davidbp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidbp/32/463_2.png) [@davidbp](https://discourse.julialang.org/u/davidbp)\
**Post date:** [December 7, 2017, 9:49pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/14 "2017-12-07T21:49:23Z")

</div>

The macro `@spawnatcrhis` is in the works.

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [December 7, 2017, 9:50pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/15 "2017-12-07T21:50:27Z")

</div>

Well, the code you write in CLArrays is pretty much identical to what you would write for CUDAnative 😉

I’d love to have people write more hardware independent [code with GPUArrays anyways](https://medium.com/@sdanisch/writing-extendable-and-hardware-agnostic-gpu-libraries-b21c145a8dad), which should again almost look identical to Julia code for CUDAnative - biggest difference is, that it will just run on all platforms.

But sure, an LLVM based approach would be nicer for the future!

---

<div class="post-metadata">

**Author:** ![viralbshah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viralbshah/32/54_2.png) [@viralbshah](https://discourse.julialang.org/u/viralbshah)\
**Post date:** [December 7, 2017, 9:50pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/16 "2017-12-07T21:50:31Z")

</div>

We should write a Flux model to train on Chris.

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [December 8, 2017, 9:32am UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/17 "2017-12-08T09:32:18Z")

</div>

> [@davidbp](#):
>
> I totally agree with you observation. There is actually a lot of space to explore. For example, there is no deep learning framework that allow us to use AMD GPUs. If Julia is the first language to solve this problem it might motivate a lot of people to try the language (HBM2 AMD GPUS for 600$ vs 10000$ of the competition).

Having a library for DNN with AMD / Intel GPU support will indeed be a game changer.

I really don’t understand why Intel / AMD aren’t pushing it harder.

By the way, there is one project, though limited to Intel hardware - [Intel MKL DNN](https://github.com/01org/mkl-dnn). Though I’m not sure if one day they will use Intel GPU. Maybe…

---

<div class="post-metadata">

**Author:** ![innerlee](https://avatars.discourse-cdn.com/v4/letter/i/b19c9b/32.png) [@innerlee](https://discourse.julialang.org/u/innerlee)\
**Post date:** [December 8, 2017, 11:45am UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/18 "2017-12-08T11:45:28Z")

</div>

I can help with the translation. Where should the texts go?

---

<div class="post-metadata">

**Author:** ![viralbshah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viralbshah/32/54_2.png) [@viralbshah](https://discourse.julialang.org/u/viralbshah)\
**Post date:** [December 8, 2017, 2:22pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/19 "2017-12-08T14:22:01Z")

</div>

If you’re comfortable with GitHub, I think we can prepare a new blog post in Chinese right there. If not, send me a google doc or anything you prefer, and I can prepare the blog post myself.

-viral

---

<div class="post-metadata">

**Author:** ![innerlee](https://avatars.discourse-cdn.com/v4/letter/i/b19c9b/32.png) [@innerlee](https://discourse.julialang.org/u/innerlee)\
**Post date:** [December 8, 2017, 3:57pm UTC](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574/20 "2017-12-08T15:57:55Z")

</div>

github is okay for me. What’s the desired format, latex? or markdown?

edit: if latex, could the source file be public on the repo?

[Next page](https://discourse.julialang.org/t/on-machine-learning-and-programming-languages/7574.md?page=2)