# Starting a Deep Learning project, should I keep using Julia or jump to Python?

**URL:** <https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840>\
**Category:** Machine Learning\
**Tags:** question\
**Created:** [May 22, 2017, 2:56am UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840 "2017-05-22T02:56:51Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![raff](https://avatars.discourse-cdn.com/v4/letter/r/5e9695/32.png) [@raff](https://discourse.julialang.org/u/raff)\
**Post date:** [May 22, 2017, 2:56am UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/1 "2017-05-22T02:56:51Z")

</div>

I fully switched from Matlab to Julia to do my machine learning research two years ago. I really like Julia and never look back since then. With Julia, my workflow is more productive than before.  
But now, I will start a deep learning project. I need to write a code similar to Bidirectional LSTM - CRF, but with my own CRF-like implementation. I also need to customize the details of some layers implementation.

I found an example on how to write Bi-LSTM-CRF code in pytorch [Advanced: Making Dynamic Decisions and the Bi-LSTM CRF — PyTorch Tutorials 1.12.1+cu102 documentation](http://pytorch.org/tutorials/beginner/nlp/advanced_tutorial.html)  
which seems to be a good start for me.

I am also aware about deep learning packages in Julia: [Flux.jl](https://github.com/MikeInnes/Flux.jl) and [Knet.jl](https://github.com/denizyuret/Knet.jl). But I am not sure if those package fit well with my project. I cannot find any implementation of Bi-LSTM-CRF in those package.

Should I jump to Python and use PyTorch or keep using Julia? Any suggestion?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 22, 2017, 3:36am UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/2 "2017-05-22T03:36:38Z")

</div>

> [@raff](#):
>
> I am also aware about deep learning packages in Julia: Flux.jl and Knet.jl. But I am not sure if those package fit well with my project. I cannot find any implementation of Bi-LSTM-CRF in those package.

Well, Knet.jl doesn’t have to have it implemented. Knet simply accelerates user’s NN code using its type of GPU-based array and its autodifferentiation setup AutoGrad.jl. But the actual NN code for Knet NNs is simply Julia code. This means it can do any NN as long as you know how to mathematically define it. If you do know the mathematical definition, just implement that as the `predict` function, and it’ll work.

Flux.jl also lets you define your own layers (at least from what I remember in @MikeInnes’ videos). I’m not exactly sure how that’s done, but look up the videos.

TensorFlow.jl is similar in that it gives you the tools to build the NN, so you’ll be able to implement whatever you need.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 22, 2017, 3:40am UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/3 "2017-05-22T03:40:10Z")

</div>

MXnet.jl generally has more complete models (less toolboxy, more end-result type of NN package). Have you checked that?

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [May 22, 2017, 4:59am UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/4 "2017-05-22T04:59:52Z")

</div>

I have been experimenting with TensorFlow under Julia and Knet. I have decided to use Knet at the moment, because I have found it more intuitive under Julia, although I am sure that it will incur more coding on my side, as TF is the current hype. What I particularly like on Knet is the ease with which you can extend the autograd by new operators and the fact that your data samples does not have to be of fixed size.

Denis, the author of Knet, has put a lot of models into examples section of the project, so I am sure you will be able to find something there.

---

<div class="post-metadata">

**Author:** ![MikeInnes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikeinnes/32/3656_2.png) [@MikeInnes](https://discourse.julialang.org/u/MikeInnes)\
**Post date:** [May 22, 2017, 5:18am UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/5 "2017-05-22T05:18:35Z")

</div>

Knet is very flexible and intuitive to use from a Julia programmer’s standpoint, it would be a good choice. I would personally like to see Knet evolve a more PyTorch-like, layer-based API, and we’ll be doing some work towards that over the summer.

One factor in Flux’s favour here is its first-class support for RNNs. If you want a custom LSTM-like layer it’s trivial, bi-RNNs are as easy as `flip(rnn)` (in theory). No one else can do this and eventually I’d like for Flux to become a no-brainer for NLP and sequence-like tasks for that reason. Unfortunately, I can’t yet recommend it for something this advanced, but I will keep everyone posted.

---

<div class="post-metadata">

**Author:** ![jekbradbury](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jekbradbury/32/953_2.png) [@jekbradbury](https://discourse.julialang.org/u/jekbradbury)\
**Post date:** [May 22, 2017, 9:36am UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/6 "2017-05-22T09:36:53Z")

</div>

I’m one of the PyTorch developers (just recently starting to explore the Julia world) and I mostly want to endorse what’s been said 🙂 . The ecosystem for deep learning in Python is obviously much more mature, but Julia’s catching up quickly: MXNet.jl and TensorFlow.jl provide full-featured interfaces to their respective packages, Knet.jl aims to build a PyTorch-like define-by-run framework but isn’t as full-featured yet, and Flux.jl is the least far along but has the kind of magical Julian architecture that’s completely impossible in Python.

I’d recommend at least trying to get a CRF working in Knet based on the PyTorch tutorial, then dropping back to MXNet.jl or TensorFlow.jl if you think too much is missing. It’s the experience of people like you trying to build new models from scratch that will drive the Julia DL ecosystem forward.

---

<div class="post-metadata">

**Author:** ![raff](https://avatars.discourse-cdn.com/v4/letter/r/5e9695/32.png) [@raff](https://discourse.julialang.org/u/raff)\
**Post date:** [May 23, 2017, 3:55am UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/7 "2017-05-23T03:55:58Z")

</div>

Thanks all for your suggestions.

It seems that Julia has a bright future for programming deep learning.  
I will study PyTorch’s BiLSTM-CRF code first, and try to implement that in Knet.jl

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [May 23, 2017, 9:03pm UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/8 "2017-05-23T21:03:30Z")

</div>

I just started a small learning project in Julia, and so far I’ve been really really happy with just writing out my model as a regular Julia function and using ReverseDiff.jl to compute its gradients. MLDataPattern.jl is also extremely helpful for basic data manipulation tasks (like batching and sampling). It’s awesome to be able to build a working machine learning architecture out of general-purpose tools like these. This approach wouldn’t work as well in Python, since it lacks Julia’s speed and multiple-dispatch magic.

---

<div class="post-metadata">

**Author:** ![amellnik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amellnik/32/137_2.png) [@amellnik](https://discourse.julialang.org/u/amellnik)\
**Post date:** [May 23, 2017, 9:49pm UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/9 "2017-05-23T21:49:42Z")

</div>

I think one of the big advantages of Knet.jl at this point is its excellent documentation and examples.

---

<div class="post-metadata">

**Author:** ![Azamat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/azamat/32/6892_2.png) [@Azamat](https://discourse.julialang.org/u/Azamat)\
**Post date:** [September 24, 2019, 12:59pm UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/10 "2019-09-24T12:59:30Z")

</div>

Just a word of caution. I have written huge codebase with `Flux.jl`, which we planned to use in production. But neither `Tracker.jl` nor `Zygote.jl` are working with `CuArrays.jl`. With `Tracker.jl` I get

```julia
ERROR: MethodError: no method matching CuArray{Float32,2}(::UndefInitializer, ::Tuple{Int64})

```

(with a very deep stacktrace), when attempting to calculate the gradient and `Zygote.jl` throws an error already during precompilation and stackoverflows during gradient calculation. So now I am really stuck and have to start over with Python. If I knew this from the beginning I would go for Python and would save a lot of time and energy…

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [September 24, 2019, 1:26pm UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/11 "2019-09-24T13:26:20Z")

</div>

Have you tried using either forwarddiff or reversediff?

---

<div class="post-metadata">

**Author:** ![Azamat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/azamat/32/6892_2.png) [@Azamat](https://discourse.julialang.org/u/Azamat)\
**Post date:** [September 24, 2019, 1:35pm UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/12 "2019-09-24T13:35:11Z")

</div>

I don’t think they are compatible with Flux.jl and GPU. Besides, ForwardDiff.jl would be prohibitively slow on my R^n → R loss function.

---

<div class="post-metadata">

**Author:** ![jonathanBieler](https://avatars.discourse-cdn.com/v4/letter/j/82dd89/32.png) [@jonathanBieler](https://discourse.julialang.org/u/jonathanBieler)\
**Post date:** [September 24, 2019, 2:39pm UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/13 "2019-09-24T14:39:18Z")

</div>

cf.

[https://github.com/FluxML/Flux.jl/issues/864](https://github.com/FluxML/Flux.jl/issues/864)

[https://github.com/FluxML/Flux.jl/issues/854](https://github.com/FluxML/Flux.jl/issues/854)

(I guess)

---

<div class="post-metadata">

**Author:** ![Azamat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/azamat/32/6892_2.png) [@Azamat](https://discourse.julialang.org/u/Azamat)\
**Post date:** [September 24, 2019, 6:59pm UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/14 "2019-09-24T18:59:08Z")

</div>

Well, if you want all the details, the issues you referenced are relatively easy, and I was able to work around them (just did `y |> cpu` for `onecold` one and ignored the test failings). Then I was getting hit by `ERROR: MethodError: Cannot convert an object of type Array{Float32,2} to an object of type Adjoint{Float32,Array{Float32,2}}`.

I hunted hard for minimum reproducible example, but could not reduce it down to few lines. Asked about this error on Slack – no luck. Then I decided to file an [issue](https://github.com/FluxML/Flux.jl/issues/866) about it on GitHub with the example I had. It was greatly simplified from the original, but still a few hundred lines of code. In the end, I worked around it by using `permutedims` instead of `'` everywhere.

Next, I was getting hit by [this bug](https://github.com/FluxML/Tracker.jl/issues/45), but not when using `mapslices` – when using `Flux.gradient` instead. Then I tried `Zygote.gradient` on the master branch of `Zygote.jl` got StackOverflow. Next, I tried the latest release of `Zygote.jl` – got `ERROR: MethodError: _forward(::Zygote.Context, ::Type{Array}, ::Array{Float32,2}) is ambiguous`. Asked about it on Slack and thankfully @MikeInnes, provided a quick fix for it, for which I am very grateful to him. After incorporating his fix I’m now getting `ERROR: UndefVarError: S not defined Stacktrace: [1] show(::IOContext{REPL.Terminals.TTYTerminal}, ::Type{SYSTEM (REPL): caught exception of type UndefVarError while trying to handle a nested exception; giving up`.

All of the last issues are above my abilities to work around or fix and have very deep stack traces. At this point, I have no choice but to give up and start everything over with PyTorch. I could file issues for those bugs, but again looks like my reproducible examples would be a few hundred lines at best after putting the all the effort to simplify my code, so I am hesitant to do that.

---

<div class="post-metadata">

**Author:** ![Rana\_Ian](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rana_ian/32/13554_2.png) [@Rana\_Ian](https://discourse.julialang.org/u/Rana_Ian)\
**Post date:** [December 7, 2019, 9:27pm UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/16 "2019-12-07T21:27:42Z")

</div>

See Flux bidirectional LSTM at [https://github.com/maetshju/flux-blstm-implementation/blob/master/01-blstm.jl](https://github.com/maetshju/flux-blstm-implementation/blob/master/01-blstm.jl)

---

<div class="post-metadata">

**Author:** ![raff](https://avatars.discourse-cdn.com/v4/letter/r/5e9695/32.png) [@raff](https://discourse.julialang.org/u/raff)\
**Post date:** [December 8, 2019, 12:33am UTC](https://discourse.julialang.org/t/starting-a-deep-learning-project-should-i-keep-using-julia-or-jump-to-python/3840/17 "2019-12-08T00:33:13Z")

</div>

Thanks everyone for the replies.

It has been two years ago since I started my first deep learning project. I decided to use PyTorch at that time. But, I left deep learning for a while to work on other projects.

I just recently work with deep learning again, and chose PyTorch as a starting point. But, eventually, I realized that it’s hard for me to be productive when using Python. I then switch back to Julia and write the code in Julia.

If anyone interested, [here](https://github.com/rizalzaf/AdversarialPrediction.jl) is the link to the repo. It’s about replacing the standard cross-entropy loss with the objective that aligns better with the evaluation metric we care about.