# Julia end-to-end LSTM for one CPU

**URL:** <https://discourse.julialang.org/t/julia-end-to-end-lstm-for-one-cpu/3942>\
**Category:** Machine Learning\
**Created:** [May 26, 2017, 5:56pm UTC](https://discourse.julialang.org/t/julia-end-to-end-lstm-for-one-cpu/3942 "2017-05-26T17:56:16Z")\
**Posts on this page:** 5\
**Page:** 2

<div class="post-metadata">

**Author:** ![hpoit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hpoit/32/245_2.png) [@hpoit](https://discourse.julialang.org/u/hpoit)\
**Post date:** [May 27, 2017, 1:30am UTC](https://discourse.julialang.org/t/julia-end-to-end-lstm-for-one-cpu/3942/21 "2017-05-27T01:30:21Z")

</div>

That’s nice. Fused kernels are recommended for performance critical operations like the ReLU and Sigmoid activation functions and their gradients. It seems like the kernels are where I should draw my attention to first.

---

<div class="post-metadata">

**Author:** ![hpoit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hpoit/32/245_2.png) [@hpoit](https://discourse.julialang.org/u/hpoit)\
**Post date:** [May 28, 2017, 3:43pm UTC](https://discourse.julialang.org/t/julia-end-to-end-lstm-for-one-cpu/3942/22 "2017-05-28T15:43:25Z")

</div>

I was wrong about kernels, they’re not relevant for a single machine for now. Mathematical operations is my first step. The first file I’m working on is math\_ops.jl, where I am defining [`type TensorArray`](https://www.tensorflow.org/versions/r0.12/api_docs/python/tensor_array_ops/classes_containing_dynamically_sized_arrays_of_tensors_#TensorArray) for [`function add(x, y, name=None)`](https://www.tensorflow.org/versions/r0.12/api_docs/python/math_ops/arithmetic_operators#add) which should return a tensor from tensors `x + y` elementwise, without broadcasting for now. Eventually I’ll have a question and post it here. Some [action](https://github.com/hpoit/LSTM). @kristoffer.carlsson, I understand you’re well versed in tensors, any advice is welcome.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [May 28, 2017, 8:22pm UTC](https://discourse.julialang.org/t/julia-end-to-end-lstm-for-one-cpu/3942/23 "2017-05-28T20:22:38Z")

</div>

Great with some code. How do I run it? It seems that [https://github.com/hpoit/LSTM/blob/master/TensorArray\_ops.jl](https://github.com/hpoit/LSTM/blob/master/TensorArray_ops.jl) is filled with syntax errors (like no `end` for `type`, or no `end` for function). You are also using types that you haven’t defined like `Tensor` and `Scalar`.

---

<div class="post-metadata">

**Author:** ![hpoit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hpoit/32/245_2.png) [@hpoit](https://discourse.julialang.org/u/hpoit)\
**Post date:** [May 29, 2017, 12:12am UTC](https://discourse.julialang.org/t/julia-end-to-end-lstm-for-one-cpu/3942/24 "2017-05-29T00:12:10Z")

</div>

Thanks! I just added `type Scalar` and `type Tensor`, but no `end`s yet. I’ll only come back here after I can give a textbook definition of the types and functions. I looked at your code, it’s pretty clean, and I’m happy about that. One question that emerged is in e.g.

`function readt(index::Tensor{Int32}, name=None)`

should it be `name=None` or `name::String` and `None` be specified inside the function? That’s probably in the Julia Doc but I haven’t reached it yet. I.e. should default values be defined in the argument or in the function? I pulled `None` out of Python and the Google Doc but to me it doesn’t make sense to specify a default value in the argument line and leave out `::String`, unless there’s a way to specify both together as an argument?

---

<div class="post-metadata">

**Author:** ![hpoit](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/hpoit/32/245_2.png) [@hpoit](https://discourse.julialang.org/u/hpoit)\
**Post date:** [June 2, 2017, 10:51pm UTC](https://discourse.julialang.org/t/julia-end-to-end-lstm-for-one-cpu/3942/25 "2017-06-02T22:51:45Z")

</div>

Yes, a keyword arg with type and default value, `name::String=None` 🙂

I’m not sure `None` is the most proper default value though. I’ll find out in a bit.

[Previous page](https://discourse.julialang.org/t/julia-end-to-end-lstm-for-one-cpu/3942.md?page=1)