# TCN Temporal Convolutional Networks in Julia?

**URL:** <https://discourse.julialang.org/t/tcn-temporal-convolutional-networks-in-julia/46188>\
**Category:** Finance and Economics\
**Tags:** question\
**Created:** [September 7, 2020, 8:58am UTC](https://discourse.julialang.org/t/tcn-temporal-convolutional-networks-in-julia/46188 "2020-09-07T08:58:52Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![compleat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/compleat/32/8958_2.png) [@compleat](https://discourse.julialang.org/u/compleat)\
**Post date:** [September 7, 2020, 8:58am UTC](https://discourse.julialang.org/t/tcn-temporal-convolutional-networks-in-julia/46188/1 "2020-09-07T08:58:52Z")

</div>

TCN’s are hyped to be the best new approach in ML forecasting of Financial Time Series since LSTM, and have been claimed to be superior in performance and more efficient to compute. Has anyone managed to implement TCN’s in Julia? [In theory, it should be possible in Flux using a clever mix of CNN’s and a few other things]. I would like to run some tests to see for myself, but would bow to others with superior knowledge for the details.

---

<div class="post-metadata">

**Author:** ![kolia](https://avatars.discourse-cdn.com/v4/letter/k/c68b51/32.png) [@kolia](https://discourse.julialang.org/u/kolia)\
**Post date:** [September 8, 2020, 2:51pm UTC](https://discourse.julialang.org/t/tcn-temporal-convolutional-networks-in-julia/46188/2 "2020-09-08T14:51:18Z")

</div>

The reference implementation [https://github.com/locuslab/TCN/blob/master/TCN/tcn.py](https://github.com/locuslab/TCN/blob/master/TCN/tcn.py) works well.

Here’s a translation of it to Flux

```julia
struct TcnBlock
    chain::Chain
    shortcut
    σ
end

Flux.@functor TcnBlock

(block::TcnBlock)(input) = block.σ.(block.chain(input) + block.shortcut(input))

function tcn_block(a::Int, b::Int;
                          dilation::Int,
                          kernel_size=3,
                          batchnorm=n->BatchNorm(n, relu),
                          dropout=0.2)
    main = Chain(
        Conv((1, kernel_size), a=>b; dilation = dilation, pad = Flux.SamePad()),
        batchnorm(b),
        Dropout(dropout),
        Conv((1, kernel_size), b=>b; dilation = dilation, pad = Flux.SamePad()),
        batchnorm(b),
        Dropout(dropout),
    )
    shortcut = Conv((1,1), a=>b)
    return ResidualBlock(main, shortcut, relu)
end

function tcn(channels; kernel_size=3, batchnorm=n->BatchNorm(n, relu), dropout=0.2)
    return Chain((tcn_block(a, b; dilation=2^(i-1), kernel_size=kernel_size, batchnorm=batchnorm, dropout=dropout)
                  for (i, (a,b)) in enumerate(zip(channels[1:end-1], channels[2:end])))...)
end

the_tcn = tcn([5, 10, 10, 10, 5])

the_tcn(rand(Float32, 1, 81, 5, 32))

```

This Flux version doesn’t do any chomping, so the filtering isn’t causal but instead extends equally far into the future as well as the past. Just grab the model output shifted to the right by the right amount (depends on your `kernel_size` and length of `channels`, probably something like `(kernel_size^(length(channels)-1) - 1)/ 2` for odd `kernel_size`s), and it’ll be ‘causal’.

_Bugfix edit_: added some boilerplate so that Flux seems the `TcnBlock` shortcut params.

---

<div class="post-metadata">

**Author:** ![compleat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/compleat/32/8958_2.png) [@compleat](https://discourse.julialang.org/u/compleat)\
**Post date:** [September 9, 2020, 9:39am UTC](https://discourse.julialang.org/t/tcn-temporal-convolutional-networks-in-julia/46188/3 "2020-09-09T09:39:39Z")

</div>

Thank you so much for this! That is very helpful indeed.  
It is interesting to compare the Flux and python code. The Flux is obviously much more succinct/elegant. How does performance compare, I wonder?

Anyway, I am eager to try this out on an example.

Thanks again!

---

<div class="post-metadata">

**Author:** ![GregT](https://avatars.discourse-cdn.com/v4/letter/g/4da419/32.png) [@GregT](https://discourse.julialang.org/u/GregT)\
**Post date:** [September 15, 2020, 4:24pm UTC](https://discourse.julialang.org/t/tcn-temporal-convolutional-networks-in-julia/46188/4 "2020-09-15T16:24:18Z")

</div>

@compleat if you have an example of using TCN on a real finance dataset, it would be super helpful if you could share w/ the Julia community!

---

<div class="post-metadata">

**Author:** ![compleat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/compleat/32/8958_2.png) [@compleat](https://discourse.julialang.org/u/compleat)\
**Post date:** [September 15, 2020, 5:01pm UTC](https://discourse.julialang.org/t/tcn-temporal-convolutional-networks-in-julia/46188/5 "2020-09-15T17:01:30Z")

</div>

Hi, I was thinking the same thing, perhaps just one simple example for daily returns of the S&P or FTSE as forecast by lags. I would like to try this using Kolia’s code, but I might need some help. [Also, I didn’t understand what he wrote at the end above]

Anyway, thanks for the suggestion. If I get anything running, I will post it.