# Tensor contractions very inefficient in julia

**URL:** <https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016>\
**Category:** Performance\
**Tags:** tensors\
**Created:** [December 17, 2020, 11:04pm UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016 "2020-12-17T23:04:33Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nishan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nishan/32/37958_2.png) [@Nishan](https://discourse.julialang.org/u/Nishan)\
**Post date:** [December 17, 2020, 11:04pm UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/1 "2020-12-17T23:04:33Z")

</div>

I am new to julia (just few days) and I want to shift from python to julia by rewriting my DMRG code in julia. For Tensor contractions in python I was using np.tensordot, I searched for equivalent packages in julia and tried NCon, NDTensors etc. Unfortunately all of them are several orders of magnitude slower than np.tensordot. While it is true that the loops and function executions are faster in julia compared to python, In a tensor network code the tensor contractions are the one that dominates other operations. I was wondering if there are anyone here implementing tensor network algorithms in julia and what pakage they use for efficient tensor contractions.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [December 17, 2020, 11:15pm UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/2 "2020-12-17T23:15:49Z")

</div>

Have you tried out `Tullio`? [GitHub - mcabbott/Tullio.jl: ⅀](https://github.com/mcabbott/Tullio.jl) It has tensor contractions that often rival BLAS speed.

---

<div class="post-metadata">

**Author:** ![Nishan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nishan/32/37958_2.png) [@Nishan](https://discourse.julialang.org/u/Nishan)\
**Post date:** [December 17, 2020, 11:17pm UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/3 "2020-12-17T23:17:44Z")

</div>

Not yet, Actually I started contractions just this morning and all day I have been frustrated and thinking of going back to python. I will check your suggestion. Thank you very much.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [December 17, 2020, 11:32pm UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/4 "2020-12-17T23:32:19Z")

</div>

> [@Nishan](#):
>
> Unfortunately all of them are several orders of magnitude slower than np.tensordot.

“Several orders of magnitude” sounds like you might be doing something wrong. If you post a minimal working benchmark of interest to you, it will be easier to help you.

---

<div class="post-metadata">

**Author:** ![Nishan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nishan/32/37958_2.png) [@Nishan](https://discourse.julialang.org/u/Nishan)\
**Post date:** [December 17, 2020, 11:57pm UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/5 "2020-12-17T23:57:33Z")

</div>

for example executing this code:

using TensorOperations  
A = rand(2,3,4)  
B = rand(5,3,6)  
@time @tensor C[a1,a3,b1,b3] := A[a1,a2,a3]\*B[b1,a2,b3]

gives us  
4.724015 seconds (26.08 M allocations: 1.252 GiB, 9.18% gc time)

where as the python equivalent is:

import numpy as np  
import time

s = time.perf\_counter()  
A = np.random.rand(2,3,4)  
B = np.random.rand(5,3,6)  
C = np.tensordot(A,B, axes = [1,1])  
f = time.perf\_counter()  
print('time taken = ',f-s)

which yields

time taken = 0.02150451199850067

The package TensorOperations seems to be widely used in physics community and was suggested to my in a recent winter school. Other packages like NCon and NDTensors also give similar results.

---

<div class="post-metadata">

**Author:** ![juthohaegeman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juthohaegeman/32/8620_2.png) [@juthohaegeman](https://discourse.julialang.org/u/juthohaegeman)\
**Post date:** [December 18, 2020, 12:00am UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/6 "2020-12-18T00:00:58Z")

</div>

Have you tried timing that a second time? First thing to learn when comparing Julia to anything else is that, the first time you run a Julia function or expression, it’s being compiled and `@time` includes the compilation (and inference and code generation and …) time.

Also, there are DMRG implementations (and much more) available at [GitHub - ITensor/ITensors.jl: A Julia library for efficient tensor computations and tensor network calculations](https://github.com/ITensor/ITensors.jl) and [https://github.com/maartenvd/MPSKit.jl](https://github.com/maartenvd/MPSKit.jl) .

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [December 18, 2020, 12:01am UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/7 "2020-12-18T00:01:39Z")

</div>

> [@Nishan](#):
>
> using TensorOperations  
> A = rand(2,3,4)  
> B = rand(5,3,6)  
> @time @tensor C[a1,a3,b1,b3] := A[a1,a2,a3]\*B[b1,a2,b3]

You are measuring the compilation time there. Run it twice or, better, use the `BenchmarkTools` package.

```julia
julia> @time @tensor C[a1,a3,b1,b3] := A[a1,a2,a3]*B[b1,a2,b3];
  0.000087 seconds (85 allocations: 7.516 KiB)

```

---

<div class="post-metadata">

**Author:** ![Nishan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nishan/32/37958_2.png) [@Nishan](https://discourse.julialang.org/u/Nishan)\
**Post date:** [December 18, 2020, 12:11am UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/8 "2020-12-18T00:11:30Z")

</div>

Thank you all, I didn’t realize that @time includes the compilation time too. Like [leandromartinez98](https://discourse.julialang.org/u/leandromartinez98) suggested I used @btime using the `BenchmarkTools` and the run time is actually in microseconds.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [December 18, 2020, 12:15am UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/9 "2020-12-18T00:15:22Z")

</div>

One of the big improvements Julia 1.6 adds is that `@time` will output what percent of a timing was caused by compile time. Running your example, it shows  
`4.457462 seconds (22.67 M allocations: 1.217 GiB, 10.42% gc time, 99.66% compilation time) `

---

<div class="post-metadata">

**Author:** ![Nishan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nishan/32/37958_2.png) [@Nishan](https://discourse.julialang.org/u/Nishan)\
**Post date:** [December 18, 2020, 12:15am UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/10 "2020-12-18T00:15:31Z")

</div>

Thank you. Yes, I am aware of those packages but I wanted to write my own from the scratch as it is more flexible in the long run. Plus I am also writing time evolution for long range Hamiltonian using TDVP and also other algorithms (I have everything written in python already and just want to rewrite it in julia)

---

<div class="post-metadata">

**Author:** ![juthohaegeman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juthohaegeman/32/8620_2.png) [@juthohaegeman](https://discourse.julialang.org/u/juthohaegeman)\
**Post date:** [December 18, 2020, 12:22am UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/11 "2020-12-18T00:22:56Z")

</div>

> [@Nishan](#):
>
> Plus I am also writing time evolution for long range Hamiltonian using TDVP and also other algorithms

These are probably all things that are also included in [MPSKit.jl](https://github.com/maartenvd/MPSKit.jl), together with tensors with arbitrary (abelian or nonabelian) symmetries. But it’s certainly also very instructive to write your own code. Nonetheless, you might want to check out or be interested in [TensorKit.jl](https://github.com/Jutho/TensorKit.jl) as a package that provides a useful set of primitives for writing general tensor network algorithms.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [December 18, 2020, 12:25am UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/12 "2020-12-18T00:25:56Z")

</div>

Take a look here (and on that whole page. A lot of immediately useful information is there).

[[Performance Tips · The Julia Language](https://docs.julialang.org/en/v1/manual/performance-tips/#Measure-performance-with-%5B@time%5D(@ref)-and-pay-attention-to-memory-allocation)]([Performance Tips · The Julia Language](https://docs.julialang.org/en/v1/manual/performance-tips/#Measure-performance-with-%5B@time%5D(@ref)-and-pay-attention-to-memory-allocation))

Well, I cannot link to the section I wanted, but it is there.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [December 18, 2020, 1:01am UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/13 "2020-12-18T01:01:13Z")

</div>

[Here’s a direct link link to the docs section.](https://docs.julialang.org/en/v1/manual/performance-tips/#Measure-performance-with-%5B@time%5D(@ref)-and-pay-attention-to-memory-allocation)

---

<div class="post-metadata">

**Author:** ![Juan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juan/32/7657_2.png) [@Juan](https://discourse.julialang.org/u/Juan)\
**Post date:** [December 18, 2020, 2:21am UTC](https://discourse.julialang.org/t/tensor-contractions-very-inefficient-in-julia/52016/14 "2020-12-18T02:21:40Z")

</div>

You also have  
[https://github.com/chakravala/Grassmann.jl](https://github.com/chakravala/Grassmann.jl)