# ANN: TensorSlice.jl → TensorCast.jl

**URL:** <https://discourse.julialang.org/t/ann-tensorslice-jl-tensorcast-jl/19243>\
**Category:** Package Announcements\
**Tags:** package, announcement, broadcast, tensors\
**Created:** [January 3, 2019, 6:12pm UTC](https://discourse.julialang.org/t/ann-tensorslice-jl-tensorcast-jl/19243 "2019-01-03T18:12:36Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [January 3, 2019, 6:12pm UTC](https://discourse.julialang.org/t/ann-tensorslice-jl-tensorcast-jl/19243/1 "2019-01-03T18:12:36Z")

</div>

The [Tens-Or-Slice™](https://github.com/mcabbott/TensorSlice.jl) is a little gadget which aims to make all kinds of slicing, dicing, squeezing and splicing look easy on TV. For example, this is how you slice a 3-tensor into 3×3 SMatrix pieces

```julia
@shape B[k]{i,j} == A[i,j,k] i:3, j:3

```

And this glues them together again, using `reduce(cat,...)` as if they were ordinary matrices, and then reshapes & transposes to get an N×3 matrix:

```julia
@shape C[(j,k),i] := B[k][i,j]

```

This macro doesn’t really do any of the work, it just calls standard Julia things, and can be hooked up to [StaticArrays.jl](https://github.com/JuliaArrays/StaticArrays.jl),[JuliennedArrays.jl](https://github.com/bramtayl/JuliennedArrays.jl), and [Strided.jl](https://github.com/Jutho/Strided.jl). The notation is supposed to agree with [`@tensor`](https://github.com/Jutho/TensorOperations.jl) and [`@einsum`](https://github.com/ahwillia/Einsum.jl) (they overlap on `permutedims`) with the addition of `==` to mean a view rather than `:=` which is a copy.

This was basically a holiday project (once the advent-of-code puzzles got too long) to teach myself a little macrology, which suffered from a little scope creep. But perhaps it will be useful to some of you.

It’s not registered, as there are probably still a few bugs to catch — awkward test cases would be welcome. Installation:

```julia
] add https://github.com/mcabbott/TensorSlice.jl

```

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [January 3, 2019, 8:01pm UTC](https://discourse.julialang.org/t/ann-tensorslice-jl-tensorcast-jl/19243/2 "2019-01-03T20:01:30Z")

</div>

I would like to see something like [einops](https://github.com/arogozhnikov/einops) in Julia for tensor manipulations. I opened an issue in Flux.jl: [https://github.com/FluxML/Flux.jl/issues/534](https://github.com/FluxML/Flux.jl/issues/534)

You could compare the functionality in your package with the einops project, that would also be very helpful. Thanks for the contribution!

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [January 3, 2019, 8:28pm UTC](https://discourse.julialang.org/t/ann-tensorslice-jl-tensorcast-jl/19243/3 "2019-01-03T20:28:28Z")

</div>

Thanks, I saw this advertised & was trying yesterday to find the link again… but all I could find was a million np.einsum-for-dummies blog posts! Now that I have the link I will take another look at what it does, a bit later.

In Flux there is also [this PR](https://github.com/FluxML/Flux.jl/pull/299) for an einsum macro, which like mine, tries to make basic Julia commands, but unlike mine, is interested in `*` etc. (It’s about 10x more compact! But also gave me some wrong answers and I couldn’t see why.) I haven’t actually tried `@shape` with Flux but see no reason for it not to work, will check later.

I also hope that TensorOperations will soon understand derivatives. I have an implementation 90% done, see [this issue](https://github.com/Jutho/TensorOperations.jl/issues/47#issuecomment-449045705) if curious. But roughly speaking, once you tell it how to differentiate a trace and a contraction, then it can do arbitrary (strict Einstein) expressions.

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [January 4, 2019, 7:45am UTC](https://discourse.julialang.org/t/ann-tensorslice-jl-tensorcast-jl/19243/4 "2019-01-04T07:45:21Z")

</div>

After reading the [einops tutorial](https://github.com/arogozhnikov/einops/blob/master/docs/1-einops-basics.ipynb):

- Almost everything their `rearrange` does, can be done with `@shape`. The exception is what they call `0 axes` and I’d call constant indices, which would not be hard to add – it’s is on the readme wishlist. This lets them output arrays with say `size(A) = (20, 1, 100)`, which might be useful for playing well with broadcasting.

- Many of their `reduce` examples could be done with `@einsum`; the simple ones are just `sum(A, dims=...)` at which Einsum is actually pretty efficient, although not Flux-differentiable. But something like “In [21]: max-pooling” could not. It’s a reshape (in something quite similar to `@shape`’s notation) followed by `maximum(A, dims=(3,5))` and `dropdims`.

That seems to be about all! I don’t see anything more in their [tutorial part 2](https://github.com/arogozhnikov/einops/blob/master/docs/2-einops-for-deep-learning.ipynb) really. Apparently `@shape` can claim to implement “shufflenet” whatever that is. At “In [23]” they do “y = convolve\_2d(y)” by reshaping before and afterwards. Note that `einops` doesn’t implement contractions, only reshape/reduce type things. [This issue](https://github.com/arogozhnikov/einops/issues/15) explains that this is because the back-end functions they would like to call are a dogshow.

In fact it will not be hard to add reduction to my package. This needs far more checking… but I just pushed [a change](https://github.com/mcabbott/TensorSlice.jl/commit/1e2920b134289dce92880a4cdfb7dec445254202) which will make this work, for any function which acts like `sum(B, dims=...)`:

```julia
B = rand(2,3,5);
@reduce A[i,j] := sum(k) B[i,j,k]

```

Then I believe that the only thing missing is `@shape A[a,1,b,1] := B[a,b]`, to have all of einops’s functionality. (Making a video like theirs would, on the other hand, take me longer than the entire package…)

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [January 24, 2019, 1:55pm UTC](https://discourse.julialang.org/t/ann-tensorslice-jl-tensorcast-jl/19243/5 "2019-01-24T13:55:18Z")

</div>

This package is now registered under the name [TensorCast.jl](https://github.com/mcabbott/TensorCast.jl), because it suffered further scope creep to include arbitrary broadcasting.

You can do things like this, the word `lazy` means it uses LazyArrays.jl to avoid materialising the whole 3-array RHS:

```julia
using Statistics
x = randn(100); m = rand(1:99, 100);

@reduce y[k] := mean(i,j) m[k] * exp(-(x[i]-x[j])^2) lazy

```

And alla [xiaodai’s question](https://discourse.julialang.org/t/most-efficient-way-to-stack-vectors-on-the-fourth-dimension/19867/4), here’s an efficient way to stack 60k matrices, broadcast an operation, and re-slice them into 60 batches of 1000:

```julia
using Flux
imgs = Flux.Data.MNIST.images(); # 60_000 matrixes each 28x28

@cast batches[b][i,j,n] |= 1 - float(imgs[n\b][i,j]) n:1000;

```

The stacking operation uses the optimised `reduce(hcat,...)`, plus appropriate reshaping. Using `|=` instead of `:=` means (by another abuse of notation) that the sliced batches are copied Arrays, not views.