# Type safe and expressive "tensor" code

**URL:** <https://discourse.julialang.org/t/type-safe-and-expressive-tensor-code/19338>\
**Category:** Machine Learning\
**Created:** [January 6, 2019, 11:03pm UTC](https://discourse.julialang.org/t/type-safe-and-expressive-tensor-code/19338 "2019-01-06T23:03:15Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![datnamer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datnamer/32/3471_2.png) [@datnamer](https://discourse.julialang.org/u/datnamer)\
**Post date:** [January 6, 2019, 11:03pm UTC](https://discourse.julialang.org/t/type-safe-and-expressive-tensor-code/19338/1 "2019-01-06T23:03:15Z")

</div>

There’s been some buzz lately about more expressive, and type safe array code for deep learning. One specific initiative has been the use of arrays with named dimensions:

> **[Tensor Considered Harmful](http://nlp.seas.harvard.edu/NamedTensor)**
>
> Named tensors for better deep learning code.

[GitHub - ctongfei/nexus: Experimental tensor-typed deep learning](https://github.com/ctongfei/nexus) (also compile time type safety)  
[GitHub - ofnote/tsalib: Tensor Shape Annotation Library (numpy, tensorflow, pytorch, ...)](https://github.com/ofnote/tsalib) (uses python type annotations).

Has anyone tried using named arrays with flux on GPU?

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [January 7, 2019, 9:47am UTC](https://discourse.julialang.org/t/type-safe-and-expressive-tensor-code/19338/2 "2019-01-07T09:47:10Z")

</div>

I have not, but as a budding macrologist I have another idea: You could extend the various index-notation macros to check at parse-time (if that’s the right term) that objects defined with certain indices are used again that way.

Papers with lots of many-index tensors often adopt some convention such as `μ,ν=0...3` (spacetime directions, say) vs. `M,N=0...9` vs. `a,b,c = 1,2,3` (numbering some objects). Then if you define `g_{μν}=...` and later refer to `g_{ab}` this sets off a giant warning sign in referee #2’s brain.

We could have a similar warning if we made a version of `@einsum` etc. which, when you write `@einsum_checked g[μ,ν] := something[i,μ,i,ν]`, saves the names of `g`’s indices to some dictionary. And then complains if the next usage is different, by more than 2 places in the alphabet, or something.

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [January 7, 2019, 5:04pm UTC](https://discourse.julialang.org/t/type-safe-and-expressive-tensor-code/19338/3 "2019-01-07T17:04:01Z")

</div>

For your possible amusement: [icheck.jl](https://github.com/mcabbott/TensorSlice.jl/blob/master/src/icheck.jl)

This provides both the “referee #2” check mentioned, and (optionally) inserts run-time checks that sizes (along axes named with a particular letter) are as they were earlier. Very much the first draft, but perhaps not so far from what “tsalib” offers?

---

<div class="post-metadata">

**Author:** ![datnamer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datnamer/32/3471_2.png) [@datnamer](https://discourse.julialang.org/u/datnamer)\
**Post date:** [January 11, 2019, 1:27am UTC](https://discourse.julialang.org/t/type-safe-and-expressive-tensor-code/19338/4 "2019-01-11T01:27:19Z")

</div>

That looks very cool! I’ll dig into it some, thanks.