# 100X tensor algebra speedup!?!

**URL:** <https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807>\
**Category:** Specific Domains\
**Tags:** question\
**Created:** [October 31, 2017, 4:33pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807 "2017-10-31T16:33:20Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![tkluck](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkluck/32/15769_2.png) [@tkluck](https://discourse.julialang.org/u/tkluck)\
**Post date:** [June 1, 2019, 2:19pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/21 "2019-06-01T14:19:34Z")

</div>

@chakravala well, in that language, does Grassmann.jl make it possible to contract 3 sparse bivectors in a single kernel? My guess is that @mohamed82008 is definitely willing to learn about multivectors provided that they solve his problem.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [June 1, 2019, 2:28pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/22 "2019-06-01T14:28:58Z")

</div>

No, I haven’t investigated doing operations on 3 elements with Grassmann.jl, but it should be possible.

My post was a general response to to the entire topic, not to @mohamed82008 specifically btw

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [June 1, 2019, 2:51pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/23 "2019-06-01T14:51:42Z")

</div>

Well, my problem is more complicated than multiplying 3 sparse matrices. It’s a reduction of 4-6 1st, 2nd and 3rd order sparse tensors to a 1st order dense tensor or a 2nd order sparse tensor. And obviously any package that I use should be able to represent a 3rd order sparse tensor to begin with. If you have some tutorial or example with more than 2 operands, I would be interested to learn your package’s paradigm.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [June 1, 2019, 3:35pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/24 "2019-06-01T15:35:55Z")

</div>

For associative operations such as tensor products and its duals, it would be fairly trivial to make them n-ary. Non-associative operations such as contractions are already kernelized for 2-inputs (contractions are composite operations based on sparse tensor products and the Hodge duality). There can be both a left contraction and a right contraction, and is not in general symmetric unless the arguments are of the same grade. Thus, there would be more complication and effort with kernelizing operations involving contractions, but it should also be possible. The entire package is already built on the concept of building composite tensor operations, and for unary and binary operations this principle is already applied. The same can be done for more complex multi-argument operations, but would require more coding first.

At JuliaCon 2019 I will publish a background summary paper along with my presentation, which will explain the theoretical foundations of Grassmann.jl, which is still a work in progress and I will definitely consider adding more kernelized tensor operations, as that is what this package is built for.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [June 1, 2019, 4:04pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/25 "2019-06-01T16:04:19Z")

</div>

I don’t want to be rude here, but I’d be shocked if Grassmann.jl can rival the speed of something like TACO and I think saying that Grassmann is ‘somewhat similar’ to TACO is perhaps misleading.

TACO is a very intensive tensor compiler system that is designed to look at arbitrary sparse tensor contractions and use also sorts of numerical analysis techniques to lower that to highly efficient kernels. The sensational ‘100x tensor algebra speedup’ in the title comes from the fact that TACO is able to be very clever about lowering a given contraction to code that involves the fewest possible operations. Many of these contractions if done in the wrong order can have completely difference scaling behaviour so it’s important to optimize the contraction order using all sorts of heuristics.

Is Grassmann.jl doing that sort of analysis? Are there plans for doing that sort of analysis?

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [June 1, 2019, 4:19pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/26 "2019-06-01T16:19:44Z")

</div>

Well, I’m actually not making the claim that Grassmann.jl competes with taco…

> [@Mason](#):
>
> that TACO is able to be very clever about lowering a given contraction to code that involves the fewest possible operations

Grassmann.jl does very similar things, but probably goes about it in a completely different way. In my package I use sub-algebras on manifolds, which are cached using combinatorial principles. There are still lots of improvements possible for working by leveraging sparsity, but it is essentially the same type of thing that you are talking about with taco.

Hey, I’m just a single person making this thing completely on my own, it’s just my own playground and how I want my tensor algebra to work, which is based on a different way of thinking for foundations.

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [June 1, 2019, 7:23pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/27 "2019-06-01T19:23:23Z")

</div>

Grassmanian.jl looks quite nice and you are right to be proud. I’d add some examples of “applications” to demonstrate how this abstract formulation “adds value”.

Differential Geometry is a topic that’s oversold: most PDEs are 3D at most and so it’s never been clear to me why differential forms are any better than `div` and `curl`. So any negativity is because of this underlying context, don’t take it personal. Doing the extra effort of making real world examples will help address that.

As a counterpoint, Taco is much clearer what it does and why it might be useful, and I don’t have to learn what a “multivector” is.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [June 1, 2019, 7:57pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/28 "2019-06-01T19:57:00Z")

</div>

> [@dlfivefifty](#):
>
> Differential Geometry is a topic that’s oversold: most PDEs are 3D at most and so it’s never been clear to me why differential forms are any better than `div` and `curl` . So any negativity is because of this underlying context, don’t take it personal. Doing the extra effort of making real world examples will help address that.

The divergence and curl can be generalized using an exterior differential operator and a boundary operator, which is unified by De Rahm co/homology. This will all be detailed at my JuliaCon paper/talk. In fact, you can easily do those calculations with Grassmann.jl, but I am still working on my paper and implementation designs.

> [@dlfivefifty](#):
>
> As a counterpoint, Taco is much clearer what it does and why it might be useful, and I don’t have to learn what a “multivector” is.

The documentation of Grassmann.jl is intentionally obscure, it is intended to attract people who are willing to go on an adventure (in which they are required to change their foundational understandings).

When it is more complete and stable in design, I will be making more tutorials and applications like PDEs. At that point, documentation and accessibility increases.

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [June 1, 2019, 8:00pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/29 "2019-06-01T20:00:06Z")

</div>

> [@chakravala](#):
>
> The divergence and curl can be generalized using an exterior differential operator and a boundary operator, which is unified by De Rahm co/homology

I know. What I don’t know is what it tells us that we don’t already know in 3D. Sure, it lets us generalise 3D differential geometry to N-D, but why is this useful?

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [June 1, 2019, 8:05pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/30 "2019-06-01T20:05:44Z")

</div>

> [@dlfivefifty](#):
>
> What I don’t know is what it tells us that we don’t already know in 3D. Sure, it lets us generalise 3D differential geometry to N-D, but why is this useful?

In conformal field theory, an R^n dimensional manifold is usefully embedded into a higher dimensional space with an additional Minkowski plane. This allows the usage of symmetry and Lie groups to help linearize otherwise difficult problems. For example, you could consider the Navier-Stokes or Maxwell equations embedded in a 5D conformal field theory, also a basis for differential forms.

---

<div class="post-metadata">

**Author:** ![tkluck](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkluck/32/15769_2.png) [@tkluck](https://discourse.julialang.org/u/tkluck)\
**Post date:** [June 1, 2019, 9:33pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/31 "2019-06-01T21:33:06Z")

</div>

> [@dlfivefifty](#):
>
> Differential Geometry is a topic that’s oversold: most PDEs are 3D at most and so it’s never been clear to me why differential forms are any better than `div` and `curl` .

Let me add a very concrete application of higher-dimensional differential geometry (though maybe you know about this one already @dlfivefifty): The complete description of electricity and magnetism is given by the [Maxwell equations](https://upload.wikimedia.org/wikipedia/commons/thumb/c/c4/Maxwell%27sEquations.svg/1280px-Maxwell%27sEquations.svg.png). These are greatly simplified by using 4-dimensional differential geometry.

Instead of considering the electric and the magnetic field as two separate 3-dimensional entities, you consider them a single 2-form in 4-dimensional space. It then reduces to [a single equation](https://wikimedia.org/api/rest_v1/media/math/render/svg/dc9f3a5f93ddca93c640b39a4307c5bf47a792fc) and, even better, this immediately explains why they are so closely intertwined and why what’s an electric effect to one observer can be a magnetic effect to another.

This is very concrete – it’s the kind of physics that makes GPS tick.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [June 1, 2019, 11:14pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/32 "2019-06-01T23:14:10Z")

</div>

> [@tkluck](#):
>
> This is very concrete – it’s the kind of physics that makes GPS tick.

Do you mean Special/General Relativity?

---

<div class="post-metadata">

**Author:** ![yakir12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yakir12/32/297_2.png) [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Post date:** [June 2, 2019, 6:20am UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/33 "2019-06-02T06:20:25Z")

</div>

Or for ultra realistic ray tracing…?

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [June 2, 2019, 12:13pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/34 "2019-06-02T12:13:39Z")

</div>

Yes, Maxwell equation will be reduced to a single equation, many equations are unified by the Hodge DeRahm Laplacian and the quadratic Casimir, so you only need a single operator for “relativistic wave equations”

> **[Casimir element](https://en.m.wikipedia.org/wiki/Casimir_element)**
>
> In mathematics, a Casimir element (also known as a Casimir invariant or Casimir operator) is a distinguished element of the center of the universal enveloping algebra of a Lie algebra. A prototypical example is the squared angular momentum operator, which is a Casimir element of the three-dimensional rotation group.
> The Casimir element is named after Hendrik Casimir, who identified them in his description of rigid body dynamics in 1931.
> The most commonly-used Casimir invariant is the quadratic ...

Also ray-tracers, the best one is currently [ganja.js](https://github.com/enkimute/ganja.js), the author of it has encouraged me to also make a ray tracer.

---

<div class="post-metadata">

**Author:** ![c42f](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/c42f/32/52842_2.png) [@c42f](https://discourse.julialang.org/u/c42f)\
**Post date:** [June 3, 2019, 3:13am UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/35 "2019-06-03T03:13:49Z")

</div>

Hah the ganja.js “wedge game” is super cool.

> **[Wedge - Geometric Algebra](https://enkimute.github.io/ganja.js/examples/example_game_wedge.html)**

This too: [Algebra vs Geometry - battle 1 – Enki's blog – Math, Graphics, Programming.](https://enkimute.github.io/AlgebraVsGeometry/), it’s just a pity about the javascript syntax. Maybe Grassman.jl is the solution to that part 😃

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [June 3, 2019, 5:58am UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/36 "2019-06-03T05:58:33Z")

</div>

> [@chakravala](#):
>
> The documentation of Grassmann.jl is intentionally obscure

FWIW, statements like this would make me steer clear of a package.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [June 3, 2019, 11:50am UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/37 "2019-06-03T11:50:19Z")

</div>

> [@Tamas\_Papp](#):
>
> FWIW, statements like this would make me steer clear of a package.

It’s not my responsibility to teach the basics of math to random people. The documentation will be increased when I am done constructing the foundations. Why would I document something that is going to change? Why should I care whether you can use my package?

Please pay me, if you want me to be a teacher.

I have given my source away for free, support is not free unless it is for critical bugs or a problem for me.

And it will become easier to understand when more time passes and the foundations are fully solid.

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [June 3, 2019, 12:02pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/38 "2019-06-03T12:02:21Z")

</div>

I see your point and we definitely appreciate you sharing the source code. But documentation can get more people into the package and can increase its value to the users and possibly the developer as well if a lot of users get interested in the package and/or if an interested user has a deep pocket. Of course, the timing of getting these docs out is completely up to you. For example, if Julia had obscure docs, then many of us wouldn’t probably be here.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [June 3, 2019, 12:03pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/39 "2019-06-03T12:03:08Z")

</div>

I think the fact that you advertise the project seems to indicate that you’d like people to use it, and I think that’s where the feedback is coming from (it’s easier to use if the documentation is accessible instead of intentionally obscure).

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [June 3, 2019, 12:16pm UTC](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807/40 "2019-06-03T12:16:09Z")

</div>

> [@chakravala](#):
>
> Please pay me, if you want me to be a teacher.

I think you misunderstood something.

I am not asking you to provide any kind of instruction, I was merely commenting on your practice of making documentation **“intentionally obscure”**.

I don’t really see the purpose of this — no one is obliged to provide documentation, but to invest time in it and then make it intentionally difficult sounds like a waste of effort. I find this puzzling.

The tone of your reply has served to reinforce my earlier conclusion about your package.

[Previous page](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807.md?page=1)

[Next page](https://discourse.julialang.org/t/100x-tensor-algebra-speedup/6807.md?page=3)