Argument order in `kron(A,B)`

General Usage
1

Julia defines kron(A,B) such that

kron(A,B)*vec(C) = vec(B*C*A')

while if I were to have my way I would define kron(A,B) such that

kron(A,B)*vec(C) = vec(A*C*B')  # A and B are swapped

Given Julia’s history of taking things seriously, I am sure there are good reasons for why kron is the way it is, but I don’t see them. Can someone help me?

2

This¹ definition² considerably³ predates⁴ Julia,⁵ for better or for worse.

7 Likes
3

Thank you very much for the speedy and well researched answer! Interesting that all languages agree on this while they can’t even agree on whether matrices should be stored in row- or column-major order…

4

There was a PR to add a tensor function that would have the column-major compatible order (as well as generalize to higher dimensional arrays), https://github.com/JuliaLang/julia/pull/35150, but it was reverted in https://github.com/JuliaLang/julia/pull/35744, so check those out for some more discussion on the Julia side. This functionality is now in a package: https://github.com/JuliaMath/TensorCore.jl.

FWIW I think it’s unfortunate that we have kron in Base, but no function with the “correct” order with as much visibility, since I think it can be very confusing for newcomers as to why textbook identities like A*B ≈ reshape((kron(A, I(d)))*vec(B), d, d) don’t work in Julia, but do in row-major languages. But I think the solution now is to promote packages like TensorCore.

1 Like
5

Given that pretty much everyone uses this order, it is somewhat difficult to argue that it isn’t the “correct” one for least astonishment. If the order was reversed, a lot of users could complain, and reasonably so.

While some orders are more convenient for some applications, arguing about the “correct” one is usually a pointless exercise because the criterion is very subjective. As long as the choice is reasonable and it is documented, it should be fine.

6

Alright, I probably shouldn’t have used the word “correct” here. I can agree that the name kron means a certain order and it’s the one implemented in Base

What I meant to say is that often one arrives at kron thinking to use it as a tensor product, and for that use-case the order does not interoperate well with column-major operations like vec and reshape, and is in some way “incorrect” for that usage (although not incorrect for a function named kron). So I think it would be helpful if the alternative was more visible, so that people with this use case can find it (people who perhaps aren’t already aware of the difference between kron and a tensor product).

2 Likes