# Why column major?

**URL:** <https://discourse.julialang.org/t/why-column-major/24374>\
**Category:** General Usage\
**Tags:** question, array, linearalgebra, column-major\
**Created:** [May 19, 2019, 10:49am UTC](https://discourse.julialang.org/t/why-column-major/24374 "2019-05-19T10:49:56Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![StevenSiew](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevensiew/32/218393_2.png) [@StevenSiew](https://discourse.julialang.org/u/StevenSiew)\
**Post date:** [June 9, 2021, 10:21pm UTC](https://discourse.julialang.org/t/why-column-major/24374/21 "2021-06-09T22:21:04Z")

</div>

> Exactly. Linear algebra and text chose different conventions, and we’re paying the price for that. Either we change how linear algebra notation works or we start reading column-major like Traditional Chinese (except with columns left to right). I didn’t know images were row-major, but I guess it makes sense if they needed to be used as input for dot-matrix printers.

Simple, create a package called “Row Major Linear Algebra”. Problem solved.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [June 9, 2021, 10:21pm UTC](https://discourse.julialang.org/t/why-column-major/24374/22 "2021-06-09T22:21:50Z")

</div>

When you read a book, you read a word, then the next line, then the next page, then the next book.

With your scheme there is also no fixed meaning to the dimensions. If you start with a book, you find the specific word along the third dimension. If you start with a library, it’s the fourth (or maybe you count sections, so fifth?). If it’s a multiverse, it’s, I dunno.

There’s no fixed point when you start from the outside going in, the dimensions are just endlessly pushed outwards in a mindwarping way. Instead, start on the inside, and build out. Later dimensions are further out, referring to larger entities.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [June 9, 2021, 10:34pm UTC](https://discourse.julialang.org/t/why-column-major/24374/23 "2021-06-09T22:34:46Z")

</div>

Just to be clear: what I’m dreaming of is ‘first index major’, but the first index moves along the rows.

---

<div class="post-metadata">

**Author:** ![RGon](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rgon/32/19515_2.png) [@RGon](https://discourse.julialang.org/u/RGon)\
**Post date:** [June 9, 2021, 10:37pm UTC](https://discourse.julialang.org/t/why-column-major/24374/24 "2021-06-09T22:37:05Z")

</div>

I actually meant to reply to yha but I mis-clicked your reply button. We’re on the same page here, I totally agree with everything you’ve said. I can’t change who I reply things to, but I can edit my previous comment to emphasize that I agree with you. Sorry, what a mess.

---

<div class="post-metadata">

**Author:** ![yha](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yha/32/3502_2.png) [@yha](https://discourse.julialang.org/u/yha)\
**Post date:** [June 9, 2021, 10:44pm UTC](https://discourse.julialang.org/t/why-column-major/24374/25 "2021-06-09T22:44:23Z")

</div>

Interesting. I see your point now, though this never bothered me at all when using MSB-first indexing in numpy. Possibly because I tend to think in abstract terms of “k-th dimension” rather than rows/columns whenever the arrays are more than 2d.  
(but if I were dictator of Julia, I’d still choose to copy numpy rather than MATLAB on this issue)

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [June 9, 2021, 10:44pm UTC](https://discourse.julialang.org/t/why-column-major/24374/26 "2021-06-09T22:44:24Z")

</div>

No problem at all😄

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [June 9, 2021, 10:50pm UTC](https://discourse.julialang.org/t/why-column-major/24374/27 "2021-06-09T22:50:33Z")

</div>

I guess there’s no way to be truly consistent, sometimes it makes sense to start on the inside and build outwards, sometimes you drill down. But I really have a problem with ‘fastest index moves outwards’. I like ‘first index is fastest, period.’

---

<div class="post-metadata">

**Author:** ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)\
**Post date:** [June 9, 2021, 11:13pm UTC](https://discourse.julialang.org/t/why-column-major/24374/28 "2021-06-09T23:13:20Z")

</div>

Is there a package for “first index fastest”?

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 9, 2021, 11:43pm UTC](https://discourse.julialang.org/t/why-column-major/24374/29 "2021-06-09T23:43:10Z")

</div>

> [@DNF](#):
>
> like ‘first index is fastest, period.

but in a simple table like this:

```julia
John 54kg 1.80m
Paul 80kg 1.70m
Denis 70kg 1.60m

```

isn’t it natural that running over columns (to compute average properties, for instance) should be the fastest?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [June 10, 2021, 9:52am UTC](https://discourse.julialang.org/t/why-column-major/24374/30 "2021-06-10T09:52:57Z")

</div>

> [@lmiq](#):
>
> isn’t it natural that

What is “natural” in these discussions may be very application-dependent. Some applications benefit from either choice, or possible hybrids (eg block storage).

The Julian answer to “row or column major” is that

1. yes, the language provides a default for `Array`,

2. but the important thing is that there is an abstract interface for most operations (indexing, iteration, etc) so that users are free to define their own container types that does what they please.

(Also, a quick & dirty row major array is just a `PermutedDimsArray` away).

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [June 10, 2021, 10:08am UTC](https://discourse.julialang.org/t/why-column-major/24374/31 "2021-06-10T10:08:59Z")

</div>

Regarding your example:

> [@lmiq](#):
>
> ```julia
> John 54kg 1.80m
> Paul 80kg 1.70m
> Denis 70kg 1.60m
> 
> ```

Its transpose is as simple or natural:

```julia
        John Paul Denis
Weight 54kg 80kg 70kg
Height 1.80m 1.70m 1.60m

```

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [June 10, 2021, 10:36am UTC](https://discourse.julialang.org/t/why-column-major/24374/32 "2021-06-10T10:36:37Z")

</div>

Think about the weight and height of a population and put that in that table (or the grades of an exam). The point is that the number of values is much greater than the number of categories.

But of course this is case dependent an application dependent. My note was only because sometimes seeing one example where the option makes sense helps us remember why it is as it is (even if it could be otherwise because of other cases).

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [June 10, 2021, 12:09pm UTC](https://discourse.julialang.org/t/why-column-major/24374/33 "2021-06-10T12:09:26Z")

</div>

> [@lmiq](#):
>
> but in a simple table like this:

Sure, I was thinking of plain numeric arrays, tables would be different.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 8, 2021, 6:51pm UTC](https://discourse.julialang.org/t/why-column-major/24374/34 "2021-07-08T18:51:16Z")

</div>

Split this topic on July 8, 2021, 6:51pm UTC

7 posts were split to a new topic: [Row vs. column major for arrays of coordinate vectors](https://discourse.julialang.org/t/row-vs-column-major-for-arrays-of-coordinate-vectors/64297)

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [July 8, 2021, 7:23pm UTC](https://discourse.julialang.org/t/why-column-major/24374/35 "2021-07-08T19:23:46Z")

</div>

`A = rand(2,2)'` is a row major matrix in natural Julia syntax.

---

<div class="post-metadata">

**Author:** ![zdenek\_hurak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zdenek_hurak/32/53118_2.png) [@zdenek\_hurak](https://discourse.julialang.org/u/zdenek_hurak)\
**Post date:** [August 28, 2021, 12:50pm UTC](https://discourse.julialang.org/t/why-column-major/24374/36 "2021-08-28T12:50:03Z")

</div>

Let me add my my 2¢ too. I confess the issue and the arguments initially made my brain fry, it is indeed quite interesting. But this is how I view it now.

With the notational convention `A[i,j]` we tend to call `i` the _row index_ and `j` the _column index_. Taking into consideration that Julia is _column-major_, it seems weird (or at least inconsistent or inconvenient) to some that the first index corresponds to a row, while it is only the second index that corresponds to a column. Perceived weirdness of this convention is even magnified when another dimension is added and we have `A[i,j,k]`. This may now look inconsistent because asymmetric. On one hand we say that the prioritized (fast) dimension is columns, then rows and then… how shall we call it… say, floors (with a reference to a virtual building where we put one matrix on top of another). But on the other hand we order the indices as rows, columns, floor, as in `A[i,j,k]`. Weird.

Or not? In fact, I no longer think so. I find it quite consistent. But I think it helps to view the **`i` as an index with which we move along the first dimension** (that is, along a given column) rather than just the index of a row. Similarly `j` is used to move along a given row. And `k` is used to move… welll, vertically (again, with the reference to a virtual building). The perceived asymetry then disappears, doesn’t it? It is not _rows → columns → floors_ but rather _indexing/moving along the given column → along the row → along… well… simply moving vertically_. It seems to me perfectly consistent with column-major format.

Perhaps my displayed troubles to find a descriptive word for the third direction can partially explain where the confusion comes from. `k` is an index of a floor and by increasing `k` we are moving along… along some “vertical”. Similarly `i` is an index of a row but by increasing it we are moving along a column.

---

<div class="post-metadata">

**Author:** ![Nathan\_Boyer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nathan_boyer/32/14825_2.png) [@Nathan\_Boyer](https://discourse.julialang.org/u/Nathan_Boyer)\
**Post date:** [August 31, 2021, 2:31pm UTC](https://discourse.julialang.org/t/why-column-major/24374/37 "2021-08-31T14:31:04Z")

</div>

Okay, I’m confused now. What is the fast way to access/create `A[i,j,k]` in Julia …

1. with nested `for` loops on different lines
2. with nested `for` loops on one line
3. with an array comprehension

… and how do you reason through their order?

I understand what column major storage means, but not necessarily the syntax for accessing it.  
(Btw, I looked for this information in the Performance and Multi-dimensional Arrays sections of the docs but couldn’t find it.)

---

<div class="post-metadata">

**Author:** ![amrods](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amrods/32/2543_2.png) [@amrods](https://discourse.julialang.org/u/amrods)\
**Post date:** [August 31, 2021, 2:36pm UTC](https://discourse.julialang.org/t/why-column-major/24374/38 "2021-08-31T14:36:51Z")

</div>

this may help dispel the confusion

> [@How much can hard-coded indexing be avoided in Julia?](https://discourse.julialang.org/t/how-much-can-hard-coded-indexing-be-avoided-in-julia/55699):
>
> I’ve noticed a trend toward Julia’s new keywords trending towards using keywords and functions instead of integer indexing in Julia. For example, first(array) instead of array[1], or eachindex(array) instead of 1:length(array). Is this an intended aim for Julia style-- to avoid integer indexes in code that can avoid it? How likely is it that in the future Julia code will not need to show its 1-based integer heritage in common written code?

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [August 31, 2021, 4:15pm UTC](https://discourse.julialang.org/t/why-column-major/24374/39 "2021-08-31T16:15:03Z")

</div>

Each one remembers this as he/she wants. But one way it to remember that: run first over the first index:

```julia
julia> function update_fast!(A)
          for k in axes(A,3) 
             for j in axes(A,2) 
                 for i in axes(A,1) # <--- running first
                    A[i,j,k] = A[i,j,k] - 1.0
                 end
             end
          end
       end
update_fast! (generic function with 1 method)

julia> function update_slow!(A)
          for i in axes(A,1) 
             for j in axes(A,2) 
                 for k in axes(A,3) # <--- running first
                    A[i,j,k] = A[i,j,k] - 1.0
                end
             end
          end
       end
update_slow! (generic function with 1 method)

julia> A = rand(50,50,50);

julia> using BenchmarkTools

julia> @btime update_fast!($A);
  22.939 μs (0 allocations: 0 bytes)

julia> @btime update_slow!($A);
  138.349 μs (0 allocations: 0 bytes)

```

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [August 31, 2021, 5:58pm UTC](https://discourse.julialang.org/t/why-column-major/24374/40 "2021-08-31T17:58:55Z")

</div>

> [@Nathan\_Boyer](#):
>
> - with nested `for` loops on different lines
> - with nested `for` loops on one line

these two doesn’t generate different code, just different in style.

[Previous page](https://discourse.julialang.org/t/why-column-major/24374.md?page=1)

[Next page](https://discourse.julialang.org/t/why-column-major/24374.md?page=3)