# Vector adjoint and indexing don't commute?

**URL:** <https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980>\
**Category:** General Usage\
**Created:** [September 2, 2020, 11:37pm UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980 "2020-09-02T23:37:21Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![timmor](https://avatars.discourse-cdn.com/v4/letter/t/e19b73/32.png) [@timmor](https://discourse.julialang.org/u/timmor)\
**Post date:** [September 2, 2020, 11:37pm UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/1 "2020-09-02T23:37:22Z")

</div>

I was surprised that vector adjoint doesn’t commute with indexing.

```julia
x = [1:5;]
x[1:3]' # is a 1x3 Adjoint
x'[1:3] # is a 3 element Vector

```

For example if I set `y = x'` my mental model of y is “row vector”, and I was thinking that `y[1:3] * x[1:3]` would be a dot product. Instead I must write `y[1:3]' * x[1:3]` which feels like I’m transposing a second time. What’s the reason for indexing not maintaining the adjointness?

In a similar vein perhaps, reverse:

```julia
reverse(x)' # is a 1x5 Adjoint
reverse(x') # is an error for not specifying which dimension

```

If I’m reversing a vector adjoint, would it be sensible for reverse to default to operating along the vector?

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [September 2, 2020, 11:40pm UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/2 "2020-09-02T23:40:41Z")

</div>

instead of `y[1:3]` write `y[:, 1:3]`, the reason is that `y[1:3] == y[1, 1:3]` is just one row which will give you a vector, this is true for any-shape matrix/array

---

<div class="post-metadata">

**Author:** ![lhnguyen-vn](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lhnguyen-vn/32/15449_2.png) [@lhnguyen-vn](https://discourse.julialang.org/u/lhnguyen-vn)\
**Post date:** [September 2, 2020, 11:48pm UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/3 "2020-09-02T23:48:55Z")

</div>

This seems to be a common pitfall for newcomers to Julia. Keeping in mind that your mental model of a row vector is actually a matrix will probably help a lot!

---

<div class="post-metadata">

**Author:** ![timmor](https://avatars.discourse-cdn.com/v4/letter/t/e19b73/32.png) [@timmor](https://discourse.julialang.org/u/timmor)\
**Post date:** [September 3, 2020, 12:01am UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/4 "2020-09-03T00:01:30Z")

</div>

Right, but I thought the motivation for the Adjoint type is that it’s _not_ a matrix. It’s a map that operates on vectors to produce scalars. So `[1,2,3]' * [1,2,3]` is a dot product. But `[1 2 3] * [1,2,3]` is a matrix-vector product, and the result is a 1-element vector. Maybe this is part of the confusion. As far as indexing is concerned, we do still need to think of Adjoints as if they were matrices, although they don’t operate as matrices do.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [September 3, 2020, 7:38am UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/5 "2020-09-03T07:38:35Z")

</div>

The conflict here is that a single index also serves as the linear index into a container (which, by convention, always returns a Vector). Thus for an adjoint, this conflicts with the “normal” index and apparently the linear one wins. I’m not sure there is a way to do a “normal” index into a adjoint vector.

---

<div class="post-metadata">

**Author:** ![timmor](https://avatars.discourse-cdn.com/v4/letter/t/e19b73/32.png) [@timmor](https://discourse.julialang.org/u/timmor)\
**Post date:** [September 3, 2020, 9:47am UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/6 "2020-09-03T09:47:35Z")

</div>

Ah yes good point, it’s the same syntax for both, so for a “morally one-dimensional” array you can’t distinguish the intention.

---

<div class="post-metadata">

**Author:** ![grunst](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/grunst/32/9271_2.png) [@grunst](https://discourse.julialang.org/u/grunst)\
**Post date:** [September 3, 2020, 10:35am UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/7 "2020-09-03T10:35:33Z")

</div>

This will let `A[1,:]` return a row vector. In case it is useful.

```julia
julia> import Base: getindex

julia> getindex(A::AbstractArray,i::Int,j) = adjoint(A'[j,i])

julia> A = randn(4,4)

julia> A[1,:]
1×4 LinearAlgebra.Adjoint{Float64,Array{Float64,1}}:
 -2.12388 1.72479 -0.704399 0.183572

julia> A[1,2:3]
1×2 LinearAlgebra.Adjoint{Float64,Array{Float64,1}}:
 1.72479 -0.704399

julia> A[1,:]*A[:,1]
6.601714378026813

```

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [September 3, 2020, 10:50am UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/8 "2020-09-03T10:50:56Z")

</div>

This is [type piracy](https://docs.julialang.org/en/v1/manual/style-guide/#Avoid-type-piracy), since you don’t own any of the types involved. This will break all kinds of routines in Base, because `A[1, :]` is always assumed to return an `AbstractVector`, so use at your own risk.

---

<div class="post-metadata">

**Author:** ![grunst](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/grunst/32/9271_2.png) [@grunst](https://discourse.julialang.org/u/grunst)\
**Post date:** [September 3, 2020, 10:56am UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/9 "2020-09-03T10:56:22Z")

</div>

Yes agree it is dangerous. Adding this method breaks the `VecOrMat of Vectors` testset in `matmul.jl`.

---

<div class="post-metadata">

**Author:** ![timmor](https://avatars.discourse-cdn.com/v4/letter/t/e19b73/32.png) [@timmor](https://discourse.julialang.org/u/timmor)\
**Post date:** [September 3, 2020, 11:10am UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/10 "2020-09-03T11:10:44Z")

</div>

Having A[1,:] return a row vector sounds appealing at first, but it doesn’t generalise to indexing in higher dimensions. The issue was more that an actual vector adjoint was losing its adjointness upon indexing that puzzled me. But now I see that’s an inevitable consequence of using the same square bracket syntax for two distinct forms of indexing: linear indexing and Cartesian indexing. For most array types you can tell the two apart by the number of indices provided, but not for vector adjoints, hence the conflict.

---

<div class="post-metadata">

**Author:** ![timmor](https://avatars.discourse-cdn.com/v4/letter/t/e19b73/32.png) [@timmor](https://discourse.julialang.org/u/timmor)\
**Post date:** [September 3, 2020, 1:51pm UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/11 "2020-09-03T13:51:26Z")

</div>

Actually digging a little further it doesn’t seem quite as simple as linear indexing always wins. Consider OffsetArrays:

```julia
using OffsetArrays
offmat = OffsetArray([1 2 3 ; 4 5 6], 0:1, -1:1)
offvec = OffsetArray([1,2,3], -1:1)

offmat[1:3] # is fine, this is linear indexing
offvec[1:3] # is not fine, this is the array's custom indexing; no syntax for ordinary 1-based linear indexing just because I'm 1-dimensional?

```

This seems a bit at odds with the [documentation on this point](https://docs.julialang.org/en/v1/manual/arrays/#Linear-indexing):  
“When exactly one index i is provided, that index no longer represents a location in a particular dimension of the array. Instead, it selects the ith element using the column-major iteration order that linearly spans the entire array.” i.e. suggesting linear indexing always wins. Or is this strictly describing the base array types, and not applicable to OffsetArray?

It’s water under the bridge now, but it would have been nice to have a separate syntax for 1-based linear indexing whatever the array type. And that would have freed the existing syntax to do the right thing by adjoints. Also it would catch inadvertent under-indexing of two- or higher-dimensional arrays; if you meant linear indexing you’d have to specify it.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [September 3, 2020, 3:08pm UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/12 "2020-09-03T15:08:07Z")

</div>

If there is no reply here, you should open an issue with OffsetArrays.

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [September 3, 2020, 3:13pm UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/13 "2020-09-03T15:13:52Z")

</div>

I think the thinking is that these really are the linear indices of the offset vector. They just don’t start at 1:

```julia
julia> offmat
2×3 OffsetArray(::Array{Int64,2}, 0:1, -1:1) with eltype Int64 with indices 0:1×-1:1:
 1 2 3
 4 5 6

julia> LinearIndices(offmat)
2×3 LinearIndices{2,Tuple{OffsetArrays.IdOffsetRange{Int64,Base.OneTo{Int64}},OffsetArrays.IdOffsetRange{Int64,Base.OneTo{Int64}}}} with indices 0:1×-1:1:
 1 3 5
 2 4 6

julia> LinearIndices(offvec)
3-element LinearIndices{1,Tuple{OffsetArrays.IdOffsetRange{Int64,Base.OneTo{Int64}}}} with indices -1:1:
 -1
  0
  1

julia> parent(offvec)[1:3]
3-element Array{Int64,1}:
 1
 2
 3

```

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [September 3, 2020, 3:32pm UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/14 "2020-09-03T15:32:00Z")

</div>

I always see these posts, but I’m a bit confused why. I almost never carry around `1 x N` object in my code. I have no need. What are the use cases?

Honestly you don’t hardly even need to keep track of dimensions anymore. Things like Tullio.jl or TensorOperations.jl just let you write out what you mean. This is the way most people do linear algebra… “`Z = A*B*C`, or whatever this should be so that you end with the right dimensions for `Z`”. Or is that just me?

---

<div class="post-metadata">

**Author:** ![timmor](https://avatars.discourse-cdn.com/v4/letter/t/e19b73/32.png) [@timmor](https://discourse.julialang.org/u/timmor)\
**Post date:** [September 4, 2020, 5:04am UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/15 "2020-09-04T05:04:59Z")

</div>

Oh and just like that, the reverse issue is resolved!  
[https://github.com/JuliaLang/julia/pull/37367](https://github.com/JuliaLang/julia/pull/37367)

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 4, 2020, 7:26am UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/16 "2020-09-04T07:26:16Z")

</div>

> [@timmor](#):
>
> Or is this strictly describing the base array types, and not applicable to OffsetArray?

IMO OffsetArrays should not support linear indexing at all, because of the possibility for confusion that you demonstrate. One can always get the parent and index into that if necessary.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [September 4, 2020, 12:39pm UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/17 "2020-09-04T12:39:22Z")

</div>

PRs to straighten out docs confusion would be very welcome. FWIW we also have this:  
[https://docs.julialang.org/en/v1/devdocs/offset-arrays/#Linear-indexing-(LinearIndices)](https://docs.julialang.org/en/v1/devdocs/offset-arrays/#Linear-indexing-(LinearIndices))

Until we have a dedicated “I’m a linear index!” type, there will likely be confusion; but we still need to be able to index `OffsetVector`s so of course we have to support `::Int` index arguments.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [September 4, 2020, 12:41pm UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/18 "2020-09-04T12:41:52Z")

</div>

> [@timmor](#):
>
> Right, but I thought the motivation for the Adjoint type is that it’s _not_ a matrix. It’s a map that operates on vectors to produce scalars.

Also called a covector. See [LinearAlgebra and TensorCore future design: Be more greedy · Issue #35763 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/35763)

---

<div class="post-metadata">

**Author:** ![timmor](https://avatars.discourse-cdn.com/v4/letter/t/e19b73/32.png) [@timmor](https://discourse.julialang.org/u/timmor)\
**Post date:** [September 4, 2020, 11:37pm UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/19 "2020-09-04T23:37:59Z")

</div>

> [@tim.holy](#):
>
> PRs to straighten out docs confusion would be very welcome. FWIW we also have this:  
> [Arrays with custom indices · The Julia Language](https://docs.julialang.org/en/v1/devdocs/offset-arrays/#Linear-indexing-(LinearIndices))

That explains it very clearly indeed, thanks very much.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [September 5, 2020, 12:29pm UTC](https://discourse.julialang.org/t/vector-adjoint-and-indexing-dont-commute/45980/20 "2020-09-05T12:29:19Z")

</div>

> [@mauro3](#):
>
> I’m not sure there is a way to do a “normal” index into a adjoint vector.

A linear index can always be converted into a multi-index by using `Base._ind2sub` to convert.

```julia
function Base.getindex(A::Adjoint, I::Int)
    Base.@_inline_meta
    @inbounds getindex(A, Base._to_subscript_indices(A, I)...)
end
Base._to_subscript_indices(A::Adjoint, i::Integer) = (Base.@_inline_meta; Base._unsafe_ind2sub(A, i))
function Base._ind2sub(A::Adjoint, ind)
    Base.@_inline_meta
    Base._ind2sub(axes(A), ind)
end

```

Thus, the `Adjoint` should be able to have a linear index.

```julia
julia> [1,2,3]'[1]
1

```

And in fact, it does already have linear indexing built in.