Literal expression for array of vectors

General Usage
, , ,
1

Is there an array literal that gives me this object (and a variation with some integer elements in some of the vectors)?

julia> ...
3×5 Matrix{Vector{Int64}}:
 []  []  []  []  []
 []  []  []  []  []
 []  []  []  []  []

My use case is writing unit tests where I expect a specific matrix of vectors as output, which I wanted to write literally. Of course I can construct the matrix step by step, but for small matrices, writing the literals makes the tests more readable in my opinion.

What I tried so far:

When I have a matrix of vectors, e.g.

julia> m = [Int[] for _ in 1:3, _ in 1:5]
3×5 Matrix{Vector{Int64}}:
 []  []  []  []  []
 []  []  []  []  []
 []  []  []  []  []

I can @show it, giving

julia> @show m
m = [Int64[] Int64[] Int64[] Int64[] Int64[]; Int64[] Int64[] Int64[] Int64[] Int64[]; Int64[] Int64[] Int64[] Int64[] Int64[]]

However, pasting this literal back into the REPL, I get a different object:

julia> new_m = [Int64[] Int64[] Int64[] Int64[] Int64[]; Int64[] Int64[] Int64[] Int64[] Int64[]; Int64[] Int64[] Int64[] Int64[] Int64[]]
0×5 Matrix{Int64}

I think I get what’s going on – the individual arrays in the literal are directly concatenated together to form a “flat” array instead of an array of vectors, like here:

julia> [[1] [2] [3] [4]; [5] [6] [7] [8]]
2×4 Matrix{Int64}:
 1  2  3  4
 5  6  7  8

This (presumably?) is intended to make concatenating rows/columns into larger matrices more convenient, but I think it would be great if copy-pasting the expression from @show would give the correct object, like for many other objects. I tried different variations of the above literal to parse to a matrix of vectors, but so far without success.

Any help or pointers would be greatly appreciated! I feel like I’ve read something about this somewhere, but I couldn’t find it right now…

2

You can reshape a vector of vectors:

julia> m = [Int[i+10j] for i in 1:3, j in 1:5];

julia> vec(m) |> println
[[11], [12], [13], [21], [22], [23], [31], [32], [33], [41], [42], [43], [51], [52], [53]]

julia> reshape([[11], [12], [13], [21], [22], [23], [31], [32], [33], [41], [42], [43], [51], [52], [53]], 3, 5)
3×5 Matrix{Vector{Int64}}:
 [11]  [21]  [31]  [41]  [51]
 [12]  [22]  [32]  [42]  [52]
 [13]  [23]  [33]  [43]  [53]

Yes. The syntax goes to hvcat, and this unfortunately makes assembling block matrices easy, and matrices of arrays hard. The syntax I wrote above is Base.vect which is not a concatenation function.

It’s arguably a bug that println(m) produces something which doesn’t parse back to the same.

2 Likes
3

Right. That still requires one extra “mental step” to look at the flat vector and the dimensions and conclude that the resulting shape makes sense.

I think I am looking for some literal syntax that preserves the “visual shape” of the matrix. With reshape I can kind of make it work, but one has to take care about the right order of the dimensions (that’s exactly the extra mental step I wanted to avoid :sweat_smile: )

E.g. this is one of the matrices that I am actually testing in my code (the eltype is not relevant for this test):

5×2 Matrix{Vector{Any}}:
 [1]  [2]
 [3]  []
 []   []
 []   []
 [4]  []

This is how I would like to write it with reshape (to get visually similar to the REPL output):

julia> reshape([
           [1], [2],
           [3], [ ],
           [ ], [ ],
           [ ], [ ],
           [4], [ ],
       ], (5, 2))
5×2 Matrix{Vector{Any}}:
 [1]  []
 [2]  []
 [3]  []
 []   [4]
 []   []

But of course it’s not right, since the vector elements are parsed in column-major order. So I would need an additional transpose:

julia> reshape([
           [1], [2],
           [3], [ ],
           [ ], [ ],
           [ ], [ ],
           [4], [ ],
       ], (2, 5)) |> permutedims
5×2 Matrix{Vector{Any}}:
 [1]  [2]
 [3]  []
 []   []
 []   []
 [4]  []

This is kind of clunky, but probably good enough for me.

4

That’s what I thought, but if it turns out that having a parser with “concatenation of rows/columns is convenient” is just not compatible with “arrays of vectors are convenient”, then perhaps there is no way around this inconsistency?

5

It’s ridiculous, but you can wrap it in another layer of 1-element vectors to be hvcat-ed because it’s not recursive:

julia> [[Int64[1]] [Int64[]];
        [Int64[]] [Int64[1]]]
2×2 Matrix{Vector{Int64}}:
 [1]  []
 []   [1]

julia> [Int64[1] Int64[];
        Int64[] Int64[1]]
ERROR: DimensionMismatch: mismatched height in block row 1 (expected 1, got 0)
1 Like
6

Wow… I tried regular parentheses, but somehow nesting the vectors in another vector didn’t cross my mind. Thanks!

I’m not sure if it makes the test code more confusing in the end, but at least it’s an answer to my question :slight_smile:

On the matter of @show, it would probably also too confusing if @show would print the nested output (?)

julia> m = [[[1]] [Int64[]]; [[2]] [[3]]]
2×2 Matrix{Vector{Int64}}:
 [1]  []
 [2]  [3]

julia> @show m

# current behavior
m = [[1] Int64[]; [2] [3]]

# desired
m = [[[1]] [Int64[]]; [[2]] [[3]]]
7

You can also print tuples instead of vectors:

julia> [(11,)  (21,)  (31,)  (41,)  (51,)   # from Tuple.(m)
        (12,)  (22,)  (32,)  (42,)  (52,)
        (13,)  (23,)  (33,)  (43,)  (53, 54, 55) ] .|> collect
3×5 Matrix{Vector{Int64}}:
 [11]  [21]  [31]  [41]  [51]
 [12]  [22]  [32]  [42]  [52]
 [13]  [23]  [33]  [43]  [53, 54, 55]

If they are all the same length, then writing a literal 3D array & slicing it might be another option:

julia> eachslice([11 12 13;;; 21 22 23;;; 31 32 33;;; 41 42 43;;; 51 52 53]; dims=(2,3))
3×5 Slices{Array{Int64, 3}, Tuple{Colon, Int64, Int64}, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}, SubArray{Int64, 1, Array{Int64, 3}, Tuple{Base.Slice{Base.OneTo{Int64}}, Int64, Int64}, true}, 2}:
 [11]  [21]  [31]  [41]  [51]
 [12]  [22]  [32]  [42]  [52]
 [13]  [23]  [33]  [43]  [53]
1 Like
8

Yeah, the nicer matrix printout is display, and the one-liner @show uses print. They’re supposed to resemble each other, not how we’d write an array literal (or paste from a printout). There could be another pastable_literal_expr function, but I don’t know of one. It’s generally not always possible for a runtime value to produce an expression that can be parsed and evaluated to an equal value; for mutables, it’s trivially impossible to make an identical value.

1 Like
9

I understand. It’s just very convenient that show often does give something that is more or less the same object (e.g for comparison with == and if it’s not a mutable struct field or so), and now I know that it would also be possible to print arrays of vectors in such a way that they could be parsed back to something == the original object.

So I was just wondering if there are any downsides of changing show/print for such arrays of arrays (other than that it might look confusing with the nested vectors)?

It would be a bit closer to what the docstring of show suggests:

 The representation used by show generally includes Julia-specific formatting
  and type information, and should be parseable Julia code when possible.

(Granted, the docstring doesn’t say to what object the code should be parseable…)

10

Indeed, here’s another example with views:

julia> m = view(collect(1:10), 1:2:10)
5-element view(::Vector{Int64}, 1:2:9) with eltype Int64:
 1
 3
 5
 7
 9

julia> show(m)
[1, 3, 5, 7, 9]
julia> m2 = eval(Meta.parse(repr(m))) # repr returns show's String
5-element Vector{Int64}:
 1
 3
 5
 7
 9

julia> m == m2
true

julia> parent(m) == parent(m2) # the similarities don't go far
false

We just have to live with Julia-looking printouts we can’t expect to evaluate to an equivalent result.

11

But the example above doesn’t parse to a real object at all (I guess it parses, but without the extra nesting the concatenation just doesn’t work):

julia> @show m;
m = [[1] Int64[]; [2] [3]]

julia> m = [[1] Int64[]; [2] [3]]
ERROR: DimensionMismatch: mismatched height in block row 1 (expected 1, got 0)

My point is not whether we can get printouts to always evaluate, but whether it’s possible in this particular case (matrix/array of vectors) to get closer to “consistent” printouts. I’m fine if this is something that not many people care about though. If I find some time I might open a separate question about this or look on Github for existing issues.

Thanks for the replies!

12

It’s kind of sad that typed_hvcat can’t figure out that elements of the output type shouldn’t be iterated over for concatenation. This is less than optimal:

julia> Vector{Int}[Int[1] Int[2]; Int[3] Int[4]]
ERROR: MethodError: Cannot `convert` an object of type Int64 to an object of type Vector{Int64}
The function `convert` exists, but no method is defined for this combination of argument types.