Converting between Matrices, Tuples, and Iterators

New to Julia
1

So there is something unstated about the way matrices, tuples, and iterators (i.e., Base.Generator) work that is causing me problems. I can make the hard things work (except for a few installation issues), but I have trouble almost every day with the equivalent of for loops or vectorization.

Today’s example is that I’m trying to build a matrix. The details of the problem are complicated, but the complicated stuff works fine. The simplification of the stuff that doesn’t work is:

samps = [3, 2, 5, 8, 3]
5-element Vector{Int64}:
 3
 2
 5
 8
 3

temp = [map(x -> x^k, samps) for k = 1:3]
3-element Vector{Vector{Int64}}:
 [3, 2, 5, 8, 3]
 [9, 4, 25, 64, 9]
 [27, 8, 125, 512, 27]

Great, but I wanted a matrix. Should be a trite change, but I’ve tried about a dozen different things and have yet to figure out something that works. Two examples of things I tried:

convert(Matrix{Int64}, temp)
MethodError: no method matching Matrix{Int64}(::Vector{Vector{Int64}})

and

temp1 = vcat(map(x -> x^k, samps) for k = 1:3)
1-element Vector{Base.Generator{UnitRange{Int64}, var"#5#7"}}:
 Base.Generator{UnitRange{Int64}, var"#5#7"}(var"#5#7"(), 1:3)

collect(temp1)
1-element Vector{Base.Generator{UnitRange{Int64}, var"#5#7"}}:
 Base.Generator{UnitRange{Int64}, var"#5#7"}(var"#5#7"(), 1:3)

I also looked for a fill like solution, but didn’t find it.

The best solution would be to create the matrix directly with a one liner, without copy, and without conversion. I don’t need the fastest possible solution (this is not the Focal Point of Amdahl’s Law for the larger application), within a factor of 2 or 3 of C would be great.

2

First, here is something that does what you want (I believe)

julia> samps'.^(1:3)
3×5 Matrix{Int64}:
  3  2    5    8   3
  9  4   25   64   9
 27  8  125  512  27

Now you can decide if that needs to be sped up or not.

3

An array comprehension seems appropriate

[x^k for x in samp, k in 1:3]

Warning: this was written on my phone and is untested, but it’s close.

Your temp variable must necessarily be a vector of vectors, by construction, but you can use hcat(temp...) or reduce(hcat, temp).

1 Like
4

Oh no, I’ve oversimplified . I meant for this to be a stand-in for a set of nearly arbitrary functions, powers of integers are too simple.

5

That works. It is what I needed.

But before posting I had tried:

[x^k for x in samp, for k in 1:3]
syntax: unexpected "]"

What is the deeper understanding of the syntax (or semantics) that will generalize this example?

6

Not using map in some contexts seems like an important pattern. I see no reason to avoid something like:

[ funcTab[k](x) for x in samp, k in 1:3 ]

Using map has become a habit (:frowning_face: I wonder when that happened to me). I’ve been using a CAS (Computer Algebra System) where you do something like

[ UnArray(map(x -> x^k, samp)) for k in 0:2 ]

where UnArray converts an array to a sequence that is repackaged into the new array. This may be inherently inefficient, because it forces copying (or for some other reason) … but if I naturally end up with a row array from the inner loop is there something like this that I can (or should) use?

7

You can ‘unarray’ with the splat operator, .... I dont’t see how that would help here though. I doesn’t work in this context, but even if it did, what would be the interpretation of something like

[1, 2, 3 for i in 0:2]

?
I don’t know what that should do.

8

Apparently it doesn’t mean anything. But how do I make it mean what I want?

Before you ask what I want, let me try to restate … If for some reason that’s a bit flaky (perhaps involving other people’s code) I produce a Vector of Vectors, but I needed the corresponding matrix, what is the best way to convert?

9

If X is your vector of vectors, reduce(hcat,X) will return the transpose of what you desired in the first post.

julia> reduce(hcat,X)
5×3 Matrix{Int64}:
 3   9   27
 2   4    8
 5  25  125
 8  64  512
 3   9   27

julia> reduce(hcat,X)'
3×5 adjoint(::Matrix{Int64}) with eltype Int64:
  3  2    5    8   3
  9  4   25   64   9
 27  8  125  512  27

You can also just easily write your own function.

function tomatrix(X::Vector{Vector{T}}) where {T}
    N = length(X)
    K = length(first(X))
    Z = Matrix{T}(undef,K,N)
    @inbounds for (i,v) in pairs(X)
        for (j,y) in pairs(v)
            Z[j,i] = y
        end
    end
    return Z
end
10

Thanks. The first thing I tried was something close to this, but it must have been slightly wrong in a way that I didn’t see at the time.

Will solder on. My intuition will catch up with the rest of me, eventually.

:sleepy:

11

As I look back over my notes, I notice that I’m often wrong in my choice between hcat and vcat (though there are other issues as well). I think this arises because a Vector of Vectors displays the inner vectors as rows (the convection that is common for graphics) but they are actually columns (the convention that is common for “Linear Algebra”).

Not sure how often this confusion arises for those new to Julia; not sure what in the manual would have helped.

1 Like
12

I really am not sure if this is better but what about:

julia> reshape([ y for k in 0:2 for y in map(x -> x^k, samp) ], 3, length(samp))
3×5 Array{Int64,2}:
 1  1  2  3  25
 1  1  5  9  64
 1  3  8  4   9
13

Yeah, I can see how that could cause some confusion. I guess the thing to keep in mind is that [1, 2, 3] always represents a regular old vector (i.e., a tensor of order/rank 1). Julian vectors are neither row vectors nor column vectors. This could be confusing if you’re coming from Matlab where [1, 2, 3] == [1 2 3]. In other words, in Matlab [1, 2, 3] is a row vector (a matrix of size (1, 3)).

1 Like
14

I don’t think of myself as “coming from Matlab”, but I’ve done quite a bit of Matlab, and was thinking about vectors as “row-matrices” or “column-matrices”, and was further imagining that the way they were displayed indicated something about orientation … strange how background can affect the little things more than the big things

1 Like
15

For an arbitrary function f(x, k), you can do f.(samp, (1:3)'). For example,

samp = [3, 2, 5, 8, 3]
f(x, k) = x^k
f.(samp, (1:3)')

The result is

5×3 Matrix{Int64}:
 3   9   27
 2   4    8
 5  25  125
 8  64  512
 3   9   27

The combination of the broadcast dot syntax and the adjoint y' of a vector y is very useful for constructing matrices.