Create Matrix from vectors returned from for loop

New to Julia
,
1

Hi all!

I’m running a simulation where I’m tracking the behavior of multiple elements over time. Say like the instantaneous speed of several cars on a race track. I can use the command

sim_time, spd_c1 = get_state_series(results, ("Car 1", :speed))

to get the time steps (saved as sim_time which is type Vector{Float64}) and the speed of Car 1 (saved as spd_c1 which is type Vector{Float64}) which I can then plot with PlotlyJS using the command

plot([scatter(x=sim_time, y=spd_c1, mode="lines", name="Car 1")])

which is fine, except I have to manually specify the name of each car I want to gather results from manually. I’d like to be able to gather all the results for all the cars for plotting automatically, but I’m struggling with how to accomplish this.

I’ve tried something like:

c1 = [2, 6, 8, 2]
c2 = [3, 20, 12, 1]
c3 = [6, 12, 4, 8]
sim_time = [1, 2, 3, 4]

for n in [c1, c2, c3]
    global car_spd = similar(sim_time)
    push!(car_spd, n)
    return car_spd
end

display(car_spd)

But I get the error

ERROR: MethodError: Cannot `convert` an object of type Vector{Int64} to an object of type Int64
Closest candidates are:
  convert(::Type{T}, ::ColorTypes.Gray24) where T<:Real at ~/.julia/packages/ColorTypes/1dGw6/src/conversions.jl:114
  convert(::Type{T}, ::ColorTypes.Gray) where T<:Real at ~/.julia/packages/ColorTypes/1dGw6/src/conversions.jl:113
  convert(::Type{T}, ::Ptr) where T<:Integer at pointer.jl:23
  ...
Stacktrace:
 [1] push!(a::Vector{Int64}, item::Vector{Int64})
   @ Base ./array.jl:1057
 [2] top-level scope
   @ ./Untitled-1:8

Whereas I was hoping for a result that looked something like

[2, 6, 8, 2;
3, 20, 12, 1;
6, 12, 4, 8]

Any ideas?

2

using stack function available on 1.9:

julia> stack([c1, c2, c3],dims=1)
3×4 Matrix{Int64}:
 2   6   8  2
 3  20  12  1
 6  12   4  8

but this may not solve the overall problem. Note that using push! often needs to allocate. If you know the number of vectors, it may be best to pre-allocate the matrix and fill the vectors one by one. Ideally, filling the matrix column by column, as Julia stores columns first.

1 Like
3

The push! is often not the direct problem (as a list of vectors is just a list of references to those vectors), but those vectors have been allocated at some point, and will need to be copied into the organized matrix memory.

A more efficient method, is to send to the vector-calculating-code a view of the part of the matrix to populate, and let it use this already allocated memory.

You can often see this pattern in the standard libraries where functions come with mutating versions (with a !) and the non-mutating versions allocate and call the non-allocating mutating versions.

2 Likes
4

In my real use-case, c1, c2, and c3 won’t be explicitly defined in advance like they are here. The actual entry into the for loop will be something like

for c in get_name.(get_components(Cars, sys))
      sim_time, car_s = get_state_series(results, (c, :speed))
      global car_spd = similar(sim_time)
      push!(car_spd, car_s)
end

so stacking won’t work, I need to combine the results one at a time from inside the for loop like I attempted to do in my example. I just didn’t write this out explicitly before because any helpful responders couldn’t run the code using these functions (they are specific to my simulation package), and I was trying to create a minimum example that created the same error as I was seeing in my full code.

5

If the number of elements can be quickly calculated in advance, then a pre-allocation strategy can work. Otherwise, collecting the results 1-by-1 using push! and later arranging them in a matrix is the idiomatic solution.

6

The problem is in:

The error is because car_spd is a vector of Int instead of a vector of vectors of Int. Instead try:

global car_spd = typeof(sim_time)[]

instead of the similar call, will fix it.

7

Ooh! That is close! Using

for n in [c1, c2, c3]
    global car_spd = typeof(sim_time)[]
    push!(car_spd, n)
    return car_spd
end

I get out

1-element Vector{Vector{Int64}}:
[2, 6, 8, 2]

if I remove the return command I get

1-element Vector{Vector{Int64}}:
[6, 12, 4, 8]

Moving car_spd out of the for loop like this

car_spd = typeof(sim_time)[]
for n in [c1, c2, c3]
    push!(car_spd, n)
end

returns

3-element Vector{Vector{Int64}}:
 [2, 6, 8, 2]
 [3, 20, 12, 1]
 [6, 12, 4, 8]

which I think is close enough to what I want because I can index that in my plot code.

Alternatively I was able to find

car_spd = Array{Vector{Int64}}([sim_time])
for n in [c1, c2, c3]  
    global car_spd = push!(car_spd, n)
end

returned

4-element Vector{Vector{Int64}}:
 [1, 2, 3, 4]
 [2, 6, 8, 2]
 [3, 20, 12, 1]
 [6, 12, 4, 8]

which is also acceptable to me since I can just adjust my index to account for the vector containing my time steps. Which I only note in case it’s helpful to someone else down the line. Thanks!

1 Like
8

Yes, global car_spd = typeof(sim_time)[] in every iteration of the loop, resets car_spd to an empty vector, which is no good.

Seems this is on the way to being resolved. If it isn’t performance critical, then any efforts to optimize might be a distraction.

9

This isn’t a performance critical application, however in the interest of learning more Julia best practices, I’d be interested in a more efficient solution. My current system only has 5 “Cars” to gather results from, but down the road I could be needing to gather results from hundreds of “Cars” at a time, in which case performance might matter more.