Create an object in the first iteration, then add to it

I’m struggling with what seems like a simple coding pattern, but I seem unable to come up with a clean solution to it. Specifically, I often find myself needing to a) create a new object (such as a matrix) and then add to it inside an iteration.

What I’d like to do is something like the following:

a = Array{Float64}
while (don't exit)
    append!(a, <stuff>)

where the first line would create an empty object and append() would add values to it, even if it’s initially empty.

But of course that doesn’t work – the call to append!() barfs. What I have to do instead is something like

a = nothing
while (don't exit)
    if a == nothing
        a = Array{Float64}(<initial stuff>)
        append!(a, <new stuff>)

…which, obviously, feels really clunky.

Yes, I could - in some cases - pre-allocate an uninitialized array of the right size, but that isn’t always possible or desirable. For instance, Flux/Zygote doesn’t let you modify an array in place, because it screws with automatic differentiation. In some cases you don’t know how big the object is going to be before you start the iteration.

I’m currently having this same issue with generating a plot. I want to just create an empty plot and then add different data lines to it. But I can’t figure out how to generate a blank plot.

Am I being dumb here?

a = Array{Float64}(undef, 0) is empty array. You can push to it.

1 Like

That’s neat, thanks. But it doesn’t seem to work in more complicated cases:

> a = Array{Float64}(undef, 0, 0)
> append!(a, [1, 2])
ERROR: MethodError: no method matching append!(::Matrix{Float64}, ::Matrix{Float64})
Closest candidates are:
  append!(::StructArrays.StructVector, ::Any) at ~/.julia/packages/StructArrays/MdA9B/src/tables.jl:24
  append!(::DataStructures.MutableLinkedList, ::Any...) at ~/.julia/packages/DataStructures/vSp4s/src/mutable_list.jl:160
  append!(::BitVector, ::Any) at ~/julia-1.7.1/share/julia/base/bitarray.jl:782

> cat(a, [5 6], dim=1)
ERROR: DimensionMismatch("mismatch in dimension 2 (expected 0 got 2)")

> a = [1 2; 3 4]
> cat(a, [5 6], dim=1)
3x2 Matrix{Int64}:
 1  2
 3  4
 5  6

Only 1d arrays can be dynamically resized in Julia. However, you can build resizable multi-dimensional arrays on top of this. For example, see GitHub - JuliaArrays/ElasticArrays.jl: Resizeable multi-dimensional arrays for Julia

1 Like

Maybe this:

a =  Array{Float64}[]
append!(a, ([1, 2],))  # 1-element Vector{Array{Float64}}: [1.0, 2.0]
append!(a, ([4, 5],[6, 7]))

# can transform into other shapes needed, for example:
m = reduce(hcat, a)'

Without knowing what you are actually trying to accomplish, it’s

The basic question here is why are you dynamically appending rows to a multidimensional array (if that is indeed what you want)? What are you actually trying to accomplish? Without knowing that, it’s hard to say whether you should be using an array-of-arrays approach, ElasticArrays or similar, an array of SVector, or …

In this particular case reading data from a file. The data consists of entries, each of which is a matrix. I’d like to accumulate all of it into a three-dimensional array.

In principle, I don’t really know how big each entry is; all I know is that all the entries are of the same size. I also don’t know how many of them there are. Figuring out the size of an entry is as simple as reading the first one. Figuring out how many of them there are would require traversing the entire file.

But this is a pain point that I seem to encounter often. In another part of this codebase I’m assembling a grid of plots which will show the data. I don’t know how many plots before runtime. I’d like to create an empty plot grid, probably with a known number of columns but an unknown number of rows, and keep appending plots to the end until I’m done.

Seems to me you’re looking for

a = Matrix{Float64}[]
while (don't exit)
    append!(a, <stuff>)

to collect your matrices and e.g.

cat(a..., dims = 3)

to turn them into a 3D array, or some more efficient alternative. Also depending on what <stuff> is, push! may be preferable to append!.

Couldn’t you accumulate the matrices like this (or push!() one by one):

a =  Matrix{Float64}[]
append!(a, ([1 2; 2 1],))
append!(a, ([4 5; 6 7], [-1 0; 0 1]))

# transform into a three-dimensional array:
using TensorCast
@cast m[i,j,k] := a[i][j,k]
1 Like

The answer for plotting will depend on the plotting package, but for Plots.jl you can just put all plots in a vector and do plot(vector_of_plots...)

> using Plots
> using UnicodePlots
> unicodeplots()
> plt1 = lineplot(Plots.fakedata(10), Plots.fakedata(10));
> plt2 = lineplot(Plots.fakedata(10), Plots.fakedata(10));
> plot([plt1 plt2])

ERROR: Cannot convert Matrix{Plot{BrailleCanvas}} to series data for plotting
  [1] error(s::String)
@ Base ./error.jl:33
  [2] _prepare_series_data(x::Matrix{Plot{BrailleCanvas}})
@ RecipesPipeline ~/.julia/packages/RecipesPipeline/Bxu2O/src/series.jl:8
  [3] _series_data_vector(x::Matrix{Plot{BrailleCanvas}}, plotattributes::Dict{Symbol, Any})
@ RecipesPipeline ~/.julia/packages/RecipesPipeline/Bxu2O/src/series.jl:27
  [4] macro expansion
@ ~/.julia/packages/RecipesPipeline/Bxu2O/src/series.jl:144 [inlined]
  [5] apply_recipe(plotattributes::AbstractDict{Symbol, Any}, #unused#::Type{RecipesPipeline.SliceIt}, x::Any, y::Any, z::Any)
@ RecipesPipeline ~/.julia/packages/RecipesBase/qpxEX/src/RecipesBase.jl:289
  [6] _process_userrecipes!(plt::Any, plotattributes::Any, args::Any)
@ RecipesPipeline ~/.julia/packages/RecipesPipeline/Bxu2O/src/user_recipe.jl:36
  [7] recipe_pipeline!(plt::Any, plotattributes::Any, args::Any)
@ RecipesPipeline ~/.julia/packages/RecipesPipeline/Bxu2O/src/RecipesPipeline.jl:70
  [8] _plot!(plt::Plots.Plot, plotattributes::Any, args::Any)
@ Plots ~/.julia/packages/Plots/FI0vT/src/plot.jl:208
  [9] plot(args::Any; kw::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
@ Plots ~/.julia/packages/Plots/FI0vT/src/plot.jl:91
  [10] plot(args::Any)
@ Plots ~/.julia/packages/Plots/FI0vT/src/plot.jl:85
  [11] top-level scope
@ REPL[58]:1
  [12] top-level scope
@ ~/.julia/packages/CUDA/sCev8/src/initialization.jl:52
using Plots; unicodeplots()
plt1 = plot(Plots.fakedata(5), Plots.fakedata(5));
plt2 = plot(Plots.fakedata(5), Plots.fakedata(5));
plot(plt1, plt2)

or: plot([plt1, plt2]...)

Yes, it works if you call plt1 = plot(...). But if you call plt1 = lineplot(...) it doesn’t.

Why should it work when calling lineplot()? It seems that there are no examples in the manual for subplotting using the results from this command.
All examples use plot() or scatter() for that purpose.

I guess I would ask the opposite. Every plot is a canvas, as far as I know. Why would it work for some types of plots and not others?

It turns out that this does work:

x1 = Plots.fakedata(10)
y1 = Plots.fakedata(10)
x2 = Plots.fakedata(10)
y2 = Plots.fakedata(10)
lineplot([x1 x2], [y2 y2], layout=2)

What is lineplot?

julia> using Plots

help?> lineplot

Couldn't find lineplot
Perhaps you meant plot
  No documentation found.

  Binding lineplot does not exist.

It appears this is a function from UnicodePlots, not from Plots. You seem to misunderstand how Plots and its backends work - you just do using Plots and then choose your backend by calling the respective function as you have done above with unicodeplots(). You do not then do using UnicodePlots and mix functions defined in the backend plotting package with those defined in Plots.

Plotting multiple plots onto one figure is documented here (granted it does not include an example of using the splatting operator but then one could argue that’s just a basic language feature that Plots.jl doesn’t have to document separately).