# Accumulating mixed data and then accessing it

**URL:** <https://discourse.julialang.org/t/accumulating-mixed-data-and-then-accessing-it/12766>\
**Category:** General Usage\
**Created:** [July 31, 2018, 1:36pm UTC](https://discourse.julialang.org/t/accumulating-mixed-data-and-then-accessing-it/12766 "2018-07-31T13:36:45Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![GlenHenshaw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/glenhenshaw/32/5269_2.png) [@GlenHenshaw](https://discourse.julialang.org/u/GlenHenshaw)\
**Post date:** [July 31, 2018, 1:36pm UTC](https://discourse.julialang.org/t/accumulating-mixed-data-and-then-accessing-it/12766/1 "2018-07-31T13:36:45Z")

</div>

I have code that iterates over a long, indeterminate period of time, and at the end of each iteration produces a mixed set of data. I would like to efficiently store all of this data and then do something useful with it (like plot it) at the end.

Right now I’m attempting to do something like the following:

```
foo = []
for i=1:infinity
    a = f1(i) # This happens to be a Float64
    b = f2(i) # This happens to be a vector [x, y, z] of Float64's
    c = f3(i) # This happens to be an Int32
    push!(foo, [a, b, c])
end

```

When done, I have in foo an array of vectors. Which I can’t figure out how to do anything useful with. I can, of course, access the first vector, the second vector, etc, simply by indexing as foo[1], foo[2], etc. But what I actually need to do is pull out all of the “a” values as a single dimensional vector, all of the “b” values as a 3xN matrix, etc.

The obvious first thing to try is just indexing the foo array, ie:

```
a = foo[:,1]
b = foo[:,2]
c = foo[:,3]

```

but because foo is not, in fact, a multidimensional array, the “:” index doesn’t look into the individual vector elements of the array, so I just get the original foo array back.

A list comprehension seemed like it might do the trick:

```
(a, b, c) = [(bar[1], bar[2], bar[3]) for bar in foo]

```

but all this does is stick the _last_ value in the “a” column into a, etc.

I can make it work by doing one list comprehension per column, ie

```
a = [bar[1] for bar in foo]

```

does in fact pull out the first column that I want. But then I have to do a separate list comprehension for every column in the array. Which seems inefficient, not to mention really really kludgy.

Is there an idiomatic way of doing this that I’m just missing? This seems like it should be straightforward, but it isn’t.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [July 31, 2018, 1:43pm UTC](https://discourse.julialang.org/t/accumulating-mixed-data-and-then-accessing-it/12766/2 "2018-07-31T13:43:06Z")

</div>

Check out [https://github.com/baggepinnen/DeepGetfield.jl/](https://github.com/baggepinnen/DeepGetfield.jl/)  
It was created to solve that problem for myself. The README should have some example use cases.

---

<div class="post-metadata">

**Author:** ![platawiec](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/platawiec/32/31914_2.png) [@platawiec](https://discourse.julialang.org/u/platawiec)\
**Post date:** [July 31, 2018, 1:58pm UTC](https://discourse.julialang.org/t/accumulating-mixed-data-and-then-accessing-it/12766/3 "2018-07-31T13:58:48Z")

</div>

If I’m just prototyping a quick script, I `push!` to `foo_a`, `foo_b`, and `foo_c` independently. That way I have an explicit call to `foo_a = Float64[]` and my arrays aren’t filled with `Any`.

The other way I solve this is with a `DataFrame`. Presumably each iteration `i` is an “experiment”, which you record the results of in the columns. Your code becomes:

```julia
foo = DataFrame(a = [], b = [], c = [])
for i=1:infinity
    a = f1(i) # This happens to be a Float64
    b = f2(i) # This happens to be a vector [x, y, z] of Float64's
    c = f3(i) # This happens to be an Int32
    push!(foo, [a, b, c])
end

```

And you can access them via `foo[:a]`.