# How to create columns in a loop

**URL:** <https://discourse.julialang.org/t/how-to-create-columns-in-a-loop/85519>\
**Category:** General Usage\
**Tags:** question, inmemorydatasets\
**Created:** [August 9, 2022, 5:59am UTC](https://discourse.julialang.org/t/how-to-create-columns-in-a-loop/85519 "2022-08-09T05:59:41Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [August 9, 2022, 5:59am UTC](https://discourse.julialang.org/t/how-to-create-columns-in-a-loop/85519/1 "2022-08-09T05:59:42Z")

</div>

I have the following code:

```julia
using InMemoryDatasets
# create a demo data set of CAN bus messages

function demo_data()
    n = 100000
    time = 0.1:0.1:n*0.1
    addr = rand(Int16(0x101):Int16(0x12e), n)
    d1 = rand(UInt8(0):UInt8(255), n)
    d2 = rand(UInt8(0):UInt8(255), n)
    d3 = rand(UInt8(0):UInt8(255), n)
    d4 = rand(UInt8(0):UInt8(255), n)
    d5 = rand(UInt8(0):UInt8(255), n)
    d6 = rand(UInt8(0):UInt8(255), n)
    d7 = rand(UInt8(0):UInt8(255), n)
    d8 = rand(UInt8(0):UInt8(255), n)
    ds = Dataset(time=time, addr=addr, d1=d1, d2=d2, d3=d3, d4=d4, d5=d5, d6=d6, d7=d7, d8=d8)
    ds.addr[5] = missing
    ds
end

```

In this example I am creating 8 similar columns. How can I do that in a loop for m columns?

---

<div class="post-metadata">

**Author:** ![SteffenPL](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/steffenpl/32/206270_2.png) [@SteffenPL](https://discourse.julialang.org/u/SteffenPL)\
**Post date:** [August 9, 2022, 6:21am UTC](https://discourse.julialang.org/t/how-to-create-columns-in-a-loop/85519/2 "2022-08-09T06:21:59Z")

</div>

Maybe you could first create a `Dict` and then use the `Dataset(::Dict)` constructor, e.g.

```julia
d = Dict( "time" => 0.1:0.1:n*0.1, 
          "addr" => rand(Int16(0x101):Int16(0x12e), n))

for i in 1:42
    d["d" * string(i, pad=3)] = rand(UInt8(0):UInt8(255), n)
end

ds = Dataset(d)

```

(The leading zeros here are used to ensure that the columns are sorted by `i`.)

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [August 9, 2022, 6:45am UTC](https://discourse.julialang.org/t/how-to-create-columns-in-a-loop/85519/3 "2022-08-09T06:45:17Z")

</div>

I don’t know anything about `DataSet`, but, in case you don’t know, you can write

```julia
rand(UInt8, n)

```

instead of

```julia
rand(UInt8(0):UInt8(255), n)

```

It’s much faster, too:

```julia
julia> @btime rand(UInt8(0):UInt8(255), 1000);
  5.533 μs (1 allocation: 1.06 KiB)

julia> @btime rand(UInt8, 1000);
  171.714 ns (1 allocation: 1.06 KiB)

```

Oh, the difference gets even bigger for your array size:

```julia
julia> @btime rand(UInt8(0):UInt8(255), 100_000);
  549.300 μs (2 allocations: 97.73 KiB

julia> @btime rand(UInt8, 100_000);
  5.380 μs (2 allocations: 97.73 KiB)

```

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [August 9, 2022, 6:52am UTC](https://discourse.julialang.org/t/how-to-create-columns-in-a-loop/85519/4 "2022-08-09T06:52:17Z")

</div>

se non hai stretta necessità del loop, puoi usare uno schema di questo tipo:

```julia-auto
julia> A=rand(UInt8, 6,5)
6×5 Matrix{UInt8}:
 0x44 0x47 0x5d 0xa1 0xa4
 0x5d 0xe4 0x22 0x27 0xc3
 0x25 0x4a 0x03 0x64 0xf7
 0x5b 0xd3 0x81 0x04 0x12
 0xce 0x4e 0xb9 0x2c 0xf6
 0x27 0xbe 0x6e 0x33 0x86

julia> ds=Dataset(A, :auto)
6×5 Dataset
 Row │ x1 x2 x3 x4 x5       
     │ identity identity identity identity identity
     │ UInt8? UInt8? UInt8? UInt8? UInt8?
─────┼──────────────────────────────────────────────────
   1 │ 68 71 93 161 164
   2 │ 93 228 34 39 195
   3 │ 37 74 3 100 247
   4 │ 91 211 129 4 18
   5 │ 206 78 185 44 246
   6 │ 39 190 110 51 134

julia> rename(x-> replace(x, "x"=>"d"), ds)
6×5 Dataset
 Row │ d1 d2 d3 d4 d5       
     │ identity identity identity identity identity
     │ UInt8? UInt8? UInt8? UInt8? UInt8?
─────┼──────────────────────────────────────────────────
   1 │ 68 71 93 161 164
   2 │ 93 228 34 39 195
   3 │ 37 74 3 100 247
   4 │ 91 211 129 4 18
   5 │ 206 78 185 44 246
   6 │ 39 190 110 51 134

```

# or this scheme

```julia-auto
m,n=6, 7
A=rand(UInt8, m,n)
ds=Dataset(A, string.("d", 1:n))

```

PS  
Why aren’t UInt8 integers shown as such?

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [August 9, 2022, 7:35am UTC](https://discourse.julialang.org/t/how-to-create-columns-in-a-loop/85519/5 "2022-08-09T07:35:05Z")

</div>

I also don’t know anything about `Dataset` but this should do the same thing as your code:

```julia
function demo_data()
    n = 100000
    m = 8
    time = 0.1:0.1:n*0.1
    addr = rand(Int16(0x101):Int16(0x12e), n)
    columns = (Symbol("d", i) => rand(UInt8, n) for i in 1:m)
    ds = Dataset(;time, addr, columns...)
    ds.addr[5] = missing
    ds
end

```

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [August 9, 2022, 7:54am UTC](https://discourse.julialang.org/t/how-to-create-columns-in-a-loop/85519/6 "2022-08-09T07:54:55Z")

</div>

Very nice! Seams to work! 🙂

But what is a Generator?

```julia
julia> columns = (Symbol("d", i) => rand(UInt8, n) for i in 1:m)
Base.Generator{UnitRange{Int64}, var"#5#6"}(var"#5#6"(), 1:8)

```

And why is it needed to write columns… ?

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [August 9, 2022, 8:02am UTC](https://discourse.julialang.org/t/how-to-create-columns-in-a-loop/85519/7 "2022-08-09T08:02:58Z")

</div>

It’s not needed, if you prefer you can create an array with a comprehension instead:

```julia
columns = [Symbol("d", i) => rand(UInt8, n) for i in 1:m]

```

You can think of the generator as a “lazy” array which creates its elements on demand.

Documentation: [Multi-dimensional Arrays · The Julia Language](https://docs.julialang.org/en/v1/manual/arrays/#Generator-Expressions)