# Copy vs view of DataFrame column?

**URL:** <https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182>\
**Category:** General Usage\
**Tags:** dataframes\
**Created:** [January 18, 2023, 1:47am UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182 "2023-01-18T01:47:45Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![martinmestre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/martinmestre/32/215857_2.png) [@martinmestre](https://discourse.julialang.org/u/martinmestre)\
**Post date:** [January 18, 2023, 1:47am UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/1 "2023-01-18T01:47:45Z")

</div>

Hello, do you know an example in which a copy,` df[:,:a`], should be used instead of a view with `df[!,:a]`?  
Thanks !

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [January 18, 2023, 9:16pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/2 "2023-01-18T21:16:24Z")

</div>

Please don’t revive a three-year old thread like this. Make a new post instead.

That said, if you have a function that modifies a vector in-place, you will want to do `df[:, :a]` instead of `df[!, :a]`.

```julia
function make_ten!(x)
    x[1] = 10
end

make_ten!(df[!, :a]) # modifies df
make_ten!(df[:, :a]) # does not modify df

```

---

<div class="post-metadata">

**Author:** ![cchderrick](https://avatars.discourse-cdn.com/v4/letter/c/ecd19e/32.png) [@cchderrick](https://discourse.julialang.org/u/cchderrick)\
**Post date:** [January 18, 2023, 9:46pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/3 "2023-01-18T21:46:06Z")

</div>

your statement and comment in example contradicts?

Statement should read

> if you have a function that modifies a vector in-place, you will want to do `df[!, :a]` instead of `df[:, :a]`.

```julia
using DataFrames
df = DataFrame(rand(10,3),["a","b","c"])
function make_ten!(x)
    x[1] = 10
end
make_ten!(df[:, :a])

```

Using colon, `df` is not updated:

```julia
julia> df[!,:a]
10-element Vector{Float64}:
 0.6133749814711238
 0.9022335210145525
 0.5930273630651568
 0.25727987475397907
 0.5368154177958848
 0.9789575335373208
 0.07748891516310152
 0.8386526410191439
 0.9929637176775048
 0.5485057586874986

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [January 19, 2023, 1:18am UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/4 "2023-01-19T01:18:59Z")

</div>

This is assuming you _don’t_ want to modify `df`. If you want the changes the function makes to persist inside the data frame, then yes, use `!`.

---

<div class="post-metadata">

**Author:** ![martinmestre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/martinmestre/32/215857_2.png) [@martinmestre](https://discourse.julialang.org/u/martinmestre)\
**Post date:** [January 19, 2023, 4:12pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/5 "2023-01-19T16:12:52Z")

</div>

Thanks @pdeffebach and @cchderrick . Sorry I revived a thread. Is it a problem because it was old? I thought that it was a very related question. Now I will always create new threads.  
Regarding the question, thanks for the answers. Would it be good that the compiler gives me a warning or error if trying to do the following:

```julia
make_ten!(df[:, :a])

```

?  
Because what I should use is either

```julia
make_ten(df[:, :a])

```

or

```julia
make_ten!(df[!, :a])

```

I mean so as to avoid mistakes.  
Do you agree?

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [January 19, 2023, 4:15pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/6 "2023-01-19T16:15:17Z")

</div>

Not sure what you mean by “mistake”.

There’s nothing wrong with any of the three operations you give.

```julia
make_ten!(df[:, :a]) # Saves memory, doesn't modify data frame
make_ten(df[:, :a]) # Ultra-safe, doesn't modify data frame
make_ten!(df[!, :a]) # Saves memory, modifies data frame

```

There’s no “right answer”, its depends on what you want to do. And no, Julia’s compiler generally does not do this sort of thing.

---

<div class="post-metadata">

**Author:** ![martinmestre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/martinmestre/32/215857_2.png) [@martinmestre](https://discourse.julialang.org/u/martinmestre)\
**Post date:** [January 19, 2023, 4:22pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/7 "2023-01-19T16:22:04Z")

</div>

Thanks. By mistake I mean that I use

```julia
make_ten!(df[:, :a])

```

thinking that it will mutate `df` because the` !` in the function name.

Now, with your explanation, I see that there is a memory benefit in using that particular operation.  
Thanks!

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [January 19, 2023, 4:25pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/8 "2023-01-19T16:25:39Z")

</div>

Once you do `df[:, :a]`, that object (a vector), knows _nothing_ about `df`. It has no connection at all with the data frame.

The same goes for `df[!, :a]` in the sense that it’s behavior does not depend on being from a data frame. However `df` still shares the memory with `df[!, :a]`.

---

<div class="post-metadata">

**Author:** ![martinmestre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/martinmestre/32/215857_2.png) [@martinmestre](https://discourse.julialang.org/u/martinmestre)\
**Post date:** [January 19, 2023, 4:29pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/9 "2023-01-19T16:29:49Z")

</div>

So if within the same scope I do:

```julia
df = DataFrame(x=[1,2,3], y=[4,5,6])
df[:,:x]=[7,8,9]

```

is like doing nothing, right? I mean because the new information of `df[:,:x]` is lost automatically.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [January 19, 2023, 4:37pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/10 "2023-01-19T16:37:58Z")

</div>

No, that’s assigning (`setindex`) not retrieving (`getindex`).

```julia
df[:, :x] = [7,8,9]

```

does modify the data frame, since you are using `setindex!`, i.e. assigning the column.

```julia
df[:, :x] = ...

```

and `df[:, :x]` on it’s own do different things. But the intuition is the same with `!` and `:` when you are doing `setindex`

```julia
julia> df = DataFrame(a = [1, 2, 3]);

julia> x = [5, 6, 7];

julia> y = [8,9, 10];

julia> df[:, :x] = x;

julia> df[!, :y] = y;

julia> x[1] = 100;

julia> df
3×3 DataFrame
 Row │ a x y     
     │ Int64 Int64 Int64 
─────┼─────────────────────
   1 │ 1 5 8
   2 │ 2 6 9
   3 │ 3 7 10

julia> y[1] = 100;

julia> df
3×3 DataFrame
 Row │ a x y     
     │ Int64 Int64 Int64 
─────┼─────────────────────
   1 │ 1 5 100
   2 │ 2 6 9
   3 │ 3 7 10

```

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [January 19, 2023, 4:57pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/11 "2023-01-19T16:57:29Z")

</div>

My recommendation is the following (of course it is only a recommendation):

- by default always use `df[:, :col]` as it is safer
- use `df[!, :col]` if
  1. speed or memory consumption is important for you (and you are sure that if you modify the data you extracted you will not mess up the source data)
  2. or you want to modify the contents of the column in-place

This applies to getting the column. For setting a column the difference is that `df[!, :col] = ...` replaces the column, while `df[:, :col] = ...` updates it in-place. The difference mostly matters when you want to assign values of other type than originally stored in the given column.

---

<div class="post-metadata">

**Author:** ![martinmestre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/martinmestre/32/215857_2.png) [@martinmestre](https://discourse.julialang.org/u/martinmestre)\
**Post date:** [January 19, 2023, 5:13pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/12 "2023-01-19T17:13:16Z")

</div>

Thanks. What is the meaning of “updates in place” ?  
If I do `df[!, :col] = y`  
and then I modify `y,` then `df` will be modified also?

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [January 19, 2023, 5:20pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/13 "2023-01-19T17:20:10Z")

</div>

> [@martinmestre](#):
>
> What is the meaning of “updates in place” ?

This is a bit nuanced, but yes `df[:, :x] = ...` will modify the vector stored in column `:x` directly. The reasoning for this behavior is a complicated, but derives from the fact that this is how base julia matrices behave.

```julia
julia> df = DataFrame(x=[1,2,3], y=[4,5,6])
3×2 DataFrame
 Row │ x y     
     │ Int64 Int64 
─────┼──────────────
   1 │ 1 4
   2 │ 2 5
   3 │ 3 6

julia> t = df[!, :x];

julia> df[:, :x] = [40, 50, 60];

julia> t
3-element Vector{Int64}:
 40
 50
 60

```

---

<div class="post-metadata">

**Author:** ![martinmestre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/martinmestre/32/215857_2.png) [@martinmestre](https://discourse.julialang.org/u/martinmestre)\
**Post date:** [January 19, 2023, 5:43pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/14 "2023-01-19T17:43:17Z")

</div>

Thanks, I will have to practice all of this in the REPL because its all new for me.

---

<div class="post-metadata">

**Author:** ![martinmestre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/martinmestre/32/215857_2.png) [@martinmestre](https://discourse.julialang.org/u/martinmestre)\
**Post date:** [January 19, 2023, 7:57pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/15 "2023-01-19T19:57:48Z")

</div>

Thanks, I think I am starting to understand but have some problem with the behavior in the assignments:  
`df[,:x] = ...`  
Could you give an example of this affirmation: “The difference mostly matters when you want to assign values of other type than originally stored in the given column.” ?  
Best.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [January 19, 2023, 8:02pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/16 "2023-01-19T20:02:51Z")

</div>

It’s very niche, but the most dramatic example is characters and integers. `df[:, :b] = ...` will auto-promote to preserve type (like julia arrays), while `df[!, :b] = ...` will preserve the type of the new addition. See:

```julia
julia> using DataFrames

julia> df = DataFrame(a = [1, 2, 3], b = [4, 5, 6]);

julia> x = ['p', 'u', 'z'];

julia> df[:, :x] = x;

julia> df[!, :b] = x;

julia> df[:, :a] = x;

julia> df
3×3 DataFrame
 Row │ a b x    
     │ Int64 Char Char 
─────┼───────────────────
   1 │ 112 p p
   2 │ 117 u u
   3 │ 122 z z

```

---

<div class="post-metadata">

**Author:** ![martinmestre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/martinmestre/32/215857_2.png) [@martinmestre](https://discourse.julialang.org/u/martinmestre)\
**Post date:** [January 19, 2023, 8:07pm UTC](https://discourse.julialang.org/t/copy-vs-view-of-dataframe-column/93182/17 "2023-01-19T20:07:05Z")

</div>

Thanks, I see.