# Query @mutate changes other column type

**URL:** <https://discourse.julialang.org/t/query-mutate-changes-other-column-type/39292>\
**Category:** New to Julia\
**Tags:** query, dataframes, csv\
**Created:** [May 11, 2020, 3:48pm UTC](https://discourse.julialang.org/t/query-mutate-changes-other-column-type/39292 "2020-05-11T15:48:03Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![stej](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@stej](https://discourse.julialang.org/u/stej)\
**Post date:** [May 11, 2020, 3:48pm UTC](https://discourse.julialang.org/t/query-mutate-changes-other-column-type/39292/1 "2020-05-11T15:48:03Z")

</div>

Hi all, this is my smallest code that I fight with:

```julia
contents = """
"5674012","aa66aa66"
"5674012","9b4e08e5"
"5674012","b036aa66,b036aa67,b036aa68"
""";

batches = CSV.File(IOBuffer(contents); header = ["X1", "Splits"], delim = ',') |> DataFrame;

emptyStringArray = Array{SubString{String},1}()
batches5 = batches |> 
            @mutate(Splits = length(_.Splits) > 0 ? split(_.Splits, ',') : emptyStringArray) |> DataFrame
batches6 = batches |> 
            @mutate(Splits = length(_.Splits) > 0 ? split(_.Splits, ',') : Array{SubString{String},1}()) |> DataFrame

```

Note the assignments to `batches5` and `batches6`. The result looks like this:

```julia
julia> batches5
3×2 DataFrame
│ Row │ X1 │ Splits │
│ │ Any │ Any │
├─────┼─────────┼──────────────────────────────────────┤
│ 1 │ 5674012 │ ["aa66aa66"] │
│ 2 │ 5674012 │ ["9b4e08e5"] │
│ 3 │ 5674012 │ ["b036aa66", "b036aa67", "b036aa68"] │

julia> batches6
3×2 DataFrame
│ Row │ X1 │ Splits │
│ │ Int64 │ Array{SubString{String},1} │
├─────┼─────────┼──────────────────────────────────────┤
│ 1 │ 5674012 │ ["aa66aa66"] │
│ 2 │ 5674012 │ ["9b4e08e5"] │
│ 3 │ 5674012 │ ["b036aa66", "b036aa67", "b036aa68"] │

```

What I don’t understand:

1. Splits column type differs different - `Any` vs. `Array{SubString{String},1}`. Why? I just wanted to save memory so I stored the value (that can be repeated) to `emptyStringArray`.

2. Even if I understand that I made something bad to column `:Splits`, I don’t understand why `X1`’s type is changed to `Any` in `batches6`. I would thought that the columns are independent. Something wrong with `@mutate`?

Could somebody clarify, what’s going on here?

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [May 11, 2020, 5:29pm UTC](https://discourse.julialang.org/t/query-mutate-changes-other-column-type/39292/2 "2020-05-11T17:29:51Z")

</div>

I don’t know what `Emptystringarray` is. The reason for the `Any` is likely that there is no method `promote_type(x::Vector{<AbstractString}, y::Emptystringarray)` so julia defaults to promoting the vector to type `Any`.

---

<div class="post-metadata">

**Author:** ![stej](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@stej](https://discourse.julialang.org/u/stej)\
**Post date:** [May 11, 2020, 5:45pm UTC](https://discourse.julialang.org/t/query-mutate-changes-other-column-type/39292/3 "2020-05-11T17:45:53Z")

</div>

the `emptyStringArray` is defined in my sample code.

> [@stej](#):
>
> emptyStringArray = Array{SubString{String},1}()

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [May 11, 2020, 5:50pm UTC](https://discourse.julialang.org/t/query-mutate-changes-other-column-type/39292/4 "2020-05-11T17:50:00Z")

</div>

sorry, I didn’t see that. I’m not sure the behavior, then. This is odd.

---

<div class="post-metadata">

**Author:** ![stej](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@stej](https://discourse.julialang.org/u/stej)\
**Post date:** [May 12, 2020, 11:27am UTC](https://discourse.julialang.org/t/query-mutate-changes-other-column-type/39292/5 "2020-05-12T11:27:32Z")

</div>

Filled a bug, so let’s see… [@mutate changes other column type · Issue #305 · queryverse/Query.jl · GitHub](https://github.com/queryverse/Query.jl/issues/305)

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [May 13, 2020, 10:17am UTC](https://discourse.julialang.org/t/query-mutate-changes-other-column-type/39292/6 "2020-05-13T10:17:11Z")

</div>

The problem is that type inference sometimes breaks down if you reference a global variable in a closure, and Query.jl depends on type inference not breaking down right now 🙂

Two ways to fix this at the moment:

1. You can declare `emptyStringArray ` to be `const`, i.e. `const emptyStringArray = ...`
2. You can put `emptyStringArray` and the query into a function

The proper solution to this is to drop the dependency on type inference in Query.jl. It has been on my todo list for about 2-3 years now 🙂 There is no fundamental reason that this could not be done, but it is a bit of a pain to implement. At some point I’ll push myself to do it, but no promises.

---

<div class="post-metadata">

**Author:** ![stej](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@stej](https://discourse.julialang.org/u/stej)\
**Post date:** [May 13, 2020, 5:32pm UTC](https://discourse.julialang.org/t/query-mutate-changes-other-column-type/39292/7 "2020-05-13T17:32:37Z")

</div>

@davidanthoff I’m really newbie in this, so I’m only trying to guess what you wanted to say 😉

Anyway, here is another example, this time without global variable:

```julia
julia> batches
3×2 DataFrame
│ Row │ X1 │ Splits │
│ │ Int64 │ String │
├─────┼─────────┼────────────────────────────┤
│ 1 │ 5674012 │ aa66aa66 │
│ 2 │ 5674012 │ 9b4e08e5 │
│ 3 │ 5674012 │ b036aa66,b036aa67,b036aa68 │

julia> batches5b = batches |>
                   @mutate(Splits = length(_.Splits) > 0 ? split(_.Splits, ',') : 1.) |> DataFrame
3×2 DataFrame
│ Row │ X1 │ Splits │
│ │ Any │ Any │
├─────┼─────────┼──────────────────────────────────────┤
│ 1 │ 5674012 │ ["aa66aa66"] │
│ 2 │ 5674012 │ ["9b4e08e5"] │
│ 3 │ 5674012 │ ["b036aa66", "b036aa67", "b036aa68"] │

```

See the column types change (`batches` vs. `batches5b`). Is your comment still relevant, or is that something new?

---

<div class="post-metadata">

**Author:** ![davidanthoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davidanthoff/32/223493_2.png) [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Post date:** [May 13, 2020, 5:36pm UTC](https://discourse.julialang.org/t/query-mutate-changes-other-column-type/39292/8 "2020-05-13T17:36:21Z")

</div>

So the reason the `Splits` column here turns into `Any` is that the expression `length(_.Splits) > 0 ? split(_.Splits, ',') : 1.` can return either an array of `SubString`, or a `Float64` value. So we need to make the column type one that can handle both of these cases, which is `Any`. We call this type of situation a type instability, because that expression returns a value of a different type depending on the values of the inputs.

That the `X1` column then also turns into `Any` is at the end of the day a bug in Query.jl that is just cumbersome to fix for me 🙂 But, that is no good excuse, of course. That bug is triggered by the type instability in the other column.

---

<div class="post-metadata">

**Author:** ![stej](https://avatars.discourse-cdn.com/v4/letter/s/e495f1/32.png) [@stej](https://discourse.julialang.org/u/stej)\
**Post date:** [May 13, 2020, 5:39pm UTC](https://discourse.julialang.org/t/query-mutate-changes-other-column-type/39292/9 "2020-05-13T17:39:50Z")

</div>

Thank you. Well explained, I kind of expected something like this. Anyway, good to know 🙂 Really appreciate you clarified it to me.