# Replacing missing values in DataFrames.jl with function requiring multiple inputs

**URL:** <https://discourse.julialang.org/t/replacing-missing-values-in-dataframes-jl-with-function-requiring-multiple-inputs/97323>\
**Category:** Data\
**Tags:** question, dataframes\
**Created:** [April 11, 2023, 12:40am UTC](https://discourse.julialang.org/t/replacing-missing-values-in-dataframes-jl-with-function-requiring-multiple-inputs/97323 "2023-04-11T00:40:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![dleather](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dleather/32/43762_2.png) [@dleather](https://discourse.julialang.org/u/dleather)\
**Post date:** [April 11, 2023, 12:40am UTC](https://discourse.julialang.org/t/replacing-missing-values-in-dataframes-jl-with-function-requiring-multiple-inputs/97323/1 "2023-04-11T00:40:48Z")

</div>

The code below should create a new column in a DataFrame,

```julia
new_MaxAllwFAR = [15.0, 15.0, 15.0, 15.0].

```

Instead it creates,

```julia
new_MaxAllwFAR = [15.0, 15.0, missing, missing].

```

Can you help me understand why? And how should I debug in this instance? Is there a more efficient way to code this in DataFrames.jl?

```julia
using DataFrames

dataF = DataFrame(MaxAllwFAR = [15., 15., missing, missing], 
                  ComArea = [1000., 1000., 1000., 1000.], 
                  ResArea = [0., 0., 0., 0.], 
                  ResidFAR = [missing, missing, 10., 10.], 
                  CommFAR = [missing, missing, 15., 15.])

function xform_maxfar(max_far, com_area, res_area, resid_far, com_far)
    if ismissing(max_far)
        if com_area > res_area
            return com_far
        else
            return resid_far
        end
    else
        return max_far
    end
end

transform!(dataF, 
    [:MaxAllwFAR, :ComArea, :ResArea, :ResidFAR, :CommFAR] => 
    xform_maxfar => :new_MaxAllwFAR)

println(dataF)

```

---

<div class="post-metadata">

**Author:** ![dleather](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dleather/32/43762_2.png) [@dleather](https://discourse.julialang.org/u/dleather)\
**Post date:** [April 11, 2023, 1:50am UTC](https://discourse.julialang.org/t/replacing-missing-values-in-dataframes-jl-with-function-requiring-multiple-inputs/97323/2 "2023-04-11T01:50:56Z")

</div>

The fix was to replace

> [@dleather](#):
>
> ```julia
> transform!(dataF, 
> [:MaxAllwFAR, :ComArea, :ResArea, :ResidFAR, :CommFAR] => 
> xform_maxfar => :new_MaxAllwFAR)
> 
> ```

with

```julia
transform!(dataF, 
    [:MaxAllwFAR, :ComArea, :ResArea, :ResidFAR, :CommFAR] => 
    ByRow(xform_maxfar) => :new_MaxAllwFAR)

```

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [April 13, 2023, 11:04am UTC](https://discourse.julialang.org/t/replacing-missing-values-in-dataframes-jl-with-function-requiring-multiple-inputs/97323/3 "2023-04-13T11:04:07Z")

</div>

The following ByCol version seems to be faster (5 up to 40 X), at least for the example case.

```julia
function newMax(max_far, com_area, res_area, resid_far, com_far)
        [ismissing(max_far[i]) ? 
        com_area[i] > res_area[i] ? 
        com_far[i] : 
        resid_far[i] : 
        max_far[i] 
        for i in eachindex(max_far)]
end

julia> @btime dataF.nM=newMax(eachcol(dataF[:,[:MaxAllwFAR, :ComArea, :ResArea, :ResidFAR, :CommFAR]])...)
  1.540 μs (32 allocations: 2.59 KiB)
4-element Vector{Float64}:
 15.0
 15.0
 15.0
 15.0

julia> @btime transform(dataF,
           [:MaxAllwFAR, :ComArea, :ResArea, :ResidFAR, :CommFAR] =>       
           newMax => :new_MaxAllwFAR);
  14.400 μs (175 allocations: 9.19 KiB)

julia> @btime transform(dataF,
                  [:MaxAllwFAR, :ComArea, :ResArea, :ResidFAR, :CommFAR] =>

                  ByRow(xform_maxfar) => :new_MaxAllwFAR);
  68.700 μs (490 allocations: 32.64 KiB)

```