# Leading Zeros got truncated from df column

**URL:** <https://discourse.julialang.org/t/leading-zeros-got-truncated-from-df-column/97409>\
**Category:** Performance\
**Tags:** strings, dataframes\
**Created:** [April 12, 2023, 10:57pm UTC](https://discourse.julialang.org/t/leading-zeros-got-truncated-from-df-column/97409 "2023-04-12T22:57:19Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sandy45](https://avatars.discourse-cdn.com/v4/letter/s/3ec8ea/32.png) [@Sandy45](https://discourse.julialang.org/u/Sandy45)\
**Post date:** [April 12, 2023, 10:57pm UTC](https://discourse.julialang.org/t/leading-zeros-got-truncated-from-df-column/97409/1 "2023-04-12T22:57:19Z")

</div>

I have dataframe with integer field and it’s usually consists of time for ex. 0001, 1020, 2359 etc.

When there are leading zeros in the column they are being truncated and I’ve tried applying lpad which seems to be doesn’t work on dataframes?

```julia
Row │ a     
     │ Int64 
─────┼───────
   1 │ 1
   2 │ 21
   3 │ 333

df.a .= lpad(string(df.a),4,"0")
3-element Vector{String}:
 "[1, 21, 333]"
 "[1, 21, 333]"
 "[1, 21, 333]"

The expected results are like below:
"0001"
"0021"
"0333"

```

---

<div class="post-metadata">

**Author:** ![rafael.guerra](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rafael.guerra/32/216610_2.png) [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)\
**Post date:** [April 12, 2023, 11:09pm UTC](https://discourse.julialang.org/t/leading-zeros-got-truncated-from-df-column/97409/2 "2023-04-12T23:09:02Z")

</div>

Try broadcasting with dot syntax:

```julia
df.a .= lpad.(string.(df.a), 4, '0')

```

or with macro:

```julia
@. df.a = lpad(string(df.a), 4, '0')

```

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [April 12, 2023, 11:09pm UTC](https://discourse.julialang.org/t/leading-zeros-got-truncated-from-df-column/97409/3 "2023-04-12T23:09:36Z")

</div>

or  
`transform!(df, :a => ByRow(x -> lpad(string(x), 4, "0")) => :a)`

or with `DataFramesMeta.jl`

```julia
@rtransform!(df, :a = lpad(string(:a), 4, "0"))

```

---

<div class="post-metadata">

**Author:** ![Sandy45](https://avatars.discourse-cdn.com/v4/letter/s/3ec8ea/32.png) [@Sandy45](https://discourse.julialang.org/u/Sandy45)\
**Post date:** [April 12, 2023, 11:20pm UTC](https://discourse.julialang.org/t/leading-zeros-got-truncated-from-df-column/97409/4 "2023-04-12T23:20:58Z")

</div>

@rafael.guerra Thank you for the reply, It seems like I am using double quotes instead of single quote.

---

<div class="post-metadata">

**Author:** ![Sandy45](https://avatars.discourse-cdn.com/v4/letter/s/3ec8ea/32.png) [@Sandy45](https://discourse.julialang.org/u/Sandy45)\
**Post date:** [April 12, 2023, 11:22pm UTC](https://discourse.julialang.org/t/leading-zeros-got-truncated-from-df-column/97409/5 "2023-04-12T23:22:06Z")

</div>

@adienes Thank you, this solution also works but seems like broadcasting appears to be fastest one considering the time taken.

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [April 13, 2023, 5:49pm UTC](https://discourse.julialang.org/t/leading-zeros-got-truncated-from-df-column/97409/6 "2023-04-13T17:49:04Z")

</div>

if running time is important, consider using @rafael.guerra 's proposal in the following form

```julia
julia> using DataFrames, BenchmarkTools

julia> @btime begin
           lp=maximum(length ∘ string, df.a)+1
           [lpad(r,$lp,'0') for r in $df.a]
           end # 800ns
  302.381 ns (3 allocations: 128 bytes)
3-element Vector{String}:
 "0001"
 "0021"
 "0333"
julia> @btime [lpad(r,4,'0') for r in df.a]
  184.818 ns (2 allocations: 96 bytes)
3-element Vector{String}:
 "0001"
 "0021"
 "0333"

julia> @btime df.a .= lpad.(string.(df.a), 4, '0')
  825.581 ns (5 allocations: 304 bytes)
3-element Vector{String}:
 "0001"
 "0021"
 "0333"

```

---

<div class="post-metadata">

**Author:** ![Nathan\_Boyer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nathan_boyer/32/14825_2.png) [@Nathan\_Boyer](https://discourse.julialang.org/u/Nathan_Boyer)\
**Post date:** [April 13, 2023, 7:18pm UTC](https://discourse.julialang.org/t/leading-zeros-got-truncated-from-df-column/97409/7 "2023-04-13T19:18:10Z")

</div>

I came here to recommend you only convert your numbers to string when you go to print results using [Printf](https://docs.julialang.org/en/v1/stdlib/Printf/) or [Formatting](https://github.com/JuliaIO/Formatting.jl) (not when you load your data), so you can analyze your data using integers. However, the strings actually worked better than I expected with transformation functions.

```julia
julia> df
4×2 DataFrame
 Row │ a b
     │ Int64 String
─────┼───────────────
   1 │ 1 0001
   2 │ 10 0010
   3 │ 2 0002
   4 │ 20 0020

julia> sort(df.a)
4-element Vector{Int64}:
  1
  2
 10
 20

julia> sort(df.b)
4-element Vector{String}:
 "0001"
 "0002"
 "0010"
 "0020"

julia> subset(df, :a => ByRow(<(10)))
2×2 DataFrame
 Row │ a b
     │ Int64 String
─────┼───────────────
   1 │ 1 0001
   2 │ 2 0002

julia> subset(df, :b => ByRow(<("0010")))
2×2 DataFrame
 Row │ a b
     │ Int64 String
─────┼───────────────
   1 │ 1 0001
   2 │ 2 0002

```

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [April 13, 2023, 9:03pm UTC](https://discourse.julialang.org/t/leading-zeros-got-truncated-from-df-column/97409/8 "2023-04-13T21:03:59Z")

</div>

I think the double quotes were not the issue, rather the missing period after your lpad. [But with single quotes is likely slightly faster. I would actually like to get rid of the distinction, and eliminate Char from Julia, same as in Swift, I have an idea, that I haven’t implemented yet, for improving String handling that would do that and more.]

> [@Nathan\_Boyer](#):
>
> I came here to recommend you only convert your numbers to string when you go to print results

That may be good advise. Possibly DataFrames should have an (optional) way to format numbers, e.g. integers (like in COBOL, its PICTURE clause)?

What’s the default when you import 0 prefixed numbers? Since it may be meaningful (e.g. Excel drops them), they should be imported as strings. [Something like `lpad(100, 2, '0')` wouldn’t sort correctly, when the padding isn’t sufficient. Natural sorting for strings would fix that, what I want as defaults for me new String data type. There’s already a package for that, just opt-in; also just using integers fixes that problem.]