# Dates.format broadcasting is slow

**URL:** https://discourse.julialang.org/t/dates-format-broadcasting-is-slow/55778
**Category:** General Usage
**Created:** [February 22, 2021, 2:03pm UTC](https://discourse.julialang.org/t/dates-format-broadcasting-is-slow/55778 "2021-02-22T14:03:49Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![bernhard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bernhard/32/2619_2.png) [@bernhard](https://discourse.julialang.org/u/bernhard)
#### Post date: [February 22, 2021, 2:03pm UTC](https://discourse.julialang.org/t/dates-format-broadcasting-is-slow/55778/1 "2021-02-22T14:03:49Z")

</div>

In my use case below, `Dates.format` is rather slow when applied to a vector.  
I was wondering if there is a better way to broadcast this.  
I am well aware, that my custom function uses ‘additional knowledge’ (few unique values) and thus the comparison may not be entirely fair. Still I am surprised about the slow execution of the broadcasted version.

```julia
using Dates

export datesformatcustom 
function datesformatcustom(col,fmt)
    uqv = unique(col)
    uqvDate = Dates.format.(uqv,fmt)
    dateDict = Dict(uqv .=> uqvDate)
    
    res = map(x->dateDict[x],col)
    return res
end

sze = 10_000_000;
sze = 1_000_000;
dts = Date(2000,1,1) .+ Dates.Day.(trunc.(Int,1500 * rand(sze)));

@time Dates.format.(dts,"yyyymm"); #20 seconds for 1 mio rows / 260 seconds for 10 mio rows
@time datesformatcustom(dts,"yyyymm"); #0.3 seconds

```

---

<div class="post-metadata">

### Author: ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)
#### Post date: [February 22, 2021, 2:19pm UTC](https://discourse.julialang.org/t/dates-format-broadcasting-is-slow/55778/2 "2021-02-22T14:19:23Z")

</div>

Your implementation is very clever and will be efficient when the ratio of unique dates to total length is very small, which is the case in your example.

The `Dates` formatting does actually seem kind of slow, to be honest. I’m sure that could be optimized and sped up. But I wouldn’t expect broadcasting to do anything as clever as what `datesformatcustom` does.

---

<div class="post-metadata">

### Author: ![bernhard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bernhard/32/2619_2.png) [@bernhard](https://discourse.julialang.org/u/bernhard)
#### Post date: [February 22, 2021, 2:30pm UTC](https://discourse.julialang.org/t/dates-format-broadcasting-is-slow/55778/3 "2021-02-22T14:30:25Z")

</div>

Thanks.  
Well, I know nothing about date parsing or manipulation.  
But in the case above, the data is already parsed as a date (thus the information is “readily available in a practical format”). Even this line is considerably faster than the broadcasting

```julia
@time alternative = string.(year.(dts) * 100 .+ month.(dts)) #2.7 seconds about 6-7 times faster than broadcasting

```

EDIT: I am well aware that this will fail for years \< 1000 (and maybe years \> 9999 if that exists)

---

<div class="post-metadata">

### Author: ![bernhard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bernhard/32/2619_2.png) [@bernhard](https://discourse.julialang.org/u/bernhard)
#### Post date: [February 22, 2021, 2:31pm UTC](https://discourse.julialang.org/t/dates-format-broadcasting-is-slow/55778/4 "2021-02-22T14:31:31Z")

</div>

I guess Dates.format is slow, because it can handle a wild variety of formats, while the format I request here is actually something very simple.

---

<div class="post-metadata">

### Author: ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)
#### Post date: [February 22, 2021, 2:32pm UTC](https://discourse.julialang.org/t/dates-format-broadcasting-is-slow/55778/5 "2021-02-22T14:32:12Z")

</div>

Try

```julia
Dates.format.(dts, dateformat"yyyymm");

```

instead. See the docs:

> Creating a DateFormat object is expensive. Whenever  
> possible, create it once and use it many times or try the  
> dateformat"" string macro. Using this macro creates the  
> DateFormat object once at macro expansion time and reuses it  
> later. see @dateformat\_str.

---

<div class="post-metadata">

### Author: ![bernhard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bernhard/32/2619_2.png) [@bernhard](https://discourse.julialang.org/u/bernhard)
#### Post date: [February 22, 2021, 2:37pm UTC](https://discourse.julialang.org/t/dates-format-broadcasting-is-slow/55778/6 "2021-02-22T14:37:43Z")

</div>

Thank you kristoffer.  
I actually briefly thought about that, but did not try it out.  
Indeed this results in 4seconds for 1 million entries, which is fast enough.
