Verify if a daily file has been created

[phantom]

Suppose you have a series of data files that are added daily and named in the following format.

folder/name-01-01-23.csv 
folder/name -01-02-23.csv 
.
.
.
folder/name-03-10-23.csv

What is the best way to verify if the daily file was created? I wasn’t sure how to get the DateTime information interpolated into the file query given format of the filename so my current approach is something like

julia> cdate = Date(Dates.now())
2023-03-11

julia> dt = split(string(cdate),"-") 
3-element Vector{SubString{String}}:
 "2023"
 "03"
 "11"

isfile("folder/name-$(dt[2])-$(dt[3])-$(dt[1][3:4]).csv")

This works but I was just wondering if this was the best approach? The more I browse the posts on this site the more I realize there are generally much smarter ways of doing things.

[Dan]

Perhaps, using Dates.format:

julia> Dates.format(today(),"mm-dd-yy")
"03-11-23"

julia> isfile("folder/name-$(Dates.format(today(),"mm-dd-yy")).csv")
false

More details in the docs: https://docs.julialang.org/en/v1/stdlib/Dates/#Dates.format-Tuple{TimeType,%20AbstractString}

[Henrique_Becker]

Just an orthogonal comment, I really suggest using the default of yyyy-mm-dd as in:

julia> using Dates

julia> string(Dates.today())
"2023-03-11"

julia> Dates.Date("2023-03-11")
2023-03-11

or explicitly:

julia> using Dates

julia> Dates.format(Dates.today(), "yyyy-mm-dd")
"2023-03-11"

julia> Dates.Date("2023-03-11", dateformat"yyyy-mm-dd")
2023-03-11

Because sorting such string alphanumerically is the same as sorting by date and there is no ambiguity about the month and day components (which is which). Any other format is sowing the seeds of chaos.

[phantom]

Thanks so much guys! This is why posting on this forum is incredibly helpful.

@Dan really appreciate your example. I checked the docs first but got confused and was using DateFormat which kept returning an error. Your example really clarifies that DateFormat converts type string to type Date and Dates.format() does the reverse by converting type Date to type string. It makes a lot more sense now thanks!

@Henrique_Becker Thanks so much for pointing that out. It’s another thing I would not have thought of. There’s already an existing cache of files named in this manner but I will keep this in mind going forward.

[gustaphe]

Before you change over to ISO8601 (a suggestion I wholeheartedly agree with), change all the old files doing something like

inreg = r"(\d{2})-(\d{2})-(\d{2})" 
outreg = s"20\3-\1-\2"
for f in readdir("folder"; join=true)
    isnothing(match(inreg, f)) && continue
    mv(f, replace(f, inreg=>outreg))
end

(Backup the directory first)

[phantom]

wow thanks this is brilliant! It would’ve taken me like seven hours to figure this out. I hope you don’t mind if I make some notes here just for future reference and anyone else who may be new to this. But if it is distractingly elementary for, or not useful to the discourse I’ll delete it.

@gustaphe

inreg = r"(\d{2})-(\d{2})-(\d{2})" # r" " creates a regular expression(regex) used to search strings for the characters inside the " ".  This follows a [pcre2syntax](https://www.pcre.org/current/doc/html/pcre2syntax.html) where \d stands for decimal digits and the {2} is a quantifier.  The ( ) creates a captured group.  Which will be used in the next line.   
outreg = s"20\3-\1-\2"             # s" " creates substitution string where \n refers to the nth captured group created by the ( )
for f in readdir("folder"; join=true) # returns the names of files in "folder",  join = true returns the full file path. 
    isnothing(match(inreg, f)) && continue  # if match() returns a nothing value, skip to the next iteration
    mv(f, replace(f, inreg=>outreg))        # mv() moves file f to the new file path where inreg is replace by outreg
end

[rafael.guerra]

FWIW, here is a regex-free alternative for renaming the files:

using Dates

for file in readdir("folder"; join=true)
    name = first(splitext(basename(file)))
    s1 = last(split(name, "name-"))
    filedate = tryparse(Date, s1, dateformat"mm-dd-yy")
    isnothing(filedate) && continue
    mv(file, replace(file, s1 => string(filedate  + Year(2000))))
end