# Help me make a regex, instead of this?

**URL:** <https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878>\
**Category:** General Usage\
**Tags:** strings, regex\
**Created:** [May 5, 2019, 5:03pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878 "2019-05-05T17:03:49Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [May 5, 2019, 5:03pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/1 "2019-05-05T17:03:49Z")

</div>

Hey guys, so imagine you have an array like this when reading your directory:

```julia
603-element Array{String,1}:
 "PartFluidOut_0000.vtk"
 "PartFluidOut_0001.vtk"
 "PartFluidOut_0002.vtk"
 "PartFluidOut_0003.vtk"
 "PartFluidOut_0004.vtk"
 "PartFluidOut_0005.vtk"
 "PartFluidOut_0006.vtk"
 "PartFluidOut_0007.vtk"
 "PartFluidOut_0008.vtk"
 "PartFluidOut_0009.vtk"
 ⋮
 "PartFluid_0293.vtk"
 "PartFluid_0294.vtk"
 "PartFluid_0295.vtk"
 "PartFluid_0296.vtk"
 "PartFluid_0297.vtk"
 "PartFluid_0298.vtk"
 "PartFluid_0299.vtk"
 "PartFluid_0300.vtk"
 "_ResumeFluidOut.csv"

```

Then what I want to do is to filter in a way that I end up with the following result:

```julia
"PartFluidOut"
"PartFluid"

```

The way to this goal as I want it is:

1. Only want to look at .vtk files, ie. everyother file type should be ignored
2. Only want letters, so no numbers or underscores etc. in output

I’ve done it using this snippet of code currently:

```julia
filenames = readdir()
filterFiles = split.(filenames,"_")

k = []
for i = 1:length(filterFiles)
    global k = push!(k,filterFiles[i][1])
end

deleteat!(k,length(k))

finalNames = unique(k)
println(finalNames)

```

This works, but is a bit hardcoded since I have to look for an underscore, and I would rather just remove all non-vtk files, then remove “.vtk” and then finally all numbers. I know that some might be able to do this efficiently with a regex function - as I am new to regex I would like to see such an example.

My question is not about optimizing my code, but improving the way it works/robustness, hopefully doing this with implementing some kind of regex.

Thanks for your time.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [May 5, 2019, 6:17pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/2 "2019-05-05T18:17:22Z")

</div>

Disclaimer: This is not well tested.

```julia
function getfilestrings(namelist, pattern::Regex=".*") # default is to accept all strings
    filenames = Set{String}() # this makes sure that the list is unique
    for filename in namelist
        m = match(pattern, filename)
        isnothing(m) && continue
        str = first(m.captures)
        !isempty(str) && push!(filenames, str)
    end
    return filenames
end

list = getfilestrings(readdir(), r"([a-zA-Z]+).*\.vtk$")

```

I am also not quite sure what you want to be able to match on. This should work for your example, but if you don’t accept filenames starting with underscore, for example, then you can modify the regex to add a `^` in the beginning, like this: `r"^([a-zA-Z]+).*\.vtk$"`

I’m using the regex to capture the first group of letters in the groups `a-z` and `A-Z`, so no non-ascii characters, and require the string to end with `.vtk`, that’s what the dollar sign is for.

---

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [May 5, 2019, 6:30pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/3 "2019-05-05T18:30:58Z")

</div>

Thanks! I will try to test this for a bit and ask regarding parts I don’t completely understand.

Kind regards

---

<div class="post-metadata">

**Author:** ![natemcintosh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/natemcintosh/32/9021_2.png) [@natemcintosh](https://discourse.julialang.org/u/natemcintosh)\
**Post date:** [May 5, 2019, 6:57pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/4 "2019-05-05T18:57:11Z")

</div>

Hi Ahmed!  
Another way to do it might be along the lines of something like this

```julia
function glob_thing(pattern, to_search::AbstractVector)::AbstractVector
    filter(x -> occursin(pattern, x), to_search)
end

function name_beginning(name::AbstractString)
    names_no_extension = splitext(name)[1]
    first_part = split(names_no_extension, "_")[1]
    return first_part
end

files = readdir("/path/to/your/data/")
vtk_files = glob_thing(".vtk", files)
name_beginnings = name_beginning.(vtk_files)
unique(name_beginnings)

```

The first function `glob_thing`, filters down the data from files to only those ending with “.vtk”. I named it `glob_thing` because I’m using it to do something similar to the glob function from terminals. I got this function from @tkoolen, who helped me out with one of my [past questions](https://discourse.julialang.org/t/help-reading-code-warntype-output/20832).

Now that we have only files that end in “.vtk”, let’s remove everything in the names after the first underscore character. We do this wit the `name_beginning()` function. This uses [`splitext`](https://docs.julialang.org/en/v1/base/file/#Base.Filesystem.splitext) to split file names into everything before the file extension, and everything after. Then we split everything before the file extension on the “\_” character, and return the first element, `first_part`.

My favorite part of this solution is using dot notation to apply `name_beginning()` on all the files in `vtk_files`. For more on how this works, see [this page](https://docs.julialang.org/en/v1/manual/functions/#man-vectorized-1).

Finally, let’s single out all the unique names with the [unique](https://docs.julialang.org/en/v1/base/collections/#Base.unique) function.

---

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [May 5, 2019, 7:20pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/5 "2019-05-05T19:20:02Z")

</div>

Thanks for both of your answers, @DNF and @natemcintosh! Your explanations are very neat and it is nice to see how it is possible to approach this problem. At the end I chose DNF’s implementation since it is more robust, ie. does not care about “\_” or “-”, and it is only about 10% slower than my initial implementation (speed in this regard is not important, just a bonus), while the implementation using glob\_thing etc. ended up being in my case double as slow - also regarding the outputs, DNF and natemcintosh approach gave respectively:

```julia
Set(["PartFluidOut", "PartFluid"])

```

```julia
2-element Array{SubString{String},1}

```

Which seems to be a bit of a hassle to work with, so I changed it back to a Array{String,1}.

But thank you to both of you, it was very nice to be forced to think out of the box and having to look up some new Julia terms, ie. “continue” and related syntax, to become better at the language. My final function for anyone interested is:

```julia
function getfilestrings(namelist, pattern::Regex=".*") # default is to accept all strings
    filenames::Array{String,1} = [] # this makes sure that the list is unique
    for filename in namelist
        m = match(pattern, filename)
        isnothing(m) && continue
        str = first(m.captures) #? captures for help, it just gets the "PartFluid", component
        !isempty(str) && push!(filenames, str)
    end
    return unique(filenames)
end

```

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [May 5, 2019, 7:25pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/6 "2019-05-05T19:25:45Z")

</div>

> [@Ahmed\_Salih](#):
>
> ```julia
> filenames::Array{String,1} = []
> 
> ```

I think it would be more idiomatic to write:

```julia
filenames = String[]

```

And, though I haven’t compared performance, it seems more aesthetically appealing to write

```julia
!isempty(str) && !in(str, filenames) && push!(filenames, str)

```

instead of pushing everything and then `unique!`ing it away afterwards.

---

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [May 5, 2019, 7:37pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/7 "2019-05-05T19:37:48Z")

</div>

Thanks for your suggestions!

Regarding the first one, I see your point using `String[]`, but I have chosen to not adopt it, since Atom does not highlight “String” for me for some reason, so stuck to the former. But I agree with you.

I kept using unique as well, since it was a bit faster and had about 20% allocations.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [May 5, 2019, 7:41pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/8 "2019-05-05T19:41:32Z")

</div>

I must admit I really dislike the style of `variable::TypeAssertion = something`. Type assertions are a last resort imho. Maybe you could try

```julia
filenames = Vector{String}()

```

---

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [May 5, 2019, 7:46pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/9 "2019-05-05T19:46:11Z")

</div>

> [@DNF](#):
>
> Vector{String}()

Thanks, that works!

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [May 5, 2019, 7:52pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/10 "2019-05-05T19:52:21Z")

</div>

In this case, it’s not just a matter of style preference. There’s a functional difference between the following two functions:

```julia
function f()
    filenames::Array{String,1} = []
    return filenames
end

function g()
    filenames = String[] # or Vector{String}()
    return filenames
end

```

namely that `[]` is a shortcut for `Vector{Any}()`, so the `::Array{String,1}` actually calls `convert(Vector{String}, [])`. So the `code_warntype` is different for these two functions:

```julia
Body::Array{String,1}
1 ─ %1 = $(Expr(:foreigncall, :(:jl_alloc_array_1d), Array{Any,1}, svec(Any, Int64), :(:ccall), 2, Array{Any,1}, 0, 0))::Array{Any,1}
│ %2 = (Base.arraysize)(%1, 1)::Int64
│ %3 = $(Expr(:foreigncall, :(:jl_alloc_array_1d), Array{String,1}, svec(Any, Int64), :(:ccall), 2, Array{String,1}, :(%2), :(%2)))::Array{String,1}
│ %4 = invoke Base.copyto!($(QuoteNode(IndexLinear()))::IndexLinear, %3::Array{String,1}, $(QuoteNode(IndexLinear()))::IndexLinear, %1::Array{Any,1})::Array{String,1}
└── return %4

julia> @code_warntype g()
Body::Array{String,1}
1 ─ %1 = $(Expr(:foreigncall, :(:jl_alloc_array_1d), Array{String,1}, svec(Any, Int64), :(:ccall), 2, Array{String,1}, 0, 0))::Array{String,1}
└── return %1

```

and likewise for the `code_native`, which is much shorter for `g()`.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [May 5, 2019, 7:52pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/11 "2019-05-05T19:52:39Z")

</div>

Oh, by the way, instead of

```julia
unique(filenames)

```

you should use

```julia
unique!(filenames)

```

That’s much faster.

---

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [May 5, 2019, 7:58pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/12 "2019-05-05T19:58:28Z")

</div>

I am not so experienced in code\_warntype, but the gist of your comment is that, running with `String[]` or `Vector{String}()` is more efficient coding-wise? Will have a look through my code and fix these then.

It is actually kind of fascinating how much difference their can be at seemingly similar functions 🙂

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [May 5, 2019, 8:02pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/13 "2019-05-05T20:02:29Z")

</div>

Yes, but it’s really not a question of understanding obscure rules, it all makes logical sense.

```julia
String[]

```

or

```julia
Vector{String}()

```

directly creates a vector of strings, while

```julia
filenames::Array{String,1} = []

```

first creates `[]` and then converts it. It’s all consistent and out in the open.

---

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [May 5, 2019, 8:12pm UTC](https://discourse.julialang.org/t/help-me-make-a-regex-instead-of-this/23878/14 "2019-05-05T20:12:19Z")

</div>

Thanks, that makes sense to me.