# Cleaning up strings via piping...?

**URL:** <https://discourse.julialang.org/t/cleaning-up-strings-via-piping/29089>\
**Category:** General Usage\
**Tags:** strings\
**Created:** [September 23, 2019, 7:45pm UTC](https://discourse.julialang.org/t/cleaning-up-strings-via-piping/29089 "2019-09-23T19:45:33Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mthelm85](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mthelm85/32/224164_2.png) [@mthelm85](https://discourse.julialang.org/u/mthelm85)\
**Post date:** [September 23, 2019, 7:45pm UTC](https://discourse.julialang.org/t/cleaning-up-strings-via-piping/29089/1 "2019-09-23T19:45:33Z")

</div>

I’m hoping to get some tips for cleaning up strings (maybe with piping). For example, take the following vector:

```julia
dirtydata = ["O' Malley's Irish Pub", "El Sueño", "123 & ABC & 100%", 1.23, missing, 12]

```

What I would like to do is convert everything in this vector to a string, and then “standardize” it by removing punctuation, stripping non-ascii and non-alphanumeric characters, removing all spaces, etc.

I wrote a function that does some of these things as follows

```julia
function convert_clean(arr)
    arr = string.(arr)
    arr = Unicode.normalize.(arr, stripmark=true)
    arr = map(x -> replace(x, r"[^a-zA-Z0-9_]" => ""), arr)
    return arr
end

julia> convert_clean(dirtydata)
6-element Array{String,1}:
 "OMalleysIrishPub"
 "ElSueno"
 "123ABC100"
 "123"
 "missing"
 "12"

```

but I’m wondering if it wouldn’t be cleaner to use piping `|>`? I can’t seem to make it work…I tried several different ways to convert the above but none of them were successful. Maybe I should just nest all of these operations inside one another?

```julia
function convert_clean2(arr)
    return map(x -> replace(x, r"[^a-zA-Z0-9_]" => ""), Unicode.normalize.(string.(arr), stripmark=true))
end

```

I like that I can get it down to one line, but it seems like a nightmare in terms of readability. The first solution just doesn’t feel right either though, repeatedly re-defining `arr` inside the function.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [September 23, 2019, 8:01pm UTC](https://discourse.julialang.org/t/cleaning-up-strings-via-piping/29089/2 "2019-09-23T20:01:08Z")

</div>

In general, I would avoid creating a whole sequence of intermediate arrays. `map` with the `do` block sequence is pretty clear and avoids the intermediate arrays:

```julia
convert_clean(arr) = map(arr) do x
    s = string(x)
    s = Unicode.normalize(s, stripmark=true)
    s = replace(s, r"[^a-zA-Z0-9_]" => "")
end

```

for example.

You could also use `.|>` here to apply a bunch of functions elementwise, but it is a bit awkward because of the need to explicitly construct anonymous functions:

```julia
convert_clean(arr) = arr .|> string .|>
         s -> Unicode.normalize(s, stripmark=true) .|>
         s -> replace(s, r"[^a-zA-Z0-9_]" => "")

```

Hopefully someday you will be able to use a [magic underscore](https://github.com/JuliaLang/julia/pull/24990)

```julia
convert_clean(arr) = arr .|> string .|>
         Unicode.normalize(_, stripmark=true) .|>
         replace(_, r"[^a-zA-Z0-9_]" => "")

```

but not yet.

---

<div class="post-metadata">

**Author:** ![mthelm85](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mthelm85/32/224164_2.png) [@mthelm85](https://discourse.julialang.org/u/mthelm85)\
**Post date:** [September 23, 2019, 8:08pm UTC](https://discourse.julialang.org/t/cleaning-up-strings-via-piping/29089/3 "2019-09-23T20:08:40Z")

</div>

Thanks, Steven. I need to learn the `do` blocks once and for all. I see them in other people’s code but I’ve never really understood exactly what they `do` 🥴 and so I rarely implement them in my code but this looks like a great use case!