# Is it possible to see performant string processing without sacrificing ease of use in Julia one day?

**URL:** <https://discourse.julialang.org/t/is-it-possible-to-see-performant-string-processing-without-sacrificing-ease-of-use-in-julia-one-day/81698>\
**Category:** Performance\
**Tags:** question, strings\
**Created:** [May 26, 2022, 8:23am UTC](https://discourse.julialang.org/t/is-it-possible-to-see-performant-string-processing-without-sacrificing-ease-of-use-in-julia-one-day/81698 "2022-05-26T08:23:30Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ReneganRonin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/reneganronin/32/21738_2.png) [@ReneganRonin](https://discourse.julialang.org/u/ReneganRonin)\
**Post date:** [May 26, 2022, 8:23am UTC](https://discourse.julialang.org/t/is-it-possible-to-see-performant-string-processing-without-sacrificing-ease-of-use-in-julia-one-day/81698/1 "2022-05-26T08:23:30Z")

</div>

I have heard a lot that string performance in Julia is bad. So is it possible to see performant string processing without sacrificing ease of use in Julia one day?

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [May 26, 2022, 9:07am UTC](https://discourse.julialang.org/t/is-it-possible-to-see-performant-string-processing-without-sacrificing-ease-of-use-in-julia-one-day/81698/2 "2022-05-26T09:07:39Z")

</div>

Depends on what you mean with “string processing”. If you don’t create new strings all over the place, string operations are quite fast already in my experience.

I’m curious what specifics you heard, or was it just general sentiment without examples?

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [May 26, 2022, 10:52am UTC](https://discourse.julialang.org/t/is-it-possible-to-see-performant-string-processing-without-sacrificing-ease-of-use-in-julia-one-day/81698/3 "2022-05-26T10:52:13Z")

</div>

Yes, you can (already). Strings (since variable-length) are generally stored on the heap, in all languages. Already faster (and likely as fast as possible when strings short enough):

> **[GitHub - JuliaStrings/InlineStrings.jl: Fixed-width string types for Julia](https://github.com/JuliaStrings/InlineStrings.jl)**
>
> Fixed-width string types for Julia. Contribute to JuliaStrings/InlineStrings.jl development by creating an account on GitHub.

You can use these strings where they apply. CSV.jl uses them by now, I believe by default, or at least DataFrames does when it imports CSV files, and since then got a lot faster. It seems the alternative:

> **[GitHub - sl-solution/InMemoryDatasets.jl: Multithreaded package for working...](https://github.com/sl-solution/InMemoryDatasets.jl)**
>
> Multithreaded package for working with tabular data in Julia - GitHub - sl-solution/InMemoryDatasets.jl: Multithreaded package for working with tabular data in Julia

might not use `InlineString`s (from the docs seems uses regular String), still it claims “High performance” (would it use CSV.jl and whatever it provides?).

Right now you (or the package you use) needs to opt into using alternative string types. I had an idea of my own for a string type where the prefix of a string is stored with `InlineString` plus a pointer to the whole string (that would by NULL for all shorter strings, avoiding the heap, only allocating therefor longer strings). I hope my idea, if I or someone else, ever gets implemented it would be merged into Julia to replace the standard `String` type, so that you do not need to opt into my type.

[@Elrod mentioned recently a vector implementation in C++ (from LLVM), with that same idea, 3x faster than Julia Vector (or std:vector). It doesn’t seem like there’s any real reason it can’t be done in Julia, and while indexing is different, simpler, a lot is similar, and maybe part of the code could be shared for arrays and strings.].

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [May 26, 2022, 4:12pm UTC](https://discourse.julialang.org/t/is-it-possible-to-see-performant-string-processing-without-sacrificing-ease-of-use-in-julia-one-day/81698/4 "2022-05-26T16:12:06Z")

</div>

> [@Palli](#):
>
> [@Elrod mentioned recently a vector implementation in C++ (from LLVM), with that same idea, 3x faster than Julia Vector (or std:vector). It doesn’t seem like there’s any real reason it can’t be done in Julia, and while indexing is different, simpler, a lot is similar, and maybe part of the code could be shared for arrays and strings.]

This is something easy enough to implement at the package level in C++ (aside from the fact distributing and installing packages in C++ isn’t easy), but would need compiler support in Julia to be performant.  
That said, there are PRs like: [Move small arrays to the stack by pchintalapudi · Pull Request #43573 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/43573)  
which would help a ton of existing code, especially once the Julia compiler starts making use of escape analysis.  
Immutable arrays will help there as well.

---

<div class="post-metadata">

**Author:** ![jakobnissen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jakobnissen/32/13477_2.png) [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)\
**Post date:** [May 26, 2022, 4:15pm UTC](https://discourse.julialang.org/t/is-it-possible-to-see-performant-string-processing-without-sacrificing-ease-of-use-in-julia-one-day/81698/5 "2022-05-26T16:15:38Z")

</div>

Yes, I believe fast strings is possible one day for Julia. There is nothing in Julia’s design that prevents string being fast - on the contrary, I think the existing abstractions and interfaces of `String` have been well designed with performance in mind. It’s just some important implementations that are lacking.

---

<div class="post-metadata">

**Author:** ![brenhinkeller](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brenhinkeller/32/36648_2.png) [@brenhinkeller](https://discourse.julialang.org/u/brenhinkeller)\
**Post date:** [May 27, 2022, 3:03am UTC](https://discourse.julialang.org/t/is-it-possible-to-see-performant-string-processing-without-sacrificing-ease-of-use-in-julia-one-day/81698/6 "2022-05-27T03:03:22Z")

</div>

Is there anything in particular folks with interest in this space would suggest as a motivating example for benchmarking?

---

<div class="post-metadata">

**Author:** ![jakobnissen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jakobnissen/32/13477_2.png) [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)\
**Post date:** [May 27, 2022, 11:36am UTC](https://discourse.julialang.org/t/is-it-possible-to-see-performant-string-processing-without-sacrificing-ease-of-use-in-julia-one-day/81698/7 "2022-05-27T11:36:43Z")

</div>

One benchmark would be Heng Li’s “biofast” benchmark, the fastq one. ([biofast/fqcnt at master · lh3/biofast · GitHub](https://github.com/lh3/biofast/tree/master/fqcnt)) Although that is as much IO as strings.  
But I agree it would be nice to find a good set of benchmarks for this to measure progress!

---

<div class="post-metadata">

**Author:** ![ReneganRonin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/reneganronin/32/21738_2.png) [@ReneganRonin](https://discourse.julialang.org/u/ReneganRonin)\
**Post date:** [May 28, 2022, 2:46am UTC](https://discourse.julialang.org/t/is-it-possible-to-see-performant-string-processing-without-sacrificing-ease-of-use-in-julia-one-day/81698/8 "2022-05-28T02:46:07Z")

</div>

im just encouraging discussion 😈 . i am not very knowledgeable with it. just looking at some answers. though i wont lie that i heard some “complaints” from other julians