Avoid allocations for string concatenation

General Usage Performance
,
1

Hei together,

I realised with the allocation profiler that our model allocates a lot when concatenating strings like "a" * "b". Is there a way to avoid allocation in such cases? I tried with InternedStrings and intern("a" * "b"), but that didn’t change anything…

Happy to hear any suggestions.

Patrick

2

concatenation necessarily allocates. That said, using an IOBuffer if you are concatenating a lot of strings into 1 is a good idea.

1 Like
3

In Julia 1.8+ there’s a LazyString type which you might want to try to represent "a" * "b" instead. Note that LazyString construction still allocates the wrapper and a tuple though, so this is probably only worthwhile for concatenating larger strings.

For small strings, you might want to consider GitHub - JuliaString/ShortStrings.jl: A fast implementation of short strings of fixed size. Great for sorting and group-by operations

4

You could try StaticStrings.jl:

julia> using StaticStrings

julia> StaticString((Tuple(static"ab")..., Tuple(static"cdef")...))
static"abcdef"6

julia> @allocated StaticString((Tuple(static"ab")..., Tuple(static"cdef")...))
0

Maybe this should be the default concatenation method for StaticString.

1 Like
5

Don’t know anything about interned strings, but I think this would evaluate "a" * "b" first, and then call intern() on the result, so not surprised that it didn’t change. Can you intern("a") * intern("b") or something?

6 
julia> function Base.:*(a::StaticString{N}, b::StaticString{M}) where {N,M}
           t_a = Tuple(a)
           t_b = Tuple(b)
           StaticString(ntuple(N+M) do i
               if i <= N
                   t_a[i]
               else
                   t_b[i-N]
               end
           end)
       end

julia> static"abc" * static"defgh"
static"abcdefgh"8

julia> @allocated static"abc" * static"defgh"
0

Should we do this?