Community string benchmark suite

[nilshg]

I often work with large tables (say in the tens of GB) with many string columns, and used to frequently run into situtations where the GC was stressed to a level that made the whole session completely unresponsive.

I posted an example of how the base String compares to ShortStrings (which I was using at the time, this was before InlineStrings were a thing) here, have a look to see if that’s useful:

The state of DataFrames.jl H2O benchmark

It turns out it’s actually quite hard to produce the same magnitude of speed gains I’m seeing in my application, but here’s an example that roughly shows what I mean: using DataFrames, Random string1 = [randstring(4) for _ ∈ 1:5e4] string2 = [randstring(6)*string(rand(1:10, 3)) for _ ∈ 1:16e6] df = DataFrame( col1 = [rand(string1) for _ ∈ 1:20e6], col2 = shuffle!([string2; [missing for _ ∈ 1:4e6]]), col3 = shuffle!([string2; [missing for _ ∈ 1:4e6]]), col4 = shuffle!([string2…