Thank you very much for reporting. Mostly fixed in fix performance issue in multirow split-apply-combine by bkamins · Pull Request #2749 · JuliaData/DataFrames.jl · GitHub (maybe it still can be improved but the major sources of problems are solved):
julia> @btime test1($df1);
890.798 μs (556 allocations: 7.66 MiB)
julia> @btime test1($dfg1);
25.061 ms (65066 allocations: 54.79 MiB)