Simple Table Operation Has Very Large Compilation Time with MLJ

General Usage
, , ,
22

Under MLJBase 0.20.5 just released:

julia> using MLJ
using DataFrames

julia> using DataFrames

julia> @time arr = [[rand(1:10) for i in 1:5] for i in 1:5000];
  0.059180 seconds (200.75 k allocations: 11.360 MiB, 98.87% compilation time)

julia> @time df = DataFrames.DataFrame(arr, :auto)
  0.312929 seconds (844.21 k allocations: 46.372 MiB, 7.70% gc time, 98.93% compilation time)

julia> @time y, X = unpack(df, ==(:x1));
  2.724263 seconds (9.68 M allocations: 596.840 MiB, 5.33% gc time, 99.53% compilation time)

julia> @time y, X = unpack(df, ==(:x1));
  0.014382 seconds (27.59 k allocations: 1.977 MiB)
1 Like