I fully agree with @mkborregaard : the data developers are doing an insane amount of work!
Concerning “typed-ness”, I believe that one issue with the current DataFrame implementation (unless of course I’m missing something) is that it’s difficult to do map
and filter
on a DataFrame performantly, even though I believe those are basic manipulations. By map
I mean something that takes a DataFrame and a function from named tuples to named tuples and outputs a DataFrame, and filter
would take a a DataFrame and a predicate on named tuples. One can of course resort to external packages (such as Query), but even there the @select
statement, which would be what here I call map
has some limitations as it relies on type inference on NamedTuples to work.
Still, there has been a lot of discussion and interesting ideas on these topics and I’m curious to see what the outcome will be.