After a lot of reading, I’m still not certain about the state of data handling in Julia. My understanding is there is a migration over to DataTables due to type-safety considerations, however there are still a number of functions that are not supported? Will DataFrames eventually be deprecated com…

I think if you just pick either of DataFrames or DataTables and stick with it for a while you’ll be fine. My only caveat is that if you are dealing with datasets ≳5GB you might find the type stability issue in DataFrames to be an obstacle. I have a few simple tools for doing conversions in such in…

It’d be nice if DataFrames/DataTables added CSV as a dependency IMHO. I don’t think readtable is deprecated just yet though.

We’ve just deprecated readtable in DataTables, but not in DataFrames. Re-exporting CSV.read under some form is still a possibility, though.

[image] ExpandingMan: however @davidanthoff seems to have been working away furiously on Query.jl so I’d imagine that’s quite nice and compatible with lots of things by now. Yes, Query.jl works with DataTables. At least on master I also have an unexported, experimental method interface that m…

[image] davidanthoff: Another option would be to have some meta package like DataVerse at some point that just loads everything that one typically needs for data work. Yes, I for one would like to use a metapackage to make things easier. Obviously not a development target, but if there was ju…

It is definitely easier to understand the internals of DataFrames/DataTables, but I question the assumption that understanding the internals of a tabular data package makes it easier to use. In my opinion, a good tabular data package allows one to abstract away from internals altogether and simply p…

[image] micahjsmith: It is definitely easier to understand the internals of DataFrames/DataTables, but I question the assumption that understanding the internals of a tabular data package makes it easier to use. In my opinion, a good tabular data package allows one to abstract away from internal…

[image] ChrisRackauckas: Honestly, I find the current situation for data formats very confusing, though the Anthoff-verse is pretty nice. Agree, coming from R and non CS background, I find it painful to working with data in Julia. At work, I’m heavy user of R’s dplyr, and though we have here …

Part of the point I was making was that it is much easier to work with DataTables and DataFrames even when some of the core features are under active development than it would be to do something analogous with something like pandas. However, since pandas is pretty mature (despite the fact that they…

For what it’s worth, the released Pandas.jl is now updated to be fully functional on Julia .5 and Julia .6.

DataTables or DataFrames?

Specific Domains Data

Juan November 19, 2018, 4:27pm 31

What about Tables.jl?

Topic		Replies	Views
Announcement: An Update on DataFrames Future Plans Data announcement	41	9486	December 27, 2017
JuliaDB, dataframes: Speculations over the future of data packages Data	24	7576	August 21, 2020
Announcement: DataFrames Future Plans Data announcement	27	8109	July 4, 2017
Is there light at the end of the DataFrames tunnel? Data question	36	4584	November 24, 2017
How is the data ecosystem right now for large datasets? Data	35	6983	July 13, 2017

DataTables or DataFrames?

Related topics