# Data

**URL:** https://discourse.julialang.org/c/domain/data/16.md?page=29

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 30

---

## [First steps in JuliaDB](https://discourse.julialang.org/t/first-steps-in-juliadb/14010)

<div class="topic-metadata">

**Author:** [@jstrube](https://discourse.julialang.org/u/jstrube)\
**Replies:** 1\
**Last updated:** [February 4, 2019, 9:09pm UTC](https://discourse.julialang.org/t/first-steps-in-juliadb/14010 "2019-02-04T21:09:31Z")

</div>

I’m taking first steps with JuliaDB, but I’m already falling on my face. My first stumble was the difference between loadtable and load. Why do I need a different method name for a different input format? Moving on… I’m…

---

## [How does StatsBase.skewness work?](https://discourse.julialang.org/t/how-does-statsbase-skewness-work/20146)

<div class="topic-metadata">

**Author:** [@Gunter\_Faes](https://discourse.julialang.org/u/Gunter_Faes)\
**Replies:** 29\
**Last updated:** [January 29, 2019, 9:31pm UTC](https://discourse.julialang.org/t/how-does-statsbase-skewness-work/20146 "2019-01-29T21:31:39Z")

</div>

Hello, It is about the application of the skewness function. But I admit that I am not (yet) a Julia expert. But to my question: I have a DataFrame Rotwein and when I make the following function call … julia\> StatsBas…

---

## [What does \`groupby\` use to determine equality?](https://discourse.julialang.org/t/what-does-groupby-use-to-determine-equality/20106)

<div class="topic-metadata">

**Author:** [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Replies:** 3\
**Last updated:** [January 26, 2019, 1:39pm UTC](https://discourse.julialang.org/t/what-does-groupby-use-to-determine-equality/20106 "2019-01-26T13:39:30Z")

</div>

I have a dataframe with some custom types in one of the columns I’d like to groupby. This doesn’t seem to work correctly as the groupby seems to consider objects that are == and have identical hash to be distinct. What…

---

## [JuliaDB: GroupViews](https://discourse.julialang.org/t/juliadb-groupviews/20045)

<div class="topic-metadata">

**Author:** [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Replies:** 0\
**Last updated:** [January 24, 2019, 5:10pm UTC](https://discourse.julialang.org/t/juliadb-groupviews/20045 "2019-01-24T17:10:42Z")

</div>

I’ve found it particularly useful to view JuliaDB tables by chunks (denoted either by their primary keys, or a subset of the indexing key). Is there any plan to implement anything like the following into the library(whet…

---

## [Julia performs poorly on group-by benchmarks](https://discourse.julialang.org/t/julia-performs-poorly-on-group-by-benchmarks/16476)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 48\
**Last updated:** [January 23, 2019, 2:57am UTC](https://discourse.julialang.org/t/julia-performs-poorly-on-group-by-benchmarks/16476 "2019-01-23T02:57:58Z")

</div>

h20 has published a set of benchmarks that shows Julia’s DataFrames.jl has, in general, the worst group-by performance out of many data packages. JuliaDB.jl was not benchmarked so that may be a good addition. I have done…

---

## [JuliaDB: How to modify multiple existing columns with setcol using functions](https://discourse.julialang.org/t/juliadb-how-to-modify-multiple-existing-columns-with-setcol-using-functions/19847)

<div class="topic-metadata">

**Author:** [@pazzo83](https://discourse.julialang.org/u/pazzo83)\
**Replies:** 1\
**Last updated:** [January 20, 2019, 10:13pm UTC](https://discourse.julialang.org/t/juliadb-how-to-modify-multiple-existing-columns-with-setcol-using-functions/19847 "2019-01-20T22:13:48Z")

</div>

Hi, I am trying to modify several existing columns with one call to setcol using functions. Essentially I have a table where i need to divide 4 of the columns by 10 to get the proper value. I know this is the proper sy…

---

## [Reading huge csv files](https://discourse.julialang.org/t/reading-huge-csv-files/15924)

<div class="topic-metadata">

**Author:** [@jayce\_ram](https://discourse.julialang.org/u/jayce_ram)\
**Replies:** 5\
**Last updated:** [January 19, 2019, 10:52pm UTC](https://discourse.julialang.org/t/reading-huge-csv-files/15924 "2019-01-19T22:52:29Z")

</div>

I would like to read a huge csv file around 9G and apply filters to the rows in order to build another dataframe carrying only the values I want to. I have installed the pack DataBase and I have just tried the simple com…

---

## [Speeding Up Query with a big dataset](https://discourse.julialang.org/t/speeding-up-query-with-a-big-dataset/19508)

<div class="topic-metadata">

**Author:** [@elshera](https://discourse.julialang.org/u/elshera)\
**Replies:** 9\
**Last updated:** [January 12, 2019, 9:01am UTC](https://discourse.julialang.org/t/speeding-up-query-with-a-big-dataset/19508 "2019-01-12T09:01:09Z")

</div>

Hi, I am currently processing a dataset of 31 columns and 100.000 rows. Loading the data, describe them etc… is ok however while running some Query to match some rows is very slow. 100K rows is big but still not very bi…

---

## [Query.jl - how to select identical rows differing by one column](https://discourse.julialang.org/t/query-jl-how-to-select-identical-rows-differing-by-one-column/19367)

<div class="topic-metadata">

**Author:** [@elshera](https://discourse.julialang.org/u/elshera)\
**Replies:** 11\
**Last updated:** [January 10, 2019, 6:36am UTC](https://discourse.julialang.org/t/query-jl-how-to-select-identical-rows-differing-by-one-column/19367 "2019-01-10T06:36:33Z")

</div>

Hi, I have a dataframe of columns A,B,C, O. How can I write a query which shows me those rows which have same pattern in the columns A,B,C but different value for column O ? example: A, B, C, O 2, 3, 4, 10 2 ,3, 4,…

---

## [Taking Fitting Seriously](https://discourse.julialang.org/t/taking-fitting-seriously/13281)

<div class="topic-metadata">

**Author:** [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Replies:** 39\
**Last updated:** [December 8, 2018, 9:58am UTC](https://discourse.julialang.org/t/taking-fitting-seriously/13281 "2018-12-08T09:58:28Z")

</div>

I’ve been thinking about taking our fitting APIs seriously for a while now. (But am very busy so have not advanced it too far). I got to briefly talk to some people at JuliaCon about it, so I am quickly writing up som…

---

## [How to test missing value in CSV package?](https://discourse.julialang.org/t/how-to-test-missing-value-in-csv-package/7535)

<div class="topic-metadata">

**Author:** [@Fred](https://discourse.julialang.org/u/Fred)\
**Replies:** 11\
**Last updated:** [December 7, 2018, 6:01pm UTC](https://discourse.julialang.org/t/how-to-test-missing-value-in-csv-package/7535 "2018-12-07T18:01:30Z")

</div>

Hi, I try to test if there is a missing value at a specific position of a dataframe obtained with the CSV package and I did not find the function to do that. Thank you for your help. julia\> x = CSV.read("data/data.csv"…

---

## [Buglet? Warning on groupby](https://discourse.julialang.org/t/buglet-warning-on-groupby/18390)

<div class="topic-metadata">

**Author:** [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Replies:** 1\
**Last updated:** [December 6, 2018, 5:58pm UTC](https://discourse.julialang.org/t/buglet-warning-on-groupby/18390 "2018-12-06T17:58:37Z")

</div>

Is the Warning message a buglet? julia\> using DataFrames; df= DataFrame( n1=1:9, cat=\[ 'A','B','A', 'B','C','B', 'C','A','C' \] ); julia\> groupby(df, :cat) GroupedDataFrame{DataFrame} with 3 groups based on key: :cat Fi…

---

## [Problems about dealing with missing values, maybe connected to DataFrames.jl](https://discourse.julialang.org/t/problems-about-dealing-with-missing-values-maybe-connected-to-dataframes-jl/18282)

<div class="topic-metadata">

**Author:** [@Jaime](https://discourse.julialang.org/u/Jaime)\
**Replies:** 4\
**Last updated:** [December 4, 2018, 5:32pm UTC](https://discourse.julialang.org/t/problems-about-dealing-with-missing-values-maybe-connected-to-dataframes-jl/18282 "2018-12-04T17:32:54Z")

</div>

julia\> a=DataFrame(\[1 2 3; 4 missing 6\]) 2×3 DataFrame │ Row │ x1 │ x2 │ x3 │ │ │ Int64⍰ │ Int64⍰ │ Int64⍰ │ ├─────┼────────┼─────────┼────────┤ │ 1 │ 1 │ 2 │ 3 │ │ 2 │ 4 │ miss…

---

## [Question regarding Query.jl @join implementation](https://discourse.julialang.org/t/question-regarding-query-jl-join-implementation/18145)

<div class="topic-metadata">

**Author:** [@gbenatt92](https://discourse.julialang.org/u/gbenatt92)\
**Replies:** 1\
**Last updated:** [November 29, 2018, 5:12pm UTC](https://discourse.julialang.org/t/question-regarding-query-jl-join-implementation/18145 "2018-11-29T17:12:11Z")

</div>

Hi! I’ve sent this question over a DM to professor @davidanthoff, but it’s better suited here. Regarding the Query.jl package, I’d like to know what are the implementations of @join and @groupjoin, how efficient they a…

---

## [JuliaDB Getting Started...with save error](https://discourse.julialang.org/t/juliadb-getting-started-with-save-error/17891)

<div class="topic-metadata">

**Author:** [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Replies:** 6\
**Last updated:** [November 28, 2018, 7:59pm UTC](https://discourse.julialang.org/t/juliadb-getting-started-with-save-error/17891 "2018-11-28T19:59:57Z")

</div>

I am taking my first little steps in JuliaDB. I am planning to follow https://juliadb.org/latest, including the tutorial. I do not only want to access the data, but reduce the loading time relative to .csv . (I have l…

---

## [De-Serialization Performance](https://discourse.julialang.org/t/de-serialization-performance/17949)

<div class="topic-metadata">

**Author:** [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Replies:** 12\
**Last updated:** [November 25, 2018, 7:59pm UTC](https://discourse.julialang.org/t/de-serialization-performance/17949 "2018-11-25T19:59:10Z")

</div>

Specific Topic Related to Reading Data Is Still Too Slow . I spent the morning figuring out how serialization/deserialization perform. (I was writing more complex programs, but ultimately it whittled down to the follow…

---

## [Any fast database working with Julia?](https://discourse.julialang.org/t/any-fast-database-working-with-julia/17960)

<div class="topic-metadata">

**Author:** [@Juan](https://discourse.julialang.org/u/Juan)\
**Replies:** 3\
**Last updated:** [November 25, 2018, 12:43am UTC](https://discourse.julialang.org/t/any-fast-database-working-with-julia/17960 "2018-11-25T00:43:44Z")

</div>

What fast on-disk database (such as SciDB, MonetDB…) would you suggest to use to work with Julia. I mean something Julia can easily connect to and able to deal with large datasets. Being able to work distributed and on…

---

## [Package for tabular data](https://discourse.julialang.org/t/package-for-tabular-data/17849)

<div class="topic-metadata">

**Author:** [@Juan](https://discourse.julialang.org/u/Juan)\
**Replies:** 12\
**Last updated:** [November 23, 2018, 1:50pm UTC](https://discourse.julialang.org/t/package-for-tabular-data/17849 "2018-11-23T13:50:35Z")

</div>

Then for somebody starting to use Julia… which of that many options should we use for our new projects if we want to work with tabular data with real numbers, categorical variables and missing data as fast as possible?

---

## [Tables.jl vs TableTraits.jl (was TextParse.jl is fast again)](https://discourse.julialang.org/t/tables-jl-vs-tabletraits-jl-was-textparse-jl-is-fast-again/16982)

<div class="topic-metadata">

**Author:** [@datnamer](https://discourse.julialang.org/u/datnamer)\
**Replies:** 21\
**Last updated:** [November 12, 2018, 10:45am UTC](https://discourse.julialang.org/t/tables-jl-vs-tabletraits-jl-was-textparse-jl-is-fast-again/16982 "2018-11-12T10:45:58Z")

</div>

@davidanthoff what’s the relationship between tabletraits.jl and tables.jl? Edit: and interabletables.jl

---

## [Tables.jl: a table interface for everyone](https://discourse.julialang.org/t/tables-jl-a-table-interface-for-everyone/14071)

<div class="topic-metadata">

**Author:** [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Replies:** 19\
**Last updated:** [November 19, 2018, 5:00pm UTC](https://discourse.julialang.org/t/tables-jl-a-table-interface-for-everyone/14071 "2018-11-19T17:00:09Z")

</div>

I’ve had several people direct message me over the last couple of days asking for details about a new package I’ve been working: Tables.jl, so I thought I’d write up a quick post on what it is, where it came from, and wh…

---

## [DataTables or DataFrames?](https://discourse.julialang.org/t/datatables-or-dataframes/3160)

<div class="topic-metadata">

**Author:** [@natashawatkins](https://discourse.julialang.org/u/natashawatkins)\
**Replies:** 32\
**Last updated:** [November 19, 2018, 4:38pm UTC](https://discourse.julialang.org/t/datatables-or-dataframes/3160 "2018-11-19T16:38:26Z")

</div>

After a lot of reading, I’m still not certain about the state of data handling in Julia. My understanding is there is a migration over to DataTables due to type-safety considerations, however there are still a number of…

---

## [Disk based data manipulation framework needed](https://discourse.julialang.org/t/disk-based-data-manipulation-framework-needed/6303)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 21\
**Last updated:** [October 31, 2017, 8:20pm UTC](https://discourse.julialang.org/t/disk-based-data-manipulation-framework-needed/6303 "2017-10-31T20:20:04Z")

</div>

SAS is big in the corporate world. I work in the finance industry and SAS is still quite big. To be honest, using SAS is a pain. SAS can’t even syntax highlight its own language properly. But it’s still there because it …

---

## [read a .jxr file failed ](https://discourse.julialang.org/t/read-a-jxr-file-failed/17676)

<div class="topic-metadata">

**Author:** [@zsz00](https://discourse.julialang.org/u/zsz00)\
**Replies:** 0\
**Last updated:** [November 18, 2018, 2:47pm UTC](https://discourse.julialang.org/t/read-a-jxr-file-failed/17676 "2018-11-18T14:47:38Z")

</div>

read a .jxr file failed , what is wrong? using FileIO, ImageMagick; filename = "../test/matter/j0\_0.jxr" f1 = load(File(format"JXR", filename)) No FileIO.applicable\_loaders found for JXR

---

## [Alternative to DataFrame Readtable to read large data files with headers](https://discourse.julialang.org/t/alternative-to-dataframe-readtable-to-read-large-data-files-with-headers/15095)

<div class="topic-metadata">

**Author:** [@astrowanders](https://discourse.julialang.org/u/astrowanders)\
**Replies:** 17\
**Last updated:** [November 12, 2018, 7:30pm UTC](https://discourse.julialang.org/t/alternative-to-dataframe-readtable-to-read-large-data-files-with-headers/15095 "2018-11-12T19:30:47Z")

</div>

Hello! I’ve been using Julia to work with output from a stellar evolution code. My data files have several columns (~60 or more) and hundreds of rows. In previous versions of Julia, I used readtable in the DataFrame pac…

---

## [Query.jl @group DataFrame by AbstractType gives Method Error](https://discourse.julialang.org/t/query-jl-group-dataframe-by-abstracttype-gives-method-error/15501)

<div class="topic-metadata">

**Author:** [@JonasIsensee](https://discourse.julialang.org/u/JonasIsensee)\
**Replies:** 1\
**Last updated:** [November 12, 2018, 7:16pm UTC](https://discourse.julialang.org/t/query-jl-group-dataframe-by-abstracttype-gives-method-error/15501 "2018-11-12T19:16:03Z")

</div>

Hey there, I noticed that it is not possible to @group by an abstract typed column and I’m wondering if this is deliberately so or just a bug. julia\> df = DataFrame(foo=Real\[\]) 0×1 DataFrame julia\> push!(df, Dict(:f…

---

## [JSON Performance Tests](https://discourse.julialang.org/t/json-performance-tests/17133)

<div class="topic-metadata">

**Author:** [@samoconnor](https://discourse.julialang.org/u/samoconnor)\
**Replies:** 7\
**Last updated:** [November 6, 2018, 6:09am UTC](https://discourse.julialang.org/t/json-performance-tests/17133 "2018-11-06T06:09:10Z")

</div>

I noticed that @kristoffer.carlsson has been working on parsing performance in JSON.jl: A few performance improvements by KristofferC · Pull Request #263 · JuliaIO/JSON.jl · GitHub … and it got me thinking. When I firs…

---

## [Data files with 'quotes' around the column names](https://discourse.julialang.org/t/data-files-with-quotes-around-the-column-names/17067)

<div class="topic-metadata">

**Author:** [@GlenHenshaw](https://discourse.julialang.org/u/GlenHenshaw)\
**Replies:** 6\
**Last updated:** [November 2, 2018, 12:21am UTC](https://discourse.julialang.org/t/data-files-with-quotes-around-the-column-names/17067 "2018-11-02T00:21:48Z")

</div>

I have data files, given to me by someone else, with the following format: 'col1','col2','col3' 1,2,3 4,5,6 And so on. CSV reads them just fine, and leaves me with a DataFrame that looks like |Row | 'col1' | 'col…

---

## [File IO - Parquet File Reader](https://discourse.julialang.org/t/file-io-parquet-file-reader/15496)

<div class="topic-metadata">

**Author:** [@mdsalerno](https://discourse.julialang.org/u/mdsalerno)\
**Replies:** 4\
**Last updated:** [October 30, 2018, 9:24pm UTC](https://discourse.julialang.org/t/file-io-parquet-file-reader/15496 "2018-10-30T21:24:26Z")

</div>

Is there a Julia v1.0 supported parquet file reader that can be used to load into a DataFrame? I attempted to use the example on Queryverse.jl but it seems it requires ParquetFiles.jl. I get the following when I try to…

---

## [TextParse.jl is fast again](https://discourse.julialang.org/t/textparse-jl-is-fast-again/16664)

<div class="topic-metadata">

**Author:** [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Replies:** 13\
**Last updated:** [October 29, 2018, 6:41pm UTC](https://discourse.julialang.org/t/textparse-jl-is-fast-again/16664 "2018-10-29T18:41:31Z")

</div>

Some of you might have noticed that TextParse.jl (and CSVFiles.jl, which is a small wrapper around TextParse.jl) saw some major performance regressions initially on julia 1.0. I just fixed these and now both packages ar…

---

## [Benchmarking ways to write/load DataFrames IndexedTables to disk](https://discourse.julialang.org/t/benchmarking-ways-to-write-load-dataframes-indexedtables-to-disk/8973)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 42\
**Last updated:** [October 25, 2018, 3:55pm UTC](https://discourse.julialang.org/t/benchmarking-ways-to-write-load-dataframes-indexedtables-to-disk/8973 "2018-10-25T15:55:16Z")

</div>

Update 2018-Feb-19: added R feather and Pandas; thanks to @zhangliye for the pandas code For Julia, JLD.jl has the fastest write-solution and I have used it via the ultra-convenient FileIO.jl. However for interop with …

[Previous page](https://discourse.julialang.org/c/domain/data/16.md?page=28)

[Next page](https://discourse.julialang.org/c/domain/data/16.md?page=30)