# Data

**URL:** https://discourse.julialang.org/c/domain/data/16.md?page=31

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 32

---

## [How to (row) append two JuliaDB tables together?](https://discourse.julialang.org/t/how-to-row-append-two-juliadb-tables-together/14228)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 1\
**Last updated:** [August 29, 2018, 9:11am UTC](https://discourse.julialang.org/t/how-to-row-append-two-juliadb-tables-together/14228 "2018-08-29T09:11:45Z")

</div>

How do I (row) append two JuliaDB tables togehter? In DataFrames.jl \[data1;data2\] is sufficient, but the same syntax doesn’t work in JuliaDB. Also How do I append table 2 to table 1 by modifying table 1? See MWE using…

---

## [CSV Reader / Writer Choices](https://discourse.julialang.org/t/csv-reader-writer-choices/14218)

<div class="topic-metadata">

**Author:** [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Replies:** 1\
**Last updated:** [August 28, 2018, 11:16pm UTC](https://discourse.julialang.org/t/csv-reader-writer-choices/14218 "2018-08-28T23:16:35Z")

</div>

Has someone compared the various packages to read and write csv files (e.g., readtables, DelimitedFiles and CSV)?

---

## [Force describe() not to omit columns](https://discourse.julialang.org/t/force-describe-not-to-omit-columns/13793)

<div class="topic-metadata">

**Author:** [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Replies:** 7\
**Last updated:** [August 21, 2018, 10:06pm UTC](https://discourse.julialang.org/t/force-describe-not-to-omit-columns/13793 "2018-08-21T22:06:42Z")

</div>

is it possible to force describe not to omit columns for printing?

---

## [Dataframe functional input and output formats in julia 1.0, august 2018](https://discourse.julialang.org/t/dataframe-functional-input-and-output-formats-in-julia-1-0-august-2018/13800)

<div class="topic-metadata">

**Author:** [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Replies:** 2\
**Last updated:** [August 21, 2018, 5:58pm UTC](https://discourse.julialang.org/t/dataframe-functional-input-and-output-formats-in-julia-1-0-august-2018/13800 "2018-08-21T17:58:51Z")

</div>

I am trying to find out in what data formats Julia 1.0 can currently \[august 2018\] use to read and write data frames. for a sample, I am using something simple like julia\> df 6×4 DataFrame │ Row │ n1 │ n2 │ n3 …

---

## [A prototype for a minimalistic universal database API](https://discourse.julialang.org/t/a-prototype-for-a-minimalistic-universal-database-api/13743)

<div class="topic-metadata">

**Author:** [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Replies:** 4\
**Last updated:** [August 20, 2018, 5:19pm UTC](https://discourse.julialang.org/t/a-prototype-for-a-minimalistic-universal-database-api/13743 "2018-08-20T17:19:44Z")

</div>

Hello all, I have written an extremely minimal universal database API in the style of PEP249. It is so minimal, in fact, that by design you should not even have to import this package to implement it. My goal was to pr…

---

## [Converting all NaN's to Missing and vice-versa in relevant columns](https://discourse.julialang.org/t/converting-all-nans-to-missing-and-vice-versa-in-relevant-columns/13675)

<div class="topic-metadata">

**Author:** [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Replies:** 2\
**Last updated:** [August 18, 2018, 9:39pm UTC](https://discourse.julialang.org/t/converting-all-nans-to-missing-and-vice-versa-in-relevant-columns/13675 "2018-08-18T21:39:47Z")

</div>

I am experimenting with dataframes and missings, which have of course morphed over the last few months. my dataframes can have float columns that are already unions(missing), float columns that are not yet unions, and …

---

## [A type for metadata?](https://discourse.julialang.org/t/a-type-for-metadata/10456)

<div class="topic-metadata">

**Author:** [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Replies:** 11\
**Last updated:** [August 17, 2018, 6:24pm UTC](https://discourse.julialang.org/t/a-type-for-metadata/10456 "2018-08-17T18:24:46Z")

</div>

I’m routinely dealing with data + metadata, and I wanted to get some feedback on ideas for how to operationalize this. I’m a biologist, not a programmer, so it’s likely I’m overlooking some obvious solution. I’ll explain…

---

## [Resurrecting universal database API](https://discourse.julialang.org/t/resurrecting-universal-database-api/11412)

<div class="topic-metadata">

**Author:** [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Replies:** 60\
**Last updated:** [August 17, 2018, 4:23am UTC](https://discourse.julialang.org/t/resurrecting-universal-database-api/11412 "2018-08-17T04:23:38Z")

</div>

Based, on the other thread, it sounds like there is some interest in resurrecting the idea of a “universal Julia database API”. There are already some great Julia database packages, and thanks to ODBC.jl and JDBC.jl at …

---

## [How to index a \`CatagoricalArray\` then make a new array with the same levels](https://discourse.julialang.org/t/how-to-index-a-catagoricalarray-then-make-a-new-array-with-the-same-levels/12874)

<div class="topic-metadata">

**Author:** [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Replies:** 4\
**Last updated:** [August 15, 2018, 5:51pm UTC](https://discourse.julialang.org/t/how-to-index-a-catagoricalarray-then-make-a-new-array-with-the-same-levels/12874 "2018-08-15T17:51:59Z")

</div>

I have been looking at the source code for CategoricalArray but I can’t figure this out. vec = CategoricalArray(\[1,2,1,2\]) t = vec\[1\] # We know what levels t.pool # CategoricalArrays.CategoricalPool{Int64,UInt32}(\[1,2\]…

---

## [JLD and FileIO StackOverflowError: error when saving EPL fixtures data](https://discourse.julialang.org/t/jld-and-fileio-stackoverflowerror-error-when-saving-epl-fixtures-data/13139)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 0\
**Last updated:** [August 10, 2018, 3:05am UTC](https://discourse.julialang.org/t/jld-and-fileio-stackoverflowerror-error-when-saving-epl-fixtures-data/13139 "2018-08-10T03:05:40Z")

</div>

I run the below code, which should work on your computer if you have all the packages installed. Basically it downloads some CSV from football-data.co.uk and tries to save it as a JLD file. The last line always gives an …

---

## [In Julia \>= 0.7, will the package "Missings.jl" still be needed or will it be part of core?](https://discourse.julialang.org/t/in-julia-0-7-will-the-package-missings-jl-still-be-needed-or-will-it-be-part-of-core/12733)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 4\
**Last updated:** [August 1, 2018, 7:11am UTC](https://discourse.julialang.org/t/in-julia-0-7-will-the-package-missings-jl-still-be-needed-or-will-it-be-part-of-core/12733 "2018-08-01T07:11:04Z")

</div>

Hello, I tough that the functionality that the package Missings.jl currently provides for Julia 0.6 (the Missing type in substitution of NA from older DataFrames packages) would be in core starting from Julia 0.7/1.x, an…

---

## [File Format for Large Two-Dimensional Dataset](https://discourse.julialang.org/t/file-format-for-large-two-dimensional-dataset/12712)

<div class="topic-metadata">

**Author:** [@cortner](https://discourse.julialang.org/u/cortner)\
**Replies:** 19\
**Last updated:** [July 31, 2018, 9:38am UTC](https://discourse.julialang.org/t/file-format-for-large-two-dimensional-dataset/12712 "2018-07-31T09:38:51Z")

</div>

I need to store a moderate-to-large matrix, which (essentially) represents a least-squares system, since the matrix entries are much more expensive to evaluate than solving the actual system. I would therefore like to st…

---

## [A spreadsheet-like pivot function](https://discourse.julialang.org/t/a-spreadsheet-like-pivot-function/2701)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 5\
**Last updated:** [July 30, 2018, 9:24am UTC](https://discourse.julialang.org/t/a-spreadsheet-like-pivot-function/2701 "2018-07-30T09:24:12Z")

</div>

(crossposting from S.O.: http://stackoverflow.com/questions/42815117/a-generic-sreadsheet-like-pivot-function-for-julia-dataframes) I implemented a generic pivot function for Julia DataFrames, with some of the nice bell…

---

## [Browsing dataframes and other tables?](https://discourse.julialang.org/t/browsing-dataframes-and-other-tables/12720)

<div class="topic-metadata">

**Author:** [@Raf](https://discourse.julialang.org/u/Raf)\
**Replies:** 10\
**Last updated:** [July 30, 2018, 3:43am UTC](https://discourse.julialang.org/t/browsing-dataframes-and-other-tables/12720 "2018-07-30T03:43:21Z")

</div>

How are people browsing their dataframes and other kinds of tables? I have sets of model parameter data to explore with my supervisors (auto-generated with Flatten.jl which I’m very excited about!). The columns are thin…

---

## [CSV.read extremely slow wrt readtable](https://discourse.julialang.org/t/csv-read-extremely-slow-wrt-readtable/11726)

<div class="topic-metadata">

**Author:** [@lucabrugnolini](https://discourse.julialang.org/u/lucabrugnolini)\
**Replies:** 14\
**Last updated:** [July 27, 2018, 6:57pm UTC](https://discourse.julialang.org/t/csv-read-extremely-slow-wrt-readtable/11726 "2018-07-27T18:57:50Z")

</div>

Hello, I have found a huge difference in speed and allocation by switching from the deprecated DataFrames.readtable to CSV.read. I have noticed that this happens with large datasets (high number of columns), thus I star…

---

## [\[ANN\] RowTables.jl](https://discourse.julialang.org/t/ann-rowtables-jl/12608)

<div class="topic-metadata">

**Author:** [@jlapeyre](https://discourse.julialang.org/u/jlapeyre)\
**Replies:** 6\
**Last updated:** [July 26, 2018, 2:40am UTC](https://discourse.julialang.org/t/ann-rowtables-jl/12608 "2018-07-26T02:40:48Z")

</div>

RowTables.jl is intended to be (and often is) faster than DataFrames at row-wise operations. https://github.com/jlapeyre/RowTables.jl This package is not registered. See README.md for a few benchmarks.

---

## [Max of Date and Missing on 0.6.4?](https://discourse.julialang.org/t/max-of-date-and-missing-on-0-6-4/12542)

<div class="topic-metadata">

**Author:** [@jacobadenbaum](https://discourse.julialang.org/u/jacobadenbaum)\
**Replies:** 2\
**Last updated:** [July 21, 2018, 2:37pm UTC](https://discourse.julialang.org/t/max-of-date-and-missing-on-0-6-4/12542 "2018-07-21T14:37:33Z")

</div>

Is this a known bug in 0.6.4? julia\> using Missings julia\> max(Date(1999, 1), missing) ERROR: MethodError: no method matching ifelse(::Missings.Missing, ::Date, ::Missings.Missing) Closest candidates are: ifelse(::Bo…

---

## [Queryverse @map question](https://discourse.julialang.org/t/queryverse-map-question/12484)

<div class="topic-metadata">

**Author:** [@johann.spies](https://discourse.julialang.org/u/johann.spies)\
**Replies:** 3\
**Last updated:** [July 20, 2018, 12:17am UTC](https://discourse.julialang.org/t/queryverse-map-question/12484 "2018-07-20T00:17:04Z")

</div>

I have describe(gewig) Summary Stats: Mean: 2979.413793 Minimum: 1613.000000 1st Quartile: 2226.500000 Median: 2822.500000 3rd Quartile: 3618.250000 Maximum: 5140.000000 Length: …

---

## [\[ANN\] TexTables.jl for Building LaTeX tables in Julia](https://discourse.julialang.org/t/ann-textables-jl-for-building-latex-tables-in-julia/12089)

<div class="topic-metadata">

**Author:** [@jacobadenbaum](https://discourse.julialang.org/u/jacobadenbaum)\
**Replies:** 15\
**Last updated:** [July 13, 2018, 12:00pm UTC](https://discourse.julialang.org/t/ann-textables-jl-for-building-latex-tables-in-julia/12089 "2018-07-13T12:00:46Z")

</div>

Hi Everyone, I have been working recently on a new package for building and printing LaTeX tables with multi-level row and column indices. It’s still in Beta and unregistered (for now, although I hope to register it of…

---

## [DataDeps and Google Drive](https://discourse.julialang.org/t/datadeps-and-google-drive/12271)

<div class="topic-metadata">

**Author:** [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Replies:** 14\
**Last updated:** [July 11, 2018, 8:44am UTC](https://discourse.julialang.org/t/datadeps-and-google-drive/12271 "2018-07-11T08:44:20Z")

</div>

Hi all (and hopefully oxinabox)! I’m trying to access data on a Google Drive using the excellent DataDeps.jl. The drive has been shared with me (so I don’t own it) and the data in it isn’t zipped (there are multiple fo…

---

## [Is there a way to print all columns in a NamedArray?](https://discourse.julialang.org/t/is-there-a-way-to-print-all-columns-in-a-namedarray/12070)

<div class="topic-metadata">

**Author:** [@mwsohn](https://discourse.julialang.org/u/mwsohn)\
**Replies:** 4\
**Last updated:** [July 7, 2018, 12:11am UTC](https://discourse.julialang.org/t/is-there-a-way-to-print-all-columns-in-a-namedarray/12070 "2018-07-07T00:11:40Z")

</div>

The NamedArray I want to print has five columns. I use “print” function to print it. Only four columns are printed and the middle column is replaced by an ellipsis. Is there a way to print all columns?

---

## [How to install feather using PyCall?](https://discourse.julialang.org/t/how-to-install-feather-using-pycall/9156)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 16\
**Last updated:** [July 6, 2018, 5:42pm UTC](https://discourse.julialang.org/t/how-to-install-feather-using-pycall/9156 "2018-07-06T17:42:38Z")

</div>

I am trying to create some benchmark for writing tabular data to disk. So I wnated to install feather in Julia. But the below gives errors using PyCall using Conda, DataFrames, FileIO #Conda.add("pandas") # need to run …

---

## [Filepaths in a module](https://discourse.julialang.org/t/filepaths-in-a-module/12208)

<div class="topic-metadata">

**Author:** [@Libbum](https://discourse.julialang.org/u/Libbum)\
**Replies:** 2\
**Last updated:** [July 6, 2018, 12:02pm UTC](https://discourse.julialang.org/t/filepaths-in-a-module/12208 "2018-07-06T12:02:20Z")

</div>

I’m having difficulty finding the best way to manage external data files in a module I’m writing. Say we have a module Foo that we add to our local packages through Pkg.clone("https://repo.com/Foo.git"). In the reposito…

---

## [PostgreSQL in Julia: LibPQ.jl](https://discourse.julialang.org/t/postgresql-in-julia-libpq-jl/9379)

<div class="topic-metadata">

**Author:** [@iamed2](https://discourse.julialang.org/u/iamed2)\
**Replies:** 9\
**Last updated:** [July 5, 2018, 6:38am UTC](https://discourse.julialang.org/t/postgresql-in-julia-libpq-jl/9379 "2018-07-05T06:38:29Z")

</div>

LibPQ.jl v0.3.0 is released so I’m announcing it here! LibPQ.jl will now be the fastest and easiest way to work with PostgreSQL in Julia. There are still features to be added, but it’s ready for use. Currently your best…

---

## [Packages for DataFrame manipulation/query](https://discourse.julialang.org/t/packages-for-dataframe-manipulation-query/12148)

<div class="topic-metadata">

**Author:** [@gbenatt92](https://discourse.julialang.org/u/gbenatt92)\
**Replies:** 5\
**Last updated:** [July 4, 2018, 12:12pm UTC](https://discourse.julialang.org/t/packages-for-dataframe-manipulation-query/12148 "2018-07-04T12:12:42Z")

</div>

Hi all, newcomer here! I know there was a lot of discussion on the past over DataFrames and where it’s headed, but my question is about the current packages for dataframe manipulation. I’ve come across both DataFramesM…

---

## [Assignment of a \`missing\` value fails in DataFrames 0.11.1](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379)

<div class="topic-metadata">

**Author:** [@mwsohn](https://discourse.julialang.org/u/mwsohn)\
**Replies:** 28\
**Last updated:** [June 28, 2018, 7:15pm UTC](https://discourse.julialang.org/t/assignment-of-a-missing-value-fails-in-dataframes-0-11-1/7379 "2018-06-28T19:15:39Z")

</div>

The following simple assignment fails. I think this defeats the very purpose of using DataFrames, i.e., allowing missing as legit values. Shouldn’t we have Array{Union{Missing,T},1} as the default type to avoid this issu…

---

## [Wrappers for GLM](https://discourse.julialang.org/t/wrappers-for-glm/11761)

<div class="topic-metadata">

**Author:** [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Replies:** 11\
**Last updated:** [June 19, 2018, 7:32am UTC](https://discourse.julialang.org/t/wrappers-for-glm/11761 "2018-06-19T07:32:38Z")

</div>

Based off of Chris’s comment here this is something that I have been wondering about myself. It would be nice to get some discussion about the regression ecosystem. We have StatsModels.jl which is used to create forumul…

---

## [Table transformation in JuliaDB or Queryverse or DataFramesMeta?](https://discourse.julialang.org/t/table-transformation-in-juliadb-or-queryverse-or-dataframesmeta/11622)

<div class="topic-metadata">

**Author:** [@dmbates](https://discourse.julialang.org/u/dmbates)\
**Replies:** 5\
**Last updated:** [June 13, 2018, 1:47pm UTC](https://discourse.julialang.org/t/table-transformation-in-juliadb-or-queryverse-or-dataframesmeta/11622 "2018-06-13T13:47:34Z")

</div>

I have been experimenting with different NLopt optimizers on particular examples of models from MixedModels. The current set of results is available at https://github.com/dmbates/MixedModelExamples/blob/master/results/…

---

## [Using DataFramesMeta and Lazy in a function](https://discourse.julialang.org/t/using-dataframesmeta-and-lazy-in-a-function/11616)

<div class="topic-metadata">

**Author:** [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Replies:** 0\
**Last updated:** [June 12, 2018, 6:09pm UTC](https://discourse.julialang.org/t/using-dataframesmeta-and-lazy-in-a-function/11616 "2018-06-12T18:09:46Z")

</div>

I’m working with dplyr today and ran into something that annoys me often. If I have a number of operations to do in repeatedly on a dataframe, its useful to put them in a function. However because of R’s lazy evaluation…

---

## [Random subsample from a JuliaDB distributed table: no getindex](https://discourse.julialang.org/t/random-subsample-from-a-juliadb-distributed-table-no-getindex/11615)

<div class="topic-metadata">

**Author:** [@ElOceanografo](https://discourse.julialang.org/u/ElOceanografo)\
**Replies:** 0\
**Last updated:** [June 12, 2018, 5:57pm UTC](https://discourse.julialang.org/t/random-subsample-from-a-juliadb-distributed-table-no-getindex/11615 "2018-06-12T17:57:27Z")

</div>

I have a dataset in a JuliaDB distributed table with about 200 million rows. I’d like to take a random subsample from this table to do some quick exploratory plotting and analysis before running the whole thing. This i…

[Previous page](https://discourse.julialang.org/c/domain/data/16.md?page=30)

[Next page](https://discourse.julialang.org/c/domain/data/16.md?page=32)