# Data

**URL:** https://discourse.julialang.org/c/domain/data/16.md?page=32

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 33

---

## [Stack hierarchical data](https://discourse.julialang.org/t/stack-hierarchical-data/11522)

<div class="topic-metadata">

**Author:** [@pmarg](https://discourse.julialang.org/u/pmarg)\
**Replies:** 0\
**Last updated:** [June 7, 2018, 9:54pm UTC](https://discourse.julialang.org/t/stack-hierarchical-data/11522 "2018-06-07T21:54:28Z")

</div>

Hi, I have some survey data which are organized hierarchically and I would like to transform them into a rectangular dataset. For example using JuliaDB: t = table(\[1,1,1,2,2,2\], \[1,2,2,2,4,5\],\[10,20,10,5,20,10\], names=…

---

## [Why JuliaDB.Table?](https://discourse.julialang.org/t/why-juliadb-table/11473)

<div class="topic-metadata">

**Author:** [@jamii](https://discourse.julialang.org/u/jamii)\
**Replies:** 1\
**Last updated:** [June 6, 2018, 6:46pm UTC](https://discourse.julialang.org/t/why-juliadb-table/11473 "2018-06-06T18:46:38Z")

</div>

IIRC the original JuliaDB release only had one datastructure, which was similar to the current JuliaDB.NDSparse. I’m curious why Table was added. Is this a UX thing, or so that they can dispatch differently, or some unde…

---

## [Vcat does not append DataFrame rows as it did in 0.10](https://discourse.julialang.org/t/vcat-does-not-append-dataframe-rows-as-it-did-in-0-10/9823)

<div class="topic-metadata">

**Author:** [@pasha](https://discourse.julialang.org/u/pasha)\
**Replies:** 77\
**Last updated:** [May 31, 2018, 11:48pm UTC](https://discourse.julialang.org/t/vcat-does-not-append-dataframe-rows-as-it-did-in-0-10/9823 "2018-05-31T23:48:23Z")

</div>

In DataFrames 0.10, we could use vcat() as a way to append differently-shaped DataFrames, much like R’s dplyr::bind\_rows(). You could send two DataFrames with differing columns or column order and it would append the ro…

---

## [Ignore comments with CSV.read(...)](https://discourse.julialang.org/t/ignore-comments-with-csv-read/11302)

<div class="topic-metadata">

**Author:** [@e3c6](https://discourse.julialang.org/u/e3c6)\
**Replies:** 1\
**Last updated:** [May 31, 2018, 7:31pm UTC](https://discourse.julialang.org/t/ignore-comments-with-csv-read/11302 "2018-05-31T19:31:30Z")

</div>

When reading a csv file with CSV.read(…), is it possible to ignore lines that start with a character (for example #)? In other words, I want to read a file that contains comments.

---

## [Renaming columns in DataFramesMeta with spaces](https://discourse.julialang.org/t/renaming-columns-in-dataframesmeta-with-spaces/11101)

<div class="topic-metadata">

**Author:** [@jacobcvt12](https://discourse.julialang.org/u/jacobcvt12)\
**Replies:** 2\
**Last updated:** [May 23, 2018, 7:14pm UTC](https://discourse.julialang.org/t/renaming-columns-in-dataframesmeta-with-spaces/11101 "2018-05-23T19:14:38Z")

</div>

I’ve created the following dataset: d = DataFrame(x=randn(100)) d\[Symbol("Y 2")\] = randn(100) I can rename the second column via rename!(d, Symbol("Y 2") =\> :Y) But I would like to rename it via a DataFramesMeta chai…

---

## [Save DataFrame containing arrays](https://discourse.julialang.org/t/save-dataframe-containing-arrays/10927)

<div class="topic-metadata">

**Author:** [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)\
**Replies:** 4\
**Last updated:** [May 16, 2018, 12:52pm UTC](https://discourse.julialang.org/t/save-dataframe-containing-arrays/10927 "2018-05-16T12:52:51Z")

</div>

I want to manipulate performance experiments data, store/load these data from a file. I try to use DataFrames.jl which looks OK. When I want to store my data in a csv file, it fails if my data contains arrays of floats. …

---

## [\`@collect table\` does not work: Query.jl and JuliaDB.jl](https://discourse.julialang.org/t/collect-table-does-not-work-query-jl-and-juliadb-jl/10869)

<div class="topic-metadata">

**Author:** [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Replies:** 2\
**Last updated:** [May 15, 2018, 2:17am UTC](https://discourse.julialang.org/t/collect-table-does-not-work-query-jl-and-juliadb-jl/10869 "2018-05-15T02:17:59Z")

</div>

Is this asymmetry expected? Should I open an issue? julia\> using JuliaDB julia\> using DataFrames julia\> using Query julia\> t = table(@NT(A = \[1,2\], B = \[2,3\])); julia\> df = DataFrame(A = \[1,2\], B = \[2,3\]); julia\> x…

---

## [Apply weights in JuliaDB groupby](https://discourse.julialang.org/t/apply-weights-in-juliadb-groupby/10817)

<div class="topic-metadata">

**Author:** [@pmarg](https://discourse.julialang.org/u/pmarg)\
**Replies:** 3\
**Last updated:** [May 10, 2018, 5:33pm UTC](https://discourse.julialang.org/t/apply-weights-in-juliadb-groupby/10817 "2018-05-10T17:33:12Z")

</div>

Hi, I’ve been struggling to find a way to calculate the weighted descriptive statistics in JuliaDB using the groupby function. Assuming that “w” is a column of the table DB with sampling weights, I would like to find so…

---

## [Store arrays in juliadb table?](https://discourse.julialang.org/t/store-arrays-in-juliadb-table/10788)

<div class="topic-metadata">

**Author:** [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)\
**Replies:** 0\
**Last updated:** [May 8, 2018, 10:24pm UTC](https://discourse.julialang.org/t/store-arrays-in-juliadb-table/10788 "2018-05-08T22:24:57Z")

</div>

Hi, I would like store a collection of performance measurements consisting in arrays of floats. Each measurement is characterized by several attributes . I would like to be able to filter the measurements relatively to …

---

## [Breaking ExcelReaders.jl update](https://discourse.julialang.org/t/breaking-excelreaders-jl-update/10624)

<div class="topic-metadata">

**Author:** [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)\
**Replies:** 2\
**Last updated:** [May 7, 2018, 4:43am UTC](https://discourse.julialang.org/t/breaking-excelreaders-jl-update/10624 "2018-05-07T04:43:28Z")

</div>

Hi everyone, just a heads up that I’ve just released a new version of ExcelReaders.jl that is breaking. The new version removes all support for DataFrames.jl and no longer uses DataArrays.jl. But, fear not, you can sti…

---

## [Reading a CSV file with an unconventional string for missing values using TextParse](https://discourse.julialang.org/t/reading-a-csv-file-with-an-unconventional-string-for-missing-values-using-textparse/10631)

<div class="topic-metadata">

**Author:** [@askvorts](https://discourse.julialang.org/u/askvorts)\
**Replies:** 4\
**Last updated:** [May 2, 2018, 8:04pm UTC](https://discourse.julialang.org/t/reading-a-csv-file-with-an-unconventional-string-for-missing-values-using-textparse/10631 "2018-05-02T20:04:32Z")

</div>

I am trying to read a CSV file with two columns (Date, Float64) using TextParse. Unfortunately missing values are coded as “-” in that file. When TextParse encounters one of those “-” one gets the following error message…

---

## [Importing CSV with missing data](https://discourse.julialang.org/t/importing-csv-with-missing-data/10589)

<div class="topic-metadata">

**Author:** [@pmarg](https://discourse.julialang.org/u/pmarg)\
**Replies:** 13\
**Last updated:** [April 30, 2018, 5:10pm UTC](https://discourse.julialang.org/t/importing-csv-with-missing-data/10589 "2018-04-30T17:10:31Z")

</div>

Hello everyone, I’m having troubles with reading a CSV with missing data. If a single column has missing values then all the imported columns are of type Missing. For example importing df.csv: "X1","X2" 1, 3 2, 4 , 5 …

---

## [A proposal for \`describe\` of a DataFrame](https://discourse.julialang.org/t/a-proposal-for-describe-of-a-dataframe/10254)

<div class="topic-metadata">

**Author:** [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Replies:** 17\
**Last updated:** [April 28, 2018, 9:05am UTC](https://discourse.julialang.org/t/a-proposal-for-describe-of-a-dataframe/10254 "2018-04-28T09:05:38Z")

</div>

I am interested in making DataFrames as functional as R and Stata’s ecosystem. So far, I am really impressed with the data munging and processing ecosystem. I think a combination of DataFrames, DataFramesMeta, and Lazy r…

---

## [Unexpected behavior of Query.jl @map](https://discourse.julialang.org/t/unexpected-behavior-of-query-jl-map/10474)

<div class="topic-metadata">

**Author:** [@SebastianM-C](https://discourse.julialang.org/u/SebastianM-C)\
**Replies:** 0\
**Last updated:** [April 21, 2018, 11:27pm UTC](https://discourse.julialang.org/t/unexpected-behavior-of-query-jl-map/10474 "2018-04-21T23:27:35Z")

</div>

I tried to call a function for each selected row with Query.jl with the following code: test = DataFrame() test\[:a\] = \[1,2,3,4\] test\[:b\] = \[1,1,2,3\] test |\> @filter(\_.b==1) |\> @map(println(\_.a)) and I noticed that in J…

---

## [JLD disk space usage](https://discourse.julialang.org/t/jld-disk-space-usage/10397)

<div class="topic-metadata">

**Author:** [@gideonsimpson](https://discourse.julialang.org/u/gideonsimpson)\
**Replies:** 1\
**Last updated:** [April 18, 2018, 9:36am UTC](https://discourse.julialang.org/t/jld-disk-space-usage/10397 "2018-04-18T09:36:07Z")

</div>

If I have data stored in a JLD file is there a way to analyze it to see which variables in the .jld are taking up which fraction of the disk space?

---

## [DataFramesMeta @linq does not dispatch to StatsBase.mean?](https://discourse.julialang.org/t/dataframesmeta-linq-does-not-dispatch-to-statsbase-mean/10343)

<div class="topic-metadata">

**Author:** [@floswald](https://discourse.julialang.org/u/floswald)\
**Replies:** 2\
**Last updated:** [April 15, 2018, 8:29am UTC](https://discourse.julialang.org/t/dataframesmeta-linq-does-not-dispatch-to-statsbase-mean/10343 "2018-04-15T08:29:44Z")

</div>

I have this issue with DataFramesMeta using DataFrames, DataFramesMeta, StatsBase julia\> d = DataFrame(a=1:10,b = 2.0 \* (1:10),w=Weights(\[1,zeros(9)...\])) 10×3 DataFrames.DataFrame │ Row │ a │ b │ w │ ├─────┼────…

---

## [ANN: DataDeps.jl: BinDeps for Data](https://discourse.julialang.org/t/ann-datadeps-jl-bindeps-for-data/8457)

<div class="topic-metadata">

**Author:** [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Replies:** 16\
**Last updated:** [April 17, 2018, 5:50am UTC](https://discourse.julialang.org/t/ann-datadeps-jl-bindeps-for-data/8457 "2018-04-17T05:50:31Z")

</div>

I’ve now released DataDeps.jl, which is a package for automating the setup of data that a package or scientific script might depend upon. See the Repository: https://github.com/oxinabox/DataDeps.jl and the demo blog p…

---

## [How does juliaDB store data on disk? Is it a row store or a column store?](https://discourse.julialang.org/t/how-does-juliadb-store-data-on-disk-is-it-a-row-store-or-a-column-store/10339)

<div class="topic-metadata">

**Author:** [@Steven\_Sagaert](https://discourse.julialang.org/u/Steven_Sagaert)\
**Replies:** 1\
**Last updated:** [April 15, 2018, 6:32am UTC](https://discourse.julialang.org/t/how-does-juliadb-store-data-on-disk-is-it-a-row-store-or-a-column-store/10339 "2018-04-15T06:32:07Z")

</div>

What is the storage format? By column store I mean Column-oriented DBMS - Wikipedia

---

## [Is it possible to use "supertype" in function type declaration](https://discourse.julialang.org/t/is-it-possible-to-use-supertype-in-function-type-declaration/10320)

<div class="topic-metadata">

**Author:** [@Fred](https://discourse.julialang.org/u/Fred)\
**Replies:** 9\
**Last updated:** [April 13, 2018, 12:38pm UTC](https://discourse.julialang.org/t/is-it-possible-to-use-supertype-in-function-type-declaration/10320 "2018-04-13T12:38:30Z")

</div>

Hi, When I process strings in Julia, I have often a problem with the ::String type because sometime my function need a String and sometime a SubString{String} and I have this error no method matching processData(::Sub…

---

## [MultiResolutionInterators.jl: Tools for working with data that has macro-scale hierachical structure, where you don't always care about some of the levels](https://discourse.julialang.org/t/multiresolutioninterators-jl-tools-for-working-with-data-that-has-macro-scale-hierachical-structure-where-you-dont-always-care-about-some-of-the-levels/10315)

<div class="topic-metadata">

**Author:** [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Replies:** 1\
**Last updated:** [April 13, 2018, 10:22am UTC](https://discourse.julialang.org/t/multiresolutioninterators-jl-tools-for-working-with-data-that-has-macro-scale-hierachical-structure-where-you-dont-always-care-about-some-of-the-levels/10315 "2018-04-13T10:22:23Z")

</div>

I wanted to share this that I am working on right now. (Not yet registered) https://github.com/oxinabox/MultiResolutionIterators.jl I’m making it as a part in my reengineering of CorpusLoaders.jl. Basically a lot of …

---

## [DataFrames & IJulia?](https://discourse.julialang.org/t/dataframes-ijulia/10118)

<div class="topic-metadata">

**Author:** [@BLI](https://discourse.julialang.org/u/BLI)\
**Replies:** 9\
**Last updated:** [April 9, 2018, 6:35am UTC](https://discourse.julialang.org/t/dataframes-ijulia/10118 "2018-04-09T06:35:36Z")

</div>

I have used pandas in Python to typeset a table in Jupyter notebook (standard Anaconda installation, nothing fancy). The data consisted in a simple list of dictionaries: using pandas, this resulted in a table with dictio…

---

## [How to insert data into a field from a table using JuliaDb](https://discourse.julialang.org/t/how-to-insert-data-into-a-field-from-a-table-using-juliadb/10136)

<div class="topic-metadata">

**Author:** [@RaquelSantos](https://discourse.julialang.org/u/RaquelSantos)\
**Replies:** 1\
**Last updated:** [April 4, 2018, 12:16am UTC](https://discourse.julialang.org/t/how-to-insert-data-into-a-field-from-a-table-using-juliadb/10136 "2018-04-04T00:16:47Z")

</div>

How to insert data into a field from a table using JuliaDb?

---

## [Data set size in DataFrames with Vector{T, Missing}](https://discourse.julialang.org/t/data-set-size-in-dataframes-with-vector-t-missing/10081)

<div class="topic-metadata">

**Author:** [@mwsohn](https://discourse.julialang.org/u/mwsohn)\
**Replies:** 3\
**Last updated:** [April 1, 2018, 9:12pm UTC](https://discourse.julialang.org/t/data-set-size-in-dataframes-with-vector-t-missing/10081 "2018-04-01T21:12:34Z")

</div>

I have recently moved to DataFrames 0.11 on Julia 0.6.2 and used StatFiles.jl with FileIO.jl to convert Stata datasets to DataFrames. I was astounded by the memory used by DataFrames. It seems to me that DataFrames requi…

---

## [Is there interest in having a Julian API for the NCDC's Climate Data Online (CDO)](https://discourse.julialang.org/t/is-there-interest-in-having-a-julian-api-for-the-ncdcs-climate-data-online-cdo/9753)

<div class="topic-metadata">

**Author:** [@Nosferican](https://discourse.julialang.org/u/Nosferican)\
**Replies:** 26\
**Last updated:** [March 30, 2018, 5:34pm UTC](https://discourse.julialang.org/t/is-there-interest-in-having-a-julian-api-for-the-ncdcs-climate-data-online-cdo/9753 "2018-03-30T17:34:12Z")

</div>

For my work, I have used the National Oceanic and Atmospheric Administration (NOAA) weather and climate data extensively. I have automated several pipelines using their API in R. There is also R’s opensci/rnoaa interface…

---

## [Type of groupby(df,id) elements are Any](https://discourse.julialang.org/t/type-of-groupby-df-id-elements-are-any/9872)

<div class="topic-metadata">

**Author:** [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Replies:** 6\
**Last updated:** [March 28, 2018, 10:59pm UTC](https://discourse.julialang.org/t/type-of-groupby-df-id-elements-are-any/9872 "2018-03-28T22:59:14Z")

</div>

Can someone shed some light on this? I just don’t understand what is going wrong here. This is on DataFrames master and Julia v0.7 and v0.6.2. I don’t think this was an issue before. using DataFrames function sumall(df,…

---

## [ANN: Octo.jl: SQL Query DSL in Julia](https://discourse.julialang.org/t/ann-octo-jl-sql-query-dsl-in-julia/9912)

<div class="topic-metadata">

**Author:** [@wookyoung](https://discourse.julialang.org/u/wookyoung)\
**Replies:** 2\
**Last updated:** [March 23, 2018, 1:53pm UTC](https://discourse.julialang.org/t/ann-octo-jl-sql-query-dsl-in-julia/9912 "2018-03-23T13:53:06Z")

</div>

Hi. this package requires at least Julia 0.7-DEV, but I think that it needs some tests before to register the package. visit here and play SQL. https://github.com/wookay/Octo.jl Before using Repo, please install kind…

---

## [Status of @tranform altering columns in DataFramesMeta](https://discourse.julialang.org/t/status-of-tranform-altering-columns-in-dataframesmeta/9904)

<div class="topic-metadata">

**Author:** [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Replies:** 5\
**Last updated:** [March 22, 2018, 8:49pm UTC](https://discourse.julialang.org/t/status-of-tranform-altering-columns-in-dataframesmeta/9904 "2018-03-22T20:49:15Z")

</div>

What is the status of altering a column that has already been altered in a @tranform block with DataFramesMeta? I have a dataframe with some messy descriptions in years and months in spanish, and I need to do some clean…

---

## [\[ANN\] JDBC.jl Rejuvenated](https://discourse.julialang.org/t/ann-jdbc-jl-rejuvenated/9827)

<div class="topic-metadata">

**Author:** [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Replies:** 3\
**Last updated:** [March 20, 2018, 2:01pm UTC](https://discourse.julialang.org/t/ann-jdbc-jl-rejuvenated/9827 "2018-03-20T14:01:48Z")

</div>

Announcing JDBC.jl v0.3.0. This should provide you with a quick-and-easy way to get access to your databases with a Julia API almost immediately. It is very easy to get your data into a dataframe, for example using JD…

---

## [SASLib.jl v0.5.0 breaking changes](https://discourse.julialang.org/t/saslib-jl-v0-5-0-breaking-changes/9562)

<div class="topic-metadata">

**Author:** [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Replies:** 0\
**Last updated:** [March 7, 2018, 8:28am UTC](https://discourse.julialang.org/t/saslib-jl-v0-5-0-breaking-changes/9562 "2018-03-07T08:28:00Z")

</div>

Hi, This post is to inform everyone that the latest version of SASLib.jl (in master branch) contains breaking changes. If there isn’t much controversy, I will tag the repo within the next week. The data returned from …

---

## [CSV.read stopped working](https://discourse.julialang.org/t/csv-read-stopped-working/9159)

<div class="topic-metadata">

**Author:** [@johann.spies](https://discourse.julialang.org/u/johann.spies)\
**Replies:** 5\
**Last updated:** [March 5, 2018, 1:41pm UTC](https://discourse.julialang.org/t/csv-read-stopped-working/9159 "2018-03-05T13:41:09Z")

</div>

I have a script that I have used regularly in the past to read a CSV-file and output a PostgreSQL-dumpfile to create a table and populate it with data. This script does not work anymore and it seems that something has c…

[Previous page](https://discourse.julialang.org/c/domain/data/16.md?page=31)

[Next page](https://discourse.julialang.org/c/domain/data/16.md?page=33)