# Data

**URL:** https://discourse.julialang.org/c/domain/data/16.md?page=34

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 35

---

## [Append one line to similar DataFrame gives eltype error](https://discourse.julialang.org/t/append-one-line-to-similar-dataframe-gives-eltype-error/7763)

<div class="topic-metadata">

**Author:** [@Fred](https://discourse.julialang.org/u/Fred)\
**Replies:** 3\
**Last updated:** [December 18, 2017, 9:38am UTC](https://discourse.julialang.org/t/append-one-line-to-similar-dataframe-gives-eltype-error/7763 "2017-12-18T09:38:29Z")

</div>

Hi, I try top append a line to 2 similar DataFrames, using the strategy described here before DataFrame recent update : But I have some problem because I do not manage to use append!() function without eltype error. …

---

## [Small question about Nullable, Missing and DBAPI.jl](https://discourse.julialang.org/t/small-question-about-nullable-missing-and-dbapi-jl/7783)

<div class="topic-metadata">

**Author:** [@Liso](https://discourse.julialang.org/u/Liso)\
**Replies:** 1\
**Last updated:** [December 15, 2017, 1:57pm UTC](https://discourse.julialang.org/t/small-question-about-nullable-missing-and-dbapi-jl/7783 "2017-12-15T13:57:24Z")

</div>

DBAPI says: Null Handling All values of type T that could be null should be returned as Nullable{T}. Is it still valid?

---

## [Custom formatter for a DataFrame column?](https://discourse.julialang.org/t/custom-formatter-for-a-dataframe-column/7777)

<div class="topic-metadata">

**Author:** [@airpmb](https://discourse.julialang.org/u/airpmb)\
**Replies:** 1\
**Last updated:** [December 15, 2017, 8:57am UTC](https://discourse.julialang.org/t/custom-formatter-for-a-dataframe-column/7777 "2017-12-15T08:57:31Z")

</div>

Is there a good way to customize the formatting of the values in particular columns of a DataFrame? I have a column that is using the types from Currencies.jl, which have great functionality but their default display for…

---

## [What is the future of time-indexed dataframes?](https://discourse.julialang.org/t/what-is-the-future-of-time-indexed-dataframes/7751)

<div class="topic-metadata">

**Author:** [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Replies:** 3\
**Last updated:** [December 14, 2017, 12:39am UTC](https://discourse.julialang.org/t/what-is-the-future-of-time-indexed-dataframes/7751 "2017-12-14T00:39:49Z")

</div>

After some research today, I came across this thread discussing the pros and cons of different approaches for handling time series data in dataframe-like format: https://github.com/JuliaStats/Roadmap.jl/issues/10 Can yo…

---

## [Converting missing type into float](https://discourse.julialang.org/t/converting-missing-type-into-float/7740)

<div class="topic-metadata">

**Author:** [@Fred](https://discourse.julialang.org/u/Fred)\
**Replies:** 5\
**Last updated:** [December 13, 2017, 3:45pm UTC](https://discourse.julialang.org/t/converting-missing-type-into-float/7740 "2017-12-13T15:45:14Z")

</div>

Hi, The new “missing” type introduced in DataFrames is a source of many errors in all the Julia programs I have made previously and I would like to find a way to remove this missing type. For example the competerank() …

---

## [CSV problem reading empty string column](https://discourse.julialang.org/t/csv-problem-reading-empty-string-column/7710)

<div class="topic-metadata">

**Author:** [@Daneel](https://discourse.julialang.org/u/Daneel)\
**Replies:** 2\
**Last updated:** [December 13, 2017, 8:56am UTC](https://discourse.julialang.org/t/csv-problem-reading-empty-string-column/7710 "2017-12-13T08:56:23Z")

</div>

How can I read a text data file with an empty string column? The file rarely, if ever, has a string in the column. data.dat 12/10/2016 00:00:00 0.004 3564172800.000 23:00:00 SPEC:NH3 time, Range\_F 1\_L 1, Zero\_F 1, Rang…

---

## [Deprecated csv prefix](https://discourse.julialang.org/t/deprecated-csv-prefix/7616)

<div class="topic-metadata">

**Author:** [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Replies:** 4\
**Last updated:** [December 8, 2017, 10:22pm UTC](https://discourse.julialang.org/t/deprecated-csv-prefix/7616 "2017-12-08T22:22:19Z")

</div>

Hi, Can you please clarify the newer syntax? csv""" x, y, z 1, 2, 3 """ @csv\_str and the csv"“” syntax are deprecated. Use CSV.read(IOBuffer(…)) from the CSV package instead. Why the csv prefix was deprecated, coul…

---

## [Review of presentation](https://discourse.julialang.org/t/review-of-presentation/7615)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 12\
**Last updated:** [December 8, 2017, 9:00pm UTC](https://discourse.julialang.org/t/review-of-presentation/7615 "2017-12-08T21:00:54Z")

</div>

I made this presentation focused on the data ecosystem in Julia. I just wanted to post it here for feedback so I can polish it. Thanks

---

## [Sorting seems to have some low hanging speed fruit for sorting by single column](https://discourse.julialang.org/t/sorting-seems-to-have-some-low-hanging-speed-fruit-for-sorting-by-single-column/7568)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 7\
**Last updated:** [December 6, 2017, 10:21pm UTC](https://discourse.julialang.org/t/sorting-seems-to-have-some-low-hanging-speed-fruit-for-sorting-by-single-column/7568 "2017-12-06T22:21:31Z")

</div>

I was doing some benchmarking of data.table vs DataFrames.jl. Of course, data.table is still way faster on sorting. But I found a low hanging fruit for sorting performance; it could be the backbone of a PR. Here is an MW…

---

## [Loading/writing a single element from an array in JLD](https://discourse.julialang.org/t/loading-writing-a-single-element-from-an-array-in-jld/7414)

<div class="topic-metadata">

**Author:** [@Axel\_Gagge](https://discourse.julialang.org/u/Axel_Gagge)\
**Replies:** 3\
**Last updated:** [December 3, 2017, 3:07am UTC](https://discourse.julialang.org/t/loading-writing-a-single-element-from-an-array-in-jld/7414 "2017-12-03T03:07:56Z")

</div>

I want to save and load data with JLD.jl (or HDF5.jl or something) which is is a 1-dimensional array of large elements (themselves large arrays). I want to be able to work with one or a few of the array elements at a tim…

---

## [Is there no standard way to read files with fixed width columns in the new DataFrames ecosystem?](https://discourse.julialang.org/t/is-there-no-standard-way-to-read-files-with-fixed-width-columns-in-the-new-dataframes-ecosystem/7324)

<div class="topic-metadata">

**Author:** [@NickNack](https://discourse.julialang.org/u/NickNack)\
**Replies:** 6\
**Last updated:** [November 29, 2017, 11:32am UTC](https://discourse.julialang.org/t/is-there-no-standard-way-to-read-files-with-fixed-width-columns-in-the-new-dataframes-ecosystem/7324 "2017-11-29T11:32:25Z")

</div>

I’ve been using DataFrames.readtable() to read files with fixed width columns, but that function is now deprecated in favor of CSV.read(). However it seems that CSV.read() lacks the flag to “treat consecutive whitespace …

---

## [Pretty printing of tables](https://discourse.julialang.org/t/pretty-printing-of-tables/7254)

<div class="topic-metadata">

**Author:** [@Rafal\_Machalica](https://discourse.julialang.org/u/Rafal_Machalica)\
**Replies:** 7\
**Last updated:** [November 26, 2017, 12:06am UTC](https://discourse.julialang.org/t/pretty-printing-of-tables/7254 "2017-11-26T00:06:09Z")

</div>

I’m looking for a quick solution of printing human-readable 2D tables. I thought I found a solution - NamedArrays. It meets my needs in terms of formatting, however when the table is slightly bigger (but still could fit …

---

## [Is there light at the end of the DataFrames tunnel?](https://discourse.julialang.org/t/is-there-light-at-the-end-of-the-dataframes-tunnel/6382)

<div class="topic-metadata">

**Author:** [@dmbates](https://discourse.julialang.org/u/dmbates)\
**Replies:** 36\
**Last updated:** [November 24, 2017, 8:44pm UTC](https://discourse.julialang.org/t/is-there-light-at-the-end-of-the-dataframes-tunnel/6382 "2017-11-24T20:44:27Z")

</div>

I keep trying to experiment with a version of the “DataFrames of the future”, as I understand it. So I would like to have a development version of DataFrames 0.11.0 combined with versions of Nulls, CategoricalArrays, CS…

---

## [\[ANN\] Schemata.jl (alpha)](https://discourse.julialang.org/t/ann-schemata-jl-alpha/7287)

<div class="topic-metadata">

**Author:** [@jocklawrie](https://discourse.julialang.org/u/jocklawrie)\
**Replies:** 0\
**Last updated:** [November 24, 2017, 11:56am UTC](https://discourse.julialang.org/t/ann-schemata-jl-alpha/7287 "2017-11-24T11:56:25Z")

</div>

Hi all, Here’s a package I’ve started as part of a larger project that I am beating into shape. It’s not yet registered, so you’ll need to clone it from here. I thought I’d put it out early because there has been much …

---

## [How to drop NA values with Query.jl?](https://discourse.julialang.org/t/how-to-drop-na-values-with-query-jl/7231)

<div class="topic-metadata">

**Author:** [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Replies:** 2\
**Last updated:** [November 21, 2017, 11:04pm UTC](https://discourse.julialang.org/t/how-to-drop-na-values-with-query-jl/7231 "2017-11-21T23:04:16Z")

</div>

Could you please explain why this code snippet doesn’t work? using DataFrames, Query df = DataFrame(a=\[1,2,3\], b=@data \[NA,2,3\]) @from i in df begin @where !isequal(i.b, NA) @select i @collect DataFrame en…

---

## [Create missing columns (or rows) in a NamedArray with a default value](https://discourse.julialang.org/t/create-missing-columns-or-rows-in-a-namedarray-with-a-default-value/7128)

<div class="topic-metadata">

**Author:** [@scelles](https://discourse.julialang.org/u/scelles)\
**Replies:** 2\
**Last updated:** [November 18, 2017, 2:51pm UTC](https://discourse.julialang.org/t/create-missing-columns-or-rows-in-a-namedarray-with-a-default-value/7128 "2017-11-18T14:51:40Z")

</div>

Hello, I would like to transform a Named Array (coming for FreqTable.jl freqtable) from x = NamedArray(\[2 0; 0 1; 1 2\]); idx\_rows = \["0", "1", "2"\] idx\_cols = \["0", "2"\] setnames!(x, idx\_rows, 1); setnames!(x, idx\_cols…

---

## [Using julia vector in R without copying with RCall](https://discourse.julialang.org/t/using-julia-vector-in-r-without-copying-with-rcall/6590)

<div class="topic-metadata">

**Author:** [@vipstar](https://discourse.julialang.org/u/vipstar)\
**Replies:** 2\
**Last updated:** [October 29, 2017, 10:57pm UTC](https://discourse.julialang.org/t/using-julia-vector-in-r-without-copying-with-rcall/6590 "2017-10-29T22:57:04Z")

</div>

Hi, I can move an array from R to julia without copying the array (with reval) but I was wondering if there is a way to move a julia vector to R without copying the array using RCall. I want to move the vector into a da…

---

## [HDF5 speed?](https://discourse.julialang.org/t/hdf5-speed/6584)

<div class="topic-metadata">

**Author:** [@tuckermcclure](https://discourse.julialang.org/u/tuckermcclure)\
**Replies:** 3\
**Last updated:** [October 20, 2017, 8:06pm UTC](https://discourse.julialang.org/t/hdf5-speed/6584 "2017-10-20T20:06:32Z")

</div>

I have a long-running simulation that logs to an HDF5 file on each sample. I preallocate space, and write, say, a vector to index 1 on the first sample, then to index 2 on the next sample, etc. I’m finding that logging d…

---

## [How to define empty IndexedTables?](https://discourse.julialang.org/t/how-to-define-empty-indexedtables/6508)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 1\
**Last updated:** [October 18, 2017, 8:33am UTC](https://discourse.julialang.org/t/how-to-define-empty-indexedtables/6508 "2017-10-18T08:33:18Z")

</div>

I am unable to define empty IndexedTables, e.g. using IndexedTables, IndexedTables.Table t = Table(Columns(a=Int64\[\],b=String\[\]),Int64\[\]) t\[1,"a"\] = 1 t\[1,"b"\] = 2 t\[1,"c"\] = t\[1,"a"\] + t\[1,"b"\] BoundsError: attempt to …

---

## [Does the concept of type-stability apply to DataFrames or Tables?](https://discourse.julialang.org/t/does-the-concept-of-type-stability-apply-to-dataframes-or-tables/6434)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 21\
**Last updated:** [October 17, 2017, 8:57pm UTC](https://discourse.julialang.org/t/does-the-concept-of-type-stability-apply-to-dataframes-or-tables/6434 "2017-10-17T20:57:03Z")

</div>

I understand that if you write type stable code in Julia then it’s easier to optimize and hence will result in fast running code. But I am thinking about this in the DataFrames context, say I have a function that takes …

---

## [Best practice for streaming a template file with replaced strings](https://discourse.julialang.org/t/best-practice-for-streaming-a-template-file-with-replaced-strings/6469)

<div class="topic-metadata">

**Author:** [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Replies:** 6\
**Last updated:** [October 17, 2017, 5:29pm UTC](https://discourse.julialang.org/t/best-practice-for-streaming-a-template-file-with-replaced-strings/6469 "2017-10-17T17:29:43Z")

</div>

I have a template file with a few placeholders. I want to replace the placeholders with actual values and pipe the result forward. Currently I read the whole file, find all occurrences of the placeholders and replace th…

---

## [How to use aggregate\_vec to create two output columnd JuliaDB.jl & IndexedTables?](https://discourse.julialang.org/t/how-to-use-aggregate-vec-to-create-two-output-columnd-juliadb-jl-indexedtables/6497)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 5\
**Last updated:** [October 17, 2017, 4:44pm UTC](https://discourse.julialang.org/t/how-to-use-aggregate-vec-to-create-two-output-columnd-juliadb-jl-indexedtables/6497 "2017-10-17T16:44:24Z")

</div>

Please see my MVE for creating the data needed for this question using Distributions, PooledArrays N=Int64(2e7); K=100; pool = \[@sprintf "id%03d" k for k in 1:K\] function randstrarray(pool, N) PooledArray(PooledA…

---

## [Stack overflow in DataFrames group by](https://discourse.julialang.org/t/stack-overflow-in-dataframes-group-by/6357)

<div class="topic-metadata">

**Author:** [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Replies:** 16\
**Last updated:** [October 15, 2017, 5:28pm UTC](https://discourse.julialang.org/t/stack-overflow-in-dataframes-group-by/6357 "2017-10-15T17:28:35Z")

</div>

I have a million row data frame and I am running groupby on a column of strings: by(df, :col) do sdf DataFrame(n=size(sdf,1)) end and I get the following StackOverflowError ERROR: LoadError: StackOverflowError: St…

---

## [\`Nulls.skip\` is very slow](https://discourse.julialang.org/t/nulls-skip-is-very-slow/6351)

<div class="topic-metadata">

**Author:** [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Replies:** 4\
**Last updated:** [October 15, 2017, 12:40pm UTC](https://discourse.julialang.org/t/nulls-skip-is-very-slow/6351 "2017-10-15T12:40:46Z")

</div>

Please excuse how rough this example is (but I think it is essentially correct) using Nulls using BenchmarkTools const N = 10^7 const n = 10^5 A = convert(Vector{Union{Int64,Null}}, rand(Int64, N)) …

---

## [Which are efficient data structures for querying data by name?](https://discourse.julialang.org/t/which-are-efficient-data-structures-for-querying-data-by-name/6285)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 2\
**Last updated:** [October 14, 2017, 12:08pm UTC](https://discourse.julialang.org/t/which-are-efficient-data-structures-for-querying-data-by-name/6285 "2017-10-14T12:08:34Z")

</div>

Hello, while I guess standard Array{T,n} are the most efficient data structures for positional access, I have often the need to access data by some keys. I am currently using DataFrames to access this data, using eithe…

---

## [Enumerate(dict) behavior](https://discourse.julialang.org/t/enumerate-dict-behavior/5312)

<div class="topic-metadata">

**Author:** [@sambitdash](https://discourse.julialang.org/u/sambitdash)\
**Replies:** 10\
**Last updated:** [October 11, 2017, 3:12am UTC](https://discourse.julialang.org/t/enumerate-dict-behavior/5312 "2017-10-11T03:12:00Z")

</div>

Hi All, enumerate(iter) behavior enumerates over an index and dataItem to the associated index. For associative containers like Dict one would believe the natural iteration will be over named indexes which is the (key, …

---

## [How to get DTable column names?](https://discourse.julialang.org/t/how-to-get-dtable-column-names/6307)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 1\
**Last updated:** [October 7, 2017, 12:26pm UTC](https://discourse.julialang.org/t/how-to-get-dtable-column-names/6307 "2017-10-07T12:26:19Z")

</div>

I loaded a set of CSV using JuliaDB.jl’s loadfiles.And I got a DTable, but there does not appear to be a function that can tell me the names of the columns. There is a names functions for DataFrames it feels natural to …

---

## [HDF5: Writing to a Compound Table](https://discourse.julialang.org/t/hdf5-writing-to-a-compound-table/6265)

<div class="topic-metadata">

**Author:** [@Jeremy](https://discourse.julialang.org/u/Jeremy)\
**Replies:** 0\
**Last updated:** [October 5, 2017, 3:46pm UTC](https://discourse.julialang.org/t/hdf5-writing-to-a-compound-table/6265 "2017-10-05T15:46:47Z")

</div>

How do I write data to a compound table in an HDF5 file? I’ve figured out how to create the table. src = "/home/user/Desktop/hdf5\_test.h5" h5open(src, "w") do file cols = \["foo","bar","blah","blahblahblah"\] memtype\_…

---

## [SQLAlchemy.jl Resurrected](https://discourse.julialang.org/t/sqlalchemy-jl-resurrected/6206)

<div class="topic-metadata">

**Author:** [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Replies:** 2\
**Last updated:** [October 3, 2017, 2:03pm UTC](https://discourse.julialang.org/t/sqlalchemy-jl-resurrected/6206 "2017-10-03T14:03:55Z")

</div>

As complaining about SQL is one of my favorite pastimes, I’ve found that it made my life a whole lot easier to make SQLAlchemy.jl work again. So much work has been put into sqlalchemy that it seems crazy not to use it. …

---

## [Julia DataFrames -\> plot 48 times slower than Python Pandas](https://discourse.julialang.org/t/julia-dataframes-plot-48-times-slower-than-python-pandas/6138)

<div class="topic-metadata">

**Author:** [@johann.spies](https://discourse.julialang.org/u/johann.spies)\
**Replies:** 16\
**Last updated:** [October 3, 2017, 10:28am UTC](https://discourse.julialang.org/t/julia-dataframes-plot-48-times-slower-than-python-pandas/6138 "2017-10-03T10:28:09Z")

</div>

I use the same methods and data in both Python and Julia (except that the Python plot has much more work as far as the attributes of the plot is concerned). I am fairly new to both Python Pandas and Julia DataFrames and…

[Previous page](https://discourse.julialang.org/c/domain/data/16.md?page=33)

[Next page](https://discourse.julialang.org/c/domain/data/16.md?page=35)