# Data

**URL:** https://discourse.julialang.org/c/domain/data/16.md?page=37

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 38

---

## [Suggested formats for saving and serialization](https://discourse.julialang.org/t/suggested-formats-for-saving-and-serialization/3230)

<div class="topic-metadata">

**Author:** [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Replies:** 8\
**Last updated:** [April 17, 2017, 7:23pm UTC](https://discourse.julialang.org/t/suggested-formats-for-saving-and-serialization/3230 "2017-04-17T19:23:29Z")

</div>

I was wondering what the suggested formats for saving and serializing are these days. I am really confused about the landscape of DataTables, DataFrames, databases, etc. I was hoping to write a few functions for the Diff…

---

## [MySQL error while precompling](https://discourse.julialang.org/t/mysql-error-while-precompling/885)

<div class="topic-metadata">

**Author:** [@Christopher\_Fisher](https://discourse.julialang.org/u/Christopher_Fisher)\
**Replies:** 10\
**Last updated:** [April 15, 2017, 2:56pm UTC](https://discourse.julialang.org/t/mysql-error-while-precompling/885 "2017-04-15T14:56:57Z")

</div>

Hi all- I am using Julia .5 on Ubuntu 16.04. When I try to use MySQL I get the following message. Is there something else I need to configure? Thanks, Chris ERROR: LoadError: LoadError: MYSQL library not found in ma…

---

## [How to handle and store large amounts of (distributed) generated data?](https://discourse.julialang.org/t/how-to-handle-and-store-large-amounts-of-distributed-generated-data/3231)

<div class="topic-metadata">

**Author:** [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Replies:** 4\
**Last updated:** [April 15, 2017, 12:15pm UTC](https://discourse.julialang.org/t/how-to-handle-and-store-large-amounts-of-distributed-generated-data/3231 "2017-04-15T12:15:39Z")

</div>

A lot of (Monte Carlo) simulations can be done simultaneously and independently on many nodes of an HPC, each generating large solutions for later analysis. However, a straight pmap will try to build a giant output that …

---

## [Feather.jl, and understanding when data is loaded into RAM](https://discourse.julialang.org/t/feather-jl-and-understanding-when-data-is-loaded-into-ram/3173)

<div class="topic-metadata">

**Author:** [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Replies:** 0\
**Last updated:** [April 11, 2017, 4:50pm UTC](https://discourse.julialang.org/t/feather-jl-and-understanding-when-data-is-loaded-into-ram/3173 "2017-04-11T16:50:51Z")

</div>

Like a lot of people, I have a great deal of use for binary serialization formats for storing tabular data. I really love Feather which is based on the new Apache Arrow format. However, its usefulness is extremely limi…

---

## [JLD takes too long reading names from a file](https://discourse.julialang.org/t/jld-takes-too-long-reading-names-from-a-file/2154)

<div class="topic-metadata">

**Author:** [@LewisHein](https://discourse.julialang.org/u/LewisHein)\
**Replies:** 2\
**Last updated:** [April 11, 2017, 9:28am UTC](https://discourse.julialang.org/t/jld-takes-too-long-reading-names-from-a-file/2154 "2017-04-11T09:28:19Z")

</div>

JLD seems to take an awfully long time just reading the names present in a JLD file – not reading values, just names. julia\> outfile = jldopen("/tmp/blah.jl", "w") Julia data file version 0.1.1: /tmp/blah.jl julia\> for…

---

## [Accessing full dataframe after a join within a Query.jl query](https://discourse.julialang.org/t/accessing-full-dataframe-after-a-join-within-a-query-jl-query/3076)

<div class="topic-metadata">

**Author:** [@tlnagy](https://discourse.julialang.org/u/tlnagy)\
**Replies:** 4\
**Last updated:** [April 7, 2017, 9:37pm UTC](https://discourse.julialang.org/t/accessing-full-dataframe-after-a-join-within-a-query-jl-query/3076 "2017-04-07T21:37:20Z")

</div>

This is transferring my question from Possible to join and then do a groupby? · Issue #104 · queryverse/Query.jl · GitHub as suggested by @davidanthoff. I have two dataframes that look like this: df1 = DataFrame(id=\[1,2…

---

## [How to write data to file with JSON.jl?](https://discourse.julialang.org/t/how-to-write-data-to-file-with-json-jl/2993)

<div class="topic-metadata">

**Author:** [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Replies:** 4\
**Last updated:** [April 1, 2017, 5:29pm UTC](https://discourse.julialang.org/t/how-to-write-data-to-file-with-json-jl/2993 "2017-04-01T17:29:18Z")

</div>

Consider the following example: using JSON people = \[Dict("name"=\>"CoolGuy", "company"=\>"tech") for i=1:1000\] companies = \[Dict("name"=\>"CoolTech", "address"=\>"Bay Area") for i=1:100\] data = Dict("people"=\>people, "co…

---

## [Kafka.jl usage](https://discourse.julialang.org/t/kafka-jl-usage/2656)

<div class="topic-metadata">

**Author:** [@Oliwia\_Wojtkowska](https://discourse.julialang.org/u/Oliwia_Wojtkowska)\
**Replies:** 15\
**Last updated:** [March 31, 2017, 1:09pm UTC](https://discourse.julialang.org/t/kafka-jl-usage/2656 "2017-03-31T13:09:02Z")

</div>

Hi there ! Does anyone use a Kafka.jl package? I have a problem with connection with one of my kafka topic, below is my code: using Kafka kc = KafkaClient("my\_host", 6667) consumer = fetch(kc, "my\_topic", 0, 0) while …

---

## [What data structure to use to hold large categorial dataset for analytics?](https://discourse.julialang.org/t/what-data-structure-to-use-to-hold-large-categorial-dataset-for-analytics/2646)

<div class="topic-metadata">

**Author:** [@ilanggear](https://discourse.julialang.org/u/ilanggear)\
**Replies:** 3\
**Last updated:** [March 14, 2017, 8:53am UTC](https://discourse.julialang.org/t/what-data-structure-to-use-to-hold-large-categorial-dataset-for-analytics/2646 "2017-03-14T08:53:12Z")

</div>

I have a dataset that is approximately 1 million rows by 40 columns of mostly categorical values, and I need to hold it in memory while I build indexes (themselves needing lots of memory) and do some ad-hoc analytics on …

---

## [DataTables : convert Nullable{String}("14:00:00") to Dates.Minute](https://discourse.julialang.org/t/datatables-convert-nullable-string-1400-to-dates-minute/2543)

<div class="topic-metadata">

**Author:** [@Fred](https://discourse.julialang.org/u/Fred)\
**Replies:** 13\
**Last updated:** [March 11, 2017, 10:05am UTC](https://discourse.julialang.org/t/datatables-convert-nullable-string-1400-to-dates-minute/2543 "2017-03-11T10:05:32Z")

</div>

Hi, I have a DataTable with two columns of time and I want to compute the time duration between these 2 columns. The problem I have is to compute a duration with Nullable type. julia\> a, b (Nullable{String}("14:00:00"…

---

## [Reading binary files in pieces](https://discourse.julialang.org/t/reading-binary-files-in-pieces/2375)

<div class="topic-metadata">

**Author:** [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Replies:** 1\
**Last updated:** [February 28, 2017, 10:29pm UTC](https://discourse.julialang.org/t/reading-binary-files-in-pieces/2375 "2017-02-28T22:29:35Z")

</div>

Hello all. I suspect the following problem is very simple from a programming perspective, but unfortunately I have so little experience with serialization/deserialization code that I’m a bit baffled about where to start…

---

## [Tips on traversing/altering time-ordered cols](https://discourse.julialang.org/t/tips-on-traversing-altering-time-ordered-cols/2226)

<div class="topic-metadata">

**Author:** [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Replies:** 0\
**Last updated:** [February 22, 2017, 1:52am UTC](https://discourse.julialang.org/t/tips-on-traversing-altering-time-ordered-cols/2226 "2017-02-22T01:52:28Z")

</div>

Using one or more of the special purpose array types / time series / time-orderable data packages, I want to touch and sometimes spread or enfold or smooth or roughen the contents of one column using itself and using on…

---

## [Minimum(df\[:value\]) and maximum(df\[:value\]) not in the documentation](https://discourse.julialang.org/t/minimum-df-value-and-maximum-df-value-not-in-the-documentation/2155)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 1\
**Last updated:** [February 17, 2017, 3:23pm UTC](https://discourse.julialang.org/t/minimum-df-value-and-maximum-df-value-not-in-the-documentation/2155 "2017-02-17T15:23:39Z")

</div>

Hello, I just noticed that both minimum(df\[:value\]) and maximum(df\[:value\]) to retrive the minimum/maximum values of a column in a dataframe are not in the documentation… as they are not the standard min() max() functio…

---

## [Suggestions for a package to read tabular data](https://discourse.julialang.org/t/suggestions-for-a-package-to-read-tabular-data/2071)

<div class="topic-metadata">

**Author:** [@zhangliye](https://discourse.julialang.org/u/zhangliye)\
**Replies:** 12\
**Last updated:** [February 13, 2017, 8:55am UTC](https://discourse.julialang.org/t/suggestions-for-a-package-to-read-tabular-data/2071 "2017-02-13T08:55:12Z")

</div>

I have thousands of \*.csv files to process, which contains table data. I usually use Python + Pandas. To get high performance, I tried to transfer from Python to Julia for some data processing work. I tested DataFrames.j…

---

## [DataFrames testing error on Windows](https://discourse.julialang.org/t/dataframes-testing-error-on-windows/1969)

<div class="topic-metadata">

**Author:** [@deepakagrawal](https://discourse.julialang.org/u/deepakagrawal)\
**Replies:** 5\
**Last updated:** [February 11, 2017, 3:06pm UTC](https://discourse.julialang.org/t/dataframes-testing-error-on-windows/1969 "2017-02-11T15:06:47Z")

</div>

INFO: Testing DataFrames WARNING: Method definition ==(Base.Nullable{S}, Base.Nullable{T}) in module Base at nullable.jl:244 overwritten in module NullableArrays at C:\\Users\\agraw\\.julia\\v0.6\\NullableArrays\\src\\operators…

---

## [Various constructors and equality for DataFrame](https://discourse.julialang.org/t/various-constructors-and-equality-for-dataframe/1551)

<div class="topic-metadata">

**Author:** [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Replies:** 5\
**Last updated:** [January 18, 2017, 2:20pm UTC](https://discourse.julialang.org/t/various-constructors-and-equality-for-dataframe/1551 "2017-01-18T14:20:19Z")

</div>

I came across this problem when writing tests for a package, for a function that produces dataframes using the DataFrames(columns::AbstractArray{T\<:Any,1}, cnames::AbstractArray{Symbol,1}) constructor. Consider this MW…

---

## [How to read range of JLD file?](https://discourse.julialang.org/t/how-to-read-range-of-jld-file/954)

<div class="topic-metadata">

**Author:** [@programista](https://discourse.julialang.org/u/programista)\
**Replies:** 19\
**Last updated:** [January 18, 2017, 1:04pm UTC](https://discourse.julialang.org/t/how-to-read-range-of-jld-file/954 "2017-01-18T13:04:28Z")

</div>

julia\> using JLD julia\> D=load("D.jld","D",(:,1)) ERROR: too few arguments in println at strings/io.jl:25

---

## [Recommendations for ODBC](https://discourse.julialang.org/t/recommendations-for-odbc/1279)

<div class="topic-metadata">

**Author:** [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Replies:** 6\
**Last updated:** [January 5, 2017, 4:12am UTC](https://discourse.julialang.org/t/recommendations-for-odbc/1279 "2017-01-05T04:12:34Z")

</div>

I need to decide (soon) between using the ODBC.jl package, or using ODBC via PyCall.jl. Back mid 2015, we’d looked at ODBC.jl, but it wasn’t ready at that point, and we ended up making our own MariaDB/MySQL wrapper for …

---

## [ODBC.jl WeakRefString{Int32} error](https://discourse.julialang.org/t/odbc-jl-weakrefstring-int32-error/1033)

<div class="topic-metadata">

**Author:** [@msravi](https://discourse.julialang.org/u/msravi)\
**Replies:** 0\
**Last updated:** [December 19, 2016, 7:46am UTC](https://discourse.julialang.org/t/odbc-jl-weakrefstring-int32-error/1033 "2016-12-19T07:46:44Z")

</div>

Hi, I have two tables in a mysql database whose schema are basically identical (see schema at the end). When I attempt to query the first one using ODBC.jl, there is no problem, and data is returned as expected. However…

---

## [When colwise generate a couple of arrays instead of one array with many elements](https://discourse.julialang.org/t/when-colwise-generate-a-couple-of-arrays-instead-of-one-array-with-many-elements/663)

<div class="topic-metadata">

**Author:** [@Hongwei](https://discourse.julialang.org/u/Hongwei)\
**Replies:** 2\
**Last updated:** [December 2, 2016, 2:19pm UTC](https://discourse.julialang.org/t/when-colwise-generate-a-couple-of-arrays-instead-of-one-array-with-many-elements/663 "2016-12-02T14:19:53Z")

</div>

Hi guys, cf = colwise(mean, test) #test is a two columns data frame I got: dump(cf) Array{Any}((2,)) 1: Array{Float64}((1,)) \[0.056307\] 2: Array{Float64}((1,)) \[0.0330923\] So I have to use \[float(cf\[i\]\[1\]) for i …

---

## [Teaching data analysis with Julia - what to do about DataFrames and all that?](https://discourse.julialang.org/t/teaching-data-analysis-with-julia-what-to-do-about-dataframes-and-all-that/422)

<div class="topic-metadata">

**Author:** [@dmbates](https://discourse.julialang.org/u/dmbates)\
**Replies:** 18\
**Last updated:** [November 21, 2016, 9:16am UTC](https://discourse.julialang.org/t/teaching-data-analysis-with-julia-what-to-do-about-dataframes-and-all-that/422 "2016-11-21T09:16:27Z")

</div>

I will be teaching a couple of workshops on fitting mixed-effects models with Julia early in 2017. The audience will know some R but probably not know any Julia. Some may know about dplyr and Hadley Wickham’s approach a…

[Previous page](https://discourse.julialang.org/c/domain/data/16.md?page=36)