# Data

**URL:** https://discourse.julialang.org/c/domain/data/16.md?page=23

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 24

---

## [Aggregate deprecated use combine](https://discourse.julialang.org/t/aggregate-deprecated-use-combine/42809)

<div class="topic-metadata">

**Author:** [@jguo](https://discourse.julialang.org/u/jguo)\
**Replies:** 6\
**Last updated:** [July 9, 2020, 11:25pm UTC](https://discourse.julialang.org/t/aggregate-deprecated-use-combine/42809 "2020-07-09T23:25:28Z")

</div>

dear Julia Community, I am new to Julia. Here is my problem using aggregate function. using DataFrames df6 = DataFrame(Group = rand(\["A", "B", "C"\], 15), Variable1 = randn(15), Variable2 = rand(15)); aggregate(df6, :G…

---

## [Count cumulative number of unique elements](https://discourse.julialang.org/t/count-cumulative-number-of-unique-elements/42736)

<div class="topic-metadata">

**Author:** [@tpoisot](https://discourse.julialang.org/u/tpoisot)\
**Replies:** 1\
**Last updated:** [July 8, 2020, 3:10pm UTC](https://discourse.julialang.org/t/count-cumulative-number-of-unique-elements/42736 "2020-07-08T15:10:45Z")

</div>

I am trying to produce a species discovery curve, which is basically the number of unique species observed over time. So, at time t, the value is length(unique(\_.species\_name)) for all observations up to time t - this in…

---

## [Is there a DictTables.jl?](https://discourse.julialang.org/t/is-there-a-dicttables-jl/42700)

<div class="topic-metadata">

**Author:** [@Amin\_Yahyaabadi](https://discourse.julialang.org/u/Amin_Yahyaabadi)\
**Replies:** 7\
**Last updated:** [July 8, 2020, 1:50pm UTC](https://discourse.julialang.org/t/is-there-a-dicttables-jl/42700 "2020-07-08T13:50:54Z")

</div>

I have a problem in which the data is tabular, but one of the columns is the keys for the rest of the row. For example: │ id │ x2 │ y2 │ │ String │ Int64 │ Int64 │ ┼────────┼───────┼───────┤ │ a │ 1 │…

---

## [How to make a vector of symbols using a macro](https://discourse.julialang.org/t/how-to-make-a-vector-of-symbols-using-a-macro/42439)

<div class="topic-metadata">

**Author:** [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Replies:** 2\
**Last updated:** [July 2, 2020, 5:27pm UTC](https://discourse.julialang.org/t/how-to-make-a-vector-of-symbols-using-a-macro/42439 "2020-07-02T17:27:59Z")

</div>

How do I make a vector of symbols using a macro? macro foo() Expr(:vect, :x1, :x2) end julia\> @foo ERROR: UndefVarError: x1 not defined Clearly this making the expression \[x1, x2\]. However I would like to make the…

---

## [Cummulative death by day by country](https://discourse.julialang.org/t/cummulative-death-by-day-by-country/42080)

<div class="topic-metadata">

**Author:** [@Chen](https://discourse.julialang.org/u/Chen)\
**Replies:** 29\
**Last updated:** [June 29, 2020, 1:45pm UTC](https://discourse.julialang.org/t/cummulative-death-by-day-by-country/42080 "2020-06-29T13:45:25Z")

</div>

In R, one would use tidyverse to, for example, aggregate daily deaths of COVID-19 by day and by country \[0\]. How would one do it in Julia? Is there a package that is similar to R’s tidyverse? Or does one write a couple …

---

## [Reading large-columned data using Feather.jl is too slow](https://discourse.julialang.org/t/reading-large-columned-data-using-feather-jl-is-too-slow/40377)

<div class="topic-metadata">

**Author:** [@Jean](https://discourse.julialang.org/u/Jean)\
**Replies:** 8\
**Last updated:** [June 28, 2020, 4:09am UTC](https://discourse.julialang.org/t/reading-large-columned-data-using-feather-jl-is-too-slow/40377 "2020-06-28T04:09:07Z")

</div>

Hi, I am working with large-columned data sets. One small example is the dimension of (1452, 66584) whose data size is about 2GB. When it converted to a feather format, its size was down to 778MB. The problem is Feathe…

---

## [How do you push to production?](https://discourse.julialang.org/t/how-do-you-push-to-production/42175)

<div class="topic-metadata">

**Author:** [@MarkovChains](https://discourse.julialang.org/u/MarkovChains)\
**Replies:** 2\
**Last updated:** [June 27, 2020, 11:26pm UTC](https://discourse.julialang.org/t/how-do-you-push-to-production/42175 "2020-06-27T23:26:16Z")

</div>

So, when I write some algorithm for processing data in R, after testing, the act of pushing it to production is usually little more than basically having a pre-existing application or process call the R script. But, sin…

---

## [Repartitioning 2TB of csv into parquets](https://discourse.julialang.org/t/repartitioning-2tb-of-csv-into-parquets/27716)

<div class="topic-metadata">

**Author:** [@gabomgp](https://discourse.julialang.org/u/gabomgp)\
**Replies:** 21\
**Last updated:** [June 25, 2020, 6:09pm UTC](https://discourse.julialang.org/t/repartitioning-2tb-of-csv-into-parquets/27716 "2020-06-25T18:09:31Z")

</div>

Good morning from Colombia. I’m beginner in big data/data science, and i’m trying to do the next task: We have 2 TB of CSV from one table. We want to try to use a SQL Layer to query that data. Currently, the data is s…

---

## [Getting started with mixed models](https://discourse.julialang.org/t/getting-started-with-mixed-models/41998)

<div class="topic-metadata">

**Author:** [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Replies:** 4\
**Last updated:** [June 24, 2020, 10:36pm UTC](https://discourse.julialang.org/t/getting-started-with-mixed-models/41998 "2020-06-24T22:36:18Z")

</div>

I think I’m missing something very basic with MixedModels… julia\> using MixedModels, StatsBase julia\> t = (y = rand(100), x = rand(100)); julia\> fit(MixedModel, @formula(y ~ x), t) ERROR: MethodError: no method matchi…

---

## [Reading outlook .msg extension file through Julia](https://discourse.julialang.org/t/reading-outlook-msg-extension-file-through-julia/41930)

<div class="topic-metadata">

**Author:** [@akscse](https://discourse.julialang.org/u/akscse)\
**Replies:** 1\
**Last updated:** [June 23, 2020, 2:20pm UTC](https://discourse.julialang.org/t/reading-outlook-msg-extension-file-through-julia/41930 "2020-06-23T14:20:02Z")

</div>

I need to do text analysis on an email dump. This dump contains outlook messages in .msg extension. Could you please suggest if there is some standard library to read .msg extensions. Thanks

---

## [How to get the values in NDSparse as a matrix](https://discourse.julialang.org/t/how-to-get-the-values-in-ndsparse-as-a-matrix/41762)

<div class="topic-metadata">

**Author:** [@Stephen](https://discourse.julialang.org/u/Stephen)\
**Replies:** 0\
**Last updated:** [June 20, 2020, 2:01am UTC](https://discourse.julialang.org/t/how-to-get-the-values-in-ndsparse-as-a-matrix/41762 "2020-06-20T02:01:07Z")

</div>

julia\> Distance = loadndsparse("distance matrix.csv",indexcols = 1) 1-d NDSparse with 5 values (12 field named tuples): │ BO BR CO HA MN NA NH NL PO PR SP WO ─────┼───────────────────────────…

---

## [Julia equivalent of Python (numpy.savetxt) and Matlab (save) commands?](https://discourse.julialang.org/t/julia-equivalent-of-python-numpy-savetxt-and-matlab-save-commands/41738)

<div class="topic-metadata">

**Author:** [@dynova](https://discourse.julialang.org/u/dynova)\
**Replies:** 5\
**Last updated:** [June 19, 2020, 8:37pm UTC](https://discourse.julialang.org/t/julia-equivalent-of-python-numpy-savetxt-and-matlab-save-commands/41738 "2020-06-19T20:37:39Z")

</div>

What is the Julia equivalent of the following Python and MATLAB codes? I’m looking for identical formatting in the output. After searching the documentation (e.g. JLD, CSV, DataFrames), I have not found a satisfactory …

---

## [ANN: RegressionTables.jl produces publication-quality regression tables](https://discourse.julialang.org/t/ann-regressiontables-jl-produces-publication-quality-regression-tables/7516)

<div class="topic-metadata">

**Author:** [@jmboehm](https://discourse.julialang.org/u/jmboehm)\
**Replies:** 41\
**Last updated:** [June 19, 2020, 7:13pm UTC](https://discourse.julialang.org/t/ann-regressiontables-jl-produces-publication-quality-regression-tables/7516 "2020-06-19T19:13:27Z")

</div>

Hi all, I’ve been working on RegressionTables.jl, a package that produces regression tables as seen in scientific journals (similar to R’s stargazer and Stata’s esttab). It currently works with output from the absolute…

---

## [Some tweaks about binary I/O plus some conversions](https://discourse.julialang.org/t/some-tweaks-about-binary-i-o-plus-some-conversions/41586)

<div class="topic-metadata">

**Author:** [@Xijiang\_Yu](https://discourse.julialang.org/u/Xijiang_Yu)\
**Replies:** 4\
**Last updated:** [June 18, 2020, 10:11pm UTC](https://discourse.julialang.org/t/some-tweaks-about-binary-i-o-plus-some-conversions/41586 "2020-06-18T22:11:59Z")

</div>

Background The genotype data can be stored like: 01212 where, 0 and 2 are homozygotes, 1 is the heterozyges. Frequently they will be read into a Float64 matrix for some dot operations and GEMM things. My question is th…

---

## [Apache Zeppelin and Julia](https://discourse.julialang.org/t/apache-zeppelin-and-julia/41587)

<div class="topic-metadata">

**Author:** [@lungben](https://discourse.julialang.org/u/lungben)\
**Replies:** 0\
**Last updated:** [June 17, 2020, 12:30pm UTC](https://discourse.julialang.org/t/apache-zeppelin-and-julia/41587 "2020-06-17T12:30:15Z")

</div>

Hi, I recently came across Apache Zeppelin, which is somewhat similar to Jupyter (and actually supports Jupyter / IPython notebooks), but more focussed on big data and multi-user capabilities. Especially the latter make…

---

## [Reading parquet very slow](https://discourse.julialang.org/t/reading-parquet-very-slow/41344)

<div class="topic-metadata">

**Author:** [@freeman](https://discourse.julialang.org/u/freeman)\
**Replies:** 4\
**Last updated:** [June 14, 2020, 8:48am UTC](https://discourse.julialang.org/t/reading-parquet-very-slow/41344 "2020-06-14T08:48:59Z")

</div>

I have a parquet file, about 100mb compressed. In python, reading it with pd.read\_parquet(path) takes about 1 second. The file is 12GB in memory. In Julia this: using DataFrames, ParquetFiles df = DataFrame(lo…

---

## [Custom dataset](https://discourse.julialang.org/t/custom-dataset/41329)

<div class="topic-metadata">

**Author:** [@SubhadityaMukherjee](https://discourse.julialang.org/u/SubhadityaMukherjee)\
**Replies:** 2\
**Last updated:** [June 14, 2020, 6:42am UTC](https://discourse.julialang.org/t/custom-dataset/41329 "2020-06-14T06:42:01Z")

</div>

Hello! I am new here. I have been trying to use Metalhead.jl/Flux.jl to train custom image classifiers but I cant seem to be able to load data from a folder directly. (Not part of the standard datasets). Say I have a f…

---

## [Add object to existing BSON file?](https://discourse.julialang.org/t/add-object-to-existing-bson-file/41305)

<div class="topic-metadata">

**Author:** [@e3c6](https://discourse.julialang.org/u/e3c6)\
**Replies:** 1\
**Last updated:** [June 13, 2020, 1:44am UTC](https://discourse.julialang.org/t/add-object-to-existing-bson-file/41305 "2020-06-13T01:44:25Z")

</div>

Suppose I saved a BSON file with BSON.@save "path/to/file.bson" obj1 obj2 . Then later on I create a new obj3 and I want to also add to this file, as if I had executed BSON.@save "path/to/file.bson" obj1 obj2 obj3 …

---

## [Portability of serialized files](https://discourse.julialang.org/t/portability-of-serialized-files/24733)

<div class="topic-metadata">

**Author:** [@tencnivel](https://discourse.julialang.org/u/tencnivel)\
**Replies:** 2\
**Last updated:** [May 30, 2019, 10:25pm UTC](https://discourse.julialang.org/t/portability-of-serialized-files/24733 "2019-05-30T22:25:14Z")

</div>

I’m trying to understand what the following actually means from the documentation of julia V1.1.0 about serialization https://pkg.julialang.org/docs/julia/THl1k/1.1.0/stdlib/Serialization.html: The read-back value will…

---

## [Workflow Suggestions: Post-processing ANSYS mechanical engineering data with Julia](https://discourse.julialang.org/t/workflow-suggestions-post-processing-ansys-mechanical-engineering-data-with-julia/41014)

<div class="topic-metadata">

**Author:** [@Nathan\_Boyer](https://discourse.julialang.org/u/Nathan_Boyer)\
**Replies:** 0\
**Last updated:** [June 8, 2020, 7:29pm UTC](https://discourse.julialang.org/t/workflow-suggestions-post-processing-ansys-mechanical-engineering-data-with-julia/41014 "2020-06-08T19:29:57Z")

</div>

I am new to both ANSYS and Julia. I would like to use Julia to process static structural and heat transfer data from the commercial ANSYS finite element software. I have computed several quantities of interest within ANS…

---

## [Replacing values in a data frame](https://discourse.julialang.org/t/replacing-values-in-a-data-frame/40937)

<div class="topic-metadata">

**Author:** [@Chris\_Anderson](https://discourse.julialang.org/u/Chris_Anderson)\
**Replies:** 7\
**Last updated:** [June 7, 2020, 7:11pm UTC](https://discourse.julialang.org/t/replacing-values-in-a-data-frame/40937 "2020-06-07T19:11:13Z")

</div>

Don’t know if this is the right category for this… I have a dataframe with a column containing averages calculated with the mean function. These values go out several decimal places, but I want to use round() to set it…

---

## [Using the groupby function](https://discourse.julialang.org/t/using-the-groupby-function/40839)

<div class="topic-metadata">

**Author:** [@Chris\_Anderson](https://discourse.julialang.org/u/Chris_Anderson)\
**Replies:** 12\
**Last updated:** [June 6, 2020, 4:12pm UTC](https://discourse.julialang.org/t/using-the-groupby-function/40839 "2020-06-06T16:12:33Z")

</div>

I have a dataframe that I would like to group by a categorical variable (4 values, let’s say a, b, c and d) and then by a continuous variable (values between 0 and 40). I’d like the continuous variable to be in groups t…

---

## [Filtering a DataFrame column using Regex](https://discourse.julialang.org/t/filtering-a-dataframe-column-using-regex/40753)

<div class="topic-metadata">

**Author:** [@alejandromerchan](https://discourse.julialang.org/u/alejandromerchan)\
**Replies:** 2\
**Last updated:** [June 4, 2020, 6:43pm UTC](https://discourse.julialang.org/t/filtering-a-dataframe-column-using-regex/40753 "2020-06-04T18:43:18Z")

</div>

I have a DataFrame that has this general structure using DataFrames df = DataFrame(Sites = \["N-1", "N-1", "A", "B", "N-2"\], Other = \[randn(5)\]) How can I filter out the rows that are named “N-” on the column :Sites? I…

---

## [Dispatch on DataFrame columns](https://discourse.julialang.org/t/dispatch-on-dataframe-columns/40689)

<div class="topic-metadata">

**Author:** [@lungben](https://discourse.julialang.org/u/lungben)\
**Replies:** 6\
**Last updated:** [June 4, 2020, 5:26pm UTC](https://discourse.julialang.org/t/dispatch-on-dataframe-columns/40689 "2020-06-04T17:26:23Z")

</div>

Hi, I have a function on a DataFrame which should behave differently depending on the names / eltypes of the DataFrame columns. Is this possible using a multiple-dispatch like pattern? Example: using DataFrames functi…

---

## [Bell Curve Overlay (Gadfly)](https://discourse.julialang.org/t/bell-curve-overlay-gadfly/40640)

<div class="topic-metadata">

**Author:** [@Chris\_Anderson](https://discourse.julialang.org/u/Chris_Anderson)\
**Replies:** 4\
**Last updated:** [June 3, 2020, 11:37pm UTC](https://discourse.julialang.org/t/bell-curve-overlay-gadfly/40640 "2020-06-03T23:37:49Z")

</div>

I have data that I’ve plotted using Geom.histogram. I’m trying to determine, visually, how closely it fits a normal distribution. I realize there are other more quantitative methods to test fit to a distribution, but f…

---

## [Parsing Column values of different types](https://discourse.julialang.org/t/parsing-column-values-of-different-types/40589)

<div class="topic-metadata">

**Author:** [@Chris\_Anderson](https://discourse.julialang.org/u/Chris_Anderson)\
**Replies:** 2\
**Last updated:** [June 2, 2020, 11:47pm UTC](https://discourse.julialang.org/t/parsing-column-values-of-different-types/40589 "2020-06-02T23:47:40Z")

</div>

I have a side job/project that involves analyzing a large data set. I am leveraging this project to gain competence with programming generally and Julia specifically, meaning the learning curve is steep as I work throug…

---

## [\[DataFrames Question\]: Transpose of dataframe](https://discourse.julialang.org/t/dataframes-question-transpose-of-dataframe/28524)

<div class="topic-metadata">

**Author:** [@Patrick\_Mineault](https://discourse.julialang.org/u/Patrick_Mineault)\
**Replies:** 2\
**Last updated:** [June 2, 2020, 2:41am UTC](https://discourse.julialang.org/t/dataframes-question-transpose-of-dataframe/28524 "2020-06-02T02:41:27Z")

</div>

Question: How do I transpose a dataframe (columns become rows, rows become columns)?

---

## [\[DataFrames Question\]: Inner join DataFrame Array?](https://discourse.julialang.org/t/dataframes-question-inner-join-dataframe-array/40412)

<div class="topic-metadata">

**Author:** [@Xijiang\_Yu](https://discourse.julialang.org/u/Xijiang_Yu)\
**Replies:** 4\
**Last updated:** [May 29, 2020, 2:24pm UTC](https://discourse.julialang.org/t/dataframes-question-inner-join-dataframe-array/40412 "2020-05-29T14:24:46Z")

</div>

Question: Suppose I have an array of DataFrame t\[1:10\]. What is the neatest way to join them? innerjoin(t\[1\], t\[2\], ..., on = :tag, ...) # or df = t\[1\] for i in 2:10 df = innerjoin(df, t\[i\], on =...) end

---

## [How to convert ODBC.API.SQLGUID to String?](https://discourse.julialang.org/t/how-to-convert-odbc-api-sqlguid-to-string/39636)

<div class="topic-metadata">

**Author:** [@ryszard314159](https://discourse.julialang.org/u/ryszard314159)\
**Replies:** 1\
**Last updated:** [May 27, 2020, 4:00pm UTC](https://discourse.julialang.org/t/how-to-convert-odbc-api-sqlguid-to-string/39636 "2020-05-27T16:00:32Z")

</div>

Is there a way to convert SQLGUID to String? Apparently there convert() does not have corresponging method. julia\> x ODBC.API.SQLGUID(0x5715b932, 0x4bc5, 0x453c, (0x9d, 0xdb, 0x04, 0xe9, 0x2f, 0x4b, 0x69, 0x87)) julia\>…

---

## [Counts of unique values per group in a DataFrame](https://discourse.julialang.org/t/counts-of-unique-values-per-group-in-a-dataframe/40088)

<div class="topic-metadata">

**Author:** [@kevin.squire](https://discourse.julialang.org/u/kevin.squire)\
**Replies:** 3\
**Last updated:** [May 25, 2020, 3:25pm UTC](https://discourse.julialang.org/t/counts-of-unique-values-per-group-in-a-dataframe/40088 "2020-05-25T15:25:27Z")

</div>

Hi, I’m trying to determine unique counts of values in a column per group in a DataFrame. As an example, given the following: using DataFrames julia\> df = DataFrame(lab = \[repeat(\["Lab1"\], 4)...; repeat(\["Lab2"\], 5).…

[Previous page](https://discourse.julialang.org/c/domain/data/16.md?page=22)

[Next page](https://discourse.julialang.org/c/domain/data/16.md?page=24)