# Data

**URL:** https://discourse.julialang.org/c/domain/data/16.md?page=28

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 29

---

## [S3 object stores - current state?](https://discourse.julialang.org/t/s3-object-stores-current-state/24437)

<div class="topic-metadata">

**Author:** [@johnh](https://discourse.julialang.org/u/johnh)\
**Replies:** 7\
**Last updated:** [May 22, 2019, 1:34am UTC](https://discourse.julialang.org/t/s3-object-stores-current-state/24437 "2019-05-22T01:34:25Z")

</div>

What is the current state of packages in Julia for accessing S3 object stores. In particular CEPH. Are people just using boto with PyCall? Forgive me if I have asked this one before. Goldfish bowl memory.

---

## [JuliaDB: efficiently rank subgroups](https://discourse.julialang.org/t/juliadb-efficiently-rank-subgroups/24299)

<div class="topic-metadata">

**Author:** [@jade\_mackay](https://discourse.julialang.org/u/jade_mackay)\
**Replies:** 2\
**Last updated:** [May 20, 2019, 1:04am UTC](https://discourse.julialang.org/t/juliadb-efficiently-rank-subgroups/24299 "2019-05-20T01:04:37Z")

</div>

Hi, I would be grateful for some advice on how to create a column containing the ordinal rank of a score grouped by another column. For example: using JuliaDB using StatsBase Nx, Ny = 4, 3 x = repeat(1:Nx,inner=Ny) # l…

---

## [Joining two DataFrames with same columns](https://discourse.julialang.org/t/joining-two-dataframes-with-same-columns/5671)

<div class="topic-metadata">

**Author:** [@IljaK91](https://discourse.julialang.org/u/IljaK91)\
**Replies:** 2\
**Last updated:** [May 15, 2019, 5:49pm UTC](https://discourse.julialang.org/t/joining-two-dataframes-with-same-columns/5671 "2019-05-15T17:49:34Z")

</div>

Hi there, I would like to join two DataFrames that have the same column, let’s say: DataFrameA: country GDP Germany 1 DataFrameB: country GDP USA 2 The resulting DataFrame should be: country GDP …

---

## [CSV.read() deprecated keyword argument](https://discourse.julialang.org/t/csv-read-deprecated-keyword-argument/24095)

<div class="topic-metadata">

**Author:** [@VincentTam](https://discourse.julialang.org/u/VincentTam)\
**Replies:** 3\
**Last updated:** [May 11, 2019, 9:16am UTC](https://discourse.julialang.org/t/csv-read-deprecated-keyword-argument/24095 "2019-05-11T09:16:41Z")

</div>

In Hayden Klok’s sample code, there’s a deprecated argument for CSV.read(). # from https://github.com/h-Klok/StatsWithJuliaBook/blob/master/1\_chapter/usingR.jl#L3-L5 using CSV, DataFrames, RCall data1 = CSV.read("machin…

---

## [Bug with Union type in Query.jl?](https://discourse.julialang.org/t/bug-with-union-type-in-query-jl/23865)

<div class="topic-metadata">

**Author:** [@glwc](https://discourse.julialang.org/u/glwc)\
**Replies:** 2\
**Last updated:** [May 5, 2019, 1:10pm UTC](https://discourse.julialang.org/t/bug-with-union-type-in-query-jl/23865 "2019-05-05T13:10:05Z")

</div>

I’d appreciate help to understand the following. The particular code here with a problem is using a @select expression to extract the second number embedded in a string in DataFrame column elements which have the type …

---

## [Avoid splat (...) operator for keywords arguments?](https://discourse.julialang.org/t/avoid-splat-operator-for-keywords-arguments/23725)

<div class="topic-metadata">

**Author:** [@Eric\_Chen](https://discourse.julialang.org/u/Eric_Chen)\
**Replies:** 1\
**Last updated:** [May 1, 2019, 8:36am UTC](https://discourse.julialang.org/t/avoid-splat-operator-for-keywords-arguments/23725 "2019-05-01T08:36:30Z")

</div>

from: Performance tips Avoid the splat ( ... ) operator for keyword arguments. However, from official Julia guide: I dont find this \* Avoid the splat ( ... ) operator for keyword arguments. Question: Is the off…

---

## [Is my understanding of a data reading benchmark correct?](https://discourse.julialang.org/t/is-my-understanding-of-a-data-reading-benchmark-correct/23611)

<div class="topic-metadata">

**Author:** [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Replies:** 4\
**Last updated:** [April 28, 2019, 7:05am UTC](https://discourse.julialang.org/t/is-my-understanding-of-a-data-reading-benchmark-correct/23611 "2019-04-28T07:05:18Z")

</div>

So lately I’ve spend a lot of time developing a tool reading data from files, and I just want to understand if my benchmarking procedure is correct. Currently I do something like: using Benchmarktools @benchmark funct…

---

## [Creating a COM connection](https://discourse.julialang.org/t/creating-a-com-connection/23570)

<div class="topic-metadata">

**Author:** [@FrancisKing](https://discourse.julialang.org/u/FrancisKing)\
**Replies:** 1\
**Last updated:** [April 26, 2019, 9:44pm UTC](https://discourse.julialang.org/t/creating-a-com-connection/23570 "2019-04-26T21:44:07Z")

</div>

Hello. Since Julia is able to do all kinds of things, I was wondering if it can also do this. In Python, VBA, and C# I can connect to COM enabled software such as Microsoft Office and transport planning tools such as S…

---

## [Query.jl: how to filter on derived quantities of a group?](https://discourse.julialang.org/t/query-jl-how-to-filter-on-derived-quantities-of-a-group/23486)

<div class="topic-metadata">

**Author:** [@gajomi](https://discourse.julialang.org/u/gajomi)\
**Replies:** 3\
**Last updated:** [April 24, 2019, 10:54pm UTC](https://discourse.julialang.org/t/query-jl-how-to-filter-on-derived-quantities-of-a-group/23486 "2019-04-24T22:54:33Z")

</div>

I am having a hard time trying to figure out how to filter by a derived quantity of a group using either the standalone or linq syntax. For example, I might want to group by some column then select only rows in that grou…

---

## [Query.@join with repeated names](https://discourse.julialang.org/t/query-join-with-repeated-names/23474)

<div class="topic-metadata">

**Author:** [@heliosdrm](https://discourse.julialang.org/u/heliosdrm)\
**Replies:** 1\
**Last updated:** [April 24, 2019, 5:39pm UTC](https://discourse.julialang.org/t/query-join-with-repeated-names/23474 "2019-04-24T17:39:04Z")

</div>

Say I have this: julia\> df1 = DataFrame(id=\[1,2,3,4\], a=\[1.,2.,3.,4.\]) 4×2 DataFrame │ Row │ id │ a │ │ │ Int64 │ Float64 │ ├─────┼───────┼─────────┤ │ 1 │ 1 │ 1.0 │ │ 2 │ 2 │ 2.0 │ │ 3 …

---

## [How fast is binary reading capabilities in Julia compared with other languages?](https://discourse.julialang.org/t/how-fast-is-binary-reading-capabilities-in-julia-compared-with-other-languages/23401)

<div class="topic-metadata">

**Author:** [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Replies:** 11\
**Last updated:** [April 23, 2019, 6:18pm UTC](https://discourse.julialang.org/t/how-fast-is-binary-reading-capabilities-in-julia-compared-with-other-languages/23401 "2019-04-23T18:18:14Z")

</div>

I am specifically talking about reading binary files, so nothing with CSV. Is Julia one of the fastest in this respect or would I still see much better performance by making it in c++? Currently I am for an example abl…

---

## [What's the difference between Null, NA, #NULL, nothing, ""](https://discourse.julialang.org/t/whats-the-difference-between-null-na-null-nothing/6468)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 6\
**Last updated:** [April 17, 2019, 10:25am UTC](https://discourse.julialang.org/t/whats-the-difference-between-null-na-null-nothing/6468 "2019-04-17T10:25:42Z")

</div>

I am a bit lost on the many ways to represent the “empty” concept in Julia… What is the difference between Null (from Nulls.jl), NA (From DataFrames), #NULL from NullableArrays, Nullable container (from base), nothing (…

---

## [Select and rename columns in IndexedTables.jl / JuliaDB.jl](https://discourse.julialang.org/t/select-and-rename-columns-in-indexedtables-jl-juliadb-jl/23077)

<div class="topic-metadata">

**Author:** [@qsong](https://discourse.julialang.org/u/qsong)\
**Replies:** 0\
**Last updated:** [April 12, 2019, 1:22pm UTC](https://discourse.julialang.org/t/select-and-rename-columns-in-indexedtables-jl-juliadb-jl/23077 "2019-04-12T13:22:24Z")

</div>

With dplyr experience, I am just learning and testing IndexedTables.jl / JuliaDB.jl. The following seems verbose. (using https://github.com/oxinabox/Pipe.jl) julia\> a = table(\[1,1,1,2,2,2\], \[1,1,2,2,1,1\], \[1,4,2,5,6,7\],…

---

## [Unravelling 'nativized' xml to readable string format](https://discourse.julialang.org/t/unravelling-nativized-xml-to-readable-string-format/22869)

<div class="topic-metadata">

**Author:** [@hustf](https://discourse.julialang.org/u/hustf)\
**Replies:** 2\
**Last updated:** [April 7, 2019, 5:25pm UTC](https://discourse.julialang.org/t/unravelling-nativized-xml-to-readable-string-format/22869 "2019-04-07T17:25:11Z")

</div>

I hope for ideas or help for unravelling some probably simple obfuscation of xml files. This requires some string work intuition, and I’m just stuck. Background: Mathcad is one of those classic engineering tools we lov…

---

## [Reading (big) ascii files](https://discourse.julialang.org/t/reading-big-ascii-files/22697)

<div class="topic-metadata">

**Author:** [@Paul18fr](https://discourse.julialang.org/u/Paul18fr)\
**Replies:** 11\
**Last updated:** [April 5, 2019, 8:12am UTC](https://discourse.julialang.org/t/reading-big-ascii-files/22697 "2019-04-05T08:12:27Z")

</div>

Dear All After reading an article speaking about Julia, I decided to have a look on it both to deal with huge amount of data (through hdf5 files both in I/O) and to perform different type of calculations; nevertheless I…

---

## [Error with Openpyxl: Pyimport is not in current path](https://discourse.julialang.org/t/error-with-openpyxl-pyimport-is-not-in-current-path/22460)

<div class="topic-metadata">

**Author:** [@Rajat\_Sanyal](https://discourse.julialang.org/u/Rajat_Sanyal)\
**Replies:** 14\
**Last updated:** [April 2, 2019, 11:01am UTC](https://discourse.julialang.org/t/error-with-openpyxl-pyimport-is-not-in-current-path/22460 "2019-04-02T11:01:27Z")

</div>

I am looking for a package which can read and write an excel file cell by cell. I know openpyxl do the job. Is there any other better package in Julia which does the same job? Now, I am trying to install it, so I used t…

---

## [DataFrame type instability - Unneccesary memory allocation](https://discourse.julialang.org/t/dataframe-type-instability-unneccesary-memory-allocation/22465)

<div class="topic-metadata">

**Author:** [@Eric\_Chen](https://discourse.julialang.org/u/Eric_Chen)\
**Replies:** 1\
**Last updated:** [March 28, 2019, 5:27pm UTC](https://discourse.julialang.org/t/dataframe-type-instability-unneccesary-memory-allocation/22465 "2019-03-28T17:27:34Z")

</div>

I am running Julia 1.1, here is my Code using BenchmarkTools using DataFrames df = DataFrame(rand(10\_000\_000, 10)); Threads.nthreads() function singlethread(df) N = nrow(df) M = ncol(df) cumsum = 0.0 …

---

## [How to reduce the number of DataFrames rows printed in IJulia?](https://discourse.julialang.org/t/how-to-reduce-the-number-of-dataframes-rows-printed-in-ijulia/21953)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 3\
**Last updated:** [March 23, 2019, 10:32am UTC](https://discourse.julialang.org/t/how-to-reduce-the-number-of-dataframes-rows-printed-in-ijulia/21953 "2019-03-23T10:32:31Z")

</div>

I want to reduce the number of rows printed by default for a DataFrame in Jupyter. How do I do that?

---

## [How to obtain the number of rows in each group using DataFramesMeta?](https://discourse.julialang.org/t/how-to-obtain-the-number-of-rows-in-each-group-using-dataframesmeta/21955)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 2\
**Last updated:** [March 17, 2019, 6:50am UTC](https://discourse.julialang.org/t/how-to-obtain-the-number-of-rows-in-each-group-using-dataframesmeta/21955 "2019-03-17T06:50:16Z")

</div>

using DataFrames, DataFramesMeta a = DataFrame(a = rand(1:10, 100)) @\> begin a @by(:a, count = length(:a)) end is there a more convenient function like R’s dplyr::n() which returns the number of rows? Or even, R…

---

## [Ways to order a \`DataFrame\` by mixed order?](https://discourse.julialang.org/t/ways-to-order-a-dataframe-by-mixed-order/21952)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 2\
**Last updated:** [March 17, 2019, 4:26am UTC](https://discourse.julialang.org/t/ways-to-order-a-dataframe-by-mixed-order/21952 "2019-03-17T04:26:52Z")

</div>

I wanted to order a DataFrame by certain mixed order, e.g. i want to order a DataFrame by a\_str in descending order followed by b\_str in ascending order Setting up data using DataFrames, Query a\_str = string.(rand(UI…

---

## [Handle escaped quotes in CSVFiles/TextParse.jl](https://discourse.julialang.org/t/handle-escaped-quotes-in-csvfiles-textparse-jl/21824)

<div class="topic-metadata">

**Author:** [@chaydont](https://discourse.julialang.org/u/chaydont)\
**Replies:** 1\
**Last updated:** [March 13, 2019, 4:48pm UTC](https://discourse.julialang.org/t/handle-escaped-quotes-in-csvfiles-textparse-jl/21824 "2019-03-13T16:48:30Z")

</div>

Is it possible to read a csv that has quote-escaped quotes with CSVFiles, without keeping the 2 quotes in the string? I’m trying to do something like this : data = """ 1980-01-01T00:00:00.000Z\\t5\\tC\\t"str…

---

## [wrangling large json files](https://discourse.julialang.org/t/wrangling-large-json-files/18505)

<div class="topic-metadata">

**Author:** [@gvdr](https://discourse.julialang.org/u/gvdr)\
**Replies:** 5\
**Last updated:** [March 12, 2019, 2:48pm UTC](https://discourse.julialang.org/t/wrangling-large-json-files/18505 "2019-03-12T14:48:10Z")

</div>

Dear All, I’m handling a large JSON file (in my case, a 100GB of ndjson). I would need to query it in various and still TBD ways. In my lucky case we can rely on a fixed schema, which is nested (although not deep, we go…

---

## [DataFrame transformation question](https://discourse.julialang.org/t/dataframe-transformation-question/21742)

<div class="topic-metadata">

**Author:** [@johann.spies](https://discourse.julialang.org/u/johann.spies)\
**Replies:** 2\
**Last updated:** [March 11, 2019, 4:30pm UTC](https://discourse.julialang.org/t/dataframe-transformation-question/21742 "2019-03-11T16:30:35Z")

</div>

What I want to do is something like “unstack” but I do not know how to do it. A simple example: l = DataFrame(a = \["a", "a", "b", "b"\], c = \["ZA", "ZM", "BW", "ZA"\]) 4×2 DataFrame │ Row │ a │ c │ │ │ Str…

---

## [Special case handling - alternatives to if/else?](https://discourse.julialang.org/t/special-case-handling-alternatives-to-if-else/21608)

<div class="topic-metadata">

**Author:** [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Replies:** 5\
**Last updated:** [March 7, 2019, 11:49pm UTC](https://discourse.julialang.org/t/special-case-handling-alternatives-to-if-else/21608 "2019-03-07T23:49:20Z")

</div>

I have an unruly and monster data table from collaborators, and I’m writing a script to clean it up. The table is updated on a semi-regular basis, and there’s no option to get it in a better form, so I’m stuck trying to …

---

## [Julia 1.0.3 segfaults when trying to read a csv DataFrame with non-quoted headers and strings](https://discourse.julialang.org/t/julia-1-0-3-segfaults-when-trying-to-read-a-csv-dataframe-with-non-quoted-headers-and-strings/21547)

<div class="topic-metadata">

**Author:** [@tanhevg](https://discourse.julialang.org/u/tanhevg)\
**Replies:** 2\
**Last updated:** [March 7, 2019, 6:02am UTC](https://discourse.julialang.org/t/julia-1-0-3-segfaults-when-trying-to-read-a-csv-dataframe-with-non-quoted-headers-and-strings/21547 "2019-03-07T06:02:36Z")

</div>

Hello, My code goes: using Queryverse df = load("my\_file.csv") |\> DataFrame And the file contents are: query\_domain,template\_domain,tm\_score d1qbaa2,d2a73b2,0.52434 d1qbaa2,d2wnxa1,0.51702 d1qbaa2,d1p35a\_,0.50272 d1q…

---

## [JuliaDB: How to find the number of rows and columns?](https://discourse.julialang.org/t/juliadb-how-to-find-the-number-of-rows-and-columns/21406)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 1\
**Last updated:** [March 3, 2019, 2:11am UTC](https://discourse.julialang.org/t/juliadb-how-to-find-the-number-of-rows-and-columns/21406 "2019-03-03T02:11:31Z")

</div>

I loaded a JuliaDB table, but how do I find the number of rows and columns? size doesn’t work.

---

## [Engarde - Python defensive data analysis](https://discourse.julialang.org/t/engarde-python-defensive-data-analysis/21334)

<div class="topic-metadata">

**Author:** [@johnh](https://discourse.julialang.org/u/johnh)\
**Replies:** 3\
**Last updated:** [March 1, 2019, 10:42am UTC](https://discourse.julialang.org/t/engarde-python-defensive-data-analysis/21334 "2019-03-01T10:42:56Z")

</div>

I went to a Meet up o data engineering at https://www.quantumblack.com/ Engarde was mentioned: Engarde! — engarde 0.4.0+9.ge7ea040 documentation Looks very useful. Is there anything similar in Julia?

---

## [Sum dimensions of matrix](https://discourse.julialang.org/t/sum-dimensions-of-matrix/21098)

<div class="topic-metadata">

**Author:** [@nik.ltin](https://discourse.julialang.org/u/nik.ltin)\
**Replies:** 11\
**Last updated:** [February 23, 2019, 2:25pm UTC](https://discourse.julialang.org/t/sum-dimensions-of-matrix/21098 "2019-02-23T14:25:20Z")

</div>

Hi there, A bit new to Julia so this might be a very basic question: I have a file with a 3 dimensional array (x,y,z), and I’d like to sum certain values from the z dimension to later on pull from and graph. For example…

---

## [Query.jl: How can I write type-preserving methods for iterable tables](https://discourse.julialang.org/t/query-jl-how-can-i-write-type-preserving-methods-for-iterable-tables/19956)

<div class="topic-metadata">

**Author:** [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Replies:** 2\
**Last updated:** [February 13, 2019, 5:46am UTC](https://discourse.julialang.org/t/query-jl-how-can-i-write-type-preserving-methods-for-iterable-tables/19956 "2019-02-13T05:46:16Z")

</div>

In MLJ.jl, we are trying to write a data-agnostic machine-learning framework and are presently trying to make this work using Query, to which I am new. We would like to write methods that take an iterable table as input…

---

## [JuliaDB cross reference](https://discourse.julialang.org/t/juliadb-cross-reference/20705)

<div class="topic-metadata">

**Author:** [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Replies:** 3\
**Last updated:** [February 12, 2019, 8:00pm UTC](https://discourse.julialang.org/t/juliadb-cross-reference/20705 "2019-02-12T20:00:52Z")

</div>

Disclaimer: I do not know any SQL. JuliaDB seems really straightforward so I’m trying it instead of some SQLish solution (although I suspect that’s what I should be using). I have a few tables (currently saved as .csv f…

[Previous page](https://discourse.julialang.org/c/domain/data/16.md?page=27)

[Next page](https://discourse.julialang.org/c/domain/data/16.md?page=29)