# Data

**URL:** https://discourse.julialang.org/c/domain/data/16.md?page=27

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

**Page:** 28

---

## [Overwrite partial DataFrame Row with new Array](https://discourse.julialang.org/t/overwrite-partial-dataframe-row-with-new-array/27953)

<div class="topic-metadata">

**Author:** [@milesf](https://discourse.julialang.org/u/milesf)\
**Replies:** 3\
**Last updated:** [August 25, 2019, 10:46am UTC](https://discourse.julialang.org/t/overwrite-partial-dataframe-row-with-new-array/27953 "2019-08-25T10:46:48Z")

</div>

I’m attempting to overwrite the last two elements in a dataframe row with the contents of an array. df = DataFrame(\[1 2 3\]) df\[1,2:3\] = DataFrame(\[5 10\]) ERROR: BoundsError: attempt to access "attempt to access a data f…

---

## [(Beginner) How to access value in a DataValue](https://discourse.julialang.org/t/beginner-how-to-access-value-in-a-datavalue/27944)

<div class="topic-metadata">

**Author:** [@hmmueller](https://discourse.julialang.org/u/hmmueller)\
**Replies:** 6\
**Last updated:** [August 24, 2019, 7:47pm UTC](https://discourse.julialang.org/t/beginner-how-to-access-value-in-a-datavalue/27944 "2019-08-24T19:47:31Z")

</div>

At DataValues.jl, I am informed that " One can access or unpack the value within a DataValue either via the get(x) function, or use the x\[\] syntax." I do not know “the x\[\] syntax” - when I tried just writing (e.g.) i.key…

---

## [Combine two capabilities of DataFrames.eachcol](https://discourse.julialang.org/t/combine-two-capabilities-of-dataframes-eachcol/27923)

<div class="topic-metadata">

**Author:** [@danriggins](https://discourse.julialang.org/u/danriggins)\
**Replies:** 2\
**Last updated:** [August 24, 2019, 2:38pm UTC](https://discourse.julialang.org/t/combine-two-capabilities-of-dataframes-eachcol/27923 "2019-08-24T14:38:57Z")

</div>

In the documentation for Dataframe.eachcol, I noticed these two examples: julia\> sum.(eachcol(df)) 2-element Array{Int64,1}: 10 50 julia\> collect(eachcol(df, true)) 2-element Array{Pair{Symbol,AbstractArray{T,1} wher…

---

## [CSV in Parallel Error](https://discourse.julialang.org/t/csv-in-parallel-error/27585)

<div class="topic-metadata">

**Author:** [@austinbean](https://discourse.julialang.org/u/austinbean)\
**Replies:** 6\
**Last updated:** [August 16, 2019, 1:12pm UTC](https://discourse.julialang.org/t/csv-in-parallel-error/27585 "2019-08-16T13:12:49Z")

</div>

I have a very large (160 Gb) csv file. I had the idea to do some processing of the file in parallel, but I’m stuck on an error in CSV.jl. What I wanted to do was use CSV.Rows on non-contiguous chunks of the file across…

---

## [Is the ODBC.load function implemented?](https://discourse.julialang.org/t/is-the-odbc-load-function-implemented/27330)

<div class="topic-metadata">

**Author:** [@js135005](https://discourse.julialang.org/u/js135005)\
**Replies:** 3\
**Last updated:** [August 12, 2019, 4:12am UTC](https://discourse.julialang.org/t/is-the-odbc-load-function-implemented/27330 "2019-08-12T04:12:39Z")

</div>

I am trying to load data from a very small dataframe into an MS SQL server database. This is documented in the ODBC.jl documentation, albeit as experimental. However it does not find the load function at all when I try …

---

## [How to create \`DataFrame\` from using NamedTuple keys as column names](https://discourse.julialang.org/t/how-to-create-dataframe-from-using-namedtuple-keys-as-column-names/27405)

<div class="topic-metadata">

**Author:** [@affans](https://discourse.julialang.org/u/affans)\
**Replies:** 4\
**Last updated:** [August 11, 2019, 8:50pm UTC](https://discourse.julialang.org/t/how-to-create-dataframe-from-using-namedtuple-keys-as-column-names/27405 "2019-08-11T20:50:41Z")

</div>

I have a function that returns a NamedTuple with 7 entries. The names are obviously Symbols and the entries are Int64. As this function is run many times, I would like to store the returned data as rows in a DataFrame, b…

---

## [R session aborted when reading feather file from Julia](https://discourse.julialang.org/t/r-session-aborted-when-reading-feather-file-from-julia/25937)

<div class="topic-metadata">

**Author:** [@ElOceanografo](https://discourse.julialang.org/u/ElOceanografo)\
**Replies:** 3\
**Last updated:** [August 8, 2019, 7:30pm UTC](https://discourse.julialang.org/t/r-session-aborted-when-reading-feather-file-from-julia/25937 "2019-08-08T19:30:26Z")

</div>

I’m trying to output a DataFrame as a Feather file from a Julia script and then import it into R. When I try to load it, though, R crashes (RGui closes out immediately, RStudio gives me an “R session aborted: R encount…

---

## [Query.jl - Return all columns with @map](https://discourse.julialang.org/t/query-jl-return-all-columns-with-map/11717)

<div class="topic-metadata">

**Author:** [@joshualeond](https://discourse.julialang.org/u/joshualeond)\
**Replies:** 3\
**Last updated:** [August 5, 2019, 7:13pm UTC](https://discourse.julialang.org/t/query-jl-return-all-columns-with-map/11717 "2019-08-05T19:13:45Z")

</div>

Hey there, I recently watched @davidanthoff’s Queryverse tutorial and am giving Query.jl a go. I like the new pipe-able syntax as I’m a heavy user of dplyr. I see the @map macro would be similar to the select and mutate …

---

## [Reading Data Is Still Too Slow](https://discourse.julialang.org/t/reading-data-is-still-too-slow/17919)

<div class="topic-metadata">

**Author:** [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Replies:** 35\
**Last updated:** [August 2, 2019, 10:49pm UTC](https://discourse.julialang.org/t/reading-data-is-still-too-slow/17919 "2019-08-02T22:49:41Z")

</div>

My crsp daily stock return data is a 6.4GB csv file. Compressed into csv.gz, it is about 1.7GB. (I posted a sample with my juliadb post, https://discourse.julialang.org/t/juliadb-getting-started-with-save-error/17891/4…

---

## [Redis Support?](https://discourse.julialang.org/t/redis-support/27013)

<div class="topic-metadata">

**Author:** [@Samuel\_Stapleton](https://discourse.julialang.org/u/Samuel_Stapleton)\
**Replies:** 1\
**Last updated:** [August 2, 2019, 3:50am UTC](https://discourse.julialang.org/t/redis-support/27013 "2019-08-02T03:50:04Z")

</div>

There were previous discussions to move Redis.jl to JuliaDatabases and to tag a version that supports Julia 1.0, but it appears that the package author never heard back about this original request. Are there plans to con…

---

## [Data storage/loading for data produced by algorithms and metadata](https://discourse.julialang.org/t/data-storage-loading-for-data-produced-by-algorithms-and-metadata/27029)

<div class="topic-metadata">

**Author:** [@jamblejoe](https://discourse.julialang.org/u/jamblejoe)\
**Replies:** 4\
**Last updated:** [August 1, 2019, 10:58am UTC](https://discourse.julialang.org/t/data-storage-loading-for-data-produced-by-algorithms-and-metadata/27029 "2019-08-01T10:58:57Z")

</div>

Hi, I am sorry, if this question was answered somewhere else already or there is a package especially for my need but the more I searched for solutions to my problem the more confused I got. I have algorithms producing…

---

## [Managing large dataset in package intended for beginners use](https://discourse.julialang.org/t/managing-large-dataset-in-package-intended-for-beginners-use/26936)

<div class="topic-metadata">

**Author:** [@Zach\_Christensen](https://discourse.julialang.org/u/Zach_Christensen)\
**Replies:** 13\
**Last updated:** [July 31, 2019, 12:57pm UTC](https://discourse.julialang.org/t/managing-large-dataset-in-package-intended-for-beginners-use/26936 "2019-07-31T12:57:36Z")

</div>

Background I’m working on a package that’s primarily for accessing a specific and large data set. I want part of its functionality to involve summary information (basic stats and plots) on subsections of the data. I cur…

---

## [How to edit row values of a dataframe column based on condition using query.jl](https://discourse.julialang.org/t/how-to-edit-row-values-of-a-dataframe-column-based-on-condition-using-query-jl/26437)

<div class="topic-metadata">

**Author:** [@vivekraj](https://discourse.julialang.org/u/vivekraj)\
**Replies:** 7\
**Last updated:** [July 17, 2019, 6:47pm UTC](https://discourse.julialang.org/t/how-to-edit-row-values-of-a-dataframe-column-based-on-condition-using-query-jl/26437 "2019-07-17T18:47:30Z")

</div>

i have filtered unique values from a column :CarCompany from dataframe: df |\> @map(\_.CarCompany) |\> @unique() |\> collect; the output shows fields with spelling mistakes: 28-element Array{SubString{String},1}: “alfa-r…

---

## [Dynamic @select with Query.jl](https://discourse.julialang.org/t/dynamic-select-with-query-jl/25242)

<div class="topic-metadata">

**Author:** [@tencnivel](https://discourse.julialang.org/u/tencnivel)\
**Replies:** 2\
**Last updated:** [July 9, 2019, 10:12am UTC](https://discourse.julialang.org/t/dynamic-select-with-query-jl/25242 "2019-07-09T10:12:54Z")

</div>

Hello, I’m using Query,jl with LINQ style query command. I try to pass a dynamic selection of variables to @select For example, instead of writing this: data\_table = @from i in data begin @select {i.id, i.la…

---

## [How to add/edit values in an NDSparse?](https://discourse.julialang.org/t/how-to-add-edit-values-in-an-ndsparse/26135)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 9\
**Last updated:** [July 8, 2019, 3:07pm UTC](https://discourse.julialang.org/t/how-to-add-edit-values-in-an-ndsparse/26135 "2019-07-08T15:07:55Z")

</div>

(crosposting from this so question) How to add or edit values in an sparse indexed table ? From the documentation I understood that the NDSparse object itself is immutable but not the underlying data, so I “understand” …

---

## [Data stream library comparable to Python streamz or tributary](https://discourse.julialang.org/t/data-stream-library-comparable-to-python-streamz-or-tributary/25925)

<div class="topic-metadata">

**Author:** [@FemtoTrader](https://discourse.julialang.org/u/FemtoTrader)\
**Replies:** 0\
**Last updated:** [July 2, 2019, 1:32pm UTC](https://discourse.julialang.org/t/data-stream-library-comparable-to-python-streamz-or-tributary/25925 "2019-07-02T13:32:27Z")

</div>

Hello, I wonder if Julia currently have libraries to deal with data stream / real-time stream processing comparable to Python streamz or tributary. https://github.com/python-streamz/streamz https://github.com/timkpain…

---

## [Keychain in Julia](https://discourse.julialang.org/t/keychain-in-julia/25884)

<div class="topic-metadata">

**Author:** [@sambitdash](https://discourse.julialang.org/u/sambitdash)\
**Replies:** 0\
**Last updated:** [July 1, 2019, 4:29am UTC](https://discourse.julialang.org/t/keychain-in-julia/25884 "2019-07-01T04:29:28Z")

</div>

Hi All, I am looking for a keychain where I can store some cryptographic keys safely. The the keys should be encrypted for most part save actual cryptographic operations. This is mostly to avoid stealing of crypto infor…

---

## [Naive use of BSON not working as expected](https://discourse.julialang.org/t/naive-use-of-bson-not-working-as-expected/25740)

<div class="topic-metadata">

**Author:** [@ablaom](https://discourse.julialang.org/u/ablaom)\
**Replies:** 15\
**Last updated:** [June 29, 2019, 7:09pm UTC](https://discourse.julialang.org/t/naive-use-of-bson-not-working-as-expected/25740 "2019-06-29T19:09:35Z")

</div>

Sorry, I am a beginner to BSON and am a little stuck. I have a struct Foo defined entirely in some package A. Package B imports A, defines an instance f=A.Foo() and serializes that instance using BSON. Now if, in the R…

---

## [Efficiency of matrix data operations](https://discourse.julialang.org/t/efficiency-of-matrix-data-operations/25768)

<div class="topic-metadata">

**Author:** [@FujiwaraTakumiEH](https://discourse.julialang.org/u/FujiwaraTakumiEH)\
**Replies:** 10\
**Last updated:** [June 29, 2019, 1:01pm UTC](https://discourse.julialang.org/t/efficiency-of-matrix-data-operations/25768 "2019-06-29T13:01:59Z")

</div>

Background: I have the following matrix is raw data,for example: a=\\left\[ \\begin{matrix} 1 & 2 & 3 \\\\ 4 & 5 & 6 \\\\ 7 & 8 & 9 \\\\ &...& \\end{matrix} \\right\] , b=\\left\[ \\begin{matrix} 1 & 2 & 3 & 5 \\\\ …

---

## [Unfortunate deprecation of showall in DataFrames](https://discourse.julialang.org/t/unfortunate-deprecation-of-showall-in-dataframes/15627)

<div class="topic-metadata">

**Author:** [@lewis](https://discourse.julialang.org/u/lewis)\
**Replies:** 3\
**Last updated:** [June 27, 2019, 9:53am UTC](https://discourse.julialang.org/t/unfortunate-deprecation-of-showall-in-dataframes/15627 "2019-06-27T09:53:09Z")

</div>

Understandable to reduce API surface area and eliminate lots of special case functions. Whether the loss of convenience is worth the streamlining is in the eye of the beholder. This one seems unfortunate: \*\*┌ Warning:…

---

## [How to create an empty dataframe from a vector of row names and column names](https://discourse.julialang.org/t/how-to-create-an-empty-dataframe-from-a-vector-of-row-names-and-column-names/25650)

<div class="topic-metadata">

**Author:** [@Fred](https://discourse.julialang.org/u/Fred)\
**Replies:** 11\
**Last updated:** [June 27, 2019, 6:29am UTC](https://discourse.julialang.org/t/how-to-create-an-empty-dataframe-from-a-vector-of-row-names-and-column-names/25650 "2019-06-27T06:29:45Z")

</div>

Hi, I would like to create a large empty dataframe of type Floats from a vector of row names and column names. For example rn=\["row1", "row2", "row3"\] cn=\["col1", "col2", "col3"\] Thanks !

---

## [Generating type specific deserialisers for BSON.jl](https://discourse.julialang.org/t/generating-type-specific-deserialisers-for-bson-jl/25720)

<div class="topic-metadata">

**Author:** [@richiejp](https://discourse.julialang.org/u/richiejp)\
**Replies:** 0\
**Last updated:** [June 26, 2019, 1:14pm UTC](https://discourse.julialang.org/t/generating-type-specific-deserialisers-for-bson-jl/25720 "2019-06-26T13:14:56Z")

</div>

This is more of a blog post than a forum post, but I don’t maintain a blog so… I have spent quite some time trying to optimise the excellent BSON.jl library as we make heavy use of it. The result of this so far is so…

---

## [JuliaDB w/ Workers on Remote Machine](https://discourse.julialang.org/t/juliadb-w-workers-on-remote-machine/25412)

<div class="topic-metadata">

**Author:** [@versipellis](https://discourse.julialang.org/u/versipellis)\
**Replies:** 2\
**Last updated:** [June 18, 2019, 6:25pm UTC](https://discourse.julialang.org/t/juliadb-w-workers-on-remote-machine/25412 "2019-06-18T18:25:36Z")

</div>

I have a workflow set up right now where I want to use my laptop as the master process, with addprocs(machine\_spec) pointing to a remote machine I’m SSHing to (with Julia and all dependencies installed). The data is stor…

---

## [JuliaDB versus](https://discourse.julialang.org/t/juliadb-versus/20977)

<div class="topic-metadata">

**Author:** [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Replies:** 12\
**Last updated:** [June 18, 2019, 12:35am UTC](https://discourse.julialang.org/t/juliadb-versus/20977 "2019-06-18T00:35:48Z")

</div>

I’ve been using JuliaDB and absolutely loving it . It helped me restructure my data and I’m now able to process my data in about 40 LOC. It all makes sense. In my application, I’m joining and grouping sources together t…

---

## [Json file with measurement values?](https://discourse.julialang.org/t/json-file-with-measurement-values/25170)

<div class="topic-metadata">

**Author:** [@alewolf](https://discourse.julialang.org/u/alewolf)\
**Replies:** 4\
**Last updated:** [June 11, 2019, 10:32pm UTC](https://discourse.julialang.org/t/json-file-with-measurement-values/25170 "2019-06-11T22:32:31Z")

</div>

I have a json file with measurement types- any idea how to read it back to julia? ie: file looks like S.json={ "D":4.0 ± 0.2, "tspan":{"t0":0,"tf":100}, "params":{ "p":20.0 ± 4.0…

---

## [How to pass a null value to a prepared statement with LibPQ.jl](https://discourse.julialang.org/t/how-to-pass-a-null-value-to-a-prepared-statement-with-libpq-jl/25068)

<div class="topic-metadata">

**Author:** [@tencnivel](https://discourse.julialang.org/u/tencnivel)\
**Replies:** 1\
**Last updated:** [June 8, 2019, 6:28am UTC](https://discourse.julialang.org/t/how-to-pass-a-null-value-to-a-prepared-statement-with-libpq-jl/25068 "2019-06-08T06:28:07Z")

</div>

Passing ‘nothing’ to a prepared statement throws an error

---

## [How can I shade an area under a curve between two lines?](https://discourse.julialang.org/t/how-can-i-shade-an-area-under-a-curve-between-two-lines/24897)

<div class="topic-metadata">

**Author:** [@Alessandro\_Vivas](https://discourse.julialang.org/u/Alessandro_Vivas)\
**Replies:** 5\
**Last updated:** [June 6, 2019, 8:26pm UTC](https://discourse.julialang.org/t/how-can-i-shade-an-area-under-a-curve-between-two-lines/24897 "2019-06-06T20:26:27Z")

</div>

I am plotting a normal curve and I want shaded some areas in this curve. I want to shaded to values minor than -1.96 e greater than 1.96. I tried some options: fill and fill area but I didn’t get sucess. x = rand( Norma…

---

## [Any Julia library for managing data series with automatic transformation](https://discourse.julialang.org/t/any-julia-library-for-managing-data-series-with-automatic-transformation/24818)

<div class="topic-metadata">

**Author:** [@nxtruong](https://discourse.julialang.org/u/nxtruong)\
**Replies:** 7\
**Last updated:** [June 3, 2019, 6:03pm UTC](https://discourse.julialang.org/t/any-julia-library-for-managing-data-series-with-automatic-transformation/24818 "2019-06-03T18:03:36Z")

</div>

I need to manage multiple data series (each is like a sequence of values, or a column in a data frame). Associated with each is an automatic transformation of the data in and out of the storage. For example, these can be…

---

## [DataFramesMeta custom filter: by groups of A, apply filter on B](https://discourse.julialang.org/t/dataframesmeta-custom-filter-by-groups-of-a-apply-filter-on-b/24557)

<div class="topic-metadata">

**Author:** [@floswald](https://discourse.julialang.org/u/floswald)\
**Replies:** 4\
**Last updated:** [May 27, 2019, 6:44pm UTC](https://discourse.julialang.org/t/dataframesmeta-custom-filter-by-groups-of-a-apply-filter-on-b/24557 "2019-05-27T18:44:08Z")

</div>

I need to subset a dataframe by groups. that is, in each group there is a different filtering condition to apply. here is a MWE using DataFrames, DataFramesMeta julia\> df = DataFrame(A = repeat(1:3,3), B = rand(1:9,9),…

---

## [DataDeps post\_fetch\_method](https://discourse.julialang.org/t/datadeps-post-fetch-method/24649)

<div class="topic-metadata">

**Author:** [@yakir12](https://discourse.julialang.org/u/yakir12)\
**Replies:** 6\
**Last updated:** [May 27, 2019, 4:52pm UTC](https://discourse.julialang.org/t/datadeps-post-fetch-method/24649 "2019-05-27T16:52:26Z")

</div>

I have a bunch of zipped .csv files on an S3 repo. I’m using the excellent DataDeps to retrieve them. Next, I use the also excellent JuliaDB to loadtable the .csvs into tables, and then from there it’s processing etc etc…

[Previous page](https://discourse.julialang.org/c/domain/data/16.md?page=26)

[Next page](https://discourse.julialang.org/c/domain/data/16.md?page=28)