Mongoc and DataFrames

General Usage
,
1

Hello, I work with mongo and financial data and I need to

  1. read data from csv
  2. write it to mongo
  3. get from mongo DataFrame
  4. dataframe as I just read it from csv file (types, etc)

I wrote some function for that

using Mongoc
using Distributed
using Logging
using JSONTables
using DataFrames

addprocs(5)
@everywhere using Mongoc

function set_from_csv_to_mongo(collection::Mongoc.Collection, path_to_file::String, return_df::Bool=true)
    df = CSV.read(path_to_file, DataFrame)
    jsonstr = objecttable(df);
    df_bson = Mongoc.BSON(jsonstr);
    append!(collection, [df_bson])
    println("Add $path_to_file to $collection")
    if return_df
        return df
    end
end

function get_data_from_mongo(collection::Mongoc.Collection, query::String="")
    mongo_data = Mongoc.find(storage.user_data);
    df = DataFrame(collect(mongo_data))
    return df
end

Then I try to put one *.csv with simple structure Date, SharePrice1,SharePrice2,…
I received the normal data frame from the first function (obviously I just read it from CSV), I saw this data in the collection, but I could not grab it back. I just received 1 row with no columns but arrays of prices and types just crashed to Any.

Does anyone know what are the best modules to work with MongoDB and DataFrames? How can I put it and get it normally? THX