Store dataframes in structure and storing multiple database connection in connection pool

General Usage
,
1

Q1 - I am working with multiple dataframes and trying to create structure with dataframes in it so that they can be accessed easily. I’ve created struct like below and when I try to assign the data to it I got an error “setfield! fields of Types should not be changed” despite of defining mutable struct. I’ve also tried using macro @set from Setfield package.

mutable struct all_dataframes1
       map_df::DataFrame
       code_df::DataFrame
       end

s = @set all_dataframes.map_df = df

ERROR: UndefRefError: access to undefined reference
Stacktrace:
  [1] getproperty
    @ .\Base.jl:37 [inlined]
  [2] _broadcast_getindex_evalf


Q2 - I am trying to store multiple database connections (eg. MYSQL and PostgreSQL) in connection pool so that I can access at any point in my code. I've seen there is connectionpool.jl that has been created for Redis but haven't found in packages registry.
2

You need to call a constructor first to get an instance of all_dataframes1 (also, take a look at the naming conventions, it is pretty unconventional to use all_dataframes1 as the type name).

In other words, if you want to mutate your struct object, you need to have a struct object in the first place:

# assuming you already have two DataFrame objects called df1 and df2
myobj = all_dataframes1(df1, df2)

# now you can mutate your fields if you need:
myobj.map_df = df3

Also, keep in mind that Setfield.jl (and Accessors.jl) are designated for updating immutable structs (and more), and you end up constructing new objects altogether - there is no reason for using these packages if you already have a mutable struct.

A few things about structs.

When defining a struct in the following way:

mutable struct A
    fieldX::Int
    fieldY::String
end

you cannot just go and do A.fieldX = 1 right away. Instead, you need to call the default constructor:

a = A(1, "hello")

Now you are free to update the fields of the instance a of type A: a.fieldX = 10 and a.fieldY = "bye"

You can read more about this here.

Now, to answer your second question (which is hidden in the code block, by mistake, I assume).

There is an old/archived package ConnectionPools.jl; you shouldn’t expect to find it in the registry (pre-Julia 1.0).

I am not aware of any generic connection pool-related package at this point. I suggest checking the documentation of the db-related packages you already use.

If somebody else has more information about the connection pools issue, I hope they will jump into the conversation and share more.

3

Have you considered storing dataframes as nested elements of a parent dataframe?
I don’t know what kind of processing/access you have to do next, but the flexibility of the dataframes structure could be useful for your purposes, without “inventing” anything new.

4

@algunion Thanks for your clarification, Yes, that’s worked for me and I will try to read more about it.

1 Like