I am trying to run a bunch of computational experiments, different algorithms on different problems, using different sets of parameters each, pretty simply right? Well I am wondering what the best pipeline would be to store interesting statistics and data for each experiment run in a way that is easy to store and process later on. My limited SQL knowledge is forcing me to think relational databases, with a table for algorithms, one table for each algorithm’s set of parameters linking to the algorithm record, a table for the problems, and finally a table for the runs and shared summary statistics and results linking to the algorithm record, the parameters’ record and the problem record. Any algorithm-specific results can also be written in a table for each algorithm and linked into the experiment runs table. So I wonder, what is the best way to do this programmatically? I am guessing it has to do with DataFrames.jl but is there any more suitable package for organising this mess that you are aware of? I am really not familiar with the data ecosystem in Julia but I know there are too many options out there and trying all of them is currently not an option for me, so I would appreciate a nice holistic summary of what’s available from an expert.
I am also interested in any generic advice if you may. Thanks in advance!