DrWatson - the perfect sidekick to your scientific inquiries!

#21

This is is just amazing! I started to work on the exact same idea but ended up giving up!

An additional feature I had in mind was to have a GUI (based on Electron and Interact.jl) to prepare the simulation runs. This would include create/edit a template, and save/load specific config files.
I would be happy to contribute in any case!

2 Likes
#22

Great, good to have you on board!

I don’t fully understand your suggestion, so I think it is best to open a feature request issue to explain it in detail! I do want to comment though that Electron+Interact are quite heavy dependencies and also not only Julia dependencies which is something one should always think twice before adding. But of course it could be worth it.

#23

You tackle stashing files that don’t permit metadata in their format: Generate a filename that encodes parameter values. That kind of scheme has a second part: One needs to parse back the filename into its parameter values.

When I do this kind of thing in a project, then it is very annoying to generate a parameterset->filename mapping, plus regex to reconstruct the parameterset from the filename, in a way that is still human readable and does not lead to extremely long names. This is a giant ugly kludge, and kudos for trying to deal with it for us.

While I saw that you tackle name generation, I did not see any mention of parsing of names in the docs. Is that supported?

2 Likes
#24

Thank you very much for your kind words. Parsing of names is not yet implemented, however I always thought about it. I believe this is a functionality we should have and that it is also easy to implement.

We just didn’t have the manpower to do it until this time. I’ve opened up an issue that summarizes the process: https://github.com/JuliaDynamics/DrWatson.jl/issues/38 contributions would be super welcome, otherwise I will do it as time permits! :slight_smile:

#25

Interestingly , this is something I tried tackle, though inelegantly, in my DataProcessingHierarchyTools package. I basically needed a way to navigate a directory structure with the directory names defining a certain level of analysis. In my particular case, I am analysing neural data, and I have some analysis that run on an entire session, some on arrays of recording channels for that session, and some on individual cells. This tool allows me to automatically navigate to the appropriate level by defining a level parameter attached to each analysis type.
What I ended up doing for parameters was simply to attach a hash of those parameters to the file name, so that when I run analysis with identical arguments, the results are simply loaded. Of course, this means that I can’t tell what the arguments were simply by looking at the filename. Anyway, DrWatson seems to be much more polished version of this, and as I said before, I’ll try to integrate that into my workflow.

#26

DVC works quite nicely for me for versioning large files using S3 as the storage. It’s clearly focused on predictive modelling workflows but the versioning system is generic so it could fit different use cases.

1 Like
#27

This is what I had in mind (this is WIP)
That would be the template creator. Then one could just select parameters and save them in a config file.

1 Like
#28

Hey, this seems cool. But can you explain its purpose? What is this GUI supposed to achieve? (i.e. what does one do with the saved config file?)

Also, what are all these fields, like field name? I can see that the field name you wrote has space so it can’t be a Julia variable.

#29

Well the idea would be to first have a template config file for a project.
Then one could create specific config files for each experiment that would directly be fed to the workspace to run an experiment, similarly to your dict_list function.
From my experience I find it easier and less error-prone to work in a GUI to select parameters instead of manipulating dictionaries directly. It also allows to have the config file saved in the results folder as well.

And the field are not directly julia variables. They would simply be fields names.

#30

I see, but to really understand this, I would still need a usage demonstration or at least explanation. What do you do with the config file? How do you use it? What is the config file? is it XML, Julia, Toml? What’s its type? How do you actually use it in a simulation? Also, don’t you need to write a special parser for this to work?

For the dictionary all these questions are immediately answered since it is a basic Julia structure.

Yes this is a valid point, but one should consider that you may need to do these things over a cluster, or a cloud, or any other connection that won’t be able to support this. This is an advantage of the dictionary approach. A second advantage is that it works consistently with any conceivable type, existing or not (due to how we handle Vector subtypes). A final point is simply that Electron is a very heavy dependency.


Please notice: I am not bashing you or anything. From personal experience, the best way to improve something is to be as critical as you can, which is what I do here.

Do you have the code for this somewhere?

#31

Thanks for the helpful feedback. I did not think the whole thing through but my pipeline idea would be :
Create Template Config File -> Save as JSON (contains field name, default value and limits/options)
Create Config File -> Open existing Template file -> Set values -> Save as a Dict (where field names are keys) in JSON.
The config file can then be directly fed by being read as a Dict.
In short it is simply a practical (?) config file GUI maker :sweat_smile:

I advanced a bit more to make things maybe more clear

For the code it’s pretty ugly so far but you can still check it out if you want :
https://github.com/theogf/MLExps.jl
As you see I was going in the same direction as you did. You can simply check src/gui_config.jl and test/test_gui.jl

1 Like
#32

Thanks for the responce! To keep this post as on-topic as possible, I’ve continued further points in the repo you shared: https://github.com/theogf/MLExps.jl/issues/1

1 Like