Dear all, today instead of doing my work, I learned about the new artifact system , and I am absolutely fascinated. I am currently working on a project that uses some data files, which I store in my Onedrive for Business folder. My module containing all functions also lives in my Onedrive, so I can…

[image] Crown421: Can I maybe even include some authentication so that not anyone with the link can just download from my drive? The data is not really cleared for general release. I’m interested in this too. Presumably if the source API has ssh access, one could rely on key-based authenticat…

what the differences are between DataDeps.jl I would also like to know what the story is here. I think DataDeps regularizes the experience for accessing and documenting data for the user but it seems like that it is mostly semantics with the new Artifacts system.

[image] yakir12: Not to hijack the conversation, I’m wondering what the differences are between DataDeps.jl and the new artifacts system… @oxinabox can speak to this more directly, but as I understand, DataDeps was designed before Artifacts were available. There are a lot of overlaps in funct…

Cool, looking forward to @oxinabox ’s input. If this is true then it sounds like there could be a new and improved DataDeps (which btw I love) that utilizes the Artifacts mechanism…

Thanks for the mention of DataDeps, I was not aware of it. From what I can tell I see some differences (at least in the design idea?). Please feel free to correct me if I am wrong: DataDeps can be used in the REPL, artifacts have to be part of a module. Artifacts are garbage collected, so once a…

Are there any packages out there yet that use the artifacts just for data storage? I’ve read the original blog post about using artifacts but I’m still not confident that I could figure it all out without breaking something.

The Yggdrasil Repo mentions We encourage Julia developers to use JLL packages for their libraries. Here are a few examples of pull requests switching to JLL package: Cairo.jl FFTW.jl GSL.jl Gtk.jl Rsvg.jl

[image] Crown421: Alternatively, use a different Cloud Storage provider. Does anyone know a service that is nicely accessible, perhaps via SSH keys? I think AWS and Azure (probably other Enterprise cloud storage places) can be connected to with ssh keys, but probably don’t have free tiers. De…

[image] Crown421: I think for my use-case, DataDeps is more appropriate right now. However, the question of secure storage/access is still a problem. I suppose one could take onedrive-sdk-python , write a PyCall.jl wrapper around it, and create a nice API and extend the relevant methods in Pkg.…

New Artifact System, Data, and OneDrive

General Usage

yakir12 January 12, 2020, 5:40pm 3

Not to hijack the conversation, I’m wondering what the differences are between DataDeps.jl and the new artifacts system…

Topic		Replies	Views
Using Artifacts for test data General Usage question	4	663	October 25, 2022
A couple of questions about Artifacts General Usage question	11	3168	April 24, 2020
Artifacts vs datadeps General Usage	4	845	May 18, 2020
My experiences using Pkg.Artifacts for test data General Usage	1	578	March 25, 2020
Are Julia Artifacts only for tarballs? Package Management question , package , artifacts	2	645	October 21, 2022

New Artifact System, Data, and OneDrive

Related topics