Have a look at DataDeps.jl.
2 Likes
I have been thinking about the implication here and would like to ensure I understand this correctly. As I see it, this means that if a package has a binary dependency that it downloads, and then the package updated, the new version is independent of the previous version and is “unaware” of the previous download. Hence it must download the dependency again. Correct?
This is not a big deal generally. For PyCall/IJulia the Conda download can be hefty, but I use Anaconda, so once the environment variables are set in startup.jl, it is just a rebuild.
Things could potentially be a little different for something like WinRPM. Packages installed by WinRPM lie under the specific WinRPM version’s folders and would need to be reinstalled if WinRPM updates, would they not? Again, for me not a problem - I hardly ever use that functionality.
I believe I understand the reasoning behind the fundamental design and I agree with it. The reason I’m asking is that I am working on a package that will use (possibly) large binary resouce files that should be mostly stable between package versions and it may be worth the 5min of effort it would take to read up on a better way of distributing the resource files than what I had in mind. I’m sure there are several better ways already developed and waiting for me to apply.
It is mostly in an effort to speeds things up. Download speeds where I live are not exactly fanatastic - from time to time a carrier pigeon with a flash drive beats the national carrier’s broadband offering in a race halfway across the country. Downloads speeds are improving, but then so are flash drive sizes and my money stays on the pigeon.
UPDATE: As it turns out, this is NOT how it works. WinRPM just updated and it installed a new version of gcc, which is the only package I had installed via gcc. Great!
I think this is what BinaryBuilder.jl is for? If I’m misunderstanding something, let me know. I may have need of this soon as well and don’t want to take a long walk down the wrong path.
I was looking at DataDeps.jl, but there are probably many more options.