This may not be the right place to ask this question, but I think a lot of people who are very active in this forum work at/founded JuliaHub so just posting here for now.
I see that there is the ability to load datasets to JuliaHub. However unlike compute, there are no prices or limits listed anywhere on the website of what it would cost to store your datasets on JuliaHub’s cloud.
Is it a large scale solution? Could I store 1000’s of files that are like a GB each and operate on them in cluster jobs? Where can I find costs/limits?
3 Likes
I apologize in advance for resurrecting this year old thread, but I have the exact same questions about JuliaHub’s DataSets feature, and it didn’t get any responses last time.
Let’s try again.
- Somebody here must know.
- Also, it would be nice if the DataSets.jl docs mentioned JuliaHub.
2 Likes
I am also interested in knowing more about price and size limits of the Datasets feature.
2 Likes
Maybe it’s secretly free.
1 Like
Hey all — sorry I missed this topic!
It is currently free for up to 2GB worth of storage with typical usage. Folks on the free tier haven’t really pushed on it beyond that yet, but we have enterprise tier clients working with multi-TB worth of datasets on their dedicated company instances. At that scale, it makes sense to do some thinking about ingress/egress and datacenter locations in addition to the storage costs. So that’s why there’s not UIs or information published around this yet. JuliaHub’s DataSets are backed by AWS S3 and the public JuliaHub is in us-east, which can give you a sense for what happens after 2GB, where to store things, and how to optimally transfer them.
If you (or anyone else here) is interested in more, please don’t hesitate to contact us at sales@juliahub.com.
5 Likes