Question about Dagger's DTable

Specific Domains Data
#1

I am reading through the DTable docs. But I can’t seem to quite grasp it.

Can I adapt DTable so that it can read from S3 path partitioned files? And the operate on the data?

In the doc, it shows how to create a DTable but it doesn’t mention where the data is stored, and how it’s stored etc. So I am a bit lost.

Any clarification welcome!

2 Likes
#2

I think with JuliaCloud/AWSS3.jl: AWS S3 Simple Storage Service interface for Julia. (github.com) it should be possible use DTable with an S3 path.