How can I use TileDB from Julia?

Specific Domains Data
1

How can I use TileDB from Julia? Is there any port? I’ve searched on Julia observer but haven’t find anything?
I’ve also searched on JuliaDB docs but haven’t found anything.

I’m looking for a fast database to deal with large datasets and be able to add new columns on disk. My candidates are hdf5, SciDB, MonetDB, Rasdaman and TileDB, the latter is supposed to be faster and optimized to work with matrices.

1 Like
2

The Julia API is apparently planned. You could of course call it via PyCall, but it might not have such great performance.

2 Likes
3

Do you have experience with TileDB?
Do you recommend it over other options?

4

imho: Maybe you can ask this question here : https://forum.tiledb.io/

5

Hi folks, in order to gauge interest, I just created a votable feature request for Julia bindings for TileDB:

If you are interested, please give an upvote at the link above. We would also be happy for any comments if you have a specific use-case in mind (or send us a note at hello@tiledb.com).

As background, TileDB is a chunked array format + storage manager supporting N-dimensional dense and sparse arrays, with built-in support for AWS S3 and HDFS, encryption, and time-traveling, as well as integrations for Spark, Dask, GDAL, PDAL, VCF, and more. The core library provides a C API, and bindings are currently available for Python, R, Go, Java, and C++. Please see our docs for more information.

Thanks,
Isaiah (TileDB, Inc)

4 Likes
6

Do you know how fast is TileDB compared to Clickhouse?

7

Hi @Juan,

The core TileDB library is a storage engine that does not implement SQL queries. Rather, it may be used as a columnar store by SQL engines (currently PrestoDB, MariaDB and SparkSQL). The integrations we have built re-use the existing engine’s SQL capability, but push compute to storage whenever possible (currently ranges, in the future more complex processing). We have been discussing adding TileDB as a data connector to Clickhouse, which would give Clickhouse optimized AWS S3 data access. Users would also enjoy direct data access outside of Clickhouse, e.g., for running Python/R/C++/etc. code directly against the data store.

Thanks,
Isaiah

1 Like