I noticed to my delight that there is a DuckDB.jl package for DuckDB.
I now want to read in some Parquet data from an in-house S3 storage. According to the documentation I need the httpfs extension for DuckDB, but how should I run the INSTALL 'fts'; command?
This is not the right way to do it:
julia --project=. --threads=32
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.9.0 (2023-05-07)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia> using DuckDB
julia> con = DBInterface.connect(DuckDB.DB, ":memory:")
DuckDB.DB(":memory:")
julia> # list extensions
DBInterface.execute(con, "select * from duckdb_extensions()")
13×6 DataFrame
Row │ extension_name loaded installed install_path description aliases
│ String? Bool? Bool? String? String? Array…?
─────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ autocomplete false false Add supports for autocomplete in… Union{Missing, String}[]
2 │ fts false false Adds support for Full-Text Searc… Union{Missing, String}[]
3 │ httpfs false false Adds support for reading and wri… Union{Missing, String}["http", "…
4 │ icu true true (BUILT-IN) Adds support for time zones and … Union{Missing, String}[]
5 │ inet false false Adds support for IP-related data… Union{Missing, String}[]
6 │ jemalloc true true (BUILT-IN) Overwrites system allocator with… Union{Missing, String}[]
7 │ json false false Adds support for JSON operations Union{Missing, String}[]
8 │ motherduck false false Enables motherduck integration w… Union{Missing, String}["md"]
9 │ parquet true true (BUILT-IN) Adds support for reading and wri… Union{Missing, String}[]
10 │ postgres_scanner false false Adds support for reading from a … Union{Missing, String}["postgres…
11 │ sqlite_scanner false false Adds support for reading SQLite … Union{Missing, String}["sqlite",…
12 │ tpcds false false Adds TPC-DS data generation and … Union{Missing, String}[]
13 │ tpch false false Adds TPC-H data generation and q… Union{Missing, String}[]
julia> DBInterface.execute(con, "INSTALL httpfs")
ERROR: Execute of query "INSTALL httpfs" failed: HTTP Error: Failed to download extension "httpfs" at URL "http://extensions.duckdb.org/ae9bcc347c/linux_amd64/httpfs.duckdb_extension.gz"
Extension "httpfs" is an existing extension.
Are you using a development build? In this case, extensions might not (yet) be uploaded.
Stacktrace:
[1] execute(stmt::DuckDB.Stmt, params::NamedTuple{(), Tuple{}})
@ DuckDB ~/.julia/packages/DuckDB/PMwHV/src/result.jl:733
[2] execute
@ ~/.julia/packages/DuckDB/PMwHV/src/result.jl:834 [inlined]
[3] execute
@ ~/.julia/packages/DBInterface/1Gmxx/src/DBInterface.jl:130 [inlined]
[4] #execute#2
@ ~/.julia/packages/DBInterface/1Gmxx/src/DBInterface.jl:152 [inlined]
[5] execute(conn::DuckDB.DB, sql::String)
@ DBInterface ~/.julia/packages/DBInterface/1Gmxx/src/DBInterface.jl:152
[6] top-level scope
@ REPL[4]:1
julia>
I will say that I am enjoying using DuckDB but less-so with the Julia client. The CLI and the Python client are getting all of the love, so I have been using those for now.
Told you they were active. They’ve already responded:
Thanks for the report! The latest version of the Julia client is currently on a different version which does not have extensions available unfortunately. We will resolve this when we release v0.8.0 next week.
