# DuckDB.jl does not free memory

**URL:** <https://discourse.julialang.org/t/duckdb-jl-does-not-free-memory/103159>\
**Category:** General Usage\
**Created:** [August 24, 2023, 1:55pm UTC](https://discourse.julialang.org/t/duckdb-jl-does-not-free-memory/103159 "2023-08-24T13:55:19Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![danielw2904](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielw2904/32/10890_2.png) [@danielw2904](https://discourse.julialang.org/u/danielw2904)\
**Post date:** [August 24, 2023, 1:55pm UTC](https://discourse.julialang.org/t/duckdb-jl-does-not-free-memory/103159/1 "2023-08-24T13:55:19Z")

</div>

I am having an issue with memory allocation using the `DuckDB.jl` package. I am querying subsets of a large table (in a database stored in a file) and writing the results to a file. Given a table with columns `id1`, `id2`, `first_date`, `last_date` the following eventually runs out of memory:

```julia

con = db.connect(DuckDB.DB, "database.db")
DBInterface.execute(con, "PRAGMA threads=8;")
for curdate in query_dates
  DBInterface.execute(con,
          """
          COPY 
          (SELECT 
              id1, 
              id2
          FROM my_large_table
          WHERE (CAST('$curdate' AS DATE) + INTERVAL 6 DAY) >= first_date
          AND '$curdate' <= last_date)
          TO 'data/$(curdate).parquet' (COMPRESSION ZSTD);
          """
      )
end

```

An equivalent query using python works (fills cache but frees memory when needed)

```python
# %%
import duckdb
from tqdm import tqdm

con = duckdb.connect(
    'database.db', 
    read_only = True)

def make_querystring(curdate):
    return f"""
        COPY 
        (
        SELECT 
            id1, 
            id2
        FROM my_large_table
        WHERE (CAST('{curdate}' AS DATE) + INTERVAL 6 DAY) >= first_date
        AND '{curdate}' <= last_date
        )
        TO 'data/{curdate}.parquet' (COMPRESSION ZSTD);
    """

for cur_date in tqdm(date_list):
    cur_q: str = make_querystring(cur_date)
    con.sql(cur_q)

```

Any ideas how I can force `DuckDB.jl` to free memory (closing the result or the database after every query did not work in my test)?

Thanks!

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [August 24, 2023, 6:17pm UTC](https://discourse.julialang.org/t/duckdb-jl-does-not-free-memory/103159/2 "2023-08-24T18:17:49Z")

</div>

> [@danielw2904](#):
>
> closing the result or the database after every query did not work in my test

That’s because of a bug where it doesn’t actually close the database 😆

There is an existing GitHub issue and I pointed to a possible solution but evidently it wasn’t good enough (meaning that some reported my fix didn’t fix the problem).

I definitely run into problems like this. One thing I will say is that there are a lot of fixes that have been merged and will be available in the next release. Their release cadence seems unusually slow for such an early stage project. If you can bother to build the feature or master branch, it is very possible that your issue is already fixed.