I tried something similar a while ago when I needed an automation solution for work. I started to build very basic custom pipelines reading data from databases and parquet files (DuckDB.jl), cleaning and aggregating data (DataFrames.jl) and saving the results to parquet files.
I can’t find the threads now, but I remember there were some initial tries to program a framework like this. However, I think they ended up abandoned.
While I found julia quite suitable for the task, in my project I discovered myself basically reinventing dlt and dbt, so I just ended up using these existing solutions instead.