What is the best practice when you want to modify the source code of an existing package?

General Usage
1

I’m starting to use the package SQLite, and it has a function load! which writes rows from a dataframe into a sqlite database.

I want this function to replace rows instead of inserting them. My current solution is to essentially copy the code, replace the word INSERT with REPLACE, and utilize the new functions:

import SQLite

function load!(itr, db::SQLite.DB, name::AbstractString="sqlitejl_"*Random.randstring(5); kwargs...)
    # check if table exists
    db_tableinfo = SQLite.tableinfo(db, name)
    rows = Tables.rows(itr)
    sch = Tables.schema(rows)
    return load!(sch, rows, db, name, db_tableinfo; kwargs...)
end

function load!(sch::Tables.Schema, rows, db::SQLite.DB, name::AbstractString, db_tableinfo::Union{NamedTuple, Nothing};
               temp::Bool=false, ifnotexists::Bool=false, analyze::Bool=false)
    # check for case-insensitive duplicate column names (sqlite doesn't allow)
    SQLite.checkdupnames(sch.names)
    # check if `rows` column names match the existing table, or create the new one
    if db_tableinfo !== nothing
        SQLite.checknames(sch, db_tableinfo.name)
    else
        createtable!(db, name, sch; temp=temp, ifnotexists=ifnotexists)
    end
    # build insert statement
    esc_id = SQLite.esc_id
    sqlite3_step = SQLite.sqlite3_step
    sqlite3_reset = SQLite.sqlite3_reset
    columns = join(esc_id.(string.(sch.names)), ",")
    params = chop(repeat("?,", length(sch.names)))
    stmt = SQLite._Stmt(db, "REPLACE INTO $(esc_id(string(name))) ($columns) VALUES ($params)")
    # start a transaction for inserting rows
    SQLite.transaction(db) do
        for row in rows
            Tables.eachcolumn(sch, row) do val, col, _
                SQLite.bind!(stmt, col, val)
            end
            sqlite3_step(stmt.handle)
            sqlite3_reset(stmt.handle)
        end
    end
    SQLite._close!(stmt)
    analyze && execute(db, "ANALYZE $nm")
    return name
end

This works fine for now, but it feels really clunky; is there a better way to approach this?

2

If this is some functionality that you think others would commonly use, then it might be best to try to get it into the package. If all you are doing is changing the word REPLACE to INSERT, it seems like that could become a keyword argument: op=REPLACE . Opening an issue on the repo might be a good starting point.

1 Like
3

Thanks, I already did this. We’ll see if this gets into the package, but for now I want to utilize the REPLACE functionality with the package as is.

4

Then I’d say what you are doing is fine. It’s what I would do anyway.

1 Like
5

Just ]dev SQLite with your modified package checked out.

1 Like
6

You may want to work with a local copy of the package by doing ]dev SQLite (if you already loaded the package in that session, you will need to restart Julia after that). Then you can edit function you want to change, and also submit it as a PR if you want.
The downside is that you don’t get a reproducible environment, because your Manifest.toml doesn’t point to a published version. If you need a reproducible environment without waiting for a fix in a published SQLite version, I think your original solution is the best you can do.

1 Like
7

Until your PR is accepted into the package, I’d suggest to continue using your version but with a different name. Your load! acts differently, so it is basically a namespace conflict. Why not call it loadreplace!?

1 Like