LibPQ copy error when field of the data frame contains comma

Specific Domains Data
1

I am trying to load data from data frame to postgresql using LibPQ.jl. it work quite fast until it reachs field that contains “,” such as “Oil, Gas & Coal” and throw an error:

[error | LibPQ]: BadCopyFileFormat: ERROR:  extra data after last expected column
CONTEXT:  COPY table_test, line 2: "ADMR,Adaro Minerals Indonesia Tbk.,Energy, Oil, Gas & Coal"

I use below code, adapted from here

using LibPQ, DataFrames
conn = LibPQ.Connection("host=/var/run/postgresql/ dbname=dbtest")

row_upload = map(eachrow(df2)) do row
    if ismissing(row[:stock_id])
        "$(row[:stock_id]),\n"
    else
        "$(row[:stock_id]),$(row[:company]),$(row[:sector]),$(row[:subsector])\n"
    end
end

copyin = LibPQ.CopyIn("COPY table_test FROM STDIN (FORMAT CSV);", row_upload)

execute(conn, copyin)

close(conn)

I have test it by separating some part of the data frame and every thing is OK when there is no “,” in each field of the data frame.
honestly, I am quite new to programming, any suggestion how to solve it efficiently without removing the “,” from the table?

2

CC @quinnj

3

You’ll want to quote any fields that could contain commas. You can also do this when this is not necessary, so you could have this to be safe:

"\"$(row[:stock_id])\",\"$(row[:company])\",\"$(row[:sector])\",\"$(row[:subsector])\"\n"