# Given a PostgresSQL.jl connection string, what's the easiest way to create a table in the database by copying a dataframe into it?

**URL:** https://discourse.julialang.org/t/given-a-postgressql-jl-connection-string-whats-the-easiest-way-to-create-a-table-in-the-database-by-copying-a-dataframe-into-it/68131
**Category:** Data
**Created:** [September 14, 2021, 7:02am UTC](https://discourse.julialang.org/t/given-a-postgressql-jl-connection-string-whats-the-easiest-way-to-create-a-table-in-the-database-by-copying-a-dataframe-into-it/68131 "2021-09-14T07:02:29Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [September 14, 2021, 7:02am UTC](https://discourse.julialang.org/t/given-a-postgressql-jl-connection-string-whats-the-easiest-way-to-create-a-table-in-the-database-by-copying-a-dataframe-into-it/68131/1 "2021-09-14T07:02:29Z")

</div>

I have tried

```julia
using ODBC

conn = ODBC.Connection("Driver={PostgreSQL Unicode(x64)};Server=db.bit.io;Port=5432;Database=bitdotio;Uid=xxx;Pwd=xxx;")

tbl = DataFrame(a=1:3)
ODBC.load(tbl, conn, "tbl1"; append=false)

```

But it gives error

```julia
ERROR: HY000: LOG: Error occurred when processing query: relation ""."tbl1" does not exist (error ID: e6555b01-5bf7-4801-9941-3cbc2067d6d7);

```

The LibPQ insert statement might work but I am trying to upload a largish file so I want to make it as efficient as possible.

---

<div class="post-metadata">

### Author: ![lungben](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lungben/32/12314_2.png) [@lungben](https://discourse.julialang.org/u/lungben)
#### Post date: [September 14, 2021, 7:47am UTC](https://discourse.julialang.org/t/given-a-postgressql-jl-connection-string-whats-the-easiest-way-to-create-a-table-in-the-database-by-copying-a-dataframe-into-it/68131/2 "2021-09-14T07:47:56Z")

</div>

You can use [https://github.com/lungben/TableIO.jl](https://github.com/lungben/TableIO.jl) to upload large data amounts to Postgres via LibPQ.jl.  
It uses internally the Postgres COPY function to upload the data in CSV format (using CSV.jl), which is much more efficient than uploading with an SQL Insert, see [https://github.com/lungben/TableIO.jl/blob/master/src/postgresql.jl#L20](https://github.com/lungben/TableIO.jl/blob/master/src/postgresql.jl#L20).

---

<div class="post-metadata">

### Author: ![drvi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/drvi/32/13236_2.png) [@drvi](https://discourse.julialang.org/u/drvi)
#### Post date: [September 14, 2021, 7:57am UTC](https://discourse.julialang.org/t/given-a-postgressql-jl-connection-string-whats-the-easiest-way-to-create-a-table-in-the-database-by-copying-a-dataframe-into-it/68131/3 "2021-09-14T07:57:59Z")

</div>

I agree, you want to use COPY FROM stdin. This closed PR [https://github.com/invenia/LibPQ.jl/pull/172](https://github.com/invenia/LibPQ.jl/pull/172) contains some examples. When providing a CSV, be sure to quote the fields accordingly.

Edit: There is also an example in the docs [Home · LibPQ.jl](https://invenia.github.io/LibPQ.jl/dev/#COPY)

---

<div class="post-metadata">

### Author: ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [September 14, 2021, 10:25am UTC](https://discourse.julialang.org/t/given-a-postgressql-jl-connection-string-whats-the-easiest-way-to-create-a-table-in-the-database-by-copying-a-dataframe-into-it/68131/4 "2021-09-14T10:25:43Z")

</div>

> [@lungben](#):
>
> It uses internally the Postgres COPY function to upload the data in CSV format (using CSV.jl), which is much more efficient than uploading with an SQL Insert, see [TableIO.jl/postgresql.jl at master · lungben/TableIO.jl · GitHub](https://github.com/lungben/TableIO.jl/blob/master/src/postgresql.jl#L20).

Doesn’t work it says “tablename must only contain alphanumeric characters and underscores” which is not true for Postgres Tables

After fixing that function I get an error say the table doesn’t exist. I was looking for something that would create the table for me if it doesn’t exist. I guess i have to write some code now.

---

<div class="post-metadata">

### Author: ![lungben](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lungben/32/12314_2.png) [@lungben](https://discourse.julialang.org/u/lungben)
#### Post date: [September 14, 2021, 10:45am UTC](https://discourse.julialang.org/t/given-a-postgressql-jl-connection-string-whats-the-easiest-way-to-create-a-table-in-the-database-by-copying-a-dataframe-into-it/68131/5 "2021-09-14T10:45:15Z")

</div>

The CSV upload does not create a table, you need to do this beforehand.  
Usually, for a “persistent” database like Postgres you have a fixed schema which you use to create your tables and you populate your data afterwards. Thus creating a new table at the same time as filling it is not a very common use case.

---

<div class="post-metadata">

### Author: ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [September 14, 2021, 10:47am UTC](https://discourse.julialang.org/t/given-a-postgressql-jl-connection-string-whats-the-easiest-way-to-create-a-table-in-the-database-by-copying-a-dataframe-into-it/68131/6 "2021-09-14T10:47:35Z")

</div>

no worries. I wrote a function to create the table.

---

<div class="post-metadata">

### Author: ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)
#### Post date: [September 14, 2021, 3:43pm UTC](https://discourse.julialang.org/t/given-a-postgressql-jl-connection-string-whats-the-easiest-way-to-create-a-table-in-the-database-by-copying-a-dataframe-into-it/68131/7 "2021-09-14T15:43:56Z")

</div>

Just for context; ODBC.jl doesn’t automatically create a table for you since it’s just a “middleware” and there isn’t a mechanism in the ODBC spec to generically create a table in any supporting database.
