# Is there a way to stream a table with 1 billion rows from SQL Server to Parquet efficently in Julia

**URL:** <https://discourse.julialang.org/t/is-there-a-way-to-stream-a-table-with-1-billion-rows-from-sql-server-to-parquet-efficently-in-julia/7107>\
**Category:** New to Julia\
**Created:** [November 16, 2017, 7:25pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-stream-a-table-with-1-billion-rows-from-sql-server-to-parquet-efficently-in-julia/7107 "2017-11-16T19:25:23Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [November 16, 2017, 9:18pm UTC](https://discourse.julialang.org/t/is-there-a-way-to-stream-a-table-with-1-billion-rows-from-sql-server-to-parquet-efficently-in-julia/7107/2 "2017-11-16T21:18:40Z")

</div>

A couple of days ago I [described](https://discourse.julialang.org/t/importing-big-data/7030/21) how to read data from PostgreSQL using [Spark.jl](https://github.com/dfdx/Spark.jl). If you manage to do the same thing with SQL Server, saving the result to Parquet is a matter of a single call to `write_parquet()`. At the moment documentation of Spark.jl is very limited, so don’t hesitate to ask questions.

Alternatively, you can query SQL server with whatever tools you like (e.g. [JDBC.jl](https://github.com/JuliaDB/JDBC.jl) or [ODBC.jl](https://github.com/JuliaDB/ODBC.jl)) and write it using [Parquet.jl](https://github.com/JuliaComputing/Parquet.jl), although I’m don’t their current state and suitability.

---

_[View the full topic](https://discourse.julialang.org/t/is-there-a-way-to-stream-a-table-with-1-billion-rows-from-sql-server-to-parquet-efficently-in-julia/7107)._