Google BigQuery

Specific Domains Data
1

I am trying to use GCP.jl to query data from BQ table, but when I tried to install it. I got this error. Any idea how to fix it?

   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package GCP [5a343a44]:
 GCP [5a343a44] log:
 ├─possible versions are: 0.1.0-0.1.2 or uninstalled
 ├─restricted to versions * by an explicit requirement, leaving only versions: 0.1.0-0.1.2
 └─restricted by compatibility requirements with DataFrames [a93c6f00] to versions: uninstalled — no versions left
   └─DataFrames [a93c6f00] log:
     ├─possible versions are: 0.11.7-1.5.0 or uninstalled
     ├─restricted to versions * by an explicit requirement, leaving only versions: 0.11.7-1.5.0
     ├─restricted by compatibility requirements with DataFramesMeta [1313f7d8] to versions: 0.13.0-1.5.0
     │ └─DataFramesMeta [1313f7d8] log:
     │   ├─possible versions are: 0.4.0-0.14.0 or uninstalled
     │   ├─restricted to versions * by an explicit requirement, leaving only versions: 0.4.0-0.14.0
     │   └─restricted by compatibility requirements with DataFrames [a93c6f00] to versions: 0.6.0-0.14.0 or uninstalled, leaving only versions: 0.6.0-0.14.0
     │     └─DataFrames [a93c6f00] log: see above
     └─restricted by compatibility requirements with PowerSystemCaseBuilder [f00506e0] to versions: 0.22.0-1.5.0
       └─PowerSystemCaseBuilder [f00506e0] log:
         ├─possible versions are: 0.1.0-1.0.5 or uninstalled
         └─restricted to versions * by an explicit requirement, leaving only versions: 0.1.0-1.0.5
2

GCP.jl hasn’t been touched in 3 years and doesn’t provide compat bounds on its dependencies (including DataFrames).

Maybe you can use

3

Thank you, but I don’t see any support for BQ.

I simply used PyCall and it worked fine.

using PyCall
bigquery = pyimport("google.cloud.bigquery")
client = bigquery.Client()
query_job = client.query("<My Query>")
rows = query_job.result()
for row in rows
       println(row)
end
4

A little bit late, but with TidierDB.jl, you can now use use sql and query a table via Google Big Query (through Tidier.jl syntax) . (GoogleCloud.jl is the connection backend)

using TidierDB
user = "path_to_Credentials.json"
project_id = "string"
con = connect(:gbq, user, project_id)

@chain db_table(con, "dataset_name.mtcars") begin
    @filter(!starts_with(Column1, "M"))
    @group_by(cyl)
    @summarize(mpg = mean(mpg))
    @mutate(mpg_squared = mpg^2, 
               mpg_rounded = round(mpg), 
               efficiency = case_when(
                                 mpg >= cyl^2 , "efficient",
                                 mpg < 15.2 , "inefficient",
                                 "moderate"))            
    @filter(efficiency in ("moderate", "efficient"))
    #@show_query
    @collect
end

2×5 DataFrame
 Row │ cyl    mpg      mpg_squared  mpg_rounded  efficiency 
     │ Int64  Float64  Float64      Float64      String     
─────┼──────────────────────────────────────────────────────
   1 │     4  27.3444      747.719         27.0  efficient
   2 │     6  19.7333      389.404         20.0  moderate
2 Likes