I am trying to create some benchmark for writing tabular data to disk. So I wnated to install feather in Julia. But the below gives errors using PyCall using Conda, DataFrames, FileIO #Conda.add("pandas") # need to run if runs into error @pyimport pandas as pd df = DataFrame(x = [1]); FileIO.save(…

I haven’t used PyCall directly much, so I can’t help there. But there is a Feather.jl package available already, if that helps at all. Might be easier than trying to set it up via PyCall yourself.

Part of the point is to show speed difference between pycall and feather.jl

Just so you are aware, I’m currently in the process of re-writing Feather with a new Arrow.jl back-end (the arrow.jl package will handle all arrow formatted data) and am very nearly done. I haven’t done any extensive benchmarking yet.

[image] xiaodai: Conda.add("feather-format") # error This is a conda-forge package, so you need to use the conda-forge channel: Conda.add("feather-format", "conda-forge").

Now I get the below erro and I am trying to read Conda.jl’s readme, but I find it a bit hard to understand, probably because I have no background in Python. Conda.add_channel("conda-forge") Conda.add("feather-format","conda-forge") ERROR: ArgumentError: Path to conda environment is not valid. Stac…

Hm, I’m not sure I understand why the old feather would have been quite that slow. This is probably a stupid question, but are you absolutely sure you didn’t include any compile time? There certainly is a possibility that (for strings at least) serializing and deserializing from Feather is just fu…

[image] ExpandingMan: are you absolutely sure you didn’t include any compile time I used @benchmark and took the mean. As a side note, I think it’s fair to include compile time, as that is truly reflective of how an end-user experience the language. But I’ve tested this on 100 million rows of…

[image] xiaodai: I think it’s fair to include compile time, as that is truly reflective of how an end-user experience the language This I do not agree with. Suppose you have a problem of “size” n. You have two options (roughly speaking), you can interpret your code and get time complexity c…

[image] xiaodai: Conda.add_channel("conda-forge") Conda.add("feather-format","conda-forge") My mistake, it should be just Conda.add_channel("conda-forge") Conda.add("feather-format")

How to install feather using PyCall?

Specific Domains Data

xiaodai February 19, 2018, 10:59pm 7

Really looking forward to it as you can see, Feather.jl is about 2x slower than R’s feather implementation, and testing of feather for Python for me is pending resolution of this thread for me.

Topic		Replies	Views
[ANN] PythonCall and JuliaCall Package Announcements interoperability , python , pythoncall	84	11328	May 8, 2023
PyCall and Conda inside GitHub Actions Tooling pycall , github-actions , conda	10	1609	October 26, 2021
[PyCall] Pre-installing a python package required by a Julia package General Usage question , package , pycall	17	9659	March 23, 2021
Unable to build PyCall and RCall New to Julia	22	1894	February 7, 2023
Packaging a Julia library with PyJulia General Usage pycall , python , package-compiler , pyjulia	8	816	February 12, 2024

How to install feather using PyCall?

Related topics