Hello, I am trying to load python pickle object. I keep getting the below error message and I am unable to resolve the issue. The data which i am trying to load is CIFAR10 Dataset . Below is my code with which i am trying to load datasets. using PyCall @pyimport pickle function load_pickle_data(RO…

CIFAR comes in Binary version. This really should be preferred over Pickle anyway. Pickle is insecure and can run malicious code. (Eg if the authors site had been hacked) The binary version is easily parse-able using julia

[image] Saran_S: fo=open(f,“r”) datadict=pickle.load(fo) The problem is that Julia UInt8 arrays are by default converted to Python bytearray, whereas pickle only allows bytes objects for some reason (hence the unhashable type: 'bytearray' error). You can do: datadict = pickle.loads(pybytes…

[image] stevengj: The problem is that Julia UInt8 arrays are by default converted to Python bytearray, whereas pickle only allows bytes objects for some reason (hence the unhashable type: 'bytearray' error). (This should be fixed in the latest PyCall master, which fixes Python I/O with Julia I…

@stevengj Thank You very much for pointing out the mistake i had to use merge! I have made the changes which you suggested. But still i am getting error using PyCall @pyimport pickle function load_pickle_data(ROOT) datadict = Dict() for b=1:5 f=joinpath(ROOT, "data_batch_$b") fo=open(f,"r")…

[image] Saran_S: UndefVarError: readbytes not defined Sorry, it is just read(fo) in Julia 0.6.

@stevengj Tried read(fo). I still get error message. merge!(datadict,pickle.load(pybytes(read(fo)))) PyError (ccall(@pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, arg, C_NULL)) <type 'exceptions.AttributeError'> AttributeError("'str' object has no attribute 'readline'",) File "/Users/S…

@stevengj Thank You very much. pickle.loads did sort the issue. "batch_label" → "training batch 5 of 5" "labels" → Any[10000] "data" → 10000×3072 Array{UInt8,2}: "filenames" → Any[10000]

Unable to load pickle data

General Usage

stevengj July 3, 2017, 5:18pm 8

You have to use pickle.loads, not pickle.load, to read from a pybytes object. pickle.load only takes an I/O stream.

Topic		Replies	Views
PyCall setup for pickle consistency between machines General Usage question , pycall , python	0	440	June 10, 2021
How to read in Julia a "pickled" numpy multiarray? General Usage	3	2997	March 10, 2020
Pickle.Defer when loading PyTorch tensors in Julia General Usage	0	57	March 20, 2026
Module not reading properly New to Julia	3	506	June 15, 2020
Could I use Python Dict or convert Julia Dict to Python Dict in Julia? New to Julia pycall , python	2	1668	September 15, 2020

Unable to load pickle data

Related topics