PyError - with ExcelReaders perhaps

jmcastro2109 · February 11, 2021, 9:10pm

Hello,

I am getting an error with PyCall that I am having a hard time interpreting. I think that the error might be caused by using ExcelReaders . The other weird thing is that if I run my code one, it appears. If I run my code twice, it works. See the code below:



# Packages
using Distributed
using NLopt, Distributions, ExcelReaders
using Combinatorics, JLD
using LinearAlgebra, Statistics, Random
using BitOperations
using SharedArrays, Discretizers, Calculus, DelimitedFiles

#ddprocs(10, exeflags="--project=.")

rmprocs()
addprocs(2)


#Initializing packages in workers
@everywhere begin
    using NLopt, Distributions, ExcelReaders, JLD
    using Combinatorics
    using LinearAlgebra, Statistics, Random, DelimitedFiles
    using BitOperations
    using SharedArrays, Discretizers, Calculus
end


cd("C:\\Users\\vincenzi\\Documents\\vincenzi")



@everywhere begin

    global Country_Sample = "imp"

    # b) Data and Some Matrices

    Data1 = readxlsheet("gravity_$(Country_Sample).xlsx", "Sheet1")
    global Country = Data1[:,1]
    Distance = Data1[:,2]
    GravCont = Data1[:,3]
    GravLang = Data1[:,4]
    Distance=convert(Array{Float32,1}, Distance)

end

and the error:

┌ Error: Fatal error on process 2
│   exception =
│    PyError ($(Expr(:escape, :(ccall(#= C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\pyfncall.jl:43 =# @pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, pyargsptr, kw))))) <class 'TypeError'>
│    TypeError("can't pickle traceback objects",)
│
│    Stacktrace:
│     [1] pyerr_check at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\exception.jl:62 [inlined]
│     [2] pyerr_check at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\exception.jl:66 [inlined]
│     [3] _handle_error(::String) at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\exception.jl:83
│     [4] macro expansion at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\exception.jl:97 [inlined]
│     [5] #109 at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\pyfncall.jl:43 [inlined]
│     [6] disable_sigint at .\c.jl:446 [inlined]
│     [7] __pycall! at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\pyfncall.jl:42 [inlined]
│     [8] _pycall!(::PyCall.PyObject, ::PyCall.PyObject, ::Tuple{PyCall.PyObject}, ::Int64, ::Ptr{Nothing}) at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\pyfncall.jl:29
│     [9] #pycall#114 at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\pyfncall.jl:11 [inlined]
│     [10] pycall at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\pyfncall.jl:80 [inlined]
│     [11] serialize(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::PyCall.PyObject) at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\serialize.jl:14
│     [12] serialize_any(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::Any) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Serialization\src\Serialization.jl:629
│     [13] serialize(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::Any) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Serialization\src\Serialization.jl:608
│     [14] serialize(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::CapturedException) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\clusterserialize.jl:192
│     [15] serialize_any(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::Any) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Serialization\src\Serialization.jl:629
│     [16] serialize(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::Any) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Serialization\src\Serialization.jl:608
│     [17] serialize_msg(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::Distributed.ResultMsg) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\messages.jl:90
│     [18] #invokelatest#1 at .\essentials.jl:709 [inlined]
│     [19] invokelatest at .\essentials.jl:708 [inlined]
│     [20] send_msg_(::Distributed.Worker, ::Distributed.MsgHeader, ::Distributed.ResultMsg, ::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\messages.jl:185
│     [21] send_msg_now at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\messages.jl:130 [inlined]
│     [22] send_msg_now(::Sockets.TCPSocket, ::Distributed.MsgHeader, ::Distributed.ResultMsg) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\messages.jl:125
│     [23] deliver_result(::Sockets.TCPSocket, ::Symbol, ::Distributed.RRID, ::RemoteException) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\process_messages.jl:111
│     [24] macro expansion at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\process_messages.jl:302 [inlined]
│     [25] (::Distributed.var"#107#109"{Distributed.CallMsg{:call_fetch},Distributed.MsgHeader,Sockets.TCPSocket})() at .\task.jl:333
└ @ Distributed D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\process_messages.jl:115
Worker 2 terminated.IOError: stream is closed or unusable

Stacktrace:
 [1] check_open at .\stream.jl:328 [inlined]
 [2] uv_write_async(::Sockets.TCPSocket, ::Ptr{UInt8}, ::UInt64) at .\stream.jl:961
 [3] uv_write(::Sockets.TCPSocket, ::Ptr{UInt8}, ::UInt64) at .\stream.jl:924
 [4] uv_write at .\stream.jl:920 [inlined]
 [5] flush(::Sockets.TCPSocket) at .\stream.jl:1016
 [6] send_msg_(::Distributed.Worker, ::Distributed.MsgHeader, ::Distributed.RemoteDoMsg, ::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\messages.jl:191
 [7] send_msg(::Distributed.Worker, ::Distributed.MsgHeader, ::Distributed.RemoteDoMsg) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\messages.jl:134
 [8] #remote_do#155 at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\remotecall.jl:457 [inlined]
 [9] remote_do at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\remotecall.jl:457 [inlined]
 [10] flush_gc_msgs(::Distributed.Worker) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\messages.jl:155
 [11] flush_gc_msgs() at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\messages.jl:202
 [12] macro expansion at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\remotecall.jl:250 [inlined]
 [13] (::Distributed.var"#135#136")() at .\task.jl:333┌ Error: Fatal error on process 3
│   exception =
│    PyError ($(Expr(:escape, :(ccall(#= C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\pyfncall.jl:43 =# @pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, pyargsptr, kw))))) <class 'TypeError'>
│    TypeError("can't pickle traceback objects",)
│
│    Stacktrace:
│     [1] pyerr_check at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\exception.jl:62 [inlined]
│     [2] pyerr_check at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\exception.jl:66 [inlined]
│     [3] _handle_error(::String) at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\exception.jl:83
│     [4] macro expansion at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\exception.jl:97 [inlined]
│     [5] #109 at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\pyfncall.jl:43 [inlined]
│     [6] disable_sigint at .\c.jl:446 [inlined]
│     [7] __pycall! at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\pyfncall.jl:42 [inlined]
│     [8] _pycall!(::PyCall.PyObject, ::PyCall.PyObject, ::Tuple{PyCall.PyObject}, ::Int64, ::Ptr{Nothing}) at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\pyfncall.jl:29
│     [9] #pycall#114 at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\pyfncall.jl:11 [inlined]
│     [10] pycall at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\pyfncall.jl:80 [inlined]
│     [11] serialize(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::PyCall.PyObject) at C:\Users\vincenzi\.julia\packages\PyCall\tqyST\src\serialize.jl:14
│     [12] serialize_any(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::Any) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Serialization\src\Serialization.jl:629
│     [13] serialize(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::Any) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Serialization\src\Serialization.jl:608
│     [14] serialize(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::CapturedException) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\clusterserialize.jl:192
│     [15] serialize_any(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::Any) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Serialization\src\Serialization.jl:629
│     [16] serialize(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::Any) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Serialization\src\Serialization.jl:608
│     [17] serialize_msg(::Distributed.ClusterSerializer{Sockets.TCPSocket}, ::Distributed.ResultMsg) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\messages.jl:90
│     [18] #invokelatest#1 at .\essentials.jl:709 [inlined]
│     [19] invokelatest at .\essentials.jl:708 [inlined]
│     [20] send_msg_(::Distributed.Worker, ::Distributed.MsgHeader, ::Distributed.ResultMsg, ::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\messages.jl:185
│     [21] send_msg_now at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\messages.jl:130 [inlined]
│     [22] send_msg_now(::Sockets.TCPSocket, ::Distributed.MsgHeader, ::Distributed.ResultMsg) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\messages.jl:125
│     [23] deliver_result(::Sockets.TCPSocket, ::Symbol, ::Distributed.RRID, ::RemoteException) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\process_messages.jl:111
│     [24] macro expansion at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\process_messages.jl:302 [inlined]
│     [25] (::Distributed.var"#107#109"{Distributed.CallMsg{:call_fetch},Distributed.MsgHeader,Sockets.TCPSocket})() at .\task.jl:333
└ @ Distributed D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Distributed\src\process_messages.jl:115
Worker 3 terminated.

I have done the trick:

ENV["Python"] = ""
Pkg.build("PyCall")

but it does not solve anything.

nilshg · February 12, 2021, 7:12am

Sorry I can’t help with the the Python error, but have you considered just using XLSX.jl to read your Excel file? This will get you around any Python issues as it’s written entirely in Julia.