I’m trying to run a Python package named PyBLP from Julia using Pycall, and ran into an issue using @pywith.
All the PyBLP commands run just fine, however when it comes to using “parallel” through a “with” statement, I’m getting an error.
Here’s the first part of the code, which should run once you install PyBLP and uses PyBLP’s example datasets.
using PyCall using Conda pyblp = pyimport("pyblp") pd = pyimport("pandas") np = pyimport("numpy") pyblp.options.digits = 2 pyblp.options.verbose = false pyblp.__version__ product_data = pd.read_csv(pyblp.data.NEVO_PRODUCTS_LOCATION) product_data.head() logit_formulation = pyblp.Formulation("prices", absorb="C(product_ids)") problem = pyblp.Problem(logit_formulation, product_data) logit_results = problem.solve() logit_results.compute_elasticities()
The very last command “compute_elasticities()” can be implemented in parallel as per the PyBLP documentation. The exact Python command should be:
with pyblp.parallel(2): elasticities = results.compute_elasticities()
I’m trying to find the corresponding code using Pycall but haven’t been successful. I first tried:
@pywith pyblp("parallel")("2") as f begin f.(results = logit_results.compute_elasticities()) end
ERROR: PyError ($(Expr(:escape, :(ccall(#= /home/rubaiyat/.julia/packages/PyCall/BD546/src/pyfncall.jl:43 =# @pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, pyargsptr, kw))))) <class 'TypeError'> TypeError("'module' object is not callable")
I also tried:
py""" with pyblp.parallel(2) : elasticities = logit_results.compute_elasticities() """
PyError ($(Expr(:escape, :(ccall(#= /home/rubaiyat/.julia/packages/PyCall/BD546/src/pyeval.jl:34 =# @pysym(:Py_CompileString), PyPtr, (Cstring, Cstring, Cint), s, fname, input_type))))) <class 'IndentationError'> IndentationError('unexpected indent', ('/home/rubaiyat/.julia/packages/PyCall/BD546/src/pyeval.jl', 2, 4, ' with pyblp.parallel(2) : elasticities = logit_results.compute_elasticities()\n'))
An additional question before wrapping up: if I were to start 2 parallel processes as above from PyCall, would I need to start Julia using 2 threads, or do I start Julia using 1 thread and then the python call would spawn multiple threads when I run the above command?
I’d appreciate any help on this, thanks!