Embed python packages into julia

Hi, I have a problem with loading python package dragonfly into julia and call it, don’t know why, since with the same way it works for another case here

Here is my example

function black_box_function(; x)
    return -x^2 - x + 1

using PyCall

temp = pyimport("dragonfly")
max_val, max_pt, history = temp.minimise_function(black_box_function, py"[[-10,10]]", 20)
print(max_val, max_pt)

For comparison, here put the program written in python, it works well

from dragonfly import minimise_function

def func(x):
    return -x**2 - x + 1

min_val, min_pt, history = minimise_function(func, [[-10, 10]], 20)
print(min_val, min_pt)

What is the error message?
Which operating system do you use?
How did you install Python?
How did you install dragonfly?

1 Like

I use windows system, and use pip to install dragonfly, the latest version of python is installed in my computer and I use VSCode to use it, I think I already install both python and dragonfly successfully, because I can run it within python environment, the error message is as follows, althouth there are some warnings, that shouldn’t be the problem, I think

C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\site-packages\dragonfly\utils\oper_utils.py:30: UserWarning: cannot_fortran' (C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\site-packages\dragonfly\utils\direct_fortran\__init__.
Could not import Fortran direct library. Dragonfly can still be used, but might be slightly slower. To get rid of this warning. gfortran) and the python-dev package and reinstall Dragonfly.
  warn('%s\n%s'%(e, fortran_err_msg))
┌ Warning: `vendor()` is deprecated, use `BLAS.get_config()` and inspect the output instead
│   caller = npyinitialize() at numpy.jl:67
└ @ PyCall C:\Users\admin\.julia\packages\PyCall\L0fLP\src\numpy.jl:67
ERROR: LoadError: PyCall.PyError("\$(Expr(:escape, :(ccall(#= C:\\Users\\admin\\.julia\\packages\\PyCall\\L0fLP\\src\\pyfncal
PyPtr, PyPtr), o, pyargsptr, kw))))", PyObject(Ptr{PyCall.PyObject_struct} @0x00007ffb31525890), PyObject(Ptr{PyCall.PyObject.PyObject_struct} @0x000000006a402080))
  [1] pyerr_check
    @ C:\Users\admin\.julia\packages\PyCall\L0fLP\src\exception.jl:62 [inlined]
  [2] pyerr_check
    @ C:\Users\admin\.julia\packages\PyCall\L0fLP\src\exception.jl:66 [inlined]
  [4] macro expansion
    @ C:\Users\admin\.julia\packages\PyCall\L0fLP\src\exception.jl:97 [inlined]
  [5] #107
    @ C:\Users\admin\.julia\packages\PyCall\L0fLP\src\pyfncall.jl:43 [inlined]
  [6] disable_sigint
    @ .\c.jl:458 [inlined]
  [7] __pycall!
    @ C:\Users\admin\.julia\packages\PyCall\L0fLP\src\pyfncall.jl:42 [inlined]
  [8] _pycall!(ret::PyObject, o::PyObject, args::Tuple{typeof(black_box_function), Matrix{Int64}, Int64}, nargs::Int64, kw::Ptr{Nothing})
    @ PyCall C:\Users\admin\.julia\packages\PyCall\L0fLP\src\pyfncall.jl:29
  [9] _pycall!(ret::PyObject, o::PyObject, args::Tuple{typeof(black_box_function), Matrix{Int64}, Int64}, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ PyCall C:\Users\admin\.julia\packages\PyCall\L0fLP\src\pyfncall.jl:11
 [10] (::PyObject)(::Function, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ PyCall C:\Users\admin\.julia\packages\PyCall\L0fLP\src\pyfncall.jl:86
 [11] (::PyObject)(::Function, ::Vararg{Any})
    @ PyCall C:\Users\admin\.julia\packages\PyCall\L0fLP\src\pyfncall.jl:86
 [12] top-level scope
    @ C:\Users\admin\.julia\packages\PyCall\L0fLP\src\pyeval.jl:232
in expression starting at C:\Users\admin\Desktop\Bayesian Optimization\Python\test07.jl:8

Well, Python on Windows is problematic, and using pip to install packages is problematic…
Conda used to be better than pip, but there are also newer developments like GitHub - cjdoris/PythonCall.jl: Python and Julia in harmony. which keeps all Python packages separated per Julia environment.

And I see that you have a space in your folder name: “C:\Users\admin\Desktop\Bayesian Optimization\Python\test07.jl:8”.

Can you do the same test when using a folder without any space in the name?


+1 for PythonCall.jl. It just makes life so much easier.


(dragonfly) pkg> add CondaPkg, PythonCall
julia> using CondaPkg
(dragonfly) pkg> conda add numpy
(dragonfly) pkg> conda add --pip dragonfly-opt
(dragonfly) pkg> conda resolve


julia> using PythonCall

julia> @pyexec("""
       global my_min, minimise_function, func
       from dragonfly import minimise_function

       def func(x):
           return -x**2 - x + 1
       def my_min(domain, max_capital):
           return minimise_function(func, domain, max_capital)

julia> my_min(domain, max_capital) = @pyeval("my_min")(domain, max_capital)
my_min (generic function with 1 method)

julia> min_val, min_pt, history = my_min(pylist([[-10, 10]]), 20);
Optimisation with bo(ei-ucb-ttei-add_ucb) using capital 20.0 (return_value)
Capital spent on initialisation: 5.0000(0.2500).
Legend: <iteration_number> (<num_successful_queries>, <fraction_of_capital_spent>):: curr_max=<current_maximum_value>, acqs=<num_times_each_acquisition_was_used>
#013 (011, 0.550):: curr_max=109.00000, acqs=[ei:1, ucb:2, ttei:0, add_ucb:3], 
#022 (021, 1.050):: curr_max=109.00000, acqs=[ei:2, ucb:4, ttei:2, add_ucb:8],

julia> @info :Result min_val min_pt
┌ Info: Result
│   min_val = Python float: -108.99999999999993
└   min_pt = Python ndarray: array([10.])

You can also work with the python function directly:

julia> @py import dragonfly: minimise_function

julia> minimise_function
Python function: <function minimise_function at 0x7fe59d1745e0>

but that goes beyond my knowledge of how this dragonfly package works, and mixing julia and python objects can get a bit more dicey than just wrapping everything into a julia function

Edit: wow

julia> using PythonCall

julia> @py import dragonfly: minimise_function

julia> func(x) = -x[1]^2 - x[1] + 1
func (generic function with 1 method)

julia> min_val, min_pt, history = minimise_function(func, pylist([[-10, 10]]), 20);

julia> @info :Result min_val min_pt
┌ Info: Result
│   min_val = Python float: -108.99999999999993
└   min_pt = Python ndarray: array([10.])