What is the plan for Julia 1.0 and Python 3.X? In particular Conda?
My company uses Conda, and I want to introduce Julia for core algorithms. So, I want to make Julia invisible to most developers, and not require them to use one of the PyJulia workarounds.
Thanks
P
so you want them to cite / use Julia code within python?
As far as I know, pyjulia is currently the only method for calling Julia from Python. What issues/workarounds are you having trouble with?
As far as plans, there has been some discussion, but no real progress. Unfortunately, bindings from external languages arenāt the most popular thing for people to voluntarily work on, since āpeople who like to write Julia codeā donāt directly benefit themselves from that work (whereas bindings to external languages can still be useful, i.e. PyCall is handy for when there is no existing Julia library). I would guess that this is probably going to require someone to contribute either effort or money to make it happen.
Exactly. I want to write my Machine Learning in Julia, and let others process the data in Python.
So from Julia I need to be able to call Python to get the data, and from Python others need to call my classifiers in Julia.
OK. Sounds like I should should volunteer, except I am new to Julia
I am hoping that Julia can be a bridge from Research to Product in my company. Researchers all use Python/Jupyter, and the product is completely Javaā¦ and Python and Java do not play nice together.
Since Julia can talk to both, and is a superior way to develop machine learning, I would like to use it to develop the core code common to both Research and the Product.
I have already succeeded with a Julia via JNI interface for the Java product, now I am thinking about Python research code.
Itās not clear what workarounds you are talking about. Are you referring to python-jl? If your coworkers only/primary use Jupyter, one easy trick you can do is to setup kernel.json for them so that they can use PyJulia without them noticing they are using a workaround. PyCall.jl should also be configured to use their conda installation in this case.
For a better solution to conda situation, https://github.com/JuliaPy/PyCall.jl/issues/612 seems to be a good direction.
I would suggest starting with pyjulia: look into how it works, and ask for suggestions on here, Slack or the issue threads,
Unfortunately writing cross-language bindings is far from easy: you need to learn a lot about the internals of two different languages then make them do things they werenāt designed to do, such as getting reference counters to play nice with garbage collectors. But it can also be a great way to learn!
Thanks all for the help.
There is no issue with PyJulia-- it works as documented (very nice)
I want to sell Julia into a Python/Conda organization, so integration is a central issue, and telling everyone that they need to run this āspecialā Python interpreter (python-jl) is obviously a big negative.
However, reading a bit deeper, this is a problem with Python/Conda not Julia. There may be no fix at this end.
But thank you for making Julia āplay niceā with other languages
P
Iām curious what you are referring to. Conda is reasonably language-agnostic, so distributing wrapped Julia code should be a lot simpler than any potential pyjulia issues.
You might want to look at GitHub - SciML/diffeqpy: Solving differential equations in Python using DifferentialEquations.jl and the SciML Scientific Machine Learning organization for an example of a real package using pyjulia to expose Julia code to Python.
As I understand it, Conda installs a statically linked Python interpreter. PyJulia requires the Python core to be a dynamically linked library, so it can call it.
I am guessing that trying to access the internals of a statically linked executable is very hard, and likely to break.
Luckily, Anaconda distributes Python binary compiled as a position-independent executable (PIE). As I linked above, it looks PyCall.jl can load python as if it is a libpython when itās a PIE Use python binary instead of libpython when it's a PIE Ā· Issue #612 Ā· JuliaPy/PyCall.jl Ā· GitHub. There is already a proof-of-concept PR WIP: Use python binary instead of libpython when it's a PIE by tkf Ā· Pull Request #614 Ā· JuliaPy/PyCall.jl Ā· GitHub. Itās not merged yet because it needs more testing which in turn requires other PRs to be merged so that refactoring and testing are easier.
If everything works well we donāt need python-jl to use PyJulia installed in a Conda environment.
If you want to avoid PyCall/PyJulia issues at all cost, you can use libjulia through ctypes (or cffi) after compiling your Julia code using PyackageCompiler. Here is an example using ctypes: AOT compiling using PackageCompiler - #8 by tkf
But I imagine it requires a lot of boilerplate code since you have to bridge two GCāed languages manually and the values have to be passed around through C-compatible types (like int, double, pointers, etc.).
This is awesome news!!!
It looks like I just need to wait 
Thank you for the terrific work!