Tips for moving from Julia to Numpy/Python?

@orbots Sorry to see you go. I can see how being able to draw on a huge body of work has advantages, but it is not Julia’s fault. In fact, people sticking it instead of leaving will make things better for Julia programmers.

1 Like

The move from Julia to Python is for practical work purposes. I’ll still be using Julia for… impractical reasons? :wink:

Many of the Python interop tools mentioned above make it actually easier to stick with Julia for prototyping and then move the working prototype code to Python while staying the Julia REPL, which I love. Hopefully Julia catches up on the practical reasons I need before I convert to a full pythonista.

3 Likes

Some folk on slack were recommending GitHub - wesselb/plum: Multiple dispatch in Python
Wessel is a Julia user at heart who for some reason decided to use python for his PhD stuff.
(While still using Julia for his work at Invenia).
And so he decided to reimplement the best bits of julia in python.

(Python has other multiple dispatch libraries but I suspect this is the closest to julia)

3 Likes

NumPyArrays.jl might help slightly during the transition to test interop between Julia and their Python/NumPy/Buffer procotol equivalents.

I sympathize with the stalled pull requests that the OP brought up. For the most part I’ve usually had to pursue the issues myself to resolve them. Occasionally, I’ve had to refactor or fork to make an independent package that would work in the timeframe I needed it. NumPyArrays.jl is an example of one of those occasions.

It may be useful to make your code array index agnostic by using eachindex(A) rather than 1:length(A) or something similar when you have loops. When that fails, perhaps using OffsetArrays.jl on the Julia side might help make your code 0-index based rather than 1-index based.

The other thing I would consider is looking closely at Numba since it also has a basis on top of LLVM. Sometimes one wonders if one might be able to take LLVM IR for a simple function from Julia and insert into Numba.

3 Likes

I also needed to convert some analysis scripts recently from Julia to Python due to client restrictions.
Converting array-based operations to Numpy is rather straight-forward.
For loop-based logic we used Numba in Python (plain Python loops are not an option for any significant amount of data), but be aware that in Numba array-access is unsafe per default. We got a nasty bug due to an out-of-index array access (caused by a missing 1-based-index to 0-based-index conversion) which gave silently wrong results instead of an error.
Another pain point was replacing Pluto by Jupyter for interactive analysis - Pluto wigdets and interactivity are so much more powerful.

1 Like

I would consider replacing x[1] indexes with x[begin] before porting.

1 Like

Your linked library provides not only multiple dispatch but also single dispatch for Python. :grinning: It’s depressing that Python needs libraries / decorators just for single dispatch for functions.

Oh yes. I’ve hit this porting Julia code to C++. The difference in ranges x[n:m] is another gotcha (for python).

Good tip. Then the copy and paste wont compile. Starting to form a best practices guide in my head for writing Julia code that is intended to be ported. eachindex(x) as mentioned above is a good one too.

3 Likes

Some other differences to watch out for: mutating default arguments and late-binding closures and changing variables in closures (requires nonlocal in Python).

The usual way to do single dispatch in Python is to define a class with its own methods. :slight_smile:

Or with singledispatch, which is in the standard library.

2 Likes

I always worry about performance when working with “large” dataset. A couple of years ago I tried doing something with 2 billions rows of the data and I think somewhere along the line it was trying to do a copy. So things got real slow.

Have u hit any performance barriers at a certain size? Or were the dataset relatively “small”?

But that would only work for objects of that particular class, it wouldn’t let you perform singular dispatch on one or several pre-existing classes.

Yes, it’s true that if you want to add a method to somebody else’s pre-existing class, you have to make a new class that inherits from that class. I’m not saying it’s a great situation, I’m saying that’s how it’s normally done. It works fine for your own classes. Within my own python package I can define as many classes as I like with the foo and bar methods.