Tips for moving from Julia to Numpy/Python?

I think I have to switch from using Julia to Python/Numpy. I know. I’m going the wrong way! Still, it’s become increasing apparent it’s the right thing to do.

I know Julia much better than Python/Numpy, but Julia isn’t supported by or is a huge PITA to get working in the tools we use at work.

Additionally the Python/Numpy ecosystem has improved quite a bit for my domain. Geometric Computing with Python
https://taichi.graphics/

I’ve seen this: https://cheatsheets.quantecon.org/

Anyone have any tools/tips/resources for going from Julia to Python/Numpy? I have a lot of Julia code written.

Thanks.

1 Like

PyCall?

I just said Julia isn’t supported. I can’t link to Julia.

what do you mean by “link” to Julia?

dlopen() or gcc -ljulia

related:

see: JuliaPy · GitHub projects

1 Like

More generally, probably you could explain better what you mean by “moving to”. You want to translate all your Julia code to Numpy? Or just learn numpy given a Julia experience?

Probably here you will get more tips on how to better integrate your Julia development workflow with the Python packages (or the C++ packages that are in that Python package).

We use numpy and Julia extensively, here are a few things we’ve found:

  • PyJulia and PyCall are pretty great for interop, so you can keep using your existing code
  • It’s really easy to pass arrays, Dicts, and scalar values like strings, numbers, etc. between the two. It’s harder, but doable, to pass more complex data structures. So try to keep any interfaces between the two limited to those data types.
  • Julia has fast for loops, python/numpy doesn’t. So you’ll want to eliminate those as much as possible and lean heavily on vectorization
  • Numpy doesn’t really have loop fusion, so you’ll need to be more careful monitoring memory usage

That’s about it. For the most part, translation is relatively straightforward.

5 Likes

“Moving to” means, having Julia knowledge/code and probably a REPL open while prototyping but moving all that code/knowledge to python/numpy for the final product. Eventually not using Julia at all, and just writing python/numpy.

Best case scenario would be a source-to-source translator or julia->python compiler source. Next best is a good set of cheatsheets/tutorials/experiences on translating/rewriting julia to numpy The one I posted seems pretty good.

This seems to be the biggest difference between the numpy way of doing things and the Julia way. Although more recent Julia code is much more vectorized and closer to numpy in this way.

Translation should not be particularly difficult in general. It will be harder, depending on the code, to retain the performance, if the code is not easily vectorized. Perhaps you will finally have to learn some C, C++, or Numba, PyPy, to code some specific parts of the code that are more important for performance.

(as a side note, that particular package you mention above is just a front-end to a series of C++ libraries. A cool project could be to write that same front-end in Julia and have the best of both worlds).

@Orbots I don’t know if you’re still open to being convinced, but either way it would be helpful for us to know what is missing for you in Julia if you’re willing to talk about it.

You are assuming I don’t already? Strange.

I already stated I can’t use Julia! That’s the whole point of this post. I’d like to use julia, but it’s unreasonably difficult to do so.

Btw there is a recent package announcement

2 Likes

I was not assuming anything, it was just a remark about something to be considered in the migration process. Sorry if that felt differently in any way.

1 Like

The nice thing about Julia is that it’s roughly equally fast whether you use vectorized or non-vectorized code. So if you’re porting Julia code to Python, one good option is to go through, find any for loops, and rewrite them using vectorization/broadcasting instead. Then you can almost copy & paste that code to python with minor modifications (I’ve done it!), you’ll just need to get rid of the dots for broadcasting.

2 Likes

Fixes for these would be a start:

After that, smaller binary size and better guarantees that JIT wont be called.

Some things to do with package system I don’t want to get into.

Even with all that, I need to maintain a more complicated build system to use julia, compared to python, which has 0 extra steps involved.

1 Like

@orbots, have you asked here?

3 Likes

Make sure you have good test coverage before moving.

2 Likes

If you do anything that benefits running Julia and Python in parallel to compare results, IPython.jl is invaluable.

1 Like