How hard would it be to implement Numpy.jl, i.e. Numpy in Julia?

This may cut to the heart of the matter: you seem to think that people here are being defensive about Julia. Quite the opposite. If anything, your suggestion that compiling Numpy to Julia would provide performance advantages is giving too much credit to Julia and not enough to Numpy. The counterargument (which has been given a number of times) is that for the API that Numpy exposes the Numpy implementation is already pretty much optimal. The Numpy folks are really, really good at what they do. The notion that compiling to Julia will somehow make things faster is kind of insulting to the Numpy team—who aren’t here to defend themselves, so the Julia community is making that case for them. Julia doesn’t magically generate faster code that other systems can’t match (we use LLVM, they use LLVM—how would we be faster?). Julia’s secret sauce is in designing a language that feels just as expressive and flexible but which can be compiled to fast code in a general and composable way. Which doesn’t help at all when the code you’re trying to run is written in Python rather than Julia.

The case you’ve made for why it would be interesting to compile Numpy to Julia is frankly a bit weak. Here are some of the arguments I’ve been able to glean from the discussion:

  1. It would be interesting (ok, but subjective).
  2. It could be faster (it won’t).
  3. It could lead to better GPU support (possible, but unlikely)

Improved GPU support by compiling to Julia is unlikely because if you can compile Numpy to GPU via Julia, then you could even more easily compile Numpy to GPU directly. This would be simpler and faster both in terms of compile time and the resulting code speed and since Julia and Numpy both use LLVM for code gen, the end result would be very similar. The biggest problem for Numpy on GPU is likely that it’s much harder to avoid dynamic allocation in Python than in Julia—even the subset that Numpy can compile needs it—and the GPU doesn’t play well with dynamic memory allocation. Compiling to Julia wouldn’t help with that at all since you’d be still be using Pythonic APIs, not Julian ones.

All that said, the great thing about open source is that people can do what they want to! If someone—perhaps you—wants to try to reimplement Numpy in Julia, they can absolutely do so. The thing people in this thread are trying to tell you is that you are likely to be disappointed with the results.

19 Likes