How hard would it be to implement Numpy.jl, i.e. Numpy in Julia?

Yeah, I’ve been saying that for years. Julia is essentially a cleaned up C++ with a bunch of automated type computations, JIT compilation, and a nice metaprogramming AST. Generic Julia functions and C++ templates are pretty much the same, with the former being automatically generic and the latter choosing the opposite default. And the default choice is then tied to JIT compilation of course, because AOT compilation means you need to compile a bunch of code if you allow too many possibilities, while JIT compilation only compiles the specializations it needs on demand so it’s okay to define a function with nearly infinitely many specializations as long as you don’t try and compile every possibility.

So yes, that part is clear and we all agree that they are equally generic, with different pros and cons for interactivity and the ability to easily create a binary. And there’s no real performance magic either: when the two are generating their optimal code they will have the same speed (if the C++ is compiled with Clang).

But one thing that is different is the standardization. C++'s problem isn’t that standards don’t exist, it’s that far too many exist. In Julia, you can expect that any array package you find will work with Julia’s broadcast system, and this broadcast system will let different array types work together. In C++, I can’t pick up two random packages and expect a .+ b or equivalent between array types in the two packages to just work. That’s a pretty big difference usage-wise. Of course, C++ could have a central broadcast system, but it doesn’t. What I did not see before is that the xtensor xexpression interface basically allows for the definition of broadcasts in a standard way, like the Julia Base broadcast and AbstractArray interfaces. That is pretty neat and looks well done! But until you see almost all C++ packages implementing this interface, you won’t be able to just pick up a SUNDIALS NVector and broadcast it against a matrix type from Boost. On the flip side, you’d expect to have this kind of thing work between any Julia packages. Of course, this isn’t a technical issue so much as a community issue, but that’s the reason why I said “the C++ one is tied to an xtensor array type” (referring to the C++ broadcast solution) isn’t a true language-wide broadcast solution like the Julia one is.

The other thing I objected to was to think that because C++ is as generic, that implies Pythran is also completely generic

is a pretty big difference. Would you agree having to write such a mapping is pretty different than having it automatically exist through a robust number system like in C++ and Julia? Since Pythran hasn’t even finished supporting NumPy yet, it sounds like it’s a ways off before all of the Python symbolic libraries are supported. I would definitely agree that Pythran is a great project if what you need fits within the confines of what the transpiler is able to do, but some projects need more generic tooling and that is where something like C++ or Julia have an edge since having to change the compiler every time you want to do something new is not sustainable.

To link back to the OP, this generality and centralization are the reason why Julia can be advantageous to develop in. However, Steven said it best in the earlier post (How hard would it be to implement Numpy.jl, i.e. Numpy in Julia? - #32 by stevengj) that these pieces don’t translate over when you’re just looking to make a NumPy implementation, so there’s really no advantage to implement NumPy in Julia over something like C++ or Pythran.

14 Likes