Greetings!
This is my first-ever post on Julia’s discourse, so I hope it turns out ok!
I have been developing a transpiler called PyJL for my Master Thesis over the last year, which translates Python source code to human-readable Julia source code. PyJL is part of the Py2Many transpiler, which transpiles Python to many C-like programming languages.
One of the last things I would like to do, is to have some feedback on the code that I am generating. I have recently created a small quiz containing Python code fragments and their corresponding translations in Julia generated by the transpiler. The intent is to assert the quality of the generated code. If you have any spare time, we would like for you to give it a try! You also have space in the quiz to leave your own suggestions, so leave them if you feel it is necessary (And yes, we will make an effort to consider all of them ).
Link to quiz (anonymous): https://forms.gle/XZFcDBu8J8XafXAX9
Ps.: If you would like to contribute to Py2Many, feel free to check out the repository. We are always open for new contributors .
If you integrated this with PyCall and managed to transpile the python parts of pytorch, would the result be capable of running pytorch code in Julia? That would be more “native” than calling a DL framework through PyCall yourself by wrapping every single call by hand.
I’m always happy to help a student project! I have a quick question, first, if you don’t mind. The instructions are: “How good do you consider the following translations to be?” My question is: “in what way?” Presumably, you’re not just interested in correctness because you could automate that. So what human touch are you looking for? For instance, simple if-else statements in julia are idiomatically done by short-circuiting || or &&. Since the translation doesn’t do this, I consider it imperfect. Do you?
Not sure if we are at that stage yet. One of the current issues we have is related to Python’s dependencies on external API’s and their dynamic features, leaving us with sometimes incomplete data-flow analysis.
We are currently trying to transpile the pywin32-ctypes project, which is a reimplementation of pywin32 that uses Python’s ctypes FFI.
We also mapped a subset of NumPy to native Julia, which gave us good results. PyTorch is on our list as well, but even if we are aiming for a subset, it would take a long time.
What I intended was for people to analyse the generated code to see if it respects the pragmatics of Julia. Our end-goal is to make the generated code as close to how a Julia programmer would write it. That leaves a little room for personal judgement, but that is to be expected, as one can write the same thing in multiple different ways.
I also agree with your last statement and I would also consider the translation to be imperfect in that scenario.
In general, all suggestions will help guide me in improving the transpiler.
I don’t feel comfortable “grading” the translations, but I can give my comments on the translations if you want, here or wherever.
One thing I noticed looking at the first few examples is that your transpiler assumes one-based indexing when accessing array members. Although this is correct for Array, I think it would make sense to do a[begin + 1] instead of a[2] and a[(begin + 3):5] instead of a[4:5], for example. This would make the generated code easier to convert to use other AbstractArray types.
Another suggestion is to do isone(x) and iszero(x) instead of x == 1 and x == 0, although you’ll be the judge whether the added complexity is worth it for you.
I think this is fair considering they’re translating from Python code, which lacks an analog for begin and end. Of course, you’re right that the transpiler assumes that the a in Python is 0-based, specifically when incrementing indices by +1 for the 1-based a in Julia. I think this is a fair approach if a is proven to be a handful of Python built-in iterables (range, list) or a NumPy array, but something like def get_index(A, i): return A[i] cannot assume 0-based indexing always e.g. A might reference either lists or dicts. Translating to an array type with the same indices, perhaps a 0-based OffsetArray, would be safer generally; unfortunately, a lot of Julia code has yet to be updated to play nice with non-1-based arrays, and even it were, Julians would still have to convert their conventionally 1-based arrays to whatever the Python code expects, which are conventionally 0-based.
I read PyTorch is really a C++ implementation with a Python interface, so I’m guessing the intention is to transpile to a Julian interface that composes better with other Julia code? All the Python examples in your quiz make heavy use of mypy/PEP484 type hints that reflect Julian type annotations, but a lot of Python code lack them because hints don’t affect implementation as in Julia. This may not be a big problem for functions but translating classes to ::Any-annotated types will introduce a lot of type instability. I assume you know PyTorch (more than me at least), so what are your thoughts on this issue when transpiling that? How about this issue when transpiling Python code that lack type hints in general?
PyTorch and JAX use type hints pretty actively (e.g., https://github.com/pytorch/pytorch/blob/master/torch/optim/adam.py), at least whenever the argument type is a primitive type, I believe. Sometimes they define type aliases like Array = Any, and then use arg: Array in code, which is not usable directly but also, in principle, could be converted to a proper type manually.
Unfortunately there is no option to skip a question. Not every respondent feels qualified to answer all question, thus many would probably stop in the middle, like I did.
As for different options (begin vs. 1, OffsetArrays vs. index translation etc.). Some of them are not equivalent: They may both work correctly, but break on minimal code changes. Thus it could make sense to provide corresponding options to the user.
However I think you are starting a pretty big project, and the most important thing now would be to get transpiler working, i.e. producing correct results, on at least medium-sized projects. The translations you produce now are all readable enough IMO.
One small note though (as I didn’t finish the survey) - combining for x = 0:n and for i in 0:n in the same code (binomial_coef) is not esthetically pleasing. Personally I prefer the second form.
Thank you for the suggestions @nsajko.
Can you point to the examples where the transpiler is assuming one-based indexing? It is possible that I optimized (that is, I told the transpiler to try to optimize) some of the examples and that is why you are seeing the use of one-based indexing. By default, the transpiler always uses zero-based indexing.
Using a[begin + 1], and converting comparisons to isone(x) and iszero(x) are all good ideas for future implementations.
Thank you @Eben60. Regarding the use of for x = 0:n and for i in 0:n, this was my mistake when adding the code to the quiz. By default, the transpiler will use the syntax for i in 0:n. The for x = 0:n syntax is applied by the formatted we are using (see JuliaFormatter).
Hi @Benny. TBH, I haven’t looked into PyTorch and similar large libraries a lot, because our focus is to first guarantee translation correctness. I hope we will get into that in the future.
Regarding the use of type hints, we have integrated the pytype type-inference mechanism to try to decrease the amount of work required to annotate large code bases. Still, as pytype is a static inference mechanism, it also has its limitations. As an example, consider the following implementation of bubble sort written in Python:
def bubble_sort(seq):
l = len(seq)
for _ in range(l):
for n in range(1, l):
if seq[n] < seq[n - 1]:
seq[n - 1], seq[n] = seq[n], seq[n - 1]
return seq
The following is pytype’s output when analysing this function:
As you can see, this does not add any relevant type information to the code. This becomes a problem if you want to perform any operations with the function that require knowledge about its return type. In such situations, we still require programmers to manually annotate the Python source code.
As I mentioned before, this might not always be a problem for functions. Sure, def bubble_sort is not statically type-inferrable, but neither Python nor Julia are in general. Julia’s type inference only kicks in at function calls, using the types of the runtime arguments. The equivalent Julia version of def bubble_sort (with no type annotations) would generally have type-stable calls, though there is an assumption that seq is 0-based-indexable by integers.
That’s just 1 particular case, however. Python functions are often written without caring about the variables’ type stability (because like type hints, there’s no runtime benefit). An example is how often Python code prepares a summing loop with sum_seq = 0 instead of how Julia does sum_seq = zero(eltype(seq)), but an accurate translation would have that type instability.
My concern was more with classes’ __init__ being non-annotated. pytype’s static inference looks like it can help a little, but you’re probably right that those have to be manually annotated in general, or else the translated Julia mutable struct would be given ::Any fields (introduces type instabilities) or given type parameters ::T (which may require excessive compilation, or error when a field is intentionally reassigned an instance of a different type).
An aside: if possible, it would be good to show your examples in a README file (which can be put in the PyJL part of the Py2Many Github) or a linked blogpost, so readers can scroll through examples without being stopped by unskippable required answers. It’s not just a matter of encouraging readers to give feedback. I had actually completed your quiz before, but my answers are not saved for me after submission, so I was still hampered by multiple pages of unskippable answers when I tried to look at your classes examples again.
I had some time to read through some of the quiz’s comments and made some improvements. I also noticed there were some bugs that I had accidentally overlooked. The most notable ones were:
Python’s int has to be translated to Int, not Int64
Some annotations in the Fasta benchmark were left untranslated. This was a copy-paste bug, which was already fixed.
The Python expression a = [0] * m has to be translated to fill(0, m), not repeat([0], m)
Furthermore, I wanted to make clear that the intent is to evaluate the pragmatics of the Julia source code, not Python. Therefore, one should evaluate if the code is readable to Julia programmers.
Lastly, as some people also requested access to the code samples in a more accessible format, I also decided to provide a document containing all the samples. This should allow you to scroll and compare any translations.
That’s a different type, though. The type annotation would presumably be something like numpy.int32, not just int. Just guessing though, I don’t know anything about python type annotations.
Python doesn’t have type annotations or parameters. Numpy arrays have an attribute (analogous to field in Julia) called dtype that specifies the size and structure of an element. You can (but shouldn’t) reassign that attribute to accomplish something like reinterpret (though you should use .view(new_dtype) instead). It is not part of Python’s type system, but a static analyzer could distinguish the eventually-np.int64 instances in np.array calls from the BigInt-like int literals.
Yes, but those have different purpose than Julia’s annotations. Rather than being part of the type system for dispatches, assertions, or conversions, they’re an auxiliary system for static analysis that is not enforced during runtime. So they’re allowed and intended to deviate from the type system, especially to write type parameters. Numpy has an open issue 16544 drafting type parameters like StaticArrays’ for Numpy arrays, basically indicating the mutable attributes for the element type and shape. These parameters would deviate from Julia’s Array, but this transpiler is already taking some liberties with translated types e.g. lists are not always Vector{Any}, ints are not BigInts.
).