I tried to talk about an example in the repo but I failed to even find the code…
What you go to the linalg module and there’s no source code there?
And the reason why the code is so hard to read is precisely because it uses this nonlinear go-to architecture. IMO, everything should have a clear top level so the code is linear and can be read like a book, while nonlinear reading should be helped by tools (in any language). The problem with PyTorch is it doesn’t read like a book: there is no table of contents telling you what comes after another. There is no flow. You have to already understand the code in order to understand it since new code can come in from anywhere. You pick a file and try reading it, and… follow hyperlinks until you think you understand things? Well if you go to torch.linalg you don’t even find the linear algebra so good luck! (hint: it’s all global as we will see later, violating the these module rules that were argued for in the first place).
The style in OrdinaryDiffEq is linear. Here is your table of contents:
(Note, this holds for every Julia package!) It tells you exactly what comes in, the chapters in what order, and it also has the exports to tell you what will come next (it should use more import instead of using, but that’s a separate matter). You can read this start to finish in it’s intended order and nothing you don’t know will jump out at you. And actually, by design this has to be legible or else you get an error! So no go-to style of code design, instead you have one canonical way to understand the code. You could use other tools as an appendix to jump around, sure, but if you want to understand the logic you can always go back to the story.
There will always be people who prefer coding with go-tos, with a bunch of globals, and a bunch of dynamic scopes, but I think time has told us again and again that making things simple and making things constrained always is helpful sooner or later. And programming styles which make people want to just append everything to globals and import as * are (a) hard for people and (b) hard for tools.
So in in a simplified sense, I think this whole discussion is phrased incorrectly. It should be understood as, “here’s a way to making using go-to’s easier so that way as code flies in from left and right you can try and make sense of the random assortment of globals!”. But the real question to ask is, “have you tried making your code read linearly and reading code linearly?”. Because you skipped the chapters on the caches and then complained that we Game of Thrones’d the ending, and then from the Cliff Notes wrote a final essay saying that the characters were undeveloped. 