Community-driven JuliaPowerPack - Patching Pains and Incubating New Ideas

Community
, , ,
1

This idea was triggered by a recent discussion on this topic. Specifically, by @gdalle idea of a functionswith besides the existing methodswith.

Now - about the JuliaPowerPack (or JuliaExtensions - I don’t insist on a particular name, maybe someone else can propose a more appropriate one).

There are many ideas that spring from certain inconveniences Julia has, and there are many ideas that are just that - good ideas that everybody/many would love to see implemented.

I would categorize these ideas into two branches:

  • functionality that might end up finding its way into Julia’s code base
  • functionality that will always remain at the package stage

Let’s take, for example, the idea @gdalle proposed as a new feature. This is clearly a good candidate for Julia’s code base. But pushing (successfully) something to Julia’s codebase is not as simple - and is not fast. Some would love to benefit from the @gdalle’s idea today - not in a few months (or never, if that is not picked up).

I hope you got the gist of what I am trying to convey.

Before going further, I would say that we can also make this package a low-bar entry (and safe space) for those from the community that are not yet comfortable enough to contribute to more serious packages (or want to get hands-on experience before creating their own). In a way - we can commit to providing assistance as we do here, on Discourse.

JuliaPowerPack can be a Julia package that mainly extends the Base with additional methods (and macros) the community voted/requested/desires. This doesn’t exclude adding other modules where/if it seems appropriate.

Such a package would include elements like:

  • functions that improve the developer’s productivity (e.g., convenience functions/methods)
  • workarounds for known issues
  • better/cleaner formatting for various data types (or output in general)
  • an experimental corner where (show, don't tell) for ideas that are better showcased in a practical way
  • and there are many other ideas that the community could propose, of which I am completely unaware at the time of writing this piece

At the same time, this package can work as an incubator and a real-world tester for those ideas that might be considered candidates for Julia’s codebase: a successful idea that is implemented, used, and loved by the community has a better chance of being picked by Julia-Repo developers.

I don’t think this package will work with some benevolent dictator - so I am leaving this to you:

  • trash the idea - and give some reasons
  • support the idea - and propose a mechanism for voting/collecting community ideas (my take: Github issues is the first that comes to mind, and/or maybe we can get a Julia Discourse Category to make it more visible to the community).
  • support the idea - and offer to be an initial contributor to get this up and running
  • jump in and troll the topic with what you would like to see in such a package

Also, feel free to modify, add to the idea, or point out things that I might have failed to consider.

5 Likes
2

Challenge accepted! Although I see the appeal, I’m not sure this is a good idea, and here’s why :smiling_imp:

At the moment there is a central place for contributions to the language: the Julia repository itself. It includes both new features and bug fixes, as well as long-term evolutions like the new parser. If we create a separate repo to demonstrate a (presumably small) fraction of new features, it will not keep up with the latest changes, as opposed to a PR to JuliaLang/julia#master. It will however create more work to synchronize and test it, as well as keep track of the features that seem merge-ready to then copy-paste them.

Perhaps a better question would be why the PRs to the Julia repo take so long. Unfortunately, we’ve had this discussion here 100 times, and the answer is always the same: we need more people / more time.
For individual packages or organizations, this is easier to fix. Spot a contributor who opened a few issues and made one or two PRs, ask them to join the team. It’s a pyramid scheme, but it seems to work as long as fresh blood keeps nourishing the community :vampire:
For the core language, it seems like a different problem because of the larger consequences of an ill-advised merge. I consider myself an experienced Julia programmer, but I’m not sure I would want the responsibility that comes with merge rights. And on the other hand… maybe I could actually help?
Can we create an onboarding process for people like me to learn the specific requirements of language development, and thus reduce the bus factor? Can we train more people to be able to at least review PRs with reasonable confidence?

3 Likes
3

Thanks for changing my optics on this - I understand now that human resources related to language contributions are already low as they are - so no good can come from the potential of double work (even if that work is done by people who would not contribute to the core-language itself).

So I think you successfully trashed part of the idea (to the point where, seeing the larger picture, I agree with you).

Consequently, the rest of the idea is also shattered: because there is an expectation that various convenience methods will have their root cause/pain solved by the core language - so if keeping in sync with the language is not feasible, these convenience methods will only cause lots of confusion at some point.