of course I understand there are resource constraints on reviewers, and ultimately there will always be a backlog.
that being said, sometimes it feels like PRs that are “ready” for at least feedback, if not a merge, go completely forgotten (and understandably so!). I think one thing that might help is as simple as some kind of StaleBot. For each PR / issue that goes without any activity for a month, auto-bump it. if it gets bumped 4-5 times without being “locked” open, then it gets auto-closed.
I think this could both
bring older PRs back to the top of the priority queue periodically, in case just the wrong set of eyeballs saw it when originally submitted
bring down the total number of resting PRs (some of which might be totally obsolete) so that it is easier for experienced reviewers to separate signal from noise
Another idea is to allow more of the broader community to assign or suggest labels; often I see an issue I know should get a certain labels but does not have it. This can also help because I feel like users will often filter issues by these labels, so it will make the related PRs harder to forget
Reviews are easier on everyone the more tests you have. If you have very good test coverage, then the review is about code style more than correctness. I recommend everyone to just keep upping the test bar as that makes correctness checking exponentially easier every single time.
I guess the natural follow-up question to that is: where and how can we improve test coverage in Base? Which then loops back to the original topic of this thread (how to make contributions flow more easily).
Here I’m optimistic because it seems other language communities have faced the same challenge and come up with a number of ideas the Julia community could take inspiration from, e.g. quest issues as mentioned in What's the state of Automatic Differentiation in Julia January 2023? - #19 by jlapeyre. As long as there’s institutional support for such initiatives (for however much core Julia development can be considered an “institution” ), the road ahead could be promising.
Do not underestimate the power of “soft” contributions! Every answer that someone else writes on Discourse that I don’t have to write is another contribution to SciML that’s opened up. Helping maintainers with all of the other random stuff that goes on allows people to focus on the code again. So big things to keep in mind:
Star the repositories. Right now, go star every repository that you think is cool. Why? Metrics help get grants and funding. This is the simplest contribution you can do, do it now.
Open clear issues. That’s how things get triaged and fixed.
Help with training people. Blog posts, tutorials, SO answers, Discourse answers. There’s a long tail of activities associated with package maintenance, and this can be more time than actually working on the package itself. The more help this gets, the more can get done. Make Youtube videos! I’d call Doggo JL one of the biggest contributors to Julia right now.
Be a collaborator. “Hey I’m looking to use your stuff for an application. Want to collaborate on this grant or paper?” Yes! This is how we find the real support and keep careers going while the package is getting built.
and of course, help add tests. Help fix doc builds. Update the website links. Etc. There’s lots of random stuff that has to be done. Don’t be shy, we all need help.
There’s been some discussion in this related thread where I tried to start a similar initiative a year ago:
What came out of it from my end was trying to get a better default issue / PR template for the Julia repo going, which was disliked very much in the triage call, citing “We actually feel it’s good to have a very loose organizational nature on the repo, without stringent requirements on what an issue/PR should fulfill”. Maybe that has changed since then, I don’t know I certainly think that’s detrimental.
Luckily, not everything in the ecosystem is done that way - SciML in particular seems to have a very fast response time to any interaction, mostly due to @ChrisRackauckas being awesome when it comes to fixing things & getting PRs merged.
I fully endorse this view, as I think this reduces imposter syndrome among new contributors. I understand where you are coming from, and I sympathize with your concern, but I wonder if there might be a better solution than to make the process feel more formal and stringent. While a well-defined issue template will certainly help with actually resolving an issue instead of it languishing for ages, this might also reduce the number of issues that are filed, as users would rather work around their annoyance than fail to meet the bar on a public forum.
I understand your concern, but quite often, extremely basic information such as
the julia version the issue occured on
the platform the user runs their code on
what the expected behavior was
simply isn’t present (even from seasoned contributors!). I’m not asking for a detailed 20-textfield questionnaire, I’m asking for a very minimal issue template that asks the user to fill in these basic requirements to even be able to debug & reproduce the issue, and to be able to easily decide whether some behavior is intended or not. If you think asking for these things induces imposter syndrome, I really don’t know what to tell you.
I’ll ALWAYS choose an easy to navigate process for newer contributors over a perceived slow-down for contributors, because that’s pretty much the only tangible thing we as a community can do to encourage new people to contribute, thereby easing the load on existing contributors.
That’s fair, but that too is something we can easily accomodate. It’s not like we only have one issue or PR template; Github supports multiple, selectable when opening an issue/PR. I really don’t see a drawback here.
I had a pull request in that was a simple typo in a comment (it was a long time ago, I don’t even remember where it is now) - it literally made no logic changes at all. When the test suite ran, it failed all sorts of things … I didn’t really know what to do with that …
EDIT: I guess the above, coupled with the fact that Julia has almost 1000 open pull requests and 4000 issues … I figured that what very little help I could offer would be better spent elsewhere? I love working with Julia, and it seems to be working just fine without my help - but I imagine that is some of the reason you are not attracting new contributors.
The fact that Julia CI is so flakey is a big problem that we need to work on. And, if you are willing, you can help!
Find a test that has recently failed in master (click on the red x’s on this page for details of test failures on master)
Search existing issues for one that tracks that test failure
If there isn’t already an issue filed, file one, linking to a test failure or two
Open a PR to skip the flakey test, and link to the issue you opened in that PR. (Feel free to @ me in this PR)
Failing tests on CI on master typically represent Julia bugs, but we try to track julia bugs in the issue tracker, not with failing CI. Shifting flaky tests to issues helps streamline reviewing all unrelated PRs.
If you have a PR with an unrelated test failure and can find that same failure on a recent commit to master, then you can skip straight to step 2.
Failing CI has been a problem in Julia since at least 2014. It makes contributions for newcomers easier if they have a reliable CI system. In some other projects, failing CI would block a PR. Why doesn’t Julia adopt that policy?
This check can be (partly) automated. Codecov can check a PR and fail if the percentage decreases. Then there could be an automatic comment explaining that the PR cannot be merged unless the test coverage goes up.
Same with code format (which is part of “style”), thanks to JuliaFormatter.
My point is that if these checks are automatized (up to a reasonable point), and the PR author can get immediate feedback (from a bot), then the PR quality will increase naturally. Also, I think that if a human reviewer starts asking a bunch of pesky quality issues it can feel a bit discouraging, but the feeling is different if these are automatic checks (which also means ALL PRs meet the same high standards).
Of course this automating this is extra work for maintainers. But the tooling is getting nicer each day. For instance @gdalle has been doing some very nice work with PkgTemplates. This could mean that the tooling for automatic quality checks will be in-place from package creation, without extra maintainer work.
In my experience, often providing the versioninfo isn’t enough, as the next step is to perform a git bisect to find the offending commit. I wonder if this might be automated? I’m digressing a bit, discussing issues instead of PRs here.
I don’t think I am; it’s precisely this inexperience that I think is our strongest signal for giving very clear directions in what is asked of users when submitting an issue. The current wall of text you see when opening an issue on the Julia repo certainly doesn’t achieve that.
In the end, having a clearer format for new issues also makes it easier for maintainers, because the information they’re looking for is always going to be in the same place. It’s not like I’m making this up as I go, the Linux Foundation recently put out a very good report on how open source projects organize their repos & contributions, and they’re saying pretty much the same thing:
I really don’t find this sarcastic remark appropriate. In other threads we’re being encouraged to engage with others, to help out in the repo. We’re trying to do that here, and get a Clippy response, famous for annoying nagging. How are people supposed to take the earnest efforts here seriously, when we’re met with this? Do you not want us to engage & try to help out in any way we can? It even got marked as the “solution” to this topic, though I have no idea whether that wasn’t an accidental click and was meant as a instead. The buttons are too close on mobile.
), the road ahead could be promising.
I certainly think that’s detrimental.
instead. The buttons are too close on mobile.