# Making the review process more pleasant for new contributors

**URL:** <https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604>\
**Category:** General Usage\
**Tags:** github, contributing\
**Created:** [August 8, 2023, 2:00pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604 "2023-08-08T14:00:25Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [August 8, 2023, 2:00pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/1 "2023-08-08T14:00:25Z")

</div>

> [@Did Julia community do something to improve its correctness?](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/119):
>
> know that many people don’t contribute who perhaps should, but also that many who do submit PRs can have a frustrating experience waiting for reviews. I don’t know how to fix that.  
> […]  
> If you have ideas about how to fix it perhaps another topic?

of course I understand there are resource constraints on reviewers, and ultimately there will always be a backlog.

that being said, sometimes it feels like PRs that are “ready” for at least feedback, if not a merge, go completely forgotten (and understandably so!). I think one thing that might help is as simple as some kind of StaleBot. For each PR / issue that goes without any activity for a month, auto-bump it. if it gets bumped 4-5 times without being “locked” open, then it gets auto-closed.

I think this could both

- bring older PRs back to the top of the priority queue periodically, in case just the wrong set of eyeballs saw it when originally submitted
- bring down the total number of resting PRs (some of which might be totally obsolete) so that it is easier for experienced reviewers to separate signal from noise

Another idea is to allow more of the broader community to assign or suggest labels; often I see an issue I know should get a certain labels but does not have it. This can also help because I feel like users will often filter issues by these labels, so it will make the related PRs harder to forget

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [August 8, 2023, 2:05pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/2 "2023-08-08T14:05:08Z")

</div>

Reviews are easier on everyone the more tests you have. If you have very good test coverage, then the review is about code style more than correctness. I recommend everyone to just keep upping the test bar as that makes correctness checking exponentially easier every single time.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [August 8, 2023, 2:41pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/3 "2023-08-08T14:41:33Z")

</div>

I guess the natural follow-up question to that is: where and how can we improve test coverage in Base? Which then loops back to the original topic of this thread (how to make contributions flow more easily).

Here I’m optimistic because it seems other language communities have faced the same challenge and come up with a number of ideas the Julia community could take inspiration from, e.g. quest issues as mentioned in [What's the state of Automatic Differentiation in Julia January 2023? - #19 by jlapeyre](https://discourse.julialang.org/t/whats-the-state-of-automatic-differentiation-in-julia-january-2023/92473/19). As long as there’s institutional support for such initiatives (for however much core Julia development can be considered an “institution” 😉 ), the road ahead could be promising.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [August 8, 2023, 2:57pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/4 "2023-08-08T14:57:41Z")

</div>

At the end of my keynote, I mentioned that there are many ways to contribute:

 ![Screenshot 2023-08-08 at 10.50.35 AM](https://global.discourse-cdn.com/julialang/original/3X/c/b/cbe1ddd02494e2fa78ef850f1baac9603bbd9147.jpeg)

**Do not underestimate the power of “soft” contributions!** Every answer that someone else writes on Discourse that I don’t have to write is another contribution to SciML that’s opened up. Helping maintainers with all of the other random stuff that goes on allows people to focus on the code again. So big things to keep in mind:

- Star the repositories. Right now, go star every repository that you think is cool. Why? Metrics help get grants and funding. This is the simplest contribution you can do, do it now.
- Open clear issues. That’s how things get triaged and fixed.
- Help with training people. Blog posts, tutorials, SO answers, Discourse answers. There’s a long tail of activities associated with package maintenance, and this can be more time than actually working on the package itself. The more help this gets, the more can get done. Make Youtube videos! I’d call Doggo JL one of the biggest contributors to Julia right now.
- Be a collaborator. “Hey I’m looking to use your stuff for an application. Want to collaborate on this grant or paper?” Yes! This is how we find the real support and keep careers going while the package is getting built.

and of course, help add tests. Help fix doc builds. Update the website links. Etc. There’s lots of random stuff that has to be done. Don’t be shy, we all need help.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 8, 2023, 3:06pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/5 "2023-08-08T15:06:46Z")

</div>

There’s been some discussion in this related thread where I tried to start a similar initiative a year ago:

> [@Improving the Julia issue tracker](https://discourse.julialang.org/t/improving-the-julia-issue-tracker/83365):
>
> Hello! This post has (in part) come to be due to a general feeling of the issue tracker for julia being a little hard to work with as a non-core developer or absolute beginner, wishing to contribute. The goal of this post is to Identify problems users have with the way the current issue tracker functions Figure out how we can improve the experience for contributors Fix more issues that people are running hitting, that may languish for a while At the moment, opening an issue presents us with …

What came out of it from my end was trying to get a better default issue / PR template for the Julia repo going, which was disliked very much in the triage call, citing “We actually feel it’s good to have a very loose organizational nature on the repo, without stringent requirements on what an issue/PR should fulfill”. Maybe that has changed since then, I don’t know 🤷 I certainly think that’s detrimental.

Luckily, not everything in the ecosystem is done that way - SciML in particular seems to have a very fast response time to any interaction, mostly due to @ChrisRackauckas being awesome when it comes to fixing things & getting PRs merged.

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [August 8, 2023, 4:29pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/6 "2023-08-08T16:29:20Z")

</div>

> [@Sukera](#):
>
> “We actually feel it’s good to have a very loose organizational nature on the repo, without stringent requirements on what an issue/PR should fulfill”.

I fully endorse this view, as I think this reduces imposter syndrome among new contributors. I understand where you are coming from, and I sympathize with your concern, but I wonder if there might be a better solution than to make the process feel more formal and stringent. While a well-defined issue template will certainly help with actually resolving an issue instead of it languishing for ages, this might also reduce the number of issues that are filed, as users would rather work around their annoyance than fail to meet the bar on a public forum.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 8, 2023, 4:38pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/7 "2023-08-08T16:38:57Z")

</div>

> [@jishnub](#):
>
> this might also reduce the number of issues that are filed, as users would rather work around their annoyance than fail to meet the bar on a public forum.

I understand your concern, but quite often, extremely basic information such as

- the julia version the issue occured on
- the platform the user runs their code on
- what the expected behavior was

simply isn’t present (even from seasoned contributors!). I’m not asking for a detailed 20-textfield questionnaire, I’m asking for a very minimal issue template that asks the user to fill in these basic requirements to even be able to debug & reproduce the issue, and to be able to easily decide whether some behavior is intended or not. If you think asking for these things induces imposter syndrome, I really don’t know what to tell you.

I’ll ALWAYS choose an easy to navigate process for newer contributors over a perceived slow-down for contributors, because that’s pretty much the only tangible thing we as a community can do to encourage new people to contribute, thereby _easing_ the load on existing contributors.

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [August 8, 2023, 4:40pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/8 "2023-08-08T16:40:29Z")

</div>

to be honest, I don’t think having a stricter template is the main barrier to swifter reviews.

for example, docs PRs seem to languish more than any other kind, and for those the version is mostly irrelevant

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 8, 2023, 4:51pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/9 "2023-08-08T16:51:00Z")

</div>

> [@adienes](#):
>
> for example, docs PRs seem to languish more than any other kind, and for those the version is mostly irrelevant

That’s fair, but that too is something we can easily accomodate. It’s not like we only have one issue or PR template; Github supports multiple, selectable when opening an issue/PR. I really don’t see a drawback here.

---

<div class="post-metadata">

**Author:** ![frylock](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/frylock/32/50213_2.png) [@frylock](https://discourse.julialang.org/u/frylock)\
**Post date:** [August 8, 2023, 5:27pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/10 "2023-08-08T17:27:11Z")

</div>

I had a pull request in that was a simple typo in a comment (it was a long time ago, I don’t even remember where it is now) - it literally made _no logic changes at all_. When the test suite ran, it failed all sorts of things … I didn’t really know what to do with that …

EDIT: I guess the above, coupled with the fact that Julia has almost 1000 open pull requests and 4000 issues … I figured that what very little help I could offer would be better spent elsewhere? I love working with Julia, and it seems to be working just fine without my help - but I imagine that is some of the reason you are not attracting new contributors.

---

<div class="post-metadata">

**Author:** ![Lilith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lilith/32/27492_2.png) [@Lilith](https://discourse.julialang.org/u/Lilith)\
**Post date:** [August 8, 2023, 6:33pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/11 "2023-08-08T18:33:56Z")

</div>

The fact that Julia CI is so flakey is a big problem that we need to work on. And, if you are willing, you can help!

1. Find a test that has recently failed in master (click on the red x’s on [this page](https://github.com/JuliaLang/julia/commits/master) for details of test failures on master)
2. Search [existing issues](https://github.com/JuliaLang/julia/issues) for one that tracks that test failure
3. If there isn’t already an issue filed, file one, linking to a test failure or two
4. Open a PR to skip the flakey test, and link to the issue you opened in that PR. (Feel free to @ me in this PR)

Failing tests on CI on master typically represent Julia bugs, but we try to track julia bugs in the issue tracker, not with failing CI. Shifting flaky tests to issues helps streamline reviewing all unrelated PRs.

If you have a PR with an unrelated test failure and can find that same failure on a recent commit to master, then you can skip straight to step 2.

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [August 8, 2023, 6:59pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/12 "2023-08-08T18:59:43Z")

</div>

Failing CI has been a problem in Julia since [at least 2014](https://danluu.com/julialang/). It makes contributions for newcomers easier if they have a reliable CI system. In some other projects, failing CI would block a PR. Why doesn’t Julia adopt that policy?

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [August 8, 2023, 7:10pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/13 "2023-08-08T19:10:01Z")

</div>

> [@Sukera](#):
>
> If you think asking for these things induces imposter syndrome, I really don’t know what to tell you.

I feel you’re vastly overestimating how comfortable most users are with GitHub

---

<div class="post-metadata">

**Author:** ![e3c6](https://avatars.discourse-cdn.com/v4/letter/e/e79b87/32.png) [@e3c6](https://discourse.julialang.org/u/e3c6)\
**Post date:** [August 8, 2023, 7:10pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/14 "2023-08-08T19:10:19Z")

</div>

> [@ChrisRackauckas](#):
>
> Reviews are easier on everyone the more tests you have. If you have very good test coverage, then the review is about code style more than correctness. I recommend everyone to just keep upping the test bar as that makes correctness checking exponentially easier every single time.

This check can be (partly) automated. Codecov can check a PR and fail if the percentage decreases. Then there could be an automatic comment explaining that the PR cannot be merged unless the test coverage goes up.

Same with code format (which is part of “style”), thanks to JuliaFormatter.

My point is that if these checks are automatized (up to a reasonable point), and the PR author can get immediate feedback (from a bot), then the PR quality will increase naturally. Also, I think that if a human reviewer starts asking a bunch of pesky quality issues it can feel a bit discouraging, but the feeling is different if these are automatic checks (which also means ALL PRs meet the same high standards).

Of course this automating this is extra work for maintainers. But the tooling is getting nicer each day. For instance @gdalle has been doing some very nice work with PkgTemplates. This could mean that the tooling for automatic quality checks will be in-place from package creation, without extra maintainer work.

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [August 8, 2023, 7:17pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/15 "2023-08-08T19:17:41Z")

</div>

In my experience, often providing the `versioninfo` isn’t enough, as the next step is to perform a git bisect to find the offending commit. I wonder if this might be automated? I’m digressing a bit, discussing issues instead of PRs here.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 8, 2023, 7:18pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/16 "2023-08-08T19:18:33Z")

</div>

> [@jishnub](#):
>
> I feel you’re vastly overestimating how comfortable most users are with GitHub

I don’t think I am; it’s precisely this inexperience that I think is our strongest signal for giving very clear directions in what is asked of users when submitting an issue. The current wall of text you see when opening an issue on the Julia repo certainly doesn’t achieve that.

In the end, having a clearer format for new issues also makes it easier for maintainers, because the information they’re looking for is always going to be in the same place. It’s not like I’m making this up as I go, the Linux Foundation recently put out a very good report on how open source projects organize their repos & contributions, and they’re saying pretty much the same thing:

> **[Open Source Maintainers](https://www.linuxfoundation.org/research/open-source-maintainers?hsLang=en)**
>
> Open Source Maintainers

---

<div class="post-metadata">

**Author:** ![jishnub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jishnub/32/33620_2.png) [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Post date:** [August 8, 2023, 7:35pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/17 "2023-08-08T19:35:33Z")

</div>

Perhaps we need this:

 ![7v5c2i](https://global.discourse-cdn.com/julialang/original/3X/b/b/bb5c1eb77d799cb1317dbc94e97c8ad82913be12.jpeg)

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 8, 2023, 8:08pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/18 "2023-08-08T20:08:52Z")

</div>

I really don’t find this sarcastic remark appropriate. In other threads we’re being encouraged to engage with others, to help out in the repo. We’re trying to do that here, and get a Clippy response, famous for annoying nagging. How are people supposed to take the earnest efforts here seriously, when we’re met with this? Do you not want us to engage & try to help out in any way we can? It even got marked as the “solution” to this topic, though I have no idea whether that wasn’t an accidental click and was meant as a ❤ instead. The buttons are too close on mobile.

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [August 8, 2023, 8:16pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/19 "2023-08-08T20:16:00Z")

</div>

I just thought it was amusing

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [August 8, 2023, 8:48pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/20 "2023-08-08T20:48:45Z")

</div>

Never underestimate the motivation power of a little bit of fun.

[Next page](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604.md?page=2)