# Making the review process more pleasant for new contributors

**URL:** <https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604>\
**Category:** General Usage\
**Tags:** github, contributing\
**Created:** [August 8, 2023, 2:00pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604 "2023-08-08T14:00:25Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 11, 2023, 9:26pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/41 "2023-08-11T21:26:10Z")

</div>

> [@gdalle](#):
>
> If you want to upvote a new issue template:

Just so it’s shared here as well, I’ve revived the old template I had lying around in a PR. Please do try it out on my fork (as described in the PR) and leave feedback!

> <https://github.com/JuliaLang/julia/pull/50881>
>
> Implements https://github.com/JuliaLang/julia/issues/50861
> 
> This is based on t…he \[Issue forms\](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms) feature by Github. You can try it out on \[my fork\](https://github.com/Seelengrab/julia) of the repo; if you want to experiment with this yourself, note that you have to make changes on the \`master\`/main branch of your repo, since that is the only place github will look for these templates.
> 
> Note that this still preserves the ability to create a blank issue, right below the "Bug" and "Feature Request" buttons.
> 
> I also ironically don't know what to label this PR as - we don't have a "repo" or "meta" or "issue tracker" label..

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [August 12, 2023, 12:28am UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/42 "2023-08-12T00:28:26Z")

</div>

> [@gdalle](#):
>
> Yeah, this means that a documentation change can be breaking. Which is kind of messed up

IME that’s handled by a new function and giving the old one a deprecation warning, but it’s not something that anyone wants to be routine, so it’s important to weigh what goes into promises. That said, we do also document things as being experimental or unstable, so documentation can also warn people of things that can break.

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [August 14, 2023, 7:29am UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/43 "2023-08-14T07:29:47Z")

</div>

I do wonder if we would benefit from some automation around detecting first PRs.  
Maybe in the form of a bot that cross-posts their PR to a slack channel, and/or that posts a “welcome, thanks for your first PR” message.

For a couple of reasons.

1. such a message could also help set expectations for what will happen next
2. getting a fast first review, sets a good impression and once people have had their first PR or two through they kind of understand how it goes so are likely more patient
3. Reviewers knowing this is someones first PR might be extra careful to be kind.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 14, 2023, 9:06am UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/44 "2023-08-14T09:06:29Z")

</div>

I do agree with that, but I also think that for such a “what will happen next” message to be effective, we need to be clear about what actually happens next. As far as I’m aware, there simply isn’t a process for how PRs are reviewed, if they’re not already related to some work that some core developers are already doing & talking about off-repo. So at the moment, what would such a message actually be?

For me at least, this subjective feeling that there is no process for PR reviews (as well as quite serious bugs in both exported functions as well as documentation not getting fixed) is slowly leading to the impression that Julia is less being maintained, and more being actively hacked on, to put out whatever fire is currently perceived to be burning…

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [August 14, 2023, 10:09am UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/45 "2023-08-14T10:09:36Z")

</div>

> [@Sukera](#):
>
> So at the moment, what would such a message actually be?

Something like:

Congratulations on your first julia PR!  
From here the process will proceed roughly as follows.  
One or more people will review the PR, and will likely have suggestions for how can be improved, and discussions as to if the change should be merged or not. You will change the PR, and the process will continue again. This will continue until it ceases in the mode of [fixed point iteration](https://en.wikipedia.org/wiki/Fixed-point_iteration). Finally someone with merge rights will merge the PR.  
If your PR seems to be not getting responses, it is acceptable to bump it after about 10 days (you can do this by posting on it a message, e.g. “bump”). We try to keep everything moving, but sometimes things do slip off the radar.  
If your PR is a new feature, or in some way contentious, it will get the [Needs Triage](https://github.com/JuliaLang/julia/pulls?q=is%3Aopen+is%3Apr+label%3Atriage) label applied, and it will be discussed as the next triage meeting (time permitting). See [the calendar](https://julialang.org/community/#events) for when that is.  
Triage will make the final call as to if a PR should (philosphically) be merged or not.  
Everyone is welcome at the triage meetings, similarly everyone is welcome to review, you don’t need special permission.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 14, 2023, 10:46am UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/46 "2023-08-14T10:46:21Z")

</div>

Apologies to be blunt, but that sounds incredibly like “Your PR will not meet our extremely high standards”, which is subjectively a gross clash with the reality of how PRs to the repo actually happen. There are no high standards to get a PR merged, it’s mostly persistence & bumping that eventually gets PRs merged (if noone raises an objection - and even minor objections can result in huge delays in that process, due to a lack of follow up). Worse, if some change seems incredibly minor (without pointing fingers, e.g. [here](https://github.com/JuliaLang/julia/pull/49700)) there is usually no consideration for the bigger picture - to me, that indicates a lack of consistent standard being applied to PRs. Fixing that would do more than having a Bot post some generic “open source works on iterated improvement” message 😕

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [August 14, 2023, 10:51am UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/47 "2023-08-14T10:51:33Z")

</div>

It’s not meant to.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [August 14, 2023, 11:47am UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/48 "2023-08-14T11:47:58Z")

</div>

I like your proposal @oxinabox. My only concern is how to keep any promises we make. Since we (like most projects) depend on volunteer labor, can we really pledge the time of volunteers? Heck, my experience of reporting bugs in commercial software tools does not leave me entirely optimistic that a meaningful response can be guaranteed even when a company pays people whose only job is to provide customer service.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 14, 2023, 12:09pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/49 "2023-08-14T12:09:27Z")

</div>

Personally, I’d prefer a wording more like this, reducing combativeness of the response.

> Thank you for creating a PR to Julia!
> 
> Before this PR can be merged, we’ll have to check that it’s in a mergeable state, as well as  
> whether the change is desirable from a content perspective (this is usually more work for new features than for documentation changes). A reviewer will then suggest changes, if necessary, to help get the PR over the finish line. Because the Julia repository is run primarily by volunteers, this can take some time - please don’t hesitate to bump your PR after ~a week, if noone got to reviewing it yet or you’re missing some follow up information!
> 
> Once a PR is ready to be merged, it’s usually merged within a short period of time. If a PR is inconclusive/may need a larger decision, it’s labelled `triage` and discussed on the next triage call every two weeks (see [the community calendar](https://julialang.org/community/#events) for the next date).
> 
> Thank you again for contributing to Julia!

but as mentioned above, I feel like this only addresses a symptom, not the cause.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [August 14, 2023, 3:11pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/50 "2023-08-14T15:11:05Z")

</div>

Since we’re touching at the other side of the coin on how to make the review process more pleasant/less frustrating for committers and maintainers, what ideas could be explored here? Has the Julia project ever considered using GitHub’s [code owners](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners) functionality or some similar mechanism like area-specific teams (e.g. a hypothetical `@JuliaLang/LinearAlgebra`) so that more eyes/hands can share the task of seeing and triaging PRs? I’ve noticed on occasion that issues and PRs may sit stagnant because either nobody knows who would be suited to review them, or someone is pinged but doesn’t materialize (likely because notifications are buried). I’m not sure going full on bureaucracy like some bigger projects have done is a good idea, but being able to remove the single point of failure that is one person checking every one of their outstanding GH notifications seems achievable.

---

<div class="post-metadata">

**Author:** ![ParadaCarleton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paradacarleton/32/20005_2.png) [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)\
**Post date:** [August 14, 2023, 4:47pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/51 "2023-08-14T16:47:11Z")

</div>

Besides what’s been brought up in this thread, there’s also the possibility of getting GPT-4 or PaLM-2 to do a first-pass review. That should be enough to check for most mistakes, prioritize as “mostly ready/not ready,” and make sure that documentation is up to par–I’ve found GPT-4 to be very good at writing docs, in particular.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 14, 2023, 5:03pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/52 "2023-08-14T17:03:35Z")

</div>

I wouldn’t trust GPT to do that, no. GPT has no clue about bigger picture software design/maintenance, especially when it comes to Julia content.

---

<div class="post-metadata">

**Author:** ![ParadaCarleton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paradacarleton/32/20005_2.png) [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)\
**Post date:** [August 14, 2023, 7:04pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/53 "2023-08-14T19:04:50Z")

</div>

Not on its own, clearly, but I don’t think there’s any harm in adding it as a first check.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 14, 2023, 8:40pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/54 "2023-08-14T20:40:59Z")

</div>

Even then - this is surely going to add a lot of noise, with how inconsistent it getting things correct actually is.

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [August 15, 2023, 2:41am UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/55 "2023-08-15T02:41:03Z")

</div>

I think it is irksom and annoying, to be judged by a stocatic bot (vs a linter which i know how to please).  
Having experienced this on other repos, I hated it.  
And given the various ethical issues around LLMs, I can see it turning people off.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [August 15, 2023, 9:31pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/56 "2023-08-15T21:31:15Z")

</div>

Bumping these GitHub bots as a way to help with reviewer selection and prioritization. There are likely more out there

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [August 16, 2023, 5:44pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/57 "2023-08-16T17:44:25Z")

</div>

[Where should Julia feature requests go? - #19 by mbauman](https://discourse.julialang.org/t/where-should-julia-feature-requests-go/102835/19) has a great summary of how other big and successful projects in this space manage their issues. Is anyone aware of a similar analysis for PRs/code changes? It would be nice to borrow from an approach that we already know works and run trials/quick experiments to get the ball rolling.

---

<div class="post-metadata">

**Author:** ![moble](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/moble/32/23535_2.png) [@moble](https://discourse.julialang.org/u/moble)\
**Post date:** [August 16, 2023, 7:56pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/58 "2023-08-16T19:56:52Z")

</div>

> [@tim.holy](#):
>
> We don’t (deliberately) merge PRs that cause new failures. Most current causes of failures probably passed CI when they were initially submitted, but intermittent issues or hardware changes or subtle interactions with other changes might cause them to (sometimes) fail; if you have enough flaky tests and failure of just one is enough to call the CI run a failure, then CI failures are common.

I wonder if a [merge queue](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/merging-a-pull-request-with-a-merge-queue#about-merge-queues) would help with this problem.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 16, 2023, 8:24pm UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/59 "2023-08-16T20:24:16Z")

</div>

We turned it on for about a day a month ago and then turned it off (because people couldn’t agree as to what the default merge strategy should be, as well as some other issues)

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [July 18, 2024, 2:18am UTC](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604/60 "2024-07-18T02:18:03Z")

</div>

> [@Sukera](#):
>
> There are no high standards to get a PR merged, it’s mostly persistence & bumping that eventually gets PRs merged

The fact that the process for getting a PR merged is “be annoyingly persistent with frequent pings” should probably be documented somehow, because it’s not intuitive, leading to frustration for unaware new contributors. E.g., see this PR:

> <https://github.com/JuliaLang/julia/pull/54820>
>
> \`Base.Broadcast.BroadcastFunction\` not documented and \[caused some inconvenience…\](https://github.com/LuxDL/Lux.jl/issues/687#issuecomment-2170637044)
> 
> This PR add \`BroadcastFunction\` to documentation

The PR just adds a doc string for a public symbol to the docs.

> > NB: the procedure for getting a PR merged is to keep pinging until it gets reviewed and merged.
> 
> It doesn’t matter, this is the last time.  
> Feel free to close this PR

[Previous page](https://discourse.julialang.org/t/making-the-review-process-more-pleasant-for-new-contributors/102604.md?page=2)