A Gentle Guide to Your First JuliaCon Review

General Usage
,
1

Hi folks!

JuliaCon is just around the corner and I wanted to more publicly share this resource I put together for new reviewers of the (fantastic) annual JuliaCon. Hope it helps!

As there have been some questions about how to make reviews for JuliaCon, I made a small guide on making your own reviews for a proposal if you have any concerns about how to write a review. To start, here are the comments that I received for the lightning talk I gave, Using Julia for Observational Health Research:

Reviewer 1:

This sounds like an incredibly interesting talk and well worth the 10minutes (lightning talk format)

Reviewer 2:

This may be an interesting application of Julia in another area.

Reviewer 3:

It appears that this talk showcases the efforts of a large group of contributors in a variety of organizations and therefore potentially the effectiveness of composability in the Julia ecosystem.

As you can see, the reviews can be very concise. I try to make mine more direct and more detailed for the submitters depending on the familiarity I have with a particular submission topic. Here is an example of one review that I created (with relevant pieces redacted):

A word that comes to mind when I see this package is: innovative. I have not only followed the development of this package but also used it myself. As a developer who finds myself constantly within the Julia REPL, I am impressed by the extensibility brought to the Julia REPL by this package. There have been packages such as RCall.jl which introduce new modes within the REPL – this package extends the base Julia mode in ways I have never seen before. I rank this package very high in interest to the Julia community due to its technical feats but due to the niche of [redacted], its applicability may be somewhat limited. However, as the author states, even though [redacted] may be niche, the idea [redacted] is highly pervasive. I would like the author in their presentation to more explain the technical details and challenges encountered within their experience in creating this package. I would, nonetheless, recommend it for a slot as a JuliaCon Talk.

It can vary widely in terms of how much or how little you can write in a review as you can see. The main philosophy I keep in mind is this, “if I were to receive this review, would I find it useful in creating an excellent X?” (X being a poster, talk, etc.). Importantly, at the end of the day, choose kindness in writing your reviews. Hope this helps and feel free to follow-up with questions or your insights! :smile:

~ tcp :deciduous_tree:

7 Likes
2

Some simple Q&A based on points I don’t think are clear due to the unique nature of the relationship between submitters/reviewers for JuliaCon. I included the answers from the julia slack as subpoints.

  • Is reviewing a package you use/have used is okay? If I use [redacted].jl in my day-to-day work, does reviewing it constitute a conflict of interest even if I don’t have any relationship with the developer?

    • No. Otherwise some talks would get zero reviews :wink:. Just try to be fair and give your honest opinion. In particular if it’s something that everyone is using it probably does not deserve a high “topic diversity” score.

  • Some proposals may include instructions to, e.g. execute a command in your shell to run an example script perhaps using this as a substitute for a better description. Is this allowed and/or good practice?

    • I think something like this can be well taken into account in a review. Feel free to add it to your comments. We will distribute them back to the submitter (in anonymised form) once the decision has been reached.

  • On workshops: with 30min talk proposals that seem quite niche, I might a put a comment like “this might be more appropriate as a lightning talk due to its niche nature and limited time available in the conference” and put slightly lower on applicability and contribution to community. Should I evaluate workshops in a similar manner, or are even the most niche workshops to be accepted as long as they are “sound”?

    • There is definitely a cut-off for workshop nicheness. A good workshop should be accessible to an intermediate Julia user - previous years had GPU progamming, HPC, etc. which aren’t everyone’s area but should be accessible and interesting to a significant fraction of the community. For example, I wouldn’t suggest accepting a very deep workshop on a specific technique in genomics but in Julia, but a workshop giving a general survey of computational techniques in genomics would be good. You can also give feedback suggesting the authors broaden the scope of the proposal if you feel it’s too niche.

  • Should the score take into account the requested format? E.g. giving a high score for an experience talk would mean “this is a very good fit for an experience talk”, right?

    • Indeed, that is exactly the idea.

  • How should we recommend to change a full-length talk to a lightning talk?

    • Just write an appropriate sentence in the review. Our scripts for detecting a request to change from full-length to lightning in reviews react to the following keywords: “lightning”, “10” (because of 10 minutes)!

  • Since there aren’t online lightning talks, how should we approach online talks that might benefit from shortening to ~10 minutes? Should we just recommend for acceptance in that case?

    • For online talks we are less constrained (well we don’t need a slot in one of the rooms :wink:). So: We’d still want your feedback that 30 mins might be too long, but if the proposal is sound it should be accepted.
1 Like
3

@logankilpatrick or @kslimes , would you accept this as a PR somewhere to the JuliaCon website or somewhere on the Julia website? Trying to think of how we could socialize this information further.

2 Likes
4

Adding a PR to the JuliaCon site would be cool with me!

2 Likes