There’s an order of magnitude difference in the effort (and hence funds) necessary for VS Code vs. debugger.
What is the committee that organizes applications to JSoC/GSoC/NumFocus? What is the process from project proposal to selection? What if there is budget for a subset of proposals, who decides which proposal is selected?
Thank you for the clarifications so far.
Everything about decision making and participating in GSOC/JSOC is listed on the Julia JSOC page. In fact, we are always short of people to vet the projects. Broadly, the group weeds out the obviously bad ones, and then sends our wishlist to Google, which then comes back with a decision on how many they can fund. The same group naturally decides which ones to fund, based on the review process. As you can imagine, none of this is an exact science.
The NF small projects has been a bit ad-hoc, because there have been very few awards, but it is roughly the same group that does GSOC.
7 Likes
From that page, the JSoC committee that reviews applications is this one:
Is that list up-to-date? It is mostly JuliaHub again.
How to increase diversity to prioritize other areas that are not a priority currently? It is clear that Chris will prioritize projects related to SciML for example. How to mitigate conflicts of interest?
How can we help with the (s)election of an independent committee?
Alternatively, could the JSoC review of proposals start with a formal call on Discourse, Zulip, etc, where anyone in the community can give a vote to rank the best applications before they go to the committee? Could that be a reasonable strategy to mitigate biases and conflicts of interest?
Again, I pointed out that there is a review process. The review is carried out by a broad team. It would be helpful if you actually participate and point out real problems.
I have repeatedly pointed out that all these are open processes. If you would like to help out, please join and help. If you would like there to be fewer JuliaHub people, I am sure many of us will be happy to take a step back if someone wants to take responsibility and demonstrate it.
Do you have any reports of conflict of interest? You repeatedly keep complaining about JuliaHub, which only makes people who are volunteering feel crappy (not only do they volunteer but there is a stream of questions and complaints that they have to respond to), and diminishing the roles of others who are not at JuliaHub.
7 Likes
The biggest thing you can do is help find mentors for areas that you believe are a priority and get them to add those areas to the JSOC page. You can also join the review committee and help with the selection process.
The GSOC process is extremely elaborate, and needs a lot of help.
I am trying. I know that lack of diversity is not intentional. It is something that just happens when you dont monitor and act on it.
An open process can still suffer from lack of diversity. Especially when committees are built without widespread votes and consensus.
I’d be happy to participate in the committee that review JSoC proposals if there is an open spot, and if I can make the decision process more transparent and public. If I just join the committee with the current review process Ill just add my own biases to the selection of proposals.
I am sorry, but the people who are actually doing the work do not need to be monitored by another committee. The processes are well known, and there are mechanisms in place for raising complaints, bringing up improvements, making changes, etc. For example, we did not start on day one with all of this - it was patiently built by the contributions of many people over the years.
I couldn’t agree more about lack of diversity - but the way to fix that is not by monitoring - but by actually recruiting new people who will make the group diverse.
You are again saying that there is lack of diversity - but you just said you don’t even know what the review committee is. I understand you are bringing up issues and trying to help improve processes, but the best way to help is by actually participating and making incremental changes.
The review process does not have a fixed number of slots. However, the review process is private by design - because it is not a good idea to make people’s failures public.
13 Likes
Sorry, I think you misunderstood what I tried to say. I said monitoring the lack of diversity, not people’s work.
If the JSoC review process and pre-selection of proposals cannot be made open before the final decision by Google’s committee, then this is an issue to discuss. Can you elaborate on why it cannot be open?
As a matter of principle, as I already said above it is not a good idea to make someone’s failure public (and I don’t speak for the committee officially - just myself). If we make reviewer’s comments, scoring, and everything public - then anyone who didn’t get selected will have a public record of their failure. Why should that follow them around for their career?
Also, in reality, this is not much of an issue, since we pretty much have been able to fund most meaningful projects - with ones that GSOC does not fund being funded through JSOC.
2 Likes
I personally don’t see any issue with anonymous peer-review. It is not perfect but well-established in academia.
But to address biases and conflicts of interest one needs to either s(elect) a committee that is more diverse or let the entire community participate in a pre-filter/ranking of proposals.
The first proposed solution requires sending an invitation to people across countries and companies to compose a diverse pool of candidates. This can be done by the Stewards or group of trustees. The community could then vote in this pool of candidates, assuming only N committee members are needed.
The second proposed solution involves designing a multi-step review process where the community has a chance to pre-filter and/or rank the proposals before they are sent to the committee.
I read this subthread and thought “wait, didn’t we have VS Code and IDE-related project ideas in G/JSoC”? And it turns out the answer was yes, and the page is still up at VS Code projects, but the link from the main ideas page was removed in Remove VS Code JSOC projects page by pfitzseb · Pull Request #1590 · JuliaLang/www.julialang.org · GitHub. If I had to guess (and please correct me) this was a matter of not enough bandwidth as David talked about upthread.
FWIW, I remember the process for getting new ideas or modifying existing GSoC idea lists for the Julia project was pretty simple. There was more than one call for ideas earlier in the year, and the whole process was a simple GitHub PR with some basic info (summary, scope, possible mentors). I know some GSoC projects (including ones I helped submit) didn’t move forward because no students applied to work on them, but at least they were all presented equally on Projects.
5 Likes
@juliohm I repeatedly said that there is a review committee that scores the proposals. It is not only the project admins (which seems to be what you are thinking it is and is perhaps the source of confusion). I also repeatedly have invited you to participate, as well as help identify volunteers to improve the diversity. More so, I repeatedly said that we are short on people everywhere, and we have repeatedly asked for volunteers in various roles.
8 Likes
This thread seems to have reached a stage where it is going in circles. I think @juliohm has made their views clear, and @viralbshah has explained how the various parts of the community involved in “funding” are connected, and what the practical limitations of supplying funds to things like the VS code extension or the debugger are (no matter how many of us in the community would love to see more progress in these tools).
As for diversity and transparency goals, it seems to me that if @juliohm would like to effect any changes, the best thing to do is to volunteer in those more formal groups like the JuliaCon committee, and to discuss with other members of those groups if or how any particular process could be improved. I’m not sure what is to be gained by further airing vague grievances in this public forum. So maybe it’s time to wind down this thread?
9 Likes
I certainly do, happy to chat with anyone who is interested.
14 Likes
I set a timer for this topic to close in 24 hours. Feel free to open more specific topics to continue the discussion, for instance about @davidanthoff’s proposal to describe a roadmap and budget. I renamed the current one to “Development resources for Julia’s ecosystem?”, which is where the conversation ended up going.
1 Like
@viralbshah I am confused still, but your answers helped a lot already. I am continuously asking to discover the committee in charge but every time I ask the answer is different, with a slight variation of members from the same org. You can also see that many people in this thread dont know the Julia governance model well. Some think that the Stewards are the main representatives of open source Julia, then you came and corrected us saying that the JuliaCon committee is the one with more power, but not for JSoC/GSoC. Then I found the JSoC admins on the website you shared, which I now learned aren’t the ones who give scores to the proposals.
I am slowly getting it, but I am not sure I have the full picture of the review process, and who decides what. Perhaps this process is not even fully understood by committee members who ended up in the committees for no strong reason other than a closer relationship with people nearby.
Given that this topic will be closed, inhibiting a direct continuation of the discussion on the governance model, I suggest that we write down our current understanding of the different committees and the kinds of decisions they are responsible for.
In particular, I would like to draft a hierarchical structure (because that is how most societies are organized) with my current understanding of the different committees involved in the Julia governance model.
My plan is to find some time to draft a diagram/illustration in graph form where nodes are committees and edges are decisions flowing from one committee to another. @viralbshah would you help review this draft to make it ready for the governance page on the official website?
I have two profiles in mind:
- A new user/member that just arrived and doesn’t know to whom he/she should ask a specific question with issues that involve a large community. For example: “How can I help the VSCode extension? Are there resources to submit a proposal?”
- A company/organization that is seeking a clear governance model that is independent of any particular organization in order to fund projects.
The edges between the committees can indicate various different things. For instance, it would be useful to know when a certain type of decision needs to be sent up in the hierarchy. If a question can be promptly addressed by the leaves of the graph, it should not consume the time of other committees. I believe that a FAQ below the diagram will help with this.
Regarding the “same org” (JuliaHub) and diversity arguments: Note that the JuliaCon 2024 committee was, IMHO, quite diverse, with @ranjan being the only one from JuliaHub (and he wasn’t officially representing the latter but was contributing on his own behalf, as a private person).
2 Likes