The graphs ecosystem

gdalle
Fri, 26 May 2023 20:54:11 +0000
Graphs

Hey everyone,

The very fruitful discussion on fixing package fragmentation started from a remark I made on Slack about graphs, which is why I’m asking the community for ideas and support.

In my opinion, the most important things we can do (by order of priority) are:

1. List existing packages to allow comparison

2. Redesign the core interface to make it more usable

3. Ensure reliability of the main packages for the long term

What follows is a brief summary of each item, but feel free to suggest more! Sorry in advance for the long post.

1. Existing packages

At this stage, the JuliaGraphs organization is recognized as the main place to look for graph-related Julia code. But that doesn’t mean it is the only place you can find such code! Incoming users are often confused about the competing standards, and there are easy fixes.

My proposal would be to add external packages to the JuliaGraphs website, so I’ve started a list. Please help me complete it, and tag your friends who develop graphs packages in secret!
For the purposes of this list, I’m excluding packages related to reading / writing / plotting graphs, as well as optimization algorithms, to concentrate only on data structures.

The list

“Organization”: does it belong to a GitHub organization?

“Maintainer”: who is listed first on the GitHub contributors list?

“Focus”: what does the package do?

“Comp(atible)”: does it have Graphs.jl in its dependencies?

“Reg(istered)”: does it belong to the general registry?

“Act(ive)”: has there been at least one commit to main in 2023 (I know it’s arbitrary)?

Last update of the list: 31-05-2023
Main interfaces (click for more details) Metadata (click for more details) Specific fields (click for more details) Multiplicity (click for more details) Structure (click for more details) Efficiency / safety (click for more details) Linear algebra (click for more details) Archives (click for more details)
A few remarks

On the positive side:

I’m amazed at the productivity and creativity of this community

A sizeable portion abides by the Graphs.jl interface

On the negative side:

There are plenty of registered but inactive packages, which means good discoverable names go to waste

The proliferation of standards, especially for metadata, is concerning

2. Core interface

If we want the Graphs.jl interface to be even more widely accepted, we have to make it better. This starts with a more generic approach, which would make it easier to handle metadata or multigraphs. The question of vertex and edge indexing is therefore central.

You’re more than welcome to contribute to the discussion, either here or on GitHub.
A few relevant issues (click for more details)
We also have to make sure that all algorithms implemented in the code and satellite package respect the interface specifications and do not make additional assumptions. As far as I can tell, this is not necessarily the case.

3. Long term reliability

As all open source developers know, taking care of such a vast ecosystem takes a lot of effort. I really think the community we’ve built around graphs has a great deal to offer in terms of performance and flexibility, but to succeed it would need:

More guidance and discussions (like this one), to focus contributions on a few key packages instead of wasting time and energy

More packages joining the organization as a way of diluting the workload

More reactivity from maintainers (including me), to tackle issues and merge PRs

More financial and human resources, ie. people who are actually paid to work on the ecosystem

I plan to recruit a half-time master’s student in the fall to start working on these issues. I was too late for the JSoC / GSoC calls, but as I understand there are others who got projects accepted in this general area (aurorarossi at least).
On the long term, it would be great if we could secure some funding for the JuliaGraphs organization.
I’m just a junior postdoc, but if anyone has ideas of grants we could apply to, let me know! I would love to make this open source mission a sizeable portion of my work, along with more theoretical research projects (I’m currently exploring graph neural networks, ping CarloLucibello).

Enough with the monologue. What do you think? Where should we go? How do we solve the Nebraska conundrum?

PS: Discourse limited me to 10 mentions, but feel free to add more among the usernames I couldn’t tag
@mbesancon @CarloLucibello @simonschoelly @jpfairbanks @viralbshah @aurorarossi @filchristou @etienne_dg @SuperGrobi @jlapeyre

36 posts - 13 participants

Read full topic