[RFC] Solving Slack's lack of history

Problem: With an increasing number of users and messages, the number of days with saved history on Slack is currently 10 (as noted by Makie’s layout creator, Julius Krumbiegel). This means that searching for past solutions, comments, or anything is quickly becoming impossible on the free plan.

Solution: Write a Slack app that allows all users to select an interval of messages, one or more packages the questions and answers in these messages pertain to, and auto submit a PR to a standard FAQ page in the documentation of those packages. The maintainers of those packages can 1) reject the PR, 2) blindly accept the PR (to later review it more carefully perhaps), or 3) curate and edit it into the documentation (either into the FAQ page or into the actual documentation of the package).

Advantages:

  1. We don’t need to worry about the lack of history on Slack (assuming of course that the history is longer than the time span it takes questions to get answers). So no need for migrating to some other web-chat service.
  2. Packages get a free boost to their documentation: with what I believe would be 4 user-actions (message span starts here, ends there, the relevant packages are these, send auto generated PR), the amount of effort would be minimal and more users would just do it™.

Feasibility: @SaschaMann said he can do it! :grinning:
Personally, I have no idea how that would actually work (he mentioned also coupling it to github actions).

Questions:

  1. What do you think, do you like it? Do you hate it?
  2. What difficulties do you foresee? How do you think we can solve these?
  3. Can we involve Documenter.jl somehow? Any specific format or structure this FAQ page should follow?

Alternative solutions: We could solve it with money (and pay Slack for full functionality), that’s about 850 USD or 6000 USD per month (pending on an open-source discount). Alternatively, we could spin our own web-chat server, and or use Matrix, requiring a mass-migration.

See also this previous topic, and more recently, this slack thread.

7 Likes

You can add context menu entries to Slack messages. If someone clicks on it, a request is sent to your app’s server. You could respond with a dialog (apparently these have been deprecated but there’s a new alternative) where the user can choose the relevant package, add information etc. I’ve written something similar to add messages to a task manager before and it was just under 200 lines of Python. This would require a bit more logic and some state keeping to be able to group multiple messages, but it seems doable. (I did not say I have the time to write it, though :stuck_out_tongue: )

The main problem I see is that, at least right now, there’s no standard location and format that the question would be added to.

If such a standardized FAQ file would exist, this could also be coupled to a GH Actions command/trigger that adds issues to the FAQ. There are probably other places where it could be used, too. One could also use an app like this to easily crosspost questions and answers to other places like StackOverflow.

I actually think this would be nice to have as a way to copy Q&A information more easily regardless of which chat app is used.

1 Like

Since we are already Discourse users, why not build a feed from Slack to Discourse? Pick a thread or message and select a destination Discourse sub-category. If needed it can be moderated.

Spinning up our own server is probably one of the cheapest solutions. Gotta migrate away from Slack though.

I always thought that Julia Slack is intended for rather short ephemeral chatting? Longer conversation is going to Discourse and Github and thus lack of proper history not such a big issue?

In case of replacement, the open source Zulip chat sounds very interesting. They adress open source projects and host for free (scroll down to FAQs). There is a pending issue to read history without the need to log in.

(This said, I never used Zulip myself, just heard about it. Slack I use when necessary, but not fond of it and I’m not signed up to Julia Slack atm)

2 Likes

The only advantage of that would be that one could search and find this information again in Discourse. I argue that it would be better for this info to end up in the documentation of the package/s it relates to (even if it initially is in a category-less place such as a FAQ page) rather than in Discourse. I also believe it would be easier for the contributors to specify a package/s that the slack messages relate to than Discourse’s sub-categories – most Slack channels are package (or organization) specific. Lastly, from the point of view of the package maintainers, the torrent of imported messages from Slack would be a lot easier to ignore if it ends up in Discourse than if it ends up as PRs on your package’s repo. So it is more likely that these messages will end up in a package’s documentation if it goes directly to the package.

Regardless of what Slack is intended for, there are always some messages that are worth saving.

At the risk of restating the obvious, why not just use a platform that preserves history? Github/Gitlab (latter can be self-hosted), Discourse (we already have it), etc.

2 Likes

For sure, the question being, what would be easier, this Slack app, or building and hosting our own platform and then migrating to it? I honestly don’t know the answer to that, but if the only thing you’ll migrate for is the lack of history then this app should resolve that (plus a few other advantages I mentioned before).

I don’t think you need to build anything. There are many platforms for discussion which remember history, I am typing this on one :sunglasses:. AFAIK

is also used as an open source alternative.

Since there is no history, there is not much to migrate. :wink:

Incidentally, since Slack makes money from making history available, I am not sure what their policy is about making an app that does this.

2 Likes

That might be a real problem. I’d argue though that we’re not saving everything, just snippets. Still, there might be a legal issue here.

But then, on a more fundamental level, I don’t see the proprietary Slack app as a good place to store such messages.

With Zulip, Julia can apply for open source hosting and wouldn’t need to host themself. From Hacker News I get the impression that Zulip is well liked (but as said no personal experience). Maybe it needs a little bit more time, but I see it as a potential nice Julia chat environment.

:smile:

2 Likes

I must have been unclear: the messages (with the questions and answers that are worth saving) are gonna be auto packed into a PR to a FAQ page of the package/s that these Q&A pertain to. As in, the doc folder in the package online (git- hub/lab or whatnot). So this app will just pipe the info to the package/s’ documentation. It won’t retain anything.

If Zulip (or any other alternative) is significantly better then great. I was under the impression though that agreeing on an alternative and moving there was difficult (telling from the last survey).

BTW, with migration I only mean moving the users there, not the history.

2 Likes