[ANN] FeedBackAndForth.jl

Package Announcements
1

Hi all,

For me, one of the great benefits of the current AI tools has been that I can get fast and quite useful feedback on my drafts. I recently worked on judgment aggregation and crowd wisdom, which gave me the idea of organizing a kind of panel review of my drafts, where multiple LLMs (which could be both cloud models and local models) can discuss a draft before making final recommendations. The results were beyond what I had hoped for.

Organizing the panel review wasn’t very difficult, but I was able to turn this into a Julia package only with the help of Claude. The result can now be found here. I wasn’t able to find anything exactly like it on the Internet, but I could easily have missed things. (I know there is refine.ink, which gives very detailed feedback but which is quite expensive; the package creates reviews at very low cost, or even zero cost if you only run local models.) If you know of similar software, I’d love to hear about that. If you give the package a try, feedback and contributions are very welcome.

6 Likes
2

Seems great… I think this is an example that would benefit to be converted to an app or even as a standalone compiled app, as it is not intended to be used as a package for code development..

2 Likes
3

I tried it and it’s very cool. I often ask AI models for writing feedback, and it’s interesting how the same AI’s output hits differently when in mock review form. Also quite fun to see generated reviewer interplay. I used Gemini and DeepSeek, and they made good points, found useful weaknesses, and offered constructive feedback, based significantly on fact! I could literally go through them point by point and get a significantly improved manuscript, no joke.

I do think this package is lacking in realism, however :slight_smile: For one, AFAIK human reviewers don’t read manuscripts these days, they just hallucinate their reviews. (The high quality here makes me wonder why AI-generated reviews have been so terrible to date.) Also I think the conception of journal “quality” here is a bit fanciful. Reviews are so full of (human) slop these days, I’m not sure what differentiates a “premier” journal except more desk rejection and shorter reviews. So I think the AI temperature needs to be turned way up here. (This paragraph is not really about FeedBackAndForth.jl, just musings on the state of modern publishing.)

OP I look forward to your next submission, FeedBackAndForthAndBack.jl which completes the full submission-review-revision loop, so we can get rid of human authors altogether!

1 Like
4

Thanks for the useful feedback and kind words, @sylvaticus and @apo383. I’ve added a couple of features (possibility to let the meta-reviewer be an LLM which did not participate in the panel review, and the possibility to set level of detail of the review).

@sylvaticus, I agree, it would be nice to have this as an app, but I leave that to others.

@apo383, I agree with what you say about refereeing (though I still prefer to write my own reports). But let’s try to keep the human author for as long as possible (which may not be very long).