RJrjakob·Jun 1, 2025
We also provide feedback on rigor across 7 different categories: https://github.com/robertjakob/rigorous/tree/main/Agent1_Pee...
In Show HN: AI Peer Reviewer – Multiagent system for scientific manuscript analysisAgentic-Systems-Lab
rigorous
Python2025–2025
Agentic-Systems-Lab / rigorous
rigorous
A comprehensive suite of tools, built to liberate science by making the creation, evaluation, and dissemination of research more transparent, affordable, and efficient.
3Distinct people
7Mentions
0Explicit recommenders
253GitHub stars
What to know before exploring
What it is
A comprehensive suite of tools, built to liberate science by making the creation, evaluation, and dissemination of research more transparent, affordable, and efficient.
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 1:49:46 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by hour7 dated mentions
Observed mentions: 2025-05-31 13:51:16 UTC to 2025-06-01 03:01:37 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| May 31, 13:00 | 2025-05-31 13:00:00 UTC | 2025-05-31 14:00:00 UTC | 1 | 1 | 0 |
| May 31, 14:00 | 2025-05-31 14:00:00 UTC | 2025-05-31 15:00:00 UTC | 0 | 0 | 0 |
| May 31, 15:00 | 2025-05-31 15:00:00 UTC | 2025-05-31 16:00:00 UTC | 0 | 0 | 0 |
| May 31, 16:00 | 2025-05-31 16:00:00 UTC | 2025-05-31 17:00:00 UTC | 0 | 0 | 0 |
| May 31, 17:00 | 2025-05-31 17:00:00 UTC | 2025-05-31 18:00:00 UTC | 0 | 0 | 0 |
| May 31, 18:00 | 2025-05-31 18:00:00 UTC | 2025-05-31 19:00:00 UTC | 1 | 0 | 1 |
| May 31, 19:00 | 2025-05-31 19:00:00 UTC | 2025-05-31 20:00:00 UTC | 3 | 0 | 3 |
| May 31, 20:00 | 2025-05-31 20:00:00 UTC | 2025-05-31 21:00:00 UTC | 0 | 0 | 0 |
| May 31, 21:00 | 2025-05-31 21:00:00 UTC | 2025-05-31 22:00:00 UTC | 0 | 0 | 0 |
| May 31, 22:00 | 2025-05-31 22:00:00 UTC | 2025-05-31 23:00:00 UTC | 1 | 0 | 1 |
| May 31, 23:00 | 2025-05-31 23:00:00 UTC | 2025-06-01 00:00:00 UTC | 0 | 0 | 0 |
| Jun 1, 00:00 | 2025-06-01 00:00:00 UTC | 2025-06-01 01:00:00 UTC | 0 | 0 | 0 |
| Jun 1, 01:00 | 2025-06-01 01:00:00 UTC | 2025-06-01 02:00:00 UTC | 0 | 0 | 0 |
| Jun 1, 02:00 | 2025-06-01 02:00:00 UTC | 2025-06-01 03:00:00 UTC | 0 | 0 | 0 |
| Jun 1, 03:00 | 2025-06-01 03:00:00 UTC | 2025-06-01 04:00:00 UTC | 1 | 0 | 1 |
What Hacker News says
7 sourcesHere is a description of how it works: https://github.com/robertjakob/rigorous/tree/main/Agent1_Pee...
In Show HN: AI Peer Reviewer – Multiagent System for Scientific Manuscript AnalysisWe are already looking into that: https://github.com/robertjakob/rigorous/tree/main/Agent2_Out...
Would be great to see contributions from the community!
In Show HN: AI Peer Reviewer – Multiagent system for scientific manuscript analysis...or run it themselves. The code is open source:
https://github.com/robertjakob/rigorous
Note: The current version uses the OpenAI API, but it should be adaptable to run on local models instead.
In Show HN: AI Peer Reviewer – Multiagent System for Scientific Manuscript AnalysisNo worries, I appreciate that you took the time to read and respond!
When I first read "Originality and Contribution" at [1], I actually assumed it was a plagiarism check. It did not occur to me until now that you were referring to novelty with that. Similarly, I assumed "Impact and Significance" referred to more about whether the subject was appropriate for a given journal or not (would readers of this journal find this information significant/relevant/impactful or should it be published elsewhere?). That's a question that many journals do ask of referees, independent of overall novelty, but I see how you mean a different aspect of novelty instead.
I'm not opposed to testing your system with a manuscript of my own, but currently the one manuscript that I have approaching publication is still in the internal review stage at my national
Read the rest
lab, and I don't know precisely when it will be ready for external submission. But I'll keep it in mind whenever it passes all of the internal checks.
[1] https://github.com/robertjakob/rigorous/blob/main/Agent1_Pee...
I had a quick look at the repo as I wondered what you meant by multiple specialized agents.
Funsamentally each of those 24 agents seem to be just:
"load from pdf > put text into this prompt > Call OpenAI API"
So is it actually just posting 24 different prompts to a generalist AI?
I'm also wondering about the prompts, one I read said "find 3-4 problems per section....find 10-15 problems per paper". What happens when you put up a good paper, does this force it to find meaningless, nit-picky, problems? Have you tried papers which are acknowledged to be well written on it?
From a programming perspective the code has got a lot of room from improvements.
The big one is if you'd used the same interface for each "agent" you could have had them all self register and call themselves in a loop rather than having to do what you've done in this file:
Read the rest
https://github.com/robertjakob/rigorous/blob/main/Agent1_Pee...
TBH, that's a bit of a WTF file. The `def _determine_research_type` method looks like a placeholder you've forgotten about too, as it use a pretty wonky way to determine the paper type.
Also, you really didn't need specialized classes for each prompt, you could have just had the prompts as text files a single class loaded as templates that you just replace text into. That will mean you're going to have a lot of work whenever you need to update the way your prompting works, having to change 24 files each time, probably cut/pasting which is error prone.
I've done it before where you have the templates in a folder, and the program just dynamically loads them. So you can add more really easily. Next stage is to add pre-processor directives to your loader that allows you to put some config at the top of each text file.
I'm also not looking that hard at the code, but it seems you dump the entire paper into each prompt, rather than just the section it needs to review, which seems like an easy money saver if you asked an AI to chop up the paper, then just inject the section needed to reduce your costs for tokens. Although you then run the risk of it chopping it up badly.
Finally, and this is a real nitpick but it's twitch inducing when reading the prompts, comments in javascript are two forward slashes not a hash.
Show HN: AI Peer Reviewer – Multiagent system for scientific manuscript analysis
After waiting 8 months for a journal response or two months for co-author feedback that consisted of "looks good" and a single comma change, we built an AI-powered peer review system that helps researchers improve their manuscripts rapidly before submission.
The system uses multiple specialized agents to analyze different aspects of scientific papers, from methodology to writing quality.
Key features: 24 specialized agents analyzing sections, scientific rigor, and writing quality // Detailed feedback with actionable recommendations. // PDF report generation. // Support for custom review criteria and target journals.
Two ways to use it:
1. Cloud version (free during testing):
https://www.rigorous.company
- Upload your manuscript
- Get a comprehensive PDF report within 1–2 working days
- No setup required
2. Self-hosted version (GitHub):
Read the rest
https://github.com/robertjakob/rigorous
- Use your own OpenAI API keys
- Full control over the review process
- Customize agents and criteria
- MIT licensed
The system is particularly useful for researchers preparing manuscripts before submission to co-authors or target journals.
Would love to get feedback from the HN community, especially from PhDs and researchers across all academic fields. The project is open source and we welcome contributions!
GitHub: https://github.com/robertjakob/rigorous
Cloud version: https://www.rigorous.company