Autoresearch agents are reshaping the research pipeline, but they also let flawed claims enter the literature at scale. Human advisors catch such issues on in-progress drafts through careful, traceable feedback — yet advisor-style assessment requires extensive manual effort and does not scale. To shift automated paper assessment from a judge to a diagnostician, we introduce PaperDoctor, an agent framework for pre-submission feedback. Every paper is evaluated across writing, layout, references, code, theory, prior work, and experiments through three layers: L1 surface screening runs on every submission; L2 typed verifiers route each claim to the skill that owns its evidence; and L3 reproducers rerun experiments by priority. Each finding is a triple — an observation (Why), a pointer to a specific location (Where), and a revision suggestion (How) — making every critique auditable and actionable. On a human study of junior researchers' pre-submission drafts, PaperDoctor reaches 85% agreement; across 60 manuscripts spanning ML, the natural and social sciences, it produces more grounded feedback than human and LLM reviewers, and its reproduction stage surfaces limitations invisible from the manuscript alone.
Unlike a reviewer's verdict, each PaperDoctor finding is auditable and directly actionable.
What is wrong and the reasoning behind it — flagged as an error when confidently incorrect, or a warning when it needs the author's judgment.
A pointer to the exact location: a sentence, an equation, a code line, or an external URL — so the critique can be traced without searching the paper.
A concrete revision the author can apply — not just a statement of what is wrong, but a recommendation for what to change.
PaperDoctor reframes automated paper assessment as a diagnostic process rather than a verdict.
A paper, its code, and its data are decomposed across seven dimensions and organized into three levels of increasing cost — spending effort proportional to the cost of verification.
Each finding pairs an observation (Why) with a concrete location (Where) and a revision suggestion (How), making every critique auditable and fixable.
Beyond reading, PaperDoctor selectively rebuilds and reruns experiments by claim importance and compute budget — surfacing reproducibility gaps invisible from the manuscript alone.
Cheap screens run by default; typed verifiers are routed by evidence; full reproduction is gated on the author's approval.
Low-cost, mostly objective checks that run directly on every submission. All four skills run in parallel.
Every extracted claim is dispatched to the verifier that owns its evidence. Verifiers and per-claim dispatch run in parallel.
The most expensive stage. A prioritized plan from L2 is executed only with the author's approval, spending compute where evidence is the constraint.
PaperDoctor's feedback is more grounded than human and LLM reviewers, and tracks paper quality.
Human and LLM reviewers concentrate ~76–90% of their attention on the paper body, rarely opening the released code. PaperDoctor distributes attention differently — 18% of its findings check the surrounding literature and 12% verify the released code — and by executing that code it uncovers discrepancies between reported and actual results that reading-only review cannot detect.
If you find PaperDoctor useful, please consider citing:
@article{lin2026paperdoctor,
title = {PaperDoctor: Evidence-Grounded and Actionable Feedback
for Scientific Papers in Progress},
author = {Lin, Kevin Qinghong and Hu, Siyuan and Lu, Pan and Chen, Yu and
Chen, Yanzhe and Queen, Owen and Chen, Yupeng and Yu, Jialin and
Yu, Junchi and Ding, Zifeng and Ji, Yuanfeng and Liu, Sheng and
Gu, Jindong and Li, Linjie and Shou, Mike Zheng and
Torr, Philip and Zou, James},
journal = {arXiv preprint},
year = {2026}
}