Harvard study says AI review comments comprehensively outperform high-level human peer review
Two researchers from Harvard University published a paper on October 22, comparing 134 pairs of review comments written by AI and humans; AI led comprehensively across seven evaluation dimensions. Three experts who had served as editors-in-chief of major academic journals conducted 30 blind evaluations, and in 26 of them they considered the AI review comments better; among the specific claims checked, the AI factual error rate was 5.5%, compared with 15.9% for humans. The study used Claude Opus 4.8 to generate review comments and GPT-5.5 to serve as an unaware AI judge; even without internet access, GPT-5.5 chose AI in all 134 comparisons.
#Papers/Research#Evaluations / benchmarks#Anthropic