Everything Hertz 190: What happens when you pay reviewers?
Apr 2, 2025
A lively look at experiments that pay peer reviewers and whether money speeds things up. They compare a $250 randomized trial with a contracted retainer pilot that produced much faster turnarounds. Conversation covers trial design, measuring review quality, costs and scaling, and who might realistically fund paid review models.
AI Snips
Chapters
Transcript
Episode notes
Heathers Frames Paying Reviewers As An Equity Issue
- James Heathers clarifies his intent: the push to pay reviewers is about equity, not personal gain.
- Heathers framed the idea as supporting poorly paid grad students, postdocs, and junior faculty who do much peer review.
Modest Gains From One-Off $250 Review Payments
- Offering $250 per review produced only modest improvements in speed and acceptance rates.
- In Critical Care Medicine's two-week block experiment reviews returned ~1 day faster and acceptance rose from 48% to 53%, but quality scores didn't improve.
Context Matters Because Some Journals Already Have High Engagement
- High baseline reviewer engagement can limit measurable effects of payment.
- In that journal ~50% of invitees accepted and ~50% of those completed reviews, producing unusually high conversion rates compared to many fields.