PROs, UX/UI Design for trials, and the nocebo effect | Paul Wicks, PhD

The episode moves through three connected ideas.

First: clinical trials have a UX problem.

One in five trials recruits zero patients. Half under-recruit. Paul's early work was understanding why – and fixing it. The answer was almost never the science. It was that patients had no parking, caregivers had no wifi (or babysitting!), and the protocol assumed everyone could get to Mass General by 9 AM on a Friday via public transit. Solving that required showing trial designs to actual patients before locking them, and listening to their recorded reactions rather than just summarizing their survey scores.

Second: measuring patient experience is genuinely hard.

There are no inches of insomnia. No pounds of pain. Patient Reported Outcomes – PROs – exist because the most burdensome chronic conditions don't have actual units that can be measured. There are thousands of PROs, many are outdated (the fibromyalgia questionnaire from the early 2000s asks how well you can vacuum and cook for your family), and most were designed for clinical trials, not for weekly check-ins on a mobile screen. So for you, Product Manager/Engineer/Builder: how are you going to measure whether your Care Model actually improves patient health?

Third: building PROs into a product creates specific traps.

Paul runs through the ones he has seen firsthand: copyright violations (most scales are licensed and litigation can cost millions), engagement drop-off (5–10% of users is a good day for consistent tracking), incentive fraud (cash rewards attract bot farms), the nocebo effect (asking about pain can worsen it: hey, 1-10 scale, how itchy are you right now?), and the regulatory ceiling – go too far with your tracker and you've crossed into medical device territory, with all the compliance that comes with it.

The Five Things to Know Before You Build

1. You probably don't have permission to use that questionnaire. Most validated scales are owned by universities and licensed for a fee. Saying you found it published online doesn't mean it's free. The database PROQOLID lists rights holders, and some of them are quite litigious 

2. Only about 5–10% of patients will track at the cadence you're planning. The ones who do are not representative of your broader user base. Obsessive trackers skew your data. Users with executive dysfunction may not be able to log in, let alone complete a questionnaire.

3. Cash incentives destroy your data. If you offer a financial reward for completion, expect an avalanche of bots and fraudulent responses. Survey researchers report 80–90% fraud rates in incentivized studies. Things that seem to work better: setting group goals, an AMA with the scientists behind the study, and giving users a summary of what their data revealed.

4. Asking about symptoms can worsen them. The nocebo effect is real (for example: scale of 1-10, how itchy do you feel right now?). Paul's team designed positively-framed questions ("how well are you sleeping?") rather than deficit-focused ones ("how bad is your insomnia?") specifically to reduce iatrogenic harm.

5. Interpreting a score triggers regulation. Tracking is fine. Saying that a score of 5/5 means "severe" COULD make your app a medical device and suddenly make you beholden to a lot more regulation

Brought to you by

Nirvana - Nirvana delivers real-time eligibility checks and cost estimates and plugs right into your existing workflows

For inquiries about sponsoring the podcast, email sales@outofpocket.health

Resources

• Paul Wicks on LinkedIn — linkedin.com/in/paulwicks
• Paul’s newsletter, ProofPoints
• Paul’s podcast, Prove It!
• Paul’s website, ProofStack Health
• RAAPID Inc course on Risk Adjustment: V28, AI, and multi-million dollar settlements. Apr 7-9
• Sponsor the Hardware Hackathon! Apr 17-19 in SF!
• PROQOLID — eprovide.mapi-trust.org — find the rights holder for any validated PRO scale
• PHQ-9: free, validated, widely used depression screener
• GAD-7: free, validated anxiety scale

Timestamps

• 00:00 — Paul's origin story: ALS research, patient communities, and clinical trial ethics
• 03:00 — PatientsLikeMe and showing trial protocols to patients before locking them can actually INCREASE recruitment
• 11:00 — What PROs (Patient Reported Outcomes) are, why there are so many, and why none of them agree
• 18:00 — Pharma, payers, providers, patients – and who actually cares about PRO data
• 26:00 — The fifth stakeholder: scale developers. Beware for license fees and lawsuit risk
• 30:00 — Which patients actually track data, and what to do about everyone else
• 33:00 — False starts: psoriasis body maps, crab-to-clam scales, and positive framing
• 38:00 — The regulatory ceiling and pharmacovigilance