Checking internal consistency across questions

In this episode, I cover:

* The fear and skepticism many researchers feel toward synthetic users, especially around job security and research quality.

* How a synthetic panel works in Qualtrics, step by step, including setup, question design, and early signals.

* The tension between stated advice and lived behavior in synthetic data, and how that tension becomes a clue for deeper human follow-up.

* How synthetic results can help shape hypotheses, narrow scope, and surface mental models worth examining with human participants.

* The role of experimentation, reality-checking, and ethical use when bringing synthetic insights into a human-centered research practice.

Key Takeaways:

* Synthetic users aren’t a replacement, they’re a low-stakes way to surface potential thinking paths worth exploring. Fear of being replaced is real for many UXRs, but synthetic panels don’t replicate lived experience. They can spark ideas, highlight tension in responses, and point toward questions worth asking humans, but they don’t carry nuance, emotion, memory, or contradiction. They’re an extra tool, not a takeover. 

* Synthetic panels help you see mental models earlier, especially the ones users rarely say out loud. The synthetic example in the video about routines revealed goal-driven thinking mixed with self-doubt, which is a pattern worth validating with real people. This gives researchers a head start when writing interview guides or structuring probes. It doesn’t give you truth, but it does give you direction.

* Synthetic data is great for pressure-testing your own questions before running a study. I described how running a synthetic version of a study I’d previously done with humans showed where the survey and interview questions held up and where they needed tightening. This kind of dry-run can save time, catch weak spots, and help teams narrow scope before talking to real people.

* Researchers still need to reality-check everything with humans. Synthetic outputs are predictions shaped by large datasets, not lived stories. Human sessions reveal timing, emotion, contradictions, and subtle meaning shifts that synthetic models can’t replicate. You can use synthetic to form hypotheses, but every hypothesis needs human evidence behind it. 

* Ethical and intentional use must lead the way. Researchers should be the ones teaching teams how to use synthetic panels responsibly. That means knowing where they fit, where they fail, and how to protect user trust. Synthetic tools aren’t going anywhere, so UXRs benefit from learning how to guide their use with clarity and care.

The companion guide to synthetic users:

Want to learn even more about synthetic users? Check out the companion guide to this video which goes in-depth about responsible, intentional, and ethical synthetic user usage.

Try Qualtrics:

Want to try this out on Qualtrics? You can request a demo below:

Interested in sponsoring the podcast?

Interested in sponsoring or advertising on this podcast? I’m always looking to partner with brands and businesses that align with my audience. Reach out to me at nikki@userresearchacademy.com to learn more about sponsorship opportunities!

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.userresearchstrategist.com/subscribe