Synthetic Research Explained: A Powerful Tool To Support, Not Replace, Human Insight

4 snips

Mar 29, 2026

Jordan Harper, Senior Principal Thought Leader at Qualtrics with a physics and tech background, explains synthetic research and synthetic audiences. He breaks down what synthetic panels are and are not. He discusses validation versus human data, how wording shapes responses, and when synthetic approaches are best used alongside real people.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

LLM Trained On Survey Responses Mimics Human Messiness

Synthetic panels simulate how humans answer surveys by training an LLM on millions of real survey responses.
Qualtrics spins up a unique LLM instance per synthetic respondent that walks through the survey and returns row-level responses mirroring human distributions.

INSIGHT

Synthetics Reveal Different Signals By Lacking Human Bias

Synthetic respondents avoid common human survey biases like social desirability and priming, producing different signal than humans.
Example: priming about smartphones shifted human cohorts but synthetics produced a fuzzy, non-directional distribution instead.

INSIGHT

Generic LLMs Give Overconfident, Stereotyped Results

Generic LLMs like ChatGPT mean-revert to stereotypical answers and lack human-like response distributions.
In tests ChatGPT answered 'yes' to climate change nearly 100% of the time, whereas the tuned model matched human distribution.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

How far can we trust research that is generated without asking a single human being?

In this episode, I sat down with Jordan Harper from Qualtrics to unpack one of the most talked-about developments at the Qualtrics X4 Summit, synthetic research. It is a topic that sparks curiosity, excitement, and a fair amount of skepticism in equal measure. And honestly, that tension is exactly why this conversation matters.

Jordan brings a rare mix of scientific thinking and real-world technology experience, which makes him well placed to cut through the hype. We explored what synthetic panels actually are, and just as importantly, what they are not.

While many assume this is simply about asking a large language model for answers, the reality is far more nuanced. The approach Jordan and his team are building is grounded in how humans respond to surveys, trained on vast datasets to reflect the inconsistencies, biases, and unpredictability that make human insight valuable in the first place.

What stood out throughout our conversation was the idea that synthetic research should be seen as additive rather than a replacement. It offers speed, flexibility, and the ability to test ideas quickly, but it does not replace the depth and lived experience that only real people can provide. In fact, some of the most interesting insights come from comparing synthetic responses with human ones, revealing patterns, biases, and even blind spots in traditional research methods.

We also got into the practical side of things. From controlling for issues like survey fatigue and social desirability bias, to experimenting with question design in ways that would be difficult with human respondents, synthetic research opens up new ways of working. At the same time, it raises important questions about validation, trust, and where to draw the line when decisions carry real-world consequences.

For me, this episode is about perspective. In a world where AI is accelerating everything, it can be tempting to look for shortcuts. But as Jordan explains, the real value comes from using these tools thoughtfully, alongside human insight rather than in place of it.

So as this technology continues to evolve, how should researchers and business leaders strike that balance? And where could synthetic research help you ask better questions before you make your next big decision?