Synthetic research panels: where they work, where they ship you off a cliff

Synthetic panels do not fail uniformly; they fail by question type, and the failure modes are predictable enough to plan around. What follows is our editorial assessment, not a head-to-head study we ran.

Where synthetic panels tend to track real panels closely: brand awareness, category usage, demographic-stable preferences, and copy-test top-box scores in familiar categories. These are the lower-risk applications.

Where they diverged badly: anything emotional or identity-laden, anything in a new category the model has weak priors for, and any question with a socially desirable answer. The synthetic panel will confidently agree with whatever the question implies.

The recommendation is not 'never use them.' It's 'use them where the failure mode is acceptable.' Concept screening at the funnel top, where false positives cost a meeting and false negatives cost an idea, yes. Final go/no-go calls on a $2M campaign, no.