A number worth writing down came out of the market research industry this week. Adriana Rocha, founder of Wortya, won a bronze at the 2026 Tony Cowling Foundation Innovation Award for research that asked a plain question: if you give AI models two years of real grocery purchasing data from 2,500 households, how well can they simulate what those households actually buy next? The answer, reported in the Foundation's interview with her, was at most 20% fidelity.
Twenty percent. With two years of ground truth to learn from. On groceries, the most repetitive, habit-driven purchase category that exists.
What does 20% fidelity actually mean?
It means that when the model predicted what a real household would put in the basket, it was wrong about four times out of five. Not wrong about an opinion or a preference statement. Wrong about recorded behaviour, measured against receipts. This is the strongest possible setup for a synthetic consumer, dense historical data on a stable habit, and the ceiling still sat at one in five.
Now compare that setup to how founders actually use synthetic users. No purchase history. A product that does not exist yet. A category with no habit to learn. If the best case is 20%, the startup case is not better.
Why do simulated buyers miss so much?
Because buying is not a language task. Models are trained on what people write and say, and people mostly write and say their stated preferences. Purchases run on things nobody types into a dataset: the budget conversation at home, the brand the store was out of, the price that felt wrong that day, the habit nobody can explain. What people say is not what they do or pay, and a model trained on saying inherits the gap.
There is a second problem we have written about before: synthetic respondents agree too easily. A simulated user has no budget, no switching cost and no embarrassment. It can like your idea at no cost to itself, which is exactly the property that makes human compliments worthless as evidence.
So are synthetic users useless?
No, and Rocha's own framing is the honest one: a filter, not an oracle. Synthetic research is good at the cheap early work. It can stress-test how you describe the problem, surface objections you had not considered, and kill the obviously weak version of an idea before you spend a real person's time on it. That is real value, and it is preparation.
The failure mode is substitution. The moment a synthetic panel's enthusiasm is treated as demand, you have built your plan on the 20% machine and called it the market.
What should a founder do with this number?
Use the machine for what it is good at, then go where the other 80% lives. The other 80% is a stranger with their own money, their own time and no reason to be nice to you. Ask them what they do about the problem today. Ask what it costs them. Then ask for a commitment and watch what happens to the enthusiasm. That reaction is the only data source the model was missing.
Key takeaways
- AI models with two years of real purchase data simulated actual buying with at most 20% fidelity, per research reported by the Tony Cowling Foundation.
- That was the best case: dense history, habitual category. A new product with no history starts below it.
- Synthetic users are a filter for sharpening questions, not an oracle for predicting demand.
- The missing 80% of buying behaviour only appears when a real person reacts with real money or time.
A model that is wrong four times out of five is a fine sparring partner. It is a terrible judge.