A market researcher posted something this week that deserves to be printed and taped to every roadmap. Her trigger phrase was three words: we already know that. Her answer was that what we think we know and what customers are actually thinking are not always the same thing, and sometimes the thing everyone knows is a comment a customer made five years ago that somehow became an undisputed fact inside the organization.
Every company has these. Two or three load bearing anecdotes that nobody can source anymore. Someone important heard a customer say it, once, and the quote got promoted from observation to doctrine without ever standing for re-election.
Does validation evidence expire?
Yes, and faster than you want. Evidence is a photograph of a market at one moment, taken from one angle. The customer who said it may have changed jobs, tools, budgets or opinions. The alternatives available to them have certainly changed. A quote from 2021 about what they would never pay for predates most of the products they now pay for.
The strange part is that teams treat evidence as permanent precisely because collecting it was painful. The interview round took a month, so its conclusions get amortized over five years. The cost of gathering the evidence becomes the argument for never gathering it again.
How does a quote become a fact?
The promotion path is always the same. First it is a data point in a research doc. Then it is a slide in a strategy deck, with the customer's name still attached. Then it is a sentence in an onboarding doc, name gone. Finally it is just how things are. Our users will not pay for this. Our buyers do not read email. Enterprise will never accept that flow. By the last stage, questioning the sentence sounds like questioning the company.
The same week, a podcast retrospective about Calm resurfaced a perfect counterexample. The team noticed a spike in meditation listening around bedtime and initially read it as people using the product wrong. Their own assumption said meditation happens in the morning. The behavior said otherwise, and looking again at that spike led to Sleep Stories, one of the biggest product decisions in the company's history. The assumption was not stupid. It was just old, and the behavior was current.
Which assumptions should you re-test first?
The ones nobody questions. Not because they are the most likely to be wrong, but because they have been unexamined the longest, and because the cost of being wrong about them compounds quietly. A fresh assumption gets challenged in every meeting. A five year old one is invisible, which is exactly what makes it load bearing.
A practical filter: write down the three sentences about your customers that your team repeats most often. For each one, ask two questions. Who exactly said this, and when. If the answer is somebody, at some point, you are not holding evidence. You are holding folklore.
Then re-test the sentence the same way you would test a brand new idea: against current strangers, with their current alternatives, watching what they do rather than collecting what they say. The point is not that the old quote was wrong. It might have been right then and wrong now, which is a different failure with the same price.
Key takeaways
- Evidence is a photograph of a market at one moment. It expires, and nobody dates the photograph.
- Old quotes get promoted from observation to doctrine because collecting them was expensive.
- The most dangerous assumptions are the ones that stopped being checked when they started being believed.
- If you cannot name who said it and when, it is folklore, not evidence. Re-test it like a new idea.
Why this today: a market insights researcher wrote this week that the assumptions nobody questions are the ones she most wants to explore, and a Calm retrospective showed what re-reading one old assumption can be worth.