A new tool launched this week that reads every review of an app and ranks what to fix first, by how many people hit each complaint instead of who shouted loudest. That is a genuinely useful ranking, and the founder was honest about where it shines: apps with hundreds of reviews, ratings that dropped after an update, competitors whose unhappy users are a free roadmap.
The interesting question is the one review mining cannot answer, no matter how good the tool gets. Reviews are a census of survivors. Demand lives mostly among the people who never showed up.
Who actually writes app reviews?
A sliver of users, at emotional peaks. Industry estimates put reviewers around one percent of an app's users, and that one percent is not a random sample. People write when they are delighted, furious, or asked at exactly the right moment. So a review corpus over-represents anger, habit-breaking bugs and payment surprises, and under-represents the quiet middle who shrugged and moved on. Ranking complaints by frequency fixes the loudest-voice bias. It cannot fix the sample.
What do reviews measure well?
Friction inside a product that people already chose. Why subscribers cancel. Which update broke trust. What a competitor's users wish were different. For a live product with real distribution, that is operational gold, and mining it beats guessing. Every one of those signals is conditional on the same event: the person found the product, wanted it enough to install it, and used it enough to have an opinion.
Why can't reviews validate what to build next?
Because the people who define your opportunity never make it into the dataset. The person who searched, saw the screenshots and left. The person who has the problem but never thought software could solve it. The person using a spreadsheet at 11pm instead of any app at all. None of them write reviews anywhere. A founder who studies three competitors' one-star reviews learns how to lose those competitors' least happy users slightly less often. That is not the same as learning whether a new product has a market.
There is a second trap: reviews describe the product category as it exists. If the honest answer to a market is a different category entirely, no amount of review mining will say so, because nobody reviews the thing that does not exist yet.
What fills the gap reviews leave?
Evidence from the people who are not in any review section: the matched humans who have the problem today. What they actually do about it, what they pay now, what they tried and abandoned, what they would need to switch. That takes asking specific, verified people before building, not reading what strangers wrote after someone else built. The two instruments answer different questions. Reviews tell you why people leave. Only real people can tell you why they would come.
Key takeaways
Reviews are written by a tiny, emotional, self-selected slice of existing users. They measure friction inside a chosen product, not demand for a future one. The people who never arrived leave no reviews, and they are where new opportunities live. Competitor review mining improves retention playbooks, not validation. To know why people would come, ask matched humans before building, and weigh what they do over what they write.
The line that matters
A review section is the market grading your execution. It will never grade your idea, because everyone who rejected the idea left before the form.