Validating
Log in

The market just repriced human judgment

The biggest user-testing platform just rebranded around humans in the AI loop. What the repricing of human judgment means for founders validating ideas.

The largest paid user-testing platform rebranded this week. UserTesting, a name that described exactly what it sold for nearly two decades, is now Auros, with a stated mission of bringing humans to the AI experience. In the forums where its testers compare payouts, the first reaction was suspicion about what changes for them. The more interesting question is what the rebrand says about everyone else.

Companies do not rename themselves around a resource that is getting cheaper. They rename themselves around the thing customers will pay a premium for next. The incumbent of an entire industry just bet that the premium ingredient of the AI era is human judgment.

What changed in the economics of building?

Paul Graham wrote this morning that a lot of things only worked because there is a limit to how fast humans can operate, and that we are about to find out what all of them are as they break. Building was the clearest one. The effort of shipping used to be a filter: if something existed, someone had cared enough to make it. Agents removed that filter. The same day, the CEO of Linear described running coding agents inside the product at raw token prices, with no margin on the build step at all. When the people selling the shovels stop marking up the digging, the digging is no longer where the value is.

Why does judgment gain value as generation gets cheap?

Because the two were always a pair, and one side just collapsed in price. Every generated thing, a feature, a landing page, a product, an answer, is a claim that somebody will want it. Generation produces claims. Judgment prices them. When producing claims cost months of engineering, the volume stayed low and judgment could be informal, a founder's instinct, a few user calls. At agent speed, the claims arrive faster than any informal process can price them, and unpriced claims pile up as risk.

Scarcity moved. It did not disappear. What is scarce now is a person with the right context who will tell you the truth about whether a thing is worth wanting, and whose answer costs them something to give.

Is AI not supposed to replace exactly this?

That was the pitch of synthetic research: simulated users, persona agents, instant feedback at zero marginal cost. The problem is structural. Models are trained on what people said, and the whole discipline of validation exists because what people say diverges from what they do and pay. A simulated customer is a very fluent average of stated preferences. It cannot be disappointed, it has no budget, and it never has to live with the purchase. The companies closest to this market appear to have reached the same conclusion, which is why the rebrand says bringing humans to AI, not replacing them with it.

What should founders do with this?

Stop treating building as progress and start treating it as the cheap part. If an agent can produce your prototype in a weekend, the prototype proves nothing about demand, and the discipline moves upstream. Budget for judgment the way you used to budget for engineering. Real people, matched to the market you are claiming, looking at the idea before the build, with an incentive to be honest rather than polite. That used to be a luxury reserved for funded teams. It is quickly becoming the only part of the process that was never automated.

The rate limit on making things is gone. The rate limit on knowing what is worth making was never a rate limit. It was always people.

Key takeaways

  • Incumbents rebrand around what customers will pay a premium for next. The research incumbent just chose human judgment.
  • Generation produces claims, judgment prices them. Agent speed floods the first side and starves the second.
  • Synthetic users are fluent averages of stated preferences. They cannot be disappointed and never pay.
  • Budget for judgment like you once budgeted for engineering. The build step stopped being the expensive one.

Cheap building did not make validation optional. It made it the whole game.