A team builds a prototype, runs five usability testing sessions, watches participants complete the core task without much trouble, and concludes the product is ready to build. The usability test did its job correctly — it answered whether people can use the interface. It never asked, and cannot answer, whether the underlying concept addresses something people actually need. Those are two different research questions, and treating a good usability result as validation of the concept is one of the most common substitution errors in UX research.
Usability testing assumes the concept is already right
Usability testing is built around a specific, narrower question: given this design, can representative users accomplish a defined task, and where do they struggle? The method inherently assumes the task itself is worth doing and the concept behind it is sound — it's testing execution, not premise. A usability test can return excellent results (fast task completion, no confusion, high satisfaction) for a product that nobody actually wants, because the test was never designed to ask that question in the first place.
Discovery research asks the question usability testing skips
Discovery research — problem-space interviews, contextual inquiry, opportunity assessment — is built to answer a prior, more fundamental question: is there a real, sufficiently important problem here, and does this general direction address it, independent of any specific interface. This kind of research typically doesn't involve a prototype at all, precisely because introducing a specific design too early anchors participants on evaluating that design rather than surfacing the underlying need. Discovery research that's done well often changes the entire direction of a product; usability research, by design, refines within a direction that's already been set.
Why the substitution happens, and why it's so easy to miss
Usability testing is more concrete, easier to schedule, and produces a shippable-feeling readout (87% task completion) that reads as validation, which makes it attractive to run even when the actual open risk is at the concept level, not the interface level. A team under time pressure to do the research often defaults to whichever method produces the clearest, fastest-to-run result, rather than the method that actually answers the riskiest open question — and usability testing on an already-built prototype is almost always faster to arrange than genuine discovery research with people who haven't seen any solution yet.
How to tell which one you actually need
If the open question is whether people can find, understand, and complete a specific task within a design you've already committed to, usability testing is the right method. If the open question is whether the underlying idea addresses something people would actually value enough to change behavior for, no amount of usability testing on a prototype will answer that — the study needs to start before a prototype exists, with people who haven't been shown any proposed solution yet.
What this means for how research gets planned
- Name the actual open risk before choosing a method — concept risk and usability risk require different studies, not different versions of the same one
- Don't let a shippable-looking usability metric stand in for concept validation it was never designed to measure
- Run discovery research before a prototype exists whenever the real uncertainty is about the underlying problem, not the interface
- Treat strong usability results on a weak concept as confirmation that the interface works, not confirmation that the product should exist
Both methods are legitimate and necessary at different points — the failure isn't choosing the wrong one occasionally, it's not distinguishing which question is actually open before deciding which one to run.