While user studies can tell you a lot about the usability problems in your interface and help you identify incremental improvements to your design, they can’t identify fundamental flaws and they can’t tell you whether your design is useful. This is because you define the tasks. If no one wants to complete those tasks in real life, or there are conditions that change the nature of those tasks in real life, your user study results will not reveal those things. The only way to find out if something would actually be used is to implement your design and give it to people to see if it offers real value
I think usability tests are very useful for identifying interface breakdowns. However, they can’t show whether a design truly provides value in real life. Designers often focus on whether users can complete tasks in a controlled setting. They assume that success there means success in the real world. As Ko points out, task completion alone doesn’t measure whether someone would actually want to use the product or integrate it into their routine. I’ve seen prototypes work perfectly in tests but fail when deployed because the tasks felt artificial or didn’t meet real user needs. That’s why real-world testing is so important. It’s absolutely harder and takes a lot more time, but it truly shows how people actually use a design in their daily lives.