While user studies can tell you a lot about the usability problems in your interface and help you identify incremental improvements to your design, they can’t identify fundamental flaws and they can’t tell you whether your design is useful. This is because you define the tasks. If no one wants to complete those tasks in real life, or there are conditions that change the nature of those tasks in real life, your user study results will not reveal those things. The only way to find out if something would actually be used is to implement your design and give it to people to see if it offers real value (you’d know, because they wouldn’t want you to take it away).
When thinking about designers perspectives and role this does not surprise me, however thinking from the users perspective it does as I always assumed user studies were the ultimate way to test a design, but considering what the author stated about them (at times) missing fundamental flaws which can change that perspective. It’s interesting how the author says that considering designers define the tasks, the results can’t show whether people would actually want to do those tasks in real life. It makes me realize how important it is to test a design’s real-world value, not just its usability from seeing if people would actually miss it if it were taken away.