peterme.com: Is Lab Usability Dead?

Peter Merholz: Is Lab Usability Dead?.

And now we're in a world people are interrupted an average of every 11 minutes. Where they have multiple applications, and multiple windows within those applications, open. Where people are storing passwords and bookmarks on their machines. Where people are managing multiple devices, from their computer to their cell phone to their iPod.

And what have we done? We still practice "lab usability," where we invite someone into a sterile neutral environment (either an observation facility, or a conference room), and ask them to use a strange computer to engage in a series of tasks (whether scripted or self-motivated, it doesn't matter). And so while we might understand how well someone can use our tool in this exceedingly artificial world, it falls short of helping us appreciate how this tool will fit into their ever crazier contexts.

Yeah - totally agree. I think lab-based usabilty testing has some value (and even more if you can wheel the lab to user/customer locations), but fundamentally we don't spend enough time really understanding real user needs in real user environments: we substitute ethnographic processes for surveys, focus groups, project team brainstorming because, well, they're easier and better understood (or at least more familiar to a wider set of people).