In an earlier post, I focused on the challenges of leading a good user interview. Today, I’d like to focus on one rule about asking questions to get reliable results.
##Ask open-ended questions. Otherwise, the responses you get will be either biased or useless.
##Let’s look at some interview questions (from worst to best)
They appear in order: from least useful to most useful in getting you good responses. For context, imagine you’re doing some user research and you’re in the middle of a project to design some widgets.
-
“You like this widget in green, right? We picked green because it’s calming.” This question is so bad it’s grotesque, but it may still happen to novice interviewers. Not only does the question suggest a specific answer (you like it!), but it also gives reasons why that answer might be the right one (green is calming!) The interviewee is not likely to share a genuine opinion with this much pressure to say “yes, I love it in green!”.
-
“How do you feel about this widget? Do you like its color?” Slightly better, but still of limited use. By suggesting a response and allowing the user to narrow it down to a binary yes/no, we’re getting a very limited amount of information back. In addition, likes and dislikes move the conversation into a potentially awkward area. Some people may not be comfortable telling you they don’t like something you made.
-
“How do you feel about this widget? What do you think it does?” Now, this is better. We are opening up to qualitative responses with lots of detail. By not suggesting options for what an answer might be, we are more likely to get unexpected, valuable results.
-
“Talk me through what you see here.” A super open-ended question, useful in the beginning of an interview. Do they even see the widget?
-
“How would you normally approach (widget-related task)?” This seems to be the best opening question for a user interview. While not appropriate for all circumstances, it is great at opening up areas you may not have considered to be in the scope of the project. This question can help reframe the problem you are working to solve.
-
“How did you last (widget-related task)?” Past behavior is a more reliable indicator than a behavior people might describe as their normal. So, this re-phrasing can encourage more honest answers.
You may notice that it takes more time to ask open-ended questions. You may trigger your interviewee to share a lot of extra information, not just a concise answer to the question you asked. You may have to follow up with more questions to get to the “why” behind a certain behavior. But with the increased effort comes a better result. The responses are real. They are not influenced by your opinion about the thing you’re asking, because you’ve kept that opinion outside the questions. These responses present a more nuanced picture of your users’ needs and environment.
I condensed this principle from conference talks, books and workshops I attended over the past few years. Two sources in particular:
-
Michelle Yaiser’s talk on user research at UX Camp.
-
Sound reporting : the NPR guide to audio journalism and production. This book is outside of the usual UX Design reading list, but it’s useful for interviewing skills. It shows how much effort and consideration it takes a journalist to gather information in a neutral, ethical way.
It’s almost always a better investment of your time to conduct a few in-depth, “difficult” and neutral interviews, than to rush many interviewees through surface-level questions. This is especially true at the beginning of a project, when more design options are open. Your effort to keep the process unbiased will yield quality results, and quality wins.
DockYard is a digital product agency offering exceptional user experience, design, full stack engineering, web app development, custom software, Ember, Elixir, and Phoenix services, consulting, and training.
