Avoiding Leading Questions in User Interviews

Question design for user interviews is an art. We interview users to gain insights by learning from their experiences. Ideally, we want to get them talking and keep them talking as they traverse a card sort, a prototype, or a series of questions about how they work.

Here’s the thing: the best insights can come from users, but if we’re not careful in how we ask questions, we can inadvertently bias them before they have a chance to share their own thoughts.

Psychologist Adam Phillips has said that psychoanalysis is about examining someone’s side effects, or, “what falls out of their pockets once they start speaking.” This is a great analogy for interviewing. We want to ask users just enough to get them talking, build some momentum, and observe what falls out of their pockets as they talk. When we’re caught up in confirming our biases, asking if they want red or blue, we’re not present enough to gain the insights we seek.

Starve a User, Feed a Backlog

Artful questions begin with feeding users as little information as possible to get them to tell us what they think and feel. To ask non-leading questions, you must be comfortable with being vague. Use as few words as possible to get the user talking, and once they start, they’ll usually continue without much prompting. Here are some examples:

• “How would you use this?”

• “What would you do with this information?”

• “What would you do next?” or my favorite, “What’s next?”

• “How do you feel about what you’re doing there?”

• Point to a part of the screen or the whole screen and ask, “Go clockwise around this area and tell me what each piece means.”

• “Expand on [that thing] you just mentioned.”

• Before a user clicks on something, typically while talking but indicating they’re about to click, ask: “What do you expect to see (or do) when you click [that button/link/etc.]?”

The key with these types of questions is not asking them with a specific outcome in mind; it’s about asking questions in the hope that you’ll tease things out. Remember, while interviews can help us confirm how things work, the potential to learn new things is always possible. The more specific our questions, the less likely it is we’ll learn something new.

To get even more utility from the questions you ask, it’s important to become comfortable with long pauses. After the user finishes a thought, allow a moment or two of silence to linger so they can process their thoughts and add anything else that comes to mind. We often find that the most interesting insights come from those silences.

Conversely, when we ask questions that are too leading, we put too much information in the mind of the user.

Here are some bad examples:

• “What’s missing?” (Too quiz-like, leads the user to come up with something)

• “Do you want this to be blue, or red?” (What if neither works, but we box them into choices they don’t want?)

• “Where should we move this to?” (Why not let users organically suggest a move instead of prompting them?)

• “Do you want this to be like [insert name of another product]?” (Ugh, such a rookie move)

• “Do you think you’d like [insert your idea] to happen?” (Just plain leading and looking for confirmation of your biases)

• “If I made this do [insert dreamed-up solution here], would you use it?” (Of course, they would; they want to perform well, and it requires no work to say yes to your imaginary solution)

• “Would you use this if I made this change [insert change here]?” (This comes off as desperate and would be much more powerful if they said it of their own volition)

These types of questions put ideas in users’ heads and lead them toward ideas they might not have come up with themselves. Keep in mind: we want to observe users in as clean a frame of reference as possible. As soon as we verbalize solutions or directions for them to go, we’ve lost our opportunity for them to guide us as they meander through their thoughts and feelings around a given scenario. Give users space and silence for the best results.

Bringing Teammates into the Interviews

It’s one thing to have a designer or two conducting interviews, but it’s another to bring in Product Owners or Managers. Honestly, PMs and POs can sometimes be too close to a product to prevent bias from slipping into the conversation. That said, it’s definitely something the team can practice.

In a recent project, we live-streamed the interview sessions over Zoom for the Product Owners and Managers to watch as we conducted them. This offered us designers time to work out how the interviews would be conducted, while the product team could learn, observe, and see what to expect.

When you start bringing extended team members into the interviews, simply agree that observation and listening at first are totally okay. “You aren’t learning anything when you’re talking,” as Lyndon Johnson used to say. As I mentioned above, not every moment needs to be filled with talking; it can be more productive to let silence be.

Trusting Insights Derived from Non-Leading Questions

Interviewing users is a great way to reduce project risk and assumptions, but it all starts with how you craft and ask questions. What are some other questions we could ask that remove bias from the equation? Could this be a team activity before testing? Take some time to come up with 10 or 12 questions geared towards teasing insights out without letting your bias in. If you’re doing it right, the questions are vague, but the user divulges more than you expected. Now you have more pure, trustworthy insights for designing your product.