Why Your Customers Are Lying to You And How To Fix It
Why Your Customers Are Lying to You And How To Fix It
How this 70-year-old technique unlocks what your customers really need from your products
I once spent two months building a feature that was used exactly once.
One. Single. Time.
And that's when I realized: if you think you're getting actionable info from customer interviews, you're fooling yourself.
But here's the kicker: it's not their fault – it's yours.
We’ll take a look at how human brains sabotage product development and how a little-known Air Force technique can save us from wasting time, money, and our customers' goodwill.
In the process, we’ll learn how to build fewer wasted features for problems that don’t exist.
A solution is born
We were interviewing users to uncover the next few features to address the biggest problem on our “problem”-based roadmap: overcoming internal user resistance to using our product.
Driving internal adoption for our product had been a focus for several quarters since we rolled out nationally. Fortunately, we were regularly speaking with our users. In one session, our Product Trio was speaking with a generally supportive user who was convinced they needed a very specific new feature to make our product a central part of their daily workflow.
As our product manager spoke with them, they fed off each other:
“Tell me more about this feature you want.”
“Do you really feel it will help you do your job better?”
“That’s a great idea. Never thought of that. How exactly should it work?”
“And what would you like it to do when this happens?”
“...and then how about if it did this? No? How about that instead?”
Once they had described this “perfect” solution in detail, our product manager made it our team’s #1 priority, pushing back other features on the roadmap.
Special delivery
I wasn’t convinced. To me, it seemed to be a very narrow “edge” case.
But the product manager won over the team and got stakeholder buy-in, so we started going through the extensive discovery effort required to design and deliver this feature.
Once we started peeling back the layers on what was required to deliver it, however, it soon became apparent that a great deal more work would be required.
After several false starts, and some stakeholder input, it ended up taking us double the original 3-4 week timeframe we initially thought it would take.
Grand opening
With this “magical” feature now fully built and tested, we finally launched, anticipating great fanfare and hordes of delighted users.
The initial response was muted, at best, and we soon moved on to other work.
After about six months, an analyst on our team looked at the data and discovered that our “killer” feature, which had cost our team enormous effort, had only been accessed once.
Something that had taken our 10-person team 6-8 weeks to build had essentially never been touched by the users whose lives it was meant to improve.
Major miss
Not only did we not make any progress in overcoming internal user resistance, we had missed out on months of crucial usability fixes and other flows that needed optimization.
What happened?
When we ask the wrong questions and build whatever solutions users request from us, we inevitably get burned by one or more of the big risks of product development.
Connecting with our users
As we shift from building what stakeholders tell us to build or what the competition has to building products aligned with end-user needs, our first step is to get closer to those users.
Involving customers in our discovery process is how we understand their needs and design better solutions to address them.
But we’ll need to ask a better quality of question if we’re to give them what they need, creating success for them and for our business.
The solutions they suggest are usually wrong
“If I asked people what they wanted, they’d say ‘faster horses.’”
Although Henry Ford never said this, it accurately sums up how customers think about our products.
When asked directly, they’ll always answer in terms of the solutions they already know.
Problems only start when we take those customer solution requests at face value. Because customers rarely know what’s technically possible, the solutions they suggest typically won’t help us create better products.
Work backwards
In fact, if we’re to get any closer to the truth, neither the product team nor the end-user customer should be thinking about specific solutions in any research conversation.
“Our primary research question in any interview should be: What needs, pain points, and desires matter most to this customer?”
Torres, Teresa. Continuous Discovery Habits: Discover Products that Create Customer Value and Business Value (p. 79). Product Talk LLC. Kindle Edition.
Asking more questions isn’t the answer
Many product teams fail not because they don’t do enough research, but because they go too far in the wrong direction.
This situation is frequently made worse when well-meaning and highly-trained “UX Researchers” take a “Big Bang” research approach, spending countless hours developing comprehensive interview guides and running dozens of hours of customer interviews.
But asking more of the wrong questions in more customer interviews won’t help us create better products.
The questions that lead to startup failure
Paradoxically, the worst questions to ask are the ones that seem to make the most logical sense based on what we want to learn.
For example, this question has set back many a startup before it ever had a chance to get off the ground:
“Would you use this product?”
And this is the question alone is probably responsible for evaporating more investor money than anything else:
“Would you pay for this product?”
Asking these seemingly “logical” questions about what we want to learn from our customers may be the worst mistake we can make.
Why?
There’s what people know. And then there’s what they tell us.
Because when we ask people what we’re trying to learn, we run directly into the brain’s worst cognitive biases.
People are simply incapable of reliably telling us what they might do in the future. They’re also unreliable sharing what they did in the past. It’s not so much they’re being dishonest, as that the human brain simply doesn’t do well with certain kinds of questions.
Fundamental human cognitive biases have plagued us throughout history and prevent us from easily sharing basic truths were only recently uncovered thanks to Daniel Kahneman's work.
To create better products, we need to learn what people actually do.
Not what they “say” they do or what they “might” theoretically do.
How our brains work against direct questions
What Kahneman uncovered was that our brains are structured to create logical-sounding stories at random with little to no basis in fact.
We’re fundamentally wired to answer questions more based on how we see ourselves (or how we’d like to be seen) than reality would otherwise indicate.
This means that asking customers what they might do inevitably makes them unreliable, as they’ll fabricate any number of plausible-sounding stories. This is what happened to our team in the opening story. Both our interview questions and our method made everything worse, and the “solution” we spent months building was based on “magical” thinking.
How can we build products based not on customer fabrications but on their actual behaviors and most basic needs?
Fortunately, there is a powerful technique that can help us understand those deeper needs.
Asking the right question is the answer
The interview approach that can get us the information we need has been around for over 70 years: The Critical Incident Technique (“CIT”).
In situation after situation, this scientifically proven approach has been shown to successfully short-circuit cognitive biases and get to deeper truths of human needs and behavior.
Originally developed by the Air Force, CIT was used to obtain better qualitative information from research participants. In one of its early applications, researchers were surprised that even imprecisely conducted CIT methods spurred major upgrades to Air Force training.
The secret to CIT?
Simply ask someone to tell a recent story.
Over time, CIT was applied to other contexts in industry, proving indispensable for understanding and improving people-based situations.
Here’s an example from a Westinghouse Electric plant study conducted to identify exceptionally effective foreman behaviors:
“3. Think of a time when a foreman has, in your opinion, shown definitely good foremanship—the type of action that points out the superior foreman.(Effective—substantial deviation from the norm.)”
John C. Flanagan. The Critical Incident Technique (p. 11).
But Flanagan cautions us that CIT is not a specific process to apply:
“...the critical incident technique does not consist of a single rigid set of rules governing such data collection. Rather it should be thought of as a flexible set of principles which must be modified and adapted to meet the specific situation at hand.”
John C. Flanagan. The Critical Incident Technique (p. 16).
The main foundation? Wording our questions better:
“The most crucial aspect of the data collection procedure is the questions asked the observers. Many studies have shown that a slight change in wording may produce a substantial change in the incidents reported.”
John C. Flanagan. The Critical Incident Technique (p. 29).
If you’ve recently been interviewed for a job, or interviewed someone else, you may not realize it, but you’ve already come into contact with the power of story-based interviewing to circumvent cognitive biases and uncover deeper truths.
Stories unlock who we really are
Job interviewers used to ask candidates direct, yes/no questions, which led to less-than-optimal success in matching people to the right role.
Current advanced hiring techniques depend on well-constructed, behavioral job interview questions. This question style has empowered interviewers to separate what they want to learn from what they ask, allowing them to understand likely candidate attitudes and behaviors in a future role.
And it comes down to the same secret.
Effective behavioral interview questions are nothing more than CIT-style story prompts:
“Tell me about a time when…”
There’s what you want to know. And what you ask.
This LinkedIn Business article provides excellent examples of the kinds of information interviewers need most to make better hiring decisions paired with powerful story prompts:
What the interviewer wants to learn:
How adaptable is the applicant? How open are they to having their job change and still remain flexible and successful?
What the interviewer asks:
“Recall a time when you were assigned a task outside of your job description. How did you handle the situation? What was the outcome?”
What the interviewer wants to learn:
How collaborative and patient is the applicant, and how well can they adjust their style to align with others?
What the interviewer asks:
“Tell me about a time when you were communicating with someone and they did not understand you. What did you do?”
Never make the mistake of asking what you want to know
The biggest takeaway from these examples?
The interviewer never asks,
“Are you adaptable?“
“Do you collaborate well with others?”
Yes/No, binary questions run right into the brain’s cognitive biases and are poor predictors of success in a future role.
Story-based questions are one of our best tools for getting closer to the truth in any kind of interview because they circumvent the brain’s cognitive biases and provide better insights into future behaviors, not what people feel interviewers want to hear.
If we want to apply these insights to building better products, our customers’ stories will be our key to understanding their “unmet needs, pain points, or desires.”
We call this the “Opportunity Space.”
Mapping opportunities
We can connect our solutions to customer needs during continuous discovery by mapping our users’ opportunities using Opportunity Solution Trees (“OST”).
Opportunity Solution Trees help us grok the entire flow from Business Outcome to Customer Outcome, through the Opportunity space, and down to our Solution ideas.
While it can seem messy and chaotic, this mapping can give us powerful information to understand how our customers’ needs connect to the solutions we create to deliver against them.
These opportunities will come from our customers’ stories, framed in terms of something they might say.
Keeping up a regular interview cadence helps us identify and maintain a clear mapping of the opportunities we can solve for.
Interview to discover
The Product Trio represents the core discovery leadership team, made up of a product manager, product designer, and tech lead/architect.
“Continuous discovery requires weekly touch points with customers by the team building the product.”
It’s hard to overstate how much of a major departure continuous product discovery is from traditional “big, up-front” research methods teams used in the past.
Instead of infrequent, lengthy interviews, the product trio conducts a continuous series of regular, short, targeted customer conversations.
The trio asks one question, focused on better uncovering and mapping potential customer opportunities to address.
Question flight levels
Framing our story-based questions differently can help us identify opportunities at various product levels:
Digging in to get details about specific parts of a user experience can help us optimize an existing product. (“Tell me about a time you had an easier onboarding experience with a new application.”)
Asking broader, more general story questions can help us unearth entirely new opportunities. (“Tell me about a time you successfully wound down after a long day.”)
Opportunity solution trees allow the product trio to refine their understanding of the customer and work backward from the right opportunities in connecting their potential solutions to higher-level business outcomes.
It all starts with designing the specific story-based question prompt to unlock innovation and build solutions aligned to both customer and business needs .
Summary & Takeaways
“The purpose of business is to create and keep a customer.”
If businesses exist to create and keep customers, interviews can help us uncover fundamental information about serving them more effectively.
But if we ask the wrong questions, human cognitive biases will guarantee that we get the wrong information, which can lead us to build the wrong things in our mistaken understanding of their needs.
Continuously meeting with customers and asking them story-based questions allows us to overcome basic cognitive biases and access deeper customer attitudes and behavior. This, in turn, allows us to pinpoint specific user opportunities to address.
By regularly making customers the central part of our discovery process, we can evolve our understanding of their needs and build better solutions to address those needs in ways that also work for our business.
Instead of starting with solutions we think they should have.