QUESTION:
Where on this line would you find “right”?
________________________________
Wrong. Perfect
ANSWER: From a product management perspective, it would be a nanometer to the right of wrong. Once you’re not wrong, you are right—and can continuously improve on the way to perfect.
The trick is that you must know where wrong is first. You get this data by interviewing several people in your market. The key is understanding the difference between data and opinion.
For example, Frontline Technologies provides workforce management software to a number of industries, including school districts. So when one person tells us that “teachers do this,” we record it. Then we go talk to the teachers to validate it. If the teachers say the same thing, then we treat it as data. If they don’t, it was just an opinion. Someone talking about what they do is data. Someone talking about what someone else does is opinion.
The dangers of listening to opinions, rather than data, are that people often tell you what they think you want to hear or frame what they say around how they do something, as opposed to why. You end up building a solution that really doesn’t address the problem. Products die when you don’t address market problems correctly, as the course correction is often more expensive than the build was in the first place.
One of our customers was trying to reduce call-center costs and gave us a laundry list of things that would have shaved seconds off business practices. But he was telling me what he thought he needed, not the problem he was trying to solve.
We spent a morning in the call center, asking questions and observing how they used our system. We noticed one particular step that took an average of three minutes on every call. We made a simple enhancement that took the three-minute step to about 10 seconds and released it, and they were floored that we gave them back hours each day—as opposed to minutes.
If you want your customers to be floored too, get out into the market and find out what problems they have.
START WITH THE USER
At Frontline Technologies, we start with market and persona segmentation and build a hypothesis around our user and buyer personas, focusing on the primary user of a system. From there, we can build an interview grid and an interview funnel to cover our current understanding of the market and personas. For example, here is a look at one of our markets in two dimensions: size and location.
Then we build our persona-grid hypothesis. It looks at who the users and the buyers are and identifies the primary user.
This particular example is for a teacher-evaluation system, and the principal was the primary evaluator and buyer. Other people would use the system, but they were just providing data to the process as opposed to controlling it.
We go into the interviewing process acknowledging that we have a hypothesis for the market problem and the potential solution, but recognizing that it is just an opinion without the interviews. The interviews transform our understanding of the market problems and potential solutions from opinions to actionable data.
We target interview candidates that will clarify what variances in the market problem, if any, exist in the different market segments and for different personas—as well as what the perceived value of solving the market problem is. We start with the primary user and try to get four to six interviews for each of the personas in the grid.
ESTABLISH A PROTOCOL
We use a simple interview protocol that takes from 30-60 minutes, and is designed to differentiate between data and opinion. We ask permission to audio record the interview so that we can truly listen to what our interviewee is saying, rather than being distracted by taking notes. No multitasking when conducting an interview.
I’ll demonstrate this protocol process, using a real example. For a recent product, our hypothesis for the primary user persona was a school principal. We went and visited Tina in her office.
1. Frame the context of the interview. This is critical so that you get the amount of data “just right”—not too broad or too narrow.
2. Ask them to describe their current process. You’re asking them to tell you how they do X, in relation to Y. Silence is your friend here. Many interviewees are trying to figure out what we want to hear, when all that we want to hear is their story. Our product managers slowly count to 10 after asking a question. The interviewees will fill the silence with what is on their minds. In Tina’s case, we asked her to describe the process she used to evaluate teachers, and she spoke for 20 minutes.
3. Ask what issues they have with their current process. This question is key to determining market problems. Sometimes the problem areas are obvious and sometimes they’re not. While Tina was describing her process, she mentioned some historic data. I asked how frequently she accessed that data, and she swiveled around in her chair and opened a file drawer dedicated to just one person’s historical data.
That moment helped us understand that we needed a cloud-storage component to our product. None of our competitors had that capability, and this was not part of our initial hypothesis of the market problem. If we were not in Tina’s office, we would not have known of that need—and addressing it became a key differentiator for our product. I thought of personnel evaluations as being three to five page forms, not a 300-page dissertation every year. We did not understand the storage implications of this little problem.
4. Ask what the business consequences of those issues are. These answers often provide solutions. Also, ask if there was anything different at the beginning of the year, at the end of the year or if there’s a monthly or quarterly process. It helps you get a sense of their cycles, and it prompts them about issues they might otherwise forget to mention.
5. Ask what they would do for a solution. Clearly, this could also lead to solutions. You have to provide context for this question: If you were the superintendent, how would you fix this problem? Or if you were the CEO of a company selling you a solution, how would you fix it? Context is important, because initially they may only think about what they can do in their current role. And if they could fix it, they would have already.
Once they give you the solution, ask if there are any barriers that would stop that from happening today. For example, is there a union contract or funding issue? When they start to talk about the barriers, they’re actually telling you what sales objections are going to come up in the sales process.
6. Ask them what other questions they would have asked if they were in your role. This is important, because you don’t know what you don’t know. When they provide these questions, ask them to answer them. And remember to re-ask steps 3, 4 and 5 to get at the issues, business consequences and possible solutions. If they were knowledgeable enough to know what you didn’t ask, they’re knowledgeable enough to have an answer.
7. Ask if there was anything else that could have been done for a better interview. This question is about continuous improvement of the process.
Even though not all interviews are great, all of them are valuable—either for providing new information or validating something we thought we already knew. The key is to hear what that interviewee has to say, not what we want to hear.
After conducting three to four interviews, we determine our “Interview Zero”: Which interviewee “got it,” and clearly communicated the problem and its business consequences. We start our review of the recordings with this interview.
We have a great conference room with whiteboard walls that we use for debriefing as a team. A scribe writes on the wall, and someone else controls playback of the audio. We map out the process and highlight the concerns or issues in the customers’ language. We also highlight both data and opinions. As mentioned, when interviewees describe something they do, we treat it as data. When they describe something someone else does, we treat that as an opinion that needs to be validated. We use that same wall to transcribe the other interviews, highlighting where we validate a common process or issue. It doesn’t take many interviews for patterns to become visible.
We use this process for new products, new features and to revalidate existing features. If you take the time to truly hear the customer, you get great actionable data from a knowledgeable source. And in many cases, you even get a reference, because the second sweetest thing people hear, after their own names, is confirmation that they were truly heard.
