This is the second part of a three-part series exploring how we listen to the market to become experts in what they need. The first article shows how product managers should listen to potential customers, the inactive members of our market who have not bought our products or our competitors? products (and are not looking). This second article discusses finding and listening to evaluators. The third article covers on-going communication with our existing customers. By listening to all three types, product managers become “experts on the market.”
Recent evaluators can make the truest assessment of our products. Having just evaluated the product against the competition, they can assess our company’s strengths and weaknesses. Perhaps most importantly, from evaluators, we can learn about problems with the product and also problems with the buying and selling process.
Listening to evaluators is different than listening to potential customers, but just as important. This is an activity that can be outsourced (but not to sales— win/loss analysis is a marketing function). If sales does win/loss analysis, they often won’t get to the real issues. Often the buyer does not want to reveal the real reasons for fear that the salesperson will try to overcome their objections and attempt to restart the selling process. But even if you outsource this to someone else, product managers should plan on doing at least one win/loss analysis interview personally every month.
So where do we begin?
First, we have to identify evaluators. Who are evaluators and how do we find them? Evaluators are those people in your market who have recognized they have a problem to solve and are actively looking for solutions to the problem. They are looking at your solution and at your competitors? solutions. This is the part of the market that your sales channel is most familiar with; they are active leads in the sales funnel.
Finding evaluators depends on the type of sales channel you use. The more anonymous people are during the sales cycle, the more difficult it is to find them. Here are some tips to find evaluators, based on different sales channels.
| Type of channel | Source | How you find them |
| Direct | Pipeline Reports | Buyers in the sales cycle disappear from the pipeline report when you win the deal, lose the deal, or when there is no decision. Sales management should be able to provide you with this information. |
| Sales Calls | If the salespeople are not cooperative in reporting wins and losses, do this: when you are asked to participate in a sales call, make it clear to them that you will be collecting business cards and will be doing win/loss analysis after the deal is won or lost. If this is the case, sales & marketing management should try to improve the situation–sales and marketing should have a cooperative relationship, not an adversarial one. | |
| New User Training | Go to a new user training class. Recent wins will be there. Collect business cards to follow-up with a win/loss interview later. | |
| Value Added Reseller (VAR) | Pipeline Reports | Losses are hard to find out about, particularly if your sales staff did not assist the VAR on the deal. If your internal sales staff helps the channel on deals, you should track the deals through the sales cycle. Like pipeline reports with your direct sales channel, this provides wins and losses when an account drops off of the report.
If you don’t assist your indirect channel with the sales process, try visiting some of your channel members and do win/loss analysis on a subset of their evaluators. Include both successful channel members and not-so-successful ones so you can analyze why some are more successful than others. It might be a lack of sales sophistication, lack of focus on your product, lack of training, or any number of other things. Unless you know why they win or lose deals, you won’t know how to improve the situation so they can have more wins. |
| Web | Wins are easier to track on the web. You should require minimum contact information (name and email address is sufficient) when they buy your product or service. Losses are more difficult, but you might offer a free trial or limited service for which they must register on your site. If they don’t buy after a period of time, follow-up and find out why. It is nearly impossible to track down those who never download or register with you at all. You may need to periodically conduct market research surveys where you tap into your market segment and find out if they have ever considered your products or services. |
If you are part of a large organization with multiple products and solutions in a complex sale, it is critical that win/loss analysis be coordinated. The evaluator should not get 4 phone calls from 4 different product managers asking for the same information. “Don’t you guys talk to one another?”
What are we trying to learn?
Typically, companies believe they listen to evaluators during the sales cycle. But more often, they react to individual deals.’If you put feature A in the product, I can win the deal.’ Or, ‘I could have won the deal if we’d had feature A in the product.’ The sales rep then says, ‘Put feature A in the product.’ Sometimes, the right thing to do is put feature A in the product. But sometimes, feature A is put in the product for a market of one, not a market of many. It is a reactive process, not a proactive process.
Rather than reacting to individual deals and potentially squandering precious resources solving a problem that one customer has versus solving problems a whole market segment has, win/loss analysis can provide information to validate (among many evaluators) what we’re doing right and what we’re doing wrong. Why are we winning and why are we losing? This type of analysis helps us strengthen (and institutionalize) what we do right and correct what we’re doing wrong. Many times, the reason companies lose business is not because of the product, but because of a flaw in the sales process. And yet when they react to lost business, they usually throw more features into the product rather than fix the process. Win/loss analysis looks at the whole picture: product, services, price, evaluation process, sales channel, collateral, competition, technical expertise, and interaction with the company.
Generally, we want to know:
- Why are we winning?
- Why are we losing?
Specifically, we need to learn:
- How did they find out about you?
- Who did they listen to for advice during the buying process?
- Were your communications clear?
- Were there any breakdowns during the sales cycle? What were they?
- What is the competition doing right? Are they winning because they have a better product or service? Or is it because they are outselling us?
- Do you have the right technology, the right product, the right services to solve the evaluator’s problem?
- What problem were they trying to solve with your product or service?
When do we listen?
Win/loss analysis should be performed when the deal is completely closed. Either the competition or we have solidly won the deal (the contract is signed, the ink is dry, and the check has cleared the bank). If we do this too early (and we were winning), it is possible we might raise some questions in the buyer’s mind that causes us to lose the deal (product management will be blamed, even if it wasn’t our fault).
If we do this too long after the deal is closed, the buyer’s memory will fade. We are trying to measure what happened during the sales cycle, and if they are too deep into implementation and use of the product, their answers will reflect their current views, not what they were thinking during the buying process.
In general, a win/loss call should occur two to four weeks after the deal is concluded.
Where do we listen?
Visiting evaluators onsite in their natural habitat is by far the best scenario. You will learn things onsite that you will never learn in other settings, particularly details about the competition. However, it is sometimes impractical (or difficult to get permission) to do this for every win or loss. Augment onsite win/loss analysis with other methods, such as phone interviews. If you sell over the web, a short web survey can be effective, too.
How do we listen?
Although product managers should do at least one win/loss report per month, the rest of these can be outsourced (as long as you read every report and analyze what you learn). Sometimes an objective third-party will be more successful uncovering the reasons we win and lose because the buyer trusts that they are not trying to re-open the deal.
Here are 10 tips on listening to evaluators:
- Make sure the evaluator knows you are not trying to save the deal (if you lost). Maintain this integrity by not passing the deal back to sales if you lost due to something that can be fixed. Learn from it and move on.
- Develop a questionnaire that includes both closed-ended questions (multiple-choice, rankings, yes/no) and open-ended questions.
- Ask if you can record the conversation. If the evaluator agrees, you can go back and review the conversation not only for content but also inflections and tone.
- If the reason the evaluator says they went with the competition is the price, follow-up later to really understand this. Was it because the salesperson wasn’t able to clearly articulate the difference in value between your solution and the competitor’s? Was it because the buyer didn’t need the additional value you provided for the premium price? Or is your price really too high?
- If the reason you lost is the salesperson, drill down on this. It might not be because the salesperson is incompetent. It might be a lack of credibility, which might be the result of a lack of training on the product or the industry. It might be because the salesperson needs coaching in uncovering the buyer’s needs.
- Listen! Your job is not to re-open the sales deal if the buyer went with the competition. Your purpose is to learn why. If the buyer says they went with the competition because you don’t have feature X (and it is your best feature!), you simply need to listen. Something clearly went wrong in the sales process for them to have this perception.
- Plan on a 15-minute interview. If you find the person is particularly open, ask if they’d be willing to participate in a longer, more in-depth interview at a time convenient to them.
- Record all win/loss interviews in a database so you can share and analyze the information.
- Use win/loss data to quantify features you need to add. Rather than simply reacting to a single data point from the last sales deal, analyze whether you are consistently losing business because you are lacking a key feature.
- Always ask, ‘What problem did you think you were solving with this product or service?’
Win/loss analysis can give us data to validate the things we are doing correctly and insight into what isn’t working. It is a more effective way to learn the strengths and weaknesses of the competition (not just about their products, but also about their sales expertise). In the absence of win/loss data, we typically react to anecdotes from the sales channel, which might reflect a market of one and not a market of many.
As a product manager, do at least one win/loss interview per month yourself. The market expertise and insights you will gain will far outweigh the time spent on the interview.
Stay tuned for Part 3 on Listening to Customers.
