One of the common questions I am asked when discussing win/loss analysis is, “How many deals do I need to analyze before I can start using the data?” In the past, my response has always been that you will get value from the very first win, loss, or no decision, you analyze. However, because statistics teaches us about minimum confidence levels, confidence intervals, and margins of error, people often struggle with the idea that they can start leveraging the data from their win/loss program from the very first data point.
Get away from the “survey” mindset
By just listening to the terminology executives use, it is obvious many initially base their vision of a win/loss program on their prior knowledge of market research or political polls. For instance, the term “survey” is often used to describe what is actually an in-depth “interview.” Unfortunately, this market research-based mindset often diminishes the usefulness of a win/loss program and gives the false impression the program will need hundreds (or even thousands) of data points to be accurate or useful.
Certainly, if you were to use a win/loss program to determine the general buying habits of your buyers (or the populace as a whole), you would want to interview a large number of random customers in order to achieve a representative sample and a high level of statistical confidence. However, working with hundreds of companies over the past ten years has shown that the most successful win/loss analysis programs do not try to cast such a wide net. Instead, they act more like a laser pointer that illuminates very specific information about very specific issues and questions. In other words, to get the most out of your win/loss program, it should not be seen as a survey tool (which simply gathers information), but rather a decision-making tool and an early warning tool.
If you think you can’t get value from your win/loss program, or believe you can’t begin making decisions based on win/loss data until you achieve a statistically significant number of data points, our research shows you risk missing as much as 75 percent of the potential ROI you could achieve with your win/loss analysis program. The key is to understand how and when to use the intelligence gathered through your win/loss analysis.
The decisions executives and managers make
To determine how many interviews are needed for a win/loss program to be effective, it is important to understand the kinds of decisions your organization makes and to understand how win/loss analysis can help you make better decisions.
Professor Robert A. Harris, in his VirtualSalt blog, outlines three major types of decisions: strategic, tactical, and operational.
Strategic decisions determine the long-term direction of an enterprise, product, or department. They also tend to direct and control an organization’s resources. Strategic decisions involve a high degree of uncertainty due to the scope and time frames usually associated with these types of decisions.
Real-world examples of win/loss analysis that helped clients make better strategic decisions include:
- Competitive losses indicate buyers
- want a vendor with strong off-shore capabilities in two specific countries
- had issues with key contractual terms the client requires of new customers
- Buyers in both wins and losses describe a specific set of needs that aren’t currently addressed in the market and warrant development of a new product
Tactical decisions support or achieve strategic decisions. In other words, tactical decisions are made to help bring about the results envisioned in the organization’s overall strategy. As such, they tend to be more immediate than strategic decisions, and can be made at both the managerial and executive level, whereas strategic decisions are almost exclusively determined by executives.
Real-world examples of win/loss program data that led clients to make tactical decisions include:
- Competitive wins indicate a key factor in buyers’ decisions was the presence of superior sales engineer support
- Buyers in competitive wins talked about specific sales messaging and sales collateral that helped them make their decision, while buyers in lost opportunities did not reference these things
- Buyers in losses cited over-confidence of the sales representatives and disparaging the competition as key reasons for non-selection
Operational decisions support tactical and strategic decisions. They help to ensure that daily operations go smoothly and that tactical decisions are implemented. As the name indicates, these are usually made by managers, although they can also be made by executives.
Real-world examples of win/loss program intelligence that help clients make operational decisions include:
- A buyer in a lost opportunity indicated
- the sales representative poorly presented the solution and did not understand how their product would solve the buyer’s problems
- they didn’t select the client primarily because a product feature was not available, but the client’s product actually had the feature
- they are still evaluating solutions and would like to hear from the client again (this situation happens more often than you think)
- A buyer in a won opportunity indicates they were having trouble with communication and it was having a negative impact on their decision satisfaction
The number of data points needed to support accurate decision-making increases as you move from operational (which is often immediate and one data point can be sufficient) to strategic, which may require a large amount of information to get a clear picture of the entire situation. For instance, deciding which paper supplier to use for a direct mail campaign (an operational decision) would probably not require client interviews, but if you wanted to change the focus of your company from services to software (a strategic decision), it is in your best interest to interview a representative number of current clients to determine how they would react to such a change.
Win/loss as a decision-making and decision-support tool
From my experience over the last ten years, I found that win/loss analysis provides most benefit at the tactical and operational level, although, certainly, it does provide tremendous insight at the strategic level as well. It just takes a little more time before you can start to lean on your win/loss program when making long-term, strategic decisions.
On average, we find for each win, loss, or no decision interview performed, you can expect to identify an average of ten operational or tactical decisions that can be acted upon.
When you consider that each analysis is going to help you make an average of ten operational or tactical decisions, it becomes clear very quickly that you can realize great value from the very first interview of your win/loss program. Just how much value does each of these potential decisions represent?
Let’s look at an example: one of the most common operational decisions identified in win/loss reviews are sales weaknesses for an individual sales representative that can be immediately addressed through focused training and/or coaching. If this training can help the sales representative avoid making the same mistakes in future opportunities, their individual win rate will improve.
The impact of this one decision to help a sales representative with an area of weakness identified by a buyer could be in the millions of dollars, depending on the size of your sales opportunities.
If done correctly, every win/loss interview will net key, actionable intelligence such as:
- What business needs are your buyers trying to solve?
- What could you do to better serve your buyers’ needs?
- What is the character and reputation of your company in the eyes of your buyers?
- How well do your sales teams communicate your value proposition?
- What are the strengths and weaknesses of your products?
- What are the strengths and weaknesses of your competitors’ products?
- How are your costs perceived in relation to those of the competition?
Last year, when I was reviewing some interviews we had done with our own buyers, I saw an issue emerge that definitely led to a better tactical decision. One of our respondents commented that their communication with us was almost entirely with the salesperson and they did not have sufficient contact with the Account Consultant who would be handling their program.
Since one of our strategic goals is to be a trusted consultant to our clients rather than just a provider of data, I realized this was a significant issue. After getting the same feedback in two other interviews, I determined this was not an isolated case and made the tactical decision to involve our Account Consultants and other subject matter experts early in the sales process.
The results have been good, with our clients coming out of the sales process more informed and more committed. This tactical decision was made based on three data points, but would have been entirely missed if we had not been performing win/loss reviews on our own opportunities.
So, how many data points do you need for win/loss?
The case I just described did not require many interviews or data points for the win/loss analysis to have a positive impact—in fact, it only took one to begin the decision process. This illustrates you can and will receive value from the very first opportunity you analyze. And as you add more data points, you will receive further validation and discover important, new insight.
The point is: since win/loss analysis is a dynamic process, it can and should focus on those issues that are most important to you and your buyers. This means, unlike a static survey, fewer data points are needed in a win/loss analysis to reach the insights you need to make better tactical and operational decisions to help you achieve your current strategic goals and overall success.
Don’t wait until you have a “sufficient number” of data points before you realize value in your win/loss program. Begin analyzing and using your win/loss data from your very first buyer interview and start seeing returns much more quickly.
