PRODUCT PROFESSIONALS IN TECHNOLOGY COMPANIES typically don’t spend enough time analyzing metrics. They’re focused on an overwhelming amount of tactical activities, plus the numbers can sometimes be hard to obtain. Often, when they do manage to carve out time, they don’t know where to begin.
But if you aspire to be a strategic product leader, you must start acting like one. The team depends on it. And in the fast-paced world of technology, you need to balance time-to-market pressures with risk mitigation. Analyzing the data can help you mitigate risk: risk that you’re building something no one wants, that you can’t deliver something to the market profitably or that the product is veering off the road.
In mathematical terms, there are infinite things to measure and analyze—but we can’t measure everything. Focus on a few key performance indicators (KPIs) to determine if you are on track or need to make changes before it is too late.
BUSINESS VISIBILITY
To drive your product to success, you need insight into how well it is performing in the market and how the internal systems are performing in the organization. Monitoring and analyzing external and internal metrics combined with your market experience will put you in the driver’s seat.
Map the road ahead. You need context to clearly understand the KPIs and to eliminate the blind spots. For example, product strategy, product direction and the product roadmap give context to the metrics you will be tracking and how you should establish your budgets and objectives. If you are adding functionality to go into a new market segment, when will you be able to drive revenue and new customers in that market? When do you need to start demand-generation efforts to add leads to the top of the funnel? What do you need to do with products and services to retain customers in the segment in the future? Your KPIs should reflect your strategy.
Define the goals. For each KPI, you should define the goal or forecast against which the actual metric is compared. The goal or forecast should be SMART: specific, measurable, achievable, relevant and time-based.
Compare the data. Metrics should be monitored, but never in isolation. One data point without comparison to something else is meaningless. “We acquired 300 customers this month.” Is that good? Compared to what? To gain the most intelligence from the data, compare it to multiple data points to see the trend. Compare variance amounts, growth percentages or percentage of the whole of:
- Goal (forecast) to actual
- Periods, including month to date, quarter to date, year to date, month over month, year over year
- Products
- Market segments
- Competitors
- Industry benchmarks
Indicate the situation. Use a visual indicator to show “good,” “neutral” and “bad” situations, such as a traffic light metaphor (green, yellow, red) or arrows ( éê). Document the ranges for each indicator. An example for revenue: green = 100% of plan, yellow is between 80%–99.9%, red < 80%.
Tell the story. When you are reporting metrics to your team or to the executives, communicate the meaning by telling the story of the KPIs. Why are you missing, achieving or exceeding the KPI? Have you discovered a systemic problem or an isolated problem? Use data, not opinions, to support the story.
This is the framework for sharing key metrics, but what metrics should you analyze to find insights to drive product success?
As president of your product, you need business visibility into financial, customer, product, marketing and market metrics. Work with the owners of the data to collect, analyze and report the KPIs to the executives. Each department should analyze metrics they are responsible for, but as a product leader, you should be orchestrating a periodic business review to look at the metrics holistically at the product or portfolio level.
1. FINANCIAL
Product revenue: amount and units. Compare forecast to actual revenue by amount, units and average selling price. If you are consistently missing your forecast, you need to understand why. Analyze win/loss interviews, lead/funnel metrics and sales productivity to answer questions like:
- Did you have enough sales capacity to deliver the revenue?
- Is the average selling price going down? Why?
- Can you pinpoint the shortfall by region, market segment, customer type (new, existing) and channel (direct, reseller, retail)?
- Was the forecast achievable?
Gross margin. If you sell a physical product that you ship to customers, you should monitor gross margin or gross-margin percent. To do this, you need to know the cost of goods sold (COGS), which typically includes labor, materials and overhead of the item being sold. The formula is:
Gross Margin = Revenue – Cost of Goods Sold
Gross Margin % = Gross Margin / Revenue
Product expense and profit. If you can do some basic profit trending for your product, you will have better business visibility to make key decisions that can improve profits.
Ask accounting if they can give you a simple profit and loss (P&L) statement for your product. It may be based on cost allocations, rather than actual costs, particularly if labor and other expenses are not tracked at the product level.
If you can’t obtain that information, do some simple calculations based on full-time equivalent (FTE) employees. Count the number of employees working on your product, multiply that times $100,000 for an annual cost. If resources work on multiple products, estimate how much time is spent on your product to determine the FTEs. Include development, quality assurance, release control, technical support, sales, product management and marketing.
Similar to revenue, you should track profit by:
- Region
- Market segment
- Channel (direct, reseller, retail)
2. CUSTOMER
As Peter Drucker said, “The purpose of business is to create and keep a customer.” Brilliant, I wish I had said this! More than 100 years after this business guru was born, he is still relevant.
Customer counts. How many new customers are you acquiring per month? How many customers have canceled maintenance or subscriptions per month?
These metrics tell you whether your customer base is growing or shrinking. You need counts and percentage growth/decline for both new and lost customers, by period, region, market segment and product line. Analyzing customer counts by market segment and product line will give you visibility about where to focus efforts to meet strategic objectives.
Wallet share. Monitor customer wallet share by comparing average revenue per active customer (ARPAC) year over year. Is wallet share increasing or decreasing? Customers who use multiple products tend to be “stickier” (higher-retention rates).
If applicable, monitor the number of users per customer. Acquiring more users also leads to stickiness.
Lifetime value. If you have ARPAC per year and the average retention time (months or years the customer is active), you can calculate the lifetime value of a customer. For example:
$25,000 ARPAC
x 6 Years Average Tenure
$150,000 Lifetime Value
Average lifetime value will help you calculate how much you should spend acquiring new customers and how much you should spend on customer retention programs. Comparing lifetime value by product or market segment provides additional insight about where you should focus your resources to gain the most leverage.
Customer satisfaction. Measure satisfaction at the transaction level, the product level and the company level. Periodically interview customers to gain deeper context about why they rate you the way they do. The customers may be rating their interactions with sales or support high, but may be frustrated with the usability of the product or service—which sent them to support in the first place.
According to Frederick F. Reichheld of Bain & Company, “How likely would you be to recommend us to a friend” is the ultimate question to ask customers. Using a 0–10 score where 0 is not at all and 10 is extremely likely, he claims the Net Promoter Score is the single most reliable indicator of a company’s ability to grow.
Once a year, ask every customer how likely they would be to recommend you to a friend. By dividing your customer base into four groups and surveying 25 percent every quarter, you can have a continuous source of data.
When you survey them, let them tell you why they would or wouldn’t recommend you: What’s the one thing we can do to improve?
Analyze for patterns, and then look at product, market segment and customer tenure to see whether the patterns vary by these attributes.
You might learn some market segments are more likely to recommend than others. If your strategic segments are lacking promoters, what do you need to do to change?
When you pinpoint the areas you need to improve (such as product, segment or customer tenure), be sure to interview both customers who responded to your survey and customers who did not. The quiet 80 percent of customers who don’t respond to surveys might have a different story to tell. Here is where the quantitative metrics alone might not tell the whole story.
3. PRODUCT
Cost of delayed product introduction. How long does it take from the time you define requirements until you have a finished product in the market? Is the cycle predictable?
If you have developed a clear product roadmap, do a retrospective every quarter tracking what you planned to do versus what you actually did.
An important roadmap metric is the cost of delayed products. If the product was delayed by two months, go back to the monthly forecasted revenue for the product times two months. If you missed an important market window (like Christmas), the opportunity cost will be much greater.
Percentage of approved requirements/ out-of-scope requirements. Monitor the percentage of originally approved requirements that get delivered in the final product. How many out-of-scope requirements were included? The problem may be poor requirements definition, which you can control, or it may be scope creep, which you need to communicate to your management. Report the facts and let management manage.
Open defects/time to remedy. Product quality should be monitored on an ongoing basis to make sure it doesn’t become a serious problem.
Tracking defects by type of defect and time to remedy compared to prior periods and prior releases will give you insight into the quality of the current release.
For unreleased products, ask quality assurance for the statistics. If the number of open defects is not going down as the release date approaches, the release date is likely to slip. Releasing a product with poor quality will cost you money, time and reputation—don’t even go there!
Customer support. From a product perspective, customer support should be measuring the number of calls per product, the average time to resolve an incident and the nature of the calls.
Analyze support metrics to identify areas of the product that can be improved for a better customer experience. Are there problems with installation of the product? Is the documentation too hard to understand? Is the product too hard to use?
4. MARKETING
Number of leads, lead conversion rate, revenue performance. The marketing department should be measuring every marketing program to determine the proper program mix to drive awareness and to engage with buyers early in the buying cycle.
If you have a closed-loop marketing automation system integrated with your customer relationship management system, you should have access to a dashboard of marketing and sales metrics to help you see where in the funnel you may be having difficulties if you are not meeting your sales targets.
How are you qualifying leads through the various stages of the buying cycle? Are opportunities getting stuck in a “no decision” state? If so, why?
Which marketing programs are most effective? Are the right market segments being targeted? Do the positioning and market messages resonate with the buyers?
Win/loss interviews will usually provide deeper context around what is working and not working in the buying process.
5. MARKET
Market share. Everyone wants to know what market share you have. By tracking this metric over time (year over year), you can discover whether the market is saturated or ripe for expansion. If the market is growing and yet your market share is shrinking, it indicates another competitor is growing at a faster rate than you are. If you are doing win/loss interviews, you can gain additional insights into why competitors’ market share is increasing or decreasing.
To determine market share, you can either conduct your own survey or commission a research firm to survey the market to determine what solutions the market is using. You may also be able to purchase market share information from industry analysts or trade associations for certain vertical markets.
WHAT’S NEXT?
Determine which KPIs your company is already tracking at the product level, then examine which have the greatest variances between goal and actual results. Do you already have a plan or program to change the variances or mitigate the risk? If not, who should be responsible for developing and implementing the plan?
Look for some quick wins and rally the team to solve those problems. For larger, more systemic problems, make a business case so you can get resources to solve them. They should become part of the vital objectives of your product team.
In the meantime, prioritize the remaining KPIs you are not yet tracking. Work with the department capable of generating the metrics to find out how long it will take to begin gathering them.
If you are not already doing a business review of your product with the executives on a quarterly basis, schedule one. Step up to the strategic side by providing business visibility to the key stakeholders.
