Better Metrics for New Products

By Jim Semick March 03, 2016

With so many new analytics tools available, it’s common to have a firehose of data and metrics available soon after your product launches. But the real challenge is determining which few metrics to focus on, and the sooner
the better.

By defining the right metrics early in the validation and development cycle, well before your product reaches customers, you can get better insight to guide your product decisions and your roadmap.

The Scientific Mindset

To set the right metrics and product goals early, you’ll need to think like a scientist. Scientists first describe their hypothesis, define a test and then measure. Product management can do the same by setting goals and then setting metrics for those goals. This scientific mindset is one of the best ways to guide new products to success.

For example, you may decide that a conversion metric, such as the percentage of trial customers who convert to paying customers, is important to measure. Even without solid customer data, you can create a hypothesis about what you think you will see and an ideal target. This will lead to incredibly valuable team conversations about the business model and—once the customer data begins arriving—the ability to spot challenges early.

Luckily, product management has more tools than ever to gain insight into customer behavior and business success. And these tools and approaches are by no means limited to digital products. I’ve spoken with product managers in industries as varied as education, healthcare and financial services who are taking advantage of the new breed of analytics products for data-based decisions, including:

  • Analytics tools (web analytics, product-usage analytics)
  • Customer engagement tools
  • A/B testing tools (for acquisition, product or pricing testing)
  • Usability and customer research tools
  • Portfolio management and product management software

And let’s not forget spreadsheets. Spreadsheets are indispensable in the early stages for digesting or sharing metrics. It is often easiest, in fact, to focus on using a shared spreadsheet or other simple tools before moving to a more sophisticated dashboard.

Identify Metrics for Your Product

If you don’t already have success metrics, how do you find the right ones?

Begin by researching metrics discussed in your industry. Whether you are in SaaS, retail, media or another industry, there are experts who discuss those metrics online.

Look at information about competitive products. Companies that are publicly traded will often discuss these key metrics during earnings calls.

Don’t spend much time looking at metrics that companies provide through news or PR outlets. These metrics are often vanity metrics that look good in a press release but don’t relate to the business results.

Generally speaking, business goals such as revenue, margin, acquisition cost and retention are good places to start. Customer-specific metrics such as product usage and retention are also good starting points.

Below are a few examples of metrics to help measure success of new products from a customer and business standpoint. The metrics you select will depend on your business and product. Choose only a few to start—the fewer the better—so you can focus.

Customer-Oriented Metrics
  • Product usage/adoption. Measure overall sign-in frequency, sharing and other metrics to gauge whether your users are increasing usage or adoption over time.

  • Percent of users who take a specific action. Look for actions that might be a predictor of higher customer lifetime value. For example, is their data entry increasing and is that a predictor of future license purchases? Rather than measuring every action, focus on the usage that matters.

  • Feature usage. Are there particular features that you believe will be indicators of customer success? Over time, further segment this data by understanding which customer personas are using certain features.

  • Retention or churn rate. For new products with recurring revenue, it’s essential to measure churn early as it directly relates to the lifetime value of a customer.

  • Quality. Although your product is new, begin to measure improving quality by tracking average bugs, net promoter score and other metrics.

Business-Oriented Metrics
  • Customer acquisition cost (CAC). Measuring the cost to acquire a typical new customer is essential in the early stages of a new product. CAC is your total sales and marketing cost and helps you identify where you’re spending money.

  • Lifetime value (LTV). Like CAC, the LTV of a typical customer is one of the key metrics to include for new products, especially for software. It helps ensure that you have a profitable model.

  • Monthly recurring revenue (MRR). This is an important metric for recurring revenue products. Similarly, annual recurring revenue (per customer) is important for many software businesses to ensure that you don’t lose more than you gain.

  • Average revenue per user. If MRR isn’t relevant to your product, the average revenue per customer or user might be important to reflect the depth of user engagement.

  • Conversion. For many software products, conversion rates, such as site visit to lead conversion or trial to paying customer, are an important metric to track.

These metrics are a great place to set your baseline, but ultimately you’ll want to refine them for your business. Work with your team to get consensus on the metrics that matter.

Whatever you select, be sure the metrics are actionable and tie back to the strategic goals and initiatives you put on your product roadmap. Periodically revise the goals and metrics; as the product matures, the metrics will likely need to change and grow with it.

Use Qualitative Metrics

Up to this point, we’ve discussed quantitative metrics, but there is a place for qualitative information in your metrics. This is especially useful for early stage products where customer data is scarce. It’s also where you often begin formulating your next round of hypotheses.

Track information such as customer demographics, acquisition source, pricing feedback, expected deal size, features used, customer frustrations, sales objections and other data. Use a spreadsheet to make it easier to digest the massive amount of qualitative information gathered.

These data points serve as great source material for metrics discussions and for future quantitative metrics. They also provide customer evidence for justifying product initiatives to stakeholders.

Ultimately, the metrics you choose depend on the stage of your product, your industry, the type of product and company size. But the most important consideration is to focus on a limited number of metrics that really matter. These are metrics that tie back to your organization’s top-line goals and business results.

Regardless of the metrics you choose, begin formulating your list as soon as you start validating the concept with potential customers. You’ll be able to make better decisions to guide your product and product roadmap and focus on the metrics that make a difference to the business.

Jim Semick

Jim Semick

Jim Semick is co-founder of ProductPlan, a leading provider of cloud-based roadmap software for product and marketing teams. For more than 15 years he has helped launch new products now generating hundreds of millions in revenue. He was part of the founding team at AppFolio, a vertical SaaS company. Prior to AppFolio, Jim validated and created version 1.0 product requirements for GoToMyPC and GoToMeeting (acquired by Citrix). Jim is a frequent speaker on product management and the process of discovering successful business models. He contributes at Follow Jim on Twitter at @JimSemick.

Looking for the latest in product and data science? Get our articles, webinars and podcasts.