Effectively Leveraging In-Home Testing, Especially During Turbulent Times
As people around the world shelter in place under the threat of COVID-19, demand for consumer-packaged goods (CPG) is booming. In the United States, supermarkets are struggling struggle to keep shelves stocked.
CPGs top the list of in-demand products because they tend to be safe (they don’t transmit infectious diseases), are reasonably long-lasting and are affordable in most instances. The current resurgence of CPG industries as a safety bulwark against the community spread of infectious diseases is a reminder of the strategic importance of CPG products.
But, like any product, the long-term success of CPG products and industries lies in understanding the value that these products deliver, and that understanding comes through listening to the market. Given the current circumstances the world finds itself in, in-home usage testing (iHUTs) provides an optimal method for gathering needed feedback to continually improve. Before addressing best practices for iHUT, let’s review its history.
The Golden Age of CPG
iHUT traces its origins to the development and growth of CPG industries in the United States in the late 1800s and early 1900s, through to the formation of the first corporate marketing research departments in the 1920s and 1930s.
With the end of World War II in 1945, the practice of iHUTs spread rapidly. Returning soldiers, new marriages, a baby boom and a population deprived of many consumer goods through the Great Depression and the war years fueled rapid growth in CPG industries.
The widespread adoption of iHUTs by CPG companies led to dramatic improvements in product quality and consumer acceptance of store-bought packaged foods, beverages and household consumables. The humble iHUT received little credit for CPG growth after the war, but the importance of its contributions cannot be overstated.
iHUTs were (and are) used for multiple important reasons. The most important are to:
- Achieve product superiority over competitive products
- Continuously improve product acceptance as consumer tastes change over time
- Monitor the potential threat levels posed by competitive products
- Reduce the costs of product formulations and/or processing methods while maintaining product quality
- Measure the effects of aging upon product quality (self-life studies)
- Monitor product quality from different factories and through different distribution channels
- Predict consumer acceptance of new products
Companies that are committed to rigorous iHUTs and continuous product improvement can, in most instances, achieve product superiority over the competition. In turn, this superiority helps build brand share, magnifies the positive effects of all marketing activities (e.g., advertising, promotion, selling), and often allows the superior product to command a premium price.
Unfortunately, in today’s short-term, highest-profit-margin world, many CPG corporations don’t use iHUTs like they did back in the golden age of consumer-packaged goods. Many of the iHUT corporate leaders from 75 years ago — if they’re still in business today — have lost the art of in-home usage testing or the budgets to do serious in-home research. iHUT shortcomings in most CPG companies create opportunities for the few that are dedicated to product superiority and continuous product improvement.
iHUT Best Practices
To fully exploit the value of iHUTs, there are some best practices for CPG companies to pursue.
A Systems Approach
iHUT methods and procedures should have standard operating procedures so that every product is tested the same way, including:
- Identical product preparation, product age, packaging and labeling of test products
- Identical questionnaires (though, portions of the questionnaire must be adapted for different product categories)
- Identical sampling plans from iHUT to iHUT
- Identical data preparation and tabulation methods
- Similar analytical methods
As products are tested over time, the goal is to build more normative databases so that successive iHUTs become more meaningful and valuable. The normative data, or norms, continually improve a company’s ability to correctly interpret its iHUT scores.
Same Research Company
Use one research company for every iHUT. This is the only way you can ensure all tests are conducted the same way.
If a product typically is used at home, it should be tested at home. If it’s consumed in restaurants, it should be tested in restaurants and so on. Generally, this kind of real-environment test always produces the most accurate results. For food products, an in-home use test is almost always more accurate and more predictive of future success than a taste test conducted in a laboratory or kitchen.
Sampling is a critical variable in iHUTs. For new products or low-share products, the sample should reflect the brand-share makeup of the market. For well-established, high-share (or highly differentiated) products, the sample should contain a readable sub-sample of that product’s users and a readable cell of nonusers.
If the product category is underdeveloped (e.g., a relatively new category), then the sample should include both nonusers and users of the category. That is, if a company’s brand share is very low, it’s important to assign more weight or importance to the opinions of nonusers of the brand. If brand share is high, then the thoughts of brand users are most important.
Product performance and quality must be defined from the customer’s perspective, not the manufacturer’s. Which aspects of the product are truly important to customers? These critical variables must be identified for each product category—typically with focus groups or depth interviews—and incorporated into the standardized iHUT testing system.
Careful and Cautious
The formulation of an established product should never be changed without careful testing and evaluation of the new formulation. Once you are sure you have a better product based on iHUT testing, introduce the “better” product into a limited geography for a reasonable time (i.e., several repeat purchase cycles). Then, and only then, roll the new product out to all markets. The smaller the market share, the greater the risks that one can take with a new formulation. The larger the market share, the more conservative one should be in introducing a new formulation.
The monadic, sequential monadic, paired-comparison and proto-monadic research designs are the most widely used for product testing. Monadic and sequential monadic designs are recommended as the best methods for iHUTs, especially the monadic test.
Testing a product by itself offers many benefits, and monadic testing typically is the best testing method available. Interaction effects between products—which is what occurs in paired-comparison and sequential monadic tests—are eliminated. The monadic test simulates real life, as that’s the way we usually use products—one at a time. By focusing the respondent’s attention on one product, the monadic test provides the most accurate and actionable diagnostic information. Additionally, the monadic design permits development of norms and action standards.
Sequential Monadic Designs
Often used to reduce costs, in this design each respondent evaluates two products (they use one product and evaluate it, then use a second product and evaluate it). The sequential monadic design works reasonably well in most instances and offers some of the same advantages of pure monadic testing.
However, it’s important to be aware of the “suppression effect” in sequential monadic testing. Compared to a pure monadic test, all test scores will be lower in sequential monadic design. Therefore, the results from sequential monadic tests cannot be compared to results from monadic tests.
Also, as in paired-comparison testing, an “interaction effect” is at work in sequential monadic designs. If one product is exceptionally good, the other product’s test scores are disproportionately lower and vice versa. Asking consumers to test two products in their homes runs the risk of miscommunication and confusion between the two products—a significant disadvantage compared to a monadic design.
In this testing, the consumer is asked to use two products simultaneously and determine which product is better. This appeals to our common sense—and is a wonderful design when presenting evidence to a jury because of its face value or face validity. This can be a very sensitive testing technique (i.e., it can measure very small differences between two products), and it’s often less expensive than other methods because sample sizes can be smaller in some instances.
However, paired-comparison testing is limited in value for a serious, ongoing product testing program. It doesn’t tell us when both products are bad. It doesn’t lend itself to the use of normative data. It’s heavily influenced by the interaction effect (i.e., any variations in the control product will create corresponding variance in the test product’s scores). For iHUTs, there is a great risk that the respondent will confuse the two products.
Defined differently from researcher to researcher, the proto-monadic design begins as a monadic test and is followed by a paired-comparison test. Oftentimes, sequential monadic tests are followed by a paired-comparison test, too.
The proto-monadic design yields good diagnostic data, and the paired comparison at the end can be thought of as a safety net—added insurance that the results are correct. The proto-monadic design typically is used in central-location taste testing rather than in in-home testing because of the complexity of execution in the home.
Monadic research designs are recommended for iHUTs because virtually all consumer products can be tested monadically and the results are free from interaction and suppression effects. Because only one product is tested, there is less chance for respondent confusion and error. And some products can’t be accurately tested in a paired-comparison design. For example, a product with a very strong flavor (e.g., hot sauce, alcohol) may deaden or inhibit the taste buds so that respondents aren’t accurately testing the second product in a paired-comparison test.
While most iHUTs are conducted in the food, beverage and household consumables categories, iHUT concepts and methods are applicable to many product categories, although the structure and mechanics of execution will vary.
For example, computer software, furniture, small appliances, large appliances, cosmetics, over-the-counter medicines and toys can be tested in-home. Power tools, lawnmowers, trimmers, dog food, cat food and bug spray can be tested in-home. Any product used in or around the home can be tested in-home.
The ultimate benefit of iHUTs is competitive advantage. Creating and maintaining a better product is the surest way to dominate a product category or an industry. Creating better products blocks competition from related industries. For example, the decline in iHUTs among CPG food companies in past 30 to 40 years has allowed restaurants to take a huge share of the CPG food business by offering superior products.
Companies dedicated to ongoing in-home usage testing can achieve and sustain product superiority. On the other hand, companies that ignore the learning and guidance from iHUTs may wake up one morning to find themselves on the brink of extinction from a competitor that has built a better mousetrap.
Looking for the latest in product and data science? Get our articles, webinars and podcasts.