Reduce the Risk of Product Failure


Usability testing includes a range of techniques that reveal how well your product performs from a very important perspective: your customers’.


At its core, usability testing is an opportunity to observe target customers using your product to perform specific tasks that can validate product assumptions, provide valuable feedback, or identify potential pain points or enhancements. After several tests, you’ll notice usage patterns and comments about what works and what doesn’t. This is vital information you’ll use to fine-tune your product and increase its value to customers.


The sooner you validate product assumptions, the better. There’s little excuse not to run at least a few usability tests as you develop your product; you can run them at minimal cost (from virtually free to a few hundred dollars). And, when you compare the cost of changing designs early in the process instead of after you’ve launched, you’re talking peanuts.


Some tests take only a few minutes to set up and a few seconds to run. Literally. And with the right preparation, there’s little impact on product development schedules.


Let’s take a look at which tests to run at each point in the life cycle, followed by some real-world examples to highlight situations in which usability testing has been effective. Finally, I’ve included step-by-step-instructions for four usability tests you can implement immediately.

Testing All Stages of the Customer Experience

The type of usability test you use depends on where you are in the product development life cycle and what you want to learn. You can perform tests on everything from the earliest product ideas or concepts—think paper sketches, even paper napkins—to prototypes to fully developed systems. Furthermore, you can run these tests on websites, mobile or wearable apps, and everything in between.


As customers develop a relationship with your product, usability tests will offer insights into what happens at each stage. Each test will contribute to an invaluable repository of insights into how your customers think about your product and what they value, because you’ll see the world through their eyes.


First Impressions

Watch what happens when customers first encounter your product. What gets their attention? Is anything confusing? Do they understand the marketing messages on your home page, and does the information resonate with them? Ultimately, do they feel compelled to explore more deeply?


Early Experiences

If you’ve communicated a positive initial impression and customers decide to explore your product, can they discover the features and functions that help get them up to speed? Does your interface provide clear, streamlined paths free of unnecessary instructions, distracting visual elements, or unneeded features or functionality? Finally, how well does your product help customers get the benefits they want, or even lead them to benefits they were unaware of?


Testing helps determine if customers find the terminology clear and jargon-free; whether the page and content is organized logically from their perspective; if processes—such as registration, checkout or upgrading—are efficient and easy to use; and if it’s clear how to cancel a process or navigate to a specific function or location.



Over time, discovering shortcuts and advanced functionality will help mature customers do their jobs more quickly. You’ll want to test how these options are communicated and ensure that they address the needs of these customers without impacting the experience of other customers.


Unless you test with actual users, your product team won’t get a true perspective about what your product is like to use by the people who matter most: the users themselves. This may seem obvious, but in many organizations, approximate customers—such as salespeople—are sometimes used to review products. After all, they have a lot of customer contact. But salespeople don’t have the same motivations or context as customers. At best, this approach is risky. When you test with real users, usability tests ensure that all product stakeholders get a realistic, honest view of your product’s effectiveness.


If you have never watched a customer use your product, you might be in for a surprise. “Obvious” product assumptions may be challenged, or you might find that customers think differently about what value means to them. Whatever you discover, you’ll unearth information to develop products that are more likely to be ones that your customers value.


Real-World Usability Testing

The following real-world examples illustrate ways that usability testing can improve your product.


Discover new or unanticipated product features that improve product profitability

An e-commerce website, selling specialist technical products upgraded its website design. During a usability test, the company was delighted that existing customers thought the new website design was a significant improvement. However, when customers were asked to buy particular products, several said they found it frustrating when an item was out of stock. While most users believed that alternative products were available, they weren’t sure what those were. It was a primary trigger for customer-service calls.


Based on initial testing results, the new site design was altered to include a “view similar products” option when the customer’s first choice was not available. Further testing demonstrated that the feature was an instant hit. The business increased sales, decreased the odds that customers would go to competitor sites to complete their orders, and reduced customer-service costs.


Validate assumptions to create more effective upsells and cross-sells

A major bank was interested in selling additional products to new customers during the sign-up process. Earlier testing clearly showed that when customers signed up for a specific product, they weren’t interested in being marketed other products until they completed their primary task.


The bank’s designers developed several concepts to introduce other products after sign-up completion. One concept included visually rich banners and product feature boxes on the thank-you screen. The other was a series of check boxes and brief product descriptions to indicate “other products you might be interested in.”


Many on the business team were surprised to hear customers say that they found the plainer version to be a more compelling invitation to explore other products. The reason? It more naturally followed from the application process they had just completed. They perceived the visually rich version as “more obviously selling” and found it harder to focus on specific products because of the visual competition.


Validate assumptions early to avoid the high cost of changes late in the development cycle

An online bookseller specializing in college textbooks wanted to use social media to increase self-promotion. When a book sold online, the system posted purchase details on each buyer’s Facebook page, based on the assumption that buyers would want to share this information. In addition, other social sharing features were strongly recommended.


Because no obvious messages or confirmations were provided, it became clear during testing that participants didn’t realize their purchase information was being shared. Participants became visibly annoyed when they realized that certain class titles—which might suggest deviant interests—had gone public. This problem was amplified when participants struggled to remove information and change the site’s default sharing options. The site owner’s assumption was clearly invalidated.


This testing only occurred close to launch, when the system was largely completed. As a result, the cost of making changes was significant and could have been avoided if the concepts and assumption had been tested before any coding was done.


Avoid unintentional problems by revealing the real user experience.

hardware company wanted to assess the ease of its home set-up process for a new version of its product. Customers had to connect the device to a video screen and had the option to access the Internet via Wi-Fi to enhance its capabilities.


To mimic real life in the test, customers were given brand-new devices sealed in their original packaging. They were asked to set up the device and use it. A high-definition TV and Wi-Fi Internet were provided. As the client’s product team watched participants via a remote video link, it became immediately clear that the volume of opened packaging made it difficult to find already-unwrapped items. In turn, that made the set-up process longer and more frustrating than necessary. Participants also complained that the Wi-Fi set-up was too difficult.


As a result of testing, the hardware company reviewed its packaging and took steps to streamline the Wi-Fi set-up process.


Test your branding perception and create compelling marketing communications that paint an accurate picture

XYZ Software’s product enabled clients to create custom software products from a range of modules. Their website prominently featured several customers, one of whom organized conferences. When visitors to XYZ’s site saw this featured customer, they thought XYZ was a conferencing company, not a software developer.


When XYZ Software became aware of the misperceptions around its branding, the company revamped the website to ensure that primary messaging and imagery matched what their brand actually offered customers.


Improve product performance and compare it against competitors.

ABCDE Corp. was eager to assess the effectiveness of its online product search against competitors’ systems. Target customers were invited to locate the same product on several competitive sites as remote observers timed each attempt. This time-on-task method was repeated periodically to measure site improvements.


By observing customers as they repeated the time-on-task method, ABCDE Corp. was able to tighten its online product search and make site improvements. Changes to the way the search and search results were presented, and to the underlying search engine software, drastically improved visitors’ ability to quickly find the products they were looking for.


Do-It-Yourself Usability Testing: A User’s Guide

These four usability tests will provide valuable insight into your product’s effectiveness and help build confidence in your product. Plus, they’re easy to set up and perform.


Remember, in any usability test, the less said, the better. Don’t provide additional information about the task unless a clarification is required. Screens must explain themselves, because in real life you wouldn’t be in the room. And although it isn’t necessary to test final screens, if you intend to test a mock-up, make sure it contains realistic content.


Five-Second Test

Assess which screen design elements viewers find most memorable. The results give product teams a good idea of how well the product is understood and whether it offers the value that target customers want.


For testing:

  • Branding and marketing messages
  • Value propositions
  • Calls to action
  • What customers notice
  • Content clarity


  1. Take a screenshot of the page you want to test and open it in a web browser.
  2. Minimize the browser.
  3. Locate users who fit target profiles and test one person at a time.
  4. Explain to users that you will show them a screen and then ask questions.
  5. When they’re ready, maximize the browser.
  6. After five seconds, minimize the browser.
  7. Ask the user one or two questions about the screen they just saw. Focus on the screen’s priorities. It’s okay to include a general question. Here are some examples you might use:
  • Whom does this site belong to?
  • What is this website for?
  • What does this business do?
  • What is the tagline?
  • What options are available to …?
  • What is the special offer?
  • What, if anything, caught your attention?
  • Do you recall anything that you particularly liked or disliked?
  • Do you recall anything that made you pause
    and think?


If you include several screens in your test, repeat the process and test one screen at a time. Use different questions for each screen so participants can’t anticipate your intent.

Basic Usability Test (Task Testing)

This is the most common usability test. You create realistic tasks that you’d expect customers to perform, then watch as they try to complete them. This will reveal how well your product performs.


For testing:

  • Navigation
  • Content clarity
  • Effectiveness of specific functional


  1. Prepare screens that illustrate the main steps someone would take to perform a task (think paper sketches, finalized screen concepts or live screens).
  2. Locate five people who fit a particular target profile to be your test participants (but test one person at a time).
  3. Read out one task at a time. For example, ask the participant to search for a product. Next, ask them to show you what they would do.
  4. Ask them to think aloud as they perform the task and summarize their experience when they’re done.
  5. Compare your observations from each participant to identify patterns that suggest problems.


Your tasks should describe a situation and desired outcome but not include information about how to do the task. For example, “You want to buy a waterproof hiking boot and spend no more than $100. You go to and land on the following page. What would you do?”

Content Test

This test uses existing content as the starting point to determine what customers find most valuable and how easy it is for users to find that information on your website. The test results are a great way to ensure your site’s content is appropriately focused.

For testing:

  • Content that users find most compelling
  • Whether users can easily find the content they value most


  1. Print screenshots of the website’s primary pages.
  2. Place printouts in a folder. Page order isn’t important. The goal is to provide a variety of site content.
  3. Invite users to review the folder and highlight content they think is valuable (create a new folder for each user).
  4. Take the folder from the user. From the examples they highlighted, select one at a time and ask them to locate that content on the live website.


Adobe Acrobat Pro has an option for automatically printing out a website’s pages.

Inherent Value Test

Identify what your best customers value most about the product. These power-users know your product well and often develop work-arounds and techniques to maximize its effectiveness. You can use this knowledge to create tasks for new or potential customers. Their reactions will indicate whether they recognize the main value of your product and how well your product leads them to this value.

For testing:

  • The effectiveness of communicating the product’s value and best features to new customers


  1. Invite your best customers to demonstrate—preferably in person—what they value most about your product, how they use it and explain why it is valuable to them (e.g., saves time, make a job easier to perform).
  2. Create tasks for the things your experienced customers value most. For example, if you make home-management software, an experienced customer might note that you can set up multiple tasks to occur with one trigger. You could then create a task such as “I want to wake up at 7 a.m., have my coffee ready and have the bathroom heated to 80 degrees.”
  3. Locate new or potential customers and ask how they would do the tasks you have created.
  4. Note any variations between experienced and new customers. For example, new customers may not realize they can set multiple tasks to occur from a single trigger, and may set up three separate tasks. After getting their reaction, show them the experienced users’ path and ask for their thoughts.


Even if everything is going smoothly, it’s good practice to play dumb and ask participants to explain what they’re doing and why. You may be surprised by what you hear.

These four usability tests will help you evaluate new product concepts, product upgrades, current products and competitors’ products. Not only will they provide valuable insight into your product’s effectiveness, they’ll also help you build confidence in your product.

If you would like to learn more about usability testing and how your organization can benefit from this powerful technique, register now for Pragmatic Institute’s September 17 webinar (1p.m. EDT) at You’ll learn how to improve your usability skills and increase the value your product provides to customers.


  • Peter Hughes

    Peter Hughes has spent more than 20 years conducting usability tests for industry-leading corporations and organizations such as JPMorgan Chase, MetLife, T. Rowe Price, Boeing, Honeywell, Hearst Media, Scholastic, the College Board, Nintendo and The Robert Wood Johnson Foundation. He is founder and lead consultant at Ascest, where he coaches organizations on how to bring the usability testing skills they need in-house and squeeze the most out of those tests. If you have questions about the UX process, email Peter at [email protected].

    View all posts


Tags: Agile

Other Resources in this Series

Most Recent


Exploring The Agile Product Operating Model

Dave West, CEO & Product Owner at, explores the agile product operating model and its role in integrating digital technology into organizations and its impact on operations and structure.

What is the Product Life Cycle?

What is the product life cycle, what role does it play in guiding a product to market and why is it so important to successful product management? This article answers these and other important questions. 

What is Product Planning

Discover how strategic product planning drives a product from concept to market success, ensuring it meets customer needs and aligns with business goals. 

Transforming Challenges Into Success: Michael Hirsch's Journey with Pragmatic Institute 

Learn how Michael Hirsch leveraged Pragmatic Institute's Framework to drive transformative results and advance his product career.

What is Product Leadership? (With Job Titles, Skills and Responsibilities)

Learn the essential skills, key responsibilities, and common job titles in product leadership, and explore how product leaders differ from product managers.

OTHER ArticleS


Exploring The Agile Product Operating Model

Dave West, CEO & Product Owner at, explores the agile product operating model and its role in integrating digital technology into organizations and its impact on operations and structure.

What is the Product Life Cycle?

What is the product life cycle, what role does it play in guiding a product to market and why is it so important to successful product management? This article answers these and other important questions. 

Sign up to stay up to date on the latest industry best practices.

Sign up to received invites to upcoming webinars, updates on our recent podcast episodes and the latest on industry best practices.



Pragmatic Institute Resources