AI prototypes can help product teams test a proposed solution, observe how people use it, and identify questions for further research. Customer reactions to a prototype do not, on their own, establish how widespread a problem is or whether solving it should be a business priority.
Pragmatic Institute instructor Amy Graham advises teams to define what they want to learn before building a prototype. In her classes, she sees teams testing a solution before they have agreed on the problem they are investigating.
Decide what to test before building
“Just stop for a minute. What are you trying to test?” Graham asks. She distinguishes several questions that require different evidence:
- Desirability: Do people want the proposed outcome?
- Usability: Can they use the proposed solution?
- Technical feasibility: Can the team build it?
- Operational feasibility: Can the organization run and support it?
- Viability: Does it make business sense?
Evidence about one question does not automatically answer the others. Write down the assumption you want to test and the response or behavior you will examine. Use that plan to choose the prototype and prepare the customer conversation.
Match prototype detail to the question
Graham suggests a rough concept when exploring desirability and a more detailed, interactive prototype when testing usability. Choose enough detail for the task you want someone to try.
Pragmatic instructor Clarke Smith cautions that AI tools can produce a polished design that looks like a finished product before a team has compared alternative approaches. “When you have a polished, finished-looking prototype it’s easy to slip into UX testing and lose sight of where you are in the process.” He suggests always showing multiple options when the team is still trying to understand a problem.
Graham recalls a vice president of product in her Design class who thought his team’s prototypes looked too finished. He believed reviewers assumed the decisions had already been made. The team wanted feedback, but some people asked where to sign up. His account concerned polished prototypes; it did not establish whether they were generated with AI.
Pragmatic instructor Will Scott suggests a workflow drawn with boxes and arrows when it is sufficient for the question. You can ask an AI tool for a workflow, wireframe, or alternative design instead of a finished-looking interface. Specify what you want to examine before requesting more detail.
Ask what customers are responding to
A customer may respond to the proposed feature, the workflow, or the visual design. Ask which part they mean.
Scott cautions against interpreting rejection of a solution as rejection of the underlying problem. Ask how the person handles the problem today and what does not work for them in the proposed approach.
Positive feedback also needs follow-up. Scott describes the mistake as assuming “that what I was trying to validate in my head, you just approved.” Ask what the customer likes, how they would use it, and what would still be missing. Compare the response with the assumption you planned to test.
Scott also warns about validation fatigue from repeated requests for feedback. Pragmatic instructor Cindy Cruzado notes that repeated reviews can leave a team struggling to decide. Before asking customers for another review, identify the unanswered question and how their response would inform your next step.
Gather evidence for a roadmap decision
Consider a hypothetical example from the instructor discussion. Three customers tell a B2B product team that reporting is difficult. A product manager creates an AI prototype and shows it to two of them. Both say they would use it immediately, and the director wants it on the roadmap.
Graham would first clarify the reporting problem. Is information missing? Does the report take too long to produce? Is it difficult to read? Suggested interview questions include:
- What makes reporting difficult today?
- Can you walk me through the last time that happened?
- Which part of this prototype would help, and what would still be missing?
If the prototype addresses the problem, Graham would then examine how many customers experience it and whether the proposed work supports the product strategy. A team pursuing new customers would need to assess whether the same problem affects those buyers.
Cruzado would also ask what other work the team would postpone. Before committing to development, consider implementation, support, and maintenance alongside the customer evidence. Faster prototyping can reduce the effort needed to explore an idea; these other costs still need assessment.
Use AI to analyze discovery evidence
Smith described alumni using AI agents to examine support tickets for recurring issues and sales call transcripts for recurring objections. “AI has made it possible to constantly look for themes and recurring problems in the information we collect every day so the discovery process never really stops.” These were participants’ accounts from a workshop he facilitated.
One participant had identified four problems with similar frequencies and did not know which to pursue. The counts described how often the problems appeared in those records. They did not resolve his prioritization decision.
Pragmatic’s Product Discovery with AI workshop is a related option for readers exploring AI in their discovery process.
Keep access to the original evidence when reviewing an AI-generated summary. A transcript or summary may omit pauses, expressions, and context that would prompt a follow-up question. Graham also recommends observing people using a product: their behavior can differ from their description of it in an interview.
Review your current discovery process
Dan Corbin recommends learning product discovery fundamentals when deciding how to use AI. Graham suggests documenting the process and evidence sources the team already uses.
List the questions you need to answer, the evidence you have, and the gaps you still need to investigate. Then identify a task where AI could help, such as comparing prototype options or organizing interview material. Check its output against the original evidence before using it in a decision.
For more on customer research, see Pragmatic’s Modern Product Discovery webinar.
