Treat persona prompts as controlled scenarios, then validate them with customers.
Use decision-relevant constraints
Team size, budget, location, implementation capacity, and a required feature can materially change which product fits a question. Choose constraints grounded in your customer research rather than inventing a decorative biography for the prompt.
Do not introduce personal attributes simply to make a persona feel realistic. Include information only when it is relevant to the decision you are investigating, and handle customer research under your normal privacy rules.
Build paired scenarios
Keep the underlying task stable and vary one meaningful constraint at a time. A fictional small-team scenario might ask for simple setup; a larger-team scenario might require granular administration. Save the exact wording and explain what the comparison is intended to test.
A synthetic scenario is an experiment you designed. It does not prove that people matching that description receive the same answer, or that the scenario represents their real behavior. Keep customer evidence and simulated observations separate.
Inspect suitability, not only mentions
Compare the rationale given for each recommendation. Which requirements are addressed, ignored, or incorrectly attributed to a product? Check important claims against current documentation. A brand appearing in more scenarios is not necessarily an improvement if it is recommended for unsuitable customers.
Use findings to clarify product positioning and content boundaries. Explain who the product is for, what it requires, and where an alternative may be a better fit. Those distinctions help customers even when the assistant’s behavior remains variable.
Validate with real evaluations
Review the scenarios with customer-facing teams and, where practical, customers. Ask whether the constraints and tradeoffs reflect actual decisions. Revise the research set when you discover an assumption was wrong, retaining the version history.
Report the outcome as a scenario comparison with sample sizes and examples. Avoid labeling it a demographic finding unless the research design and data genuinely support that claim.
