Skip to main content
A Test Scenario simulates a customer profile to test conversations with GenerativeAgent. Each scenario defines the mock API responses and the customer’s side of the interaction, so you can assess how GenerativeAgent handles different scenarios, edge cases, and potential issues before a change reaches production.
Use Test Scenarios to test:
  • Key happy-flows
  • Edge cases
  • Common problems or issues

​
What a Test Scenario defines

A Test Scenario brings together everything a simulated conversation needs:
  • Mock API data: The User API Profile defines the data GenerativeAgent receives during the test, such as account details or order status. Mock responses can be generated for you or defined function by function.
  • Input context: The input variables and starting task your system passes to GenerativeAgent when a conversation begins. Matching production here keeps the simulated conversation realistic.
  • Customer profile and goals: The simulation details set what the simulated customer wants, what information they know, and how they behave. The customer attempts each goal in sequence, the same way a real customer works through a chat or call.
  • Evaluations: The evaluation criteria that judge the conversation after it ends, with optional applicability criteria that control when a check applies.
You can have all of this generated from a short description or from a production conversation, or define each part yourself.

Create Test Scenarios

Generate a scenario from a description, build one from a conversation, or create one manually.

Run Test Scenarios

Run scenarios in the Scenario Runner and review the simulation results.

​
Branch Scope

Scenario tests are scoped to the branch they are created in. A scenario created on a specific branch, or on main, is only visible and runnable as a simulation on that same branch. For example, a scenario created on feature-branch does not appear or run on main or any other branch. To use it elsewhere, either export and import it, or, when creating a new branch, choose the option to carry over the scenario tests from the branch being forked.

​
Export and Import Scenarios

You can export scenarios as JSON files and import them back in bulk. This makes it possible to:
  • Back up your scenarios
  • Edit multiple scenarios at once outside the UI
  • Move scenarios between branches, such as copying scenarios from main to a new feature branch
For example, download all scenarios from main as a JSON file, adjust a few values in your editor, then upload the file to staging-branch to apply them there.

​
Next steps

Input Variables

Learn how to use Input Variables to provide context to GenerativeAgent in your test scenarios.

Enter specific task

Learn how to set a specific starting task for GenerativeAgent in your test scenarios.

System Transfer Function

Learn how to create and use System Transfer Functions to transfer control to an external system in your conversations.

Previewer

Learn how to use the Previewer to analyze conversation details and evaluation outcomes for your test scenarios.