Skip to main content
A simulation lets a simulated caller, the tester, talk to your agent. You describe the tester once. Then you run the simulation against a draft or a published version of the agent as often as you like. Checks look at every conversation of a run, so a prompt change that breaks a call shows up before a real caller hears it. You find simulations in two places:
  • Simulations in the sidebar lists every simulation of the account, with its runs on all agents.
  • The Simulations tab of an agent lists the simulations added to that agent.

​
Create a simulation

Open the agent and select the Simulations tab. Select Add new Simulation and describe what you want to test.
New Simulation dialog with the field What do you want to test? and the buttons Create Simulation manually and Continue with CharlieNew Simulation dialog with the field What do you want to test? and the buttons Create Simulation manually and Continue with Charlie
  • Select Continue with Charlie to let Charlie write the simulation. Charlie reads the agent, works out the tester, the variables, and the tool answers, and asks about anything that is unclear.
  • Select Create Simulation manually to fill in the form yourself.
Simulation form with a name, a description of the simulated caller, outbound direction, and the agent talking firstSimulation form with a name, a description of the simulated caller, outbound direction, and the agent talking first
After you select Create, the Add Checks step lets you add checks for the new simulation. Select Done to skip it. To create a simulation from a real call, open the call in Conversations and select Create simulation. The button shows once the call is processed and the caller has said something. Charlie builds the simulation from the transcript, replaces personal data with made-up data, and adds the simulation to the call’s agent. To use a simulation of another agent, open the menu next to Add new Simulation and select Add existing simulation. Both agents then share the simulation, so an edit changes it for both. To change it for one agent only, use Duplicate in the simulation’s menu, edit the copy, and select Remove from agent on the shared simulation.

​
Run a simulation

Before you run, make sure the account has an active phone number that the agent can use. No call is dialed, but each simulated call takes that number as its own. Select Run on a simulation. The Run dialog opens.
Run dialog with Version to test, Amount of conversations set to 4, and the note that simulated calls are billed like regular callsRun dialog with Version to test, Amount of conversations set to 4, and the note that simulated calls are billed like regular calls
  • Version to test is Test draft, Test published version, or a version under Older versions. The draft is selected when it has changes, otherwise the published version.
  • Amount of conversations is how many conversations the run places. Pick 1, 2, 4, or 6, or enter a custom amount up to 20.
Run all starts one run for each simulation on the agent. Its dialog asks for the Conversations per simulation and shows the total. On the account-wide Simulations page, Run also asks for the Agent. If the simulation is not on that agent yet, Add to this agent adds it. While a conversation is open next to the list, Run is in the simulation’s menu. While a run tests the draft, do not save changes to the draft. A save during the run makes its remaining conversations fail. Each run shows its conversations under it. An account runs up to 5 simulated conversations at a time. The other conversations show Waiting to start until a slot is free.

​
Review the results

Select a conversation to open it next to the list. A finished conversation shows the recording, the check results, and the Overview and Transcription tabs. While a conversation is in progress, you see the live transcript. Select Listen, the headphones button, to hear the call.
Agent Simulations tab with a running conversation and its live transcript next to the list of simulationsAgent Simulations tab with a running conversation and its live transcript next to the list of simulations
When the checks are done, the icons show the result. A conversation icon is green when every check passed, red when a check failed, and orange when a check could not be run. A run icon is green when every conversation passed, red when the checks failed in every conversation, and orange when they failed or could not be run in some. Hover over an icon for the details. Checks of this run lists each check with how many conversations passed it.
Finished simulation run with its conversations, the check results, and the transcript of one conversationFinished simulation run with its conversations, the check results, and the transcript of one conversation
From the menu of a run:
  • Run again opens the Run dialog with the version and the amount of conversations of that run. The new run uses the simulation as it is now. For a run of the draft, it selects the current draft, or the published version if the draft has no changes.
  • Cancel run hangs up the conversations in progress and skips the rest.
  • Delete removes a run that has ended, with its conversations.

​
Check conversations automatically

A check looks at every conversation of a run and marks it as passed or failed. Select Checks on the Simulations page, on an agent’s Simulations tab, or in a simulation’s menu. The dialog lists the checks for all agents and the checks for the agent or the simulation you opened it from. For other agents and simulations, it says where their checks are listed.
Checks dialog with the sections All agents, the agent, and the simulation, and one check in the simulation sectionChecks dialog with the sections All agents, the agent, and the simulation, and one check in the simulation section
Select Add Check. Applies to sets which runs the check looks at. It starts with the place you opened Checks from, and you can change it.
  • All agents applies to every simulation run in the account. If the tested agent version does not have the outcome or the tool that the check names, the run leaves the check out and shows a warning. On an agent’s tab, such a check shows Not checked here.
  • One agent applies to every run of that agent.
  • One simulation applies to the runs of that simulation, on any agent.
New Check dialog for the outcome roof_check_agreed with the expected value YesNew Check dialog for the outcome roof_check_agreed with the expected value Yes
A new, edited, or deleted check changes only runs that start after the change. Earlier runs keep their checks and results. Each result is Passed, Failed, or Could not be checked. A conversation shows Not checked when its run ended before the check, for example after Cancel run.

​
How simulated calls work

  • No phone call. The agent and the tester meet in a browser call, and no contact is created. The account still needs a phone number for the agent, as described in Run a simulation.
  • Tools. Calendar tools, SMS, WhatsApp, call-me-later, and transfers are always simulated, so nothing is booked, sent, or transferred. Your own tools never run for real. Without a set answer, each call gets a new generated answer, so two calls of the same tool can disagree. Set an answer when the result matters. Ending the call, collecting data with its address validation, knowledge base search, and web search run as in a real call. Web search returns live results.
  • Length. A conversation lasts at most as long as the simulation’s Max call length, or less if the agent’s or your plan’s maximum call length is shorter.
  • Data. Each conversation is recorded and transcribed, whatever the agent’s recording settings or the tester’s consent, and its outcomes and collected data are analyzed. Simulated calls do not show in Conversations, and they send no webhooks, emails, CRM updates, or workflow triggers. Data retention rules do not delete simulated calls.
  • Billing. Simulated calls are billed like regular calls.
  • Time. Conversations run on the current date and time.
  • Limits. The tester cannot interrupt the agent in the middle of a sentence. A Duo agent that uses the browser cannot run simulations yet.
You can also ask Charlie to create, run, and review simulations and checks for you.