- Smoke Testing
- Regression Testing
- Edge Case Testing
Smoke Testing
Smoke testing verifies that the agent’s most essential conversational functionality works as expected before deeper testing begins. Use it as the first gate in your test suite to confirm that a build or model update is stable enough for further evaluation. Run smoke tests before larger test suites, after major prompt, model, or tool changes, and to quickly verify agent health after a deployment. Keep them fast and narrowly scoped, covering only the most critical conversation paths. Passing smoke tests should be a prerequisite for deeper testing. Example: A greeting and intent test that confirms the agent responds correctly to a basic request like “Hello, what can you do?”Regression Testing
Regression testing confirms that changes such as prompt edits, model upgrades, tool updates, or bug fixes do not break previously working behaviors. As your agent evolves, regression tests ensure that earlier successes remain intact and protect against silent regressions introduced by iterative changes. Run your full test suite after every change and before production deployments. Structure tests so they can be replayed consistently and automatically. Example: If an agent correctly handles “How do I reset my password?”, a regression test verifies that this behavior still works after unrelated changes are made.Edge Case Testing
Edge case testing targets scenarios that are unusual or unlikely but plausible in real user conversations. Conversational agents can behave unpredictably when given unexpected inputs or when a conversation drifts across topics. Edge case tests are especially important when users deviate from expected flows, use informal language, or switch topics mid-conversation. Run edge case tests continuously throughout development and after deployment. Revisit them whenever real user logs reveal unexpected failures. Treat every discovered edge case as a reusable test asset. Add it to your suite immediately and prioritize by impact and likelihood. Example: Testing how the agent responds when a user asks “What’s the weather like?” in the middle of a billing support conversation, or when they use phrasing that could match multiple intents. For a full reference of edge case patterns, see Edge Cases.Putting It All Together
These three strategies are most effective when combined in a shared, living test suite:| Test Type | Purpose | When to Run |
|---|---|---|
| Smoke Testing | Catch critical failures early | Before regression testing |
| Regression Testing | Protect existing behavior | After every change |
| Edge Case Testing | Guard against unusual failures | Ongoing |