Testing & Evaluation

Testing non-deterministic AI agents is fundamentally different from testing traditional software. Since the output can vary for the exact same input, boolean assert statements are often insufficient.

The Evaluation Stack

1. Assertions (Deterministic)

For some parts of the system, traditional tests still work.

• Output Format: Is the output valid JSON?
• Tool Usage: Did the agent call the search tool with the correct arguments?
• Latency: Did the response take less than 2 seconds?

2. Reference-Based Grading (Rouge/BLEU)

Comparing the agent's output to a "gold standard" human-written answer.

• Exact Match: Rarely used for generative tasks.
• Fuzzy Match: Checking for semantic similarity.
• Limitations: An agent might be correct but phrase it completely differently, leading to a low score.

3. LLM-as-a-Judge

Using a stronger LLM (e.g., GPT-4) to evaluate the output of the agent locally.

• Rubrics: Giving the Judge LLM a specific rubric (e.g., "Is the answer helpful? Is it polite?").
• Pros: Scalable, fast, and captures semantic nuance better than code.
• Cons: Can be expensive and bias towards its own "style."

4. Human Evaluation

The gold standard, but the most expensive.

• Thumb up/down: Simple binary feedback from users.
• A/B Testing: Showing two versions to users and seeing which performs better.

What to Test?

Correctness

Does the agent answer the question factually? This is crucial for RAG systems to avoid hallucinations.

Robustness

How does the agent handle:

• Jailbreaks: Attempts to make it do something bad.
• Garbage Input: Random characters or nonsense.
• Language Mixing: Switching languages mid-sentence.

Tool Use Accuracy

• Selection: Did it pick the right tool?
• Arguments: Did it provide the correct parameters?
• Error Handling: What happens if the tool API fails?

Frameworks for Eval

• Ragas: Specifically for RAG pipeline evaluation.
• DeepEval: Unit testing for LLMs.
• Promptfoo: CLI tool for testing prompts matrix-style.