ring%3atesting-agents-with-subagents（Ring%3 正在测试具有子 Agent 的 Agent）

Testing Agents With Subagents

Overview

Testing agents is TDD applied to AI worker definitions.

You run agents with known test inputs (RED - observe incorrect outputs), fix the agent definition (GREEN - outputs now correct), then handle edge cases (REFACTOR - robust under all conditions).

Core principle: If you didn't run an agent with test inputs and verify its outputs, you don't know if the agent works correctly.

REQUIRED BACKGROUND: You MUST understand ring:test-driven-development before using this skill. That skill defines the fundamental RED-GREEN-REFACTOR cycle. This skill provides agent-specific test formats (test inputs, output verification, accuracy metrics).

Key difference from testing-skills-with-subagents:

• Skills = instructions that guide behavior; test if agent follows rules under pressure
• Agents = separate Claude instances via Task tool; test if they produce correct outputs

The Iron Law

NO AGENT DEPLOYMENT WITHOUT RED-GREEN-REFACTOR TESTING FIRST

About to deploy an agent without completing the test cycle? You have ONLY one option:

STOP. TEST FIRST. THEN DEPLOY.

You CANNOT:

• ❌ "Deploy and monitor for issues"
• ❌ "Test with first real usage"
• ❌ "Quick smoke test is enough"
• ❌ "Tested manually in Claude UI"
• ❌ "One test case passed"
• ❌ "Agent prompt looks correct"
• ❌ "Based on working template"
• ❌ "Deploy now, test in parallel"
• ❌ "Production is down, no time to test"

ZERO exceptions. Simple agent, expert confidence, time pressure, production outage - NONE override testing.

Why this is absolute: Untested agents fail in production. Every time. The question is not IF but WHEN and HOW BADLY. A 20-minute test suite prevents hours of debugging and lost trust.

When to Use

Test agents that:

• Analyze code/designs and produce findings (reviewers)
• Generate structured outputs (planners, analyzers)
• Make decisions or categorizations (severity, priority)
• Have defined output schemas that must be followed
• Are used in parallel workflows where consistency matters

Test exemptions require explicit human partner approval:

• Simple pass-through agents (just reformatting) - only if human partner confirms
• Agents without structured outputs - only if human partner confirms
• You CANNOT self-determine test exemption
• When in doubt → TEST

TDD Mapping for Agent Testing

Same cycle as code TDD, different test format.

RED Phase: Baseline Testing (Observe Failures)

Goal: Run agent with known test inputs - observe what's wrong, document exact failures.

This is identical to TDD's "write failing test first" - you MUST see what the agent actually produces before fixing the definition.

Process:

• Create test inputs (known issues, edge cases, clean inputs)
• Run agent - dispatch via Task tool with test inputs
• Compare outputs - expected vs actual
• Document failures - missing findings, wrong severity, bad format
• Identify patterns - which input types cause failures?

Test Input Categories

Minimum Test Suite Requirements

Before deploying ANY agent, you MUST have:

Fewer tests = incomplete testing = DO NOT DEPLOY.

One test case proves nothing. Three tests are suspicious. Six tests are minimum for confidence.

Example Test Suite for Code Reviewer

Running the Test

Dispatch via Task tool with test input → Document exact output verbatim (don't summarize).

GREEN Phase: Fix Agent Definition (Make Tests Pass)

Write/update agent definition addressing specific failures documented in RED phase.

Common fixes:

Example Fix: Severity Calibration

RED Failure: Agent marked hardcoded password as MEDIUM instead of CRITICAL

GREEN Fix: Add severity calibration: CRITICAL (hardcoded secrets, SQL injection, auth bypass), HIGH (missing validation, error exposure), MEDIUM (verbose errors, missing security headers), LOW (headers, deps)

Re-run Tests

After fixing, re-run ALL test cases. If any fail → continue fixing, re-test.

VERIFY GREEN: Output Verification

Goal: Confirm agent produces correct, well-structured outputs consistently.

Accuracy Metrics

Consistency Testing

Run same input 3 times → outputs should be identical. Inconsistency indicates ambiguous agent definition.

REFACTOR Phase: Edge Cases and Robustness

Agent passes basic tests? Now test edge cases.

Edge Case Categories

Stress Testing

Test edge cases: Large file (5000 lines, 20 issues), Complex nesting (15-level deep). Verify all issues found with reasonable response time.

Ambiguity Testing

Test context-dependent cases (e.g., hardcoded password with "local dev" comment). Agent should flag but acknowledge context.

Plugging Holes

For each edge case failure, add explicit handling to agent definition:

• Empty files: Return "No code to review" with PASS
• Large files: Focus on high-risk patterns first
• Minified code: Note limitations
• Context comments: Consider but don't use to dismiss issues

Testing Parallel Agent Workflows

When agents run in parallel (like 5 reviewers), test combined workflow:

• Parallel Consistency: Same input to all reviewers → check findings overlap appropriately, no contradictions
• Aggregation Testing: Same issue found by multiple reviewers → severity should be consistent; fix misalignments

Agent Testing Checklist

RED Phase: Create test inputs (known issues, clean, edge cases) → Run agent → Document failures verbatim

GREEN Phase: Update agent definition → Re-run tests → All pass

REFACTOR Phase: Test edge cases → Test stress scenarios → Add explicit handling → Verify consistency (3+ runs) → Test parallel integration (if applicable) → Re-run ALL tests after each change

Metrics (reviewer agents): True positive >95%, False positive <10%, False negative <5%, Severity accuracy >90%, Schema compliance 100%, Consistency >95%

Prohibited Testing Shortcuts

You CANNOT substitute proper testing with:

ALL require running agent with documented test inputs and comparing outputs.

Testing Agent Modifications

EVERY agent edit requires re-running the FULL test suite:

"Small change" is not an exception. One-line prompt changes can completely alter LLM behavior. Re-test always.

Common Mistakes

Rationalization Table

Red Flags - STOP and Test Now

If you catch yourself thinking ANY of these, STOP. You're about to violate the Iron Law:

• Agent edited but tests not re-run
• "Looks good" without execution
• Single test case only
• No documented baseline
• No edge case testing
• Manual verification only
• "Will test in production"
• "Based on template, should work"
• "Just a small prompt change"
• "No time to test properly"
• "One quick test is enough"
• "Agent is simple, obviously works"
• "Expert intuition says it's fine"
• "Production is down, skip testing"
• "Deploy now, test in parallel"

All of these mean: STOP. Run full RED-GREEN-REFACTOR cycle NOW.

Quick Reference (TDD Cycle for Agents)

Example: Testing a New Reviewer Agent

Step 1: Create Test Suite

Step 2: RED Phase - Run tests, document failures: Test 1 marked HIGH not CRITICAL, Test 3 missed, Test 5 errored, Test 6 dismissed.

Step 3: GREEN Phase - Fix definition: Add severity calibration (SQL=CRITICAL), hardcoded secrets pattern, empty file handling, "context comments dont dismiss issues".

Step 4: Re-run - All tests pass with correct severities and handling.

Step 5: REFACTOR - Add edge cases: minified code, 10K line file, mixed languages, nested vulnerabilities. Run, fix, repeat.

The Bottom Line

Agent testing IS TDD. Same principles, same cycle, same benefits.

If you wouldn't deploy code without tests, don't deploy agents without testing them.

RED-GREEN-REFACTOR for agents works exactly like RED-GREEN-REFACTOR for code:

1. RED: See what's wrong (run with test inputs)
2. GREEN: Fix it (update agent definition)
3. REFACTOR: Make it robust (edge cases, consistency)

Evidence before deployment. Always.

Blocker Criteria

STOP and report if:

Cannot Be Overridden

The following requirements CANNOT be waived:

• MUST run agent with documented test inputs before deployment
• MUST complete minimum test suite (4-6 cases per agent type)
• MUST re-run FULL test suite after ANY agent modification
• MUST NOT substitute manual UI testing for documented test runs
• MUST NOT deploy agent that fails accuracy metrics

Severity Calibration

Pressure Resistance

Anti-Rationalization Table