qa-plan（质量保证计划）

QA Plan

Produce the most ambitious, user-outcome-grounded QA plan possible. Take spec.json, the codebase, and the diff — produce a qa-progress.json that captures every scenario a thorough human QA engineer would want to verify, directly consumable by /qa.

This skill bridges spec-derived scenarios (what the spec *says* to test) and code-derived scenarios (what the code *actually needs* tested). It does not execute tests — it produces the plan that /qa executes. It has no execution boundaries. /qa-plan plans everything worth testing. /qa handles execution feasibility — anything that can't be verified locally gets flagged as pending human verification, never silently dropped.

User Story Fidelity Principle

Every scenario must verify what the user actually experiences, not what the code does.

• "The API returns 200" is not a scenario. "The user sees their dashboard load with real data" is.
• "The test suite passes" is not coverage. A test with mocked providers verifies code logic, not user outcomes. If the only test coverage uses mocks, stubs, or fake implementations for the service boundary — the scenario is NOT covered and must remain in the plan.
• "The webhook handler processes the payload" is not a scenario. "The user's payment status updates in real-time after Stripe confirms" is.

When writing scenarios, start from the user's perspective: what do they see, what do they experience, what outcome do they expect? Then trace backward to what needs to be verified to prove that outcome is real.

Inputs (any one is sufficient):

Output: tmp/ship/qa-progress.json — scenarios derived from SPEC.md source material and code investigation, enriched with code-grounded fields, prioritized, and directly consumable by /qa.

Autonomy

This skill supports the cross-skill autonomy convention:

Headless mode adjustments:

• Contradictions between spec and code: documented as contradiction gap type, not escalated
• Ambiguous oracle types: default to human with a note explaining why
• Scope-boundary scenarios: include with a grayArea flag rather than asking

Create workflow tasks (first action)

Before starting any work, create a task for each step using TaskCreate with addBlockedBy to enforce ordering. Derive descriptions and completion criteria from each step's own workflow text.

1. QA-Plan: Detect repo type and gather inputs
2. QA-Plan: Enrich with /worldmodel — map surfaces, personas, and silent impacts
3. QA-Plan: Derive from spec source material
4. QA-Plan: Synthesize scenarios — atomic and compositional
5. QA-Plan: Trace spec to code
6. QA-Plan: Trace code to spec
7. QA-Plan: Code investigation
8. QA-Plan: Coverage reality check
9. QA-Plan: Order scenarios
10. QA-Plan: Write qa-progress.json

Mark each task in_progress when starting and completed when its step's exit criteria are met. On re-entry, check TaskList first and resume from the first non-completed task.

Workflow

Step 0: Detect repo type

Determine what kind of project this is. The repo type drives which QA categories are relevant and what tools are expected during execution.

Detection signals:

Category emphasis by repo type:

Record the detected repo type. This table calibrates where to look hardest for scenarios. Generate scenarios for every category where a real user outcome exists — the table does not grant permission to skip categories. A CLI tool can still have usability scenarios. An SDK can still have UX-flow scenarios for the developer experience.

Step 1: Enrich with /worldmodel

Load /worldmodel skill with the feature topic + SPEC path and/or PR number as user-provided sources. This maps the feature's blast radius before you start writing scenarios — what surfaces does it touch, who is affected, and what might break silently.

Read worldmodel's output. Extract these for use in Steps 2–5:

• Surfaces (product + internal) — transforms "what files changed" into "what surfaces are affected." Each surface is a candidate for QA scenarios.
• Personas & Audiences — who is affected and how fast the change reaches them. Prioritize scenarios for personas with the widest blast radius.
• Current State — how the system works today (code-verified). Grounds your given clauses in reality.
• Patterns Observed — cross-channel divergences that may surface testing angles the SPEC/PR didn't mention.

Pay special attention to silent impacts — changes that affect users without producing obvious errors or visible behavior changes (e.g., a caching change that subtly alters data freshness, a telemetry contract change that breaks downstream dashboards). Worldmodel's Personas section classifies propagation timing; look for items marked "silent." These need explicit QA scenarios because they won't produce obvious failures.

Output of this step: A topology map of surfaces, personas, and silent impacts. Feed this into Steps 2–5 — each surface without a covering scenario is a gap to fill.

Step 2: Derive from spec source material

Read SPEC.md directly for QA-relevant source material. Read spec.json for story structure. These are inputs to your thinking — not a template for your output.

Ignore spec.json's qaScenarios[]. Those scenarios are implementation context for iteration agents — they help implementers understand what will be verified. They are mechanical 1:1 mappings from spec sections and do not represent the scope or ambition of your plan. Derive your scenarios from SPEC.md source material directly. Do not seed from, pin to, or limit your plan based on spec.json's qaScenarios.

1. Locate SPEC.md (from the provided path, or infer from spec.json's context)
2. If SPEC.md exists, read these sections:

- §5 User journeys — happy paths, failure & recovery paths, ongoing usage per persona - §5 Interaction state matrix — feature states AND transitions between states - §6 Acceptance criteria — each criterion is a testable condition - §9 Data flow shadow paths — nil, empty, wrong type, timeout, conflict, partial failure - §9 Failure modes table — individual failures, detection, recovery, user impact - §9 Affected routes/pages — routes and what to verify - §13 Deployment/rollout considerations

1. Read spec.json for story structure — which stories exist, their acceptance criteria, their ordering
2. If neither exists: "No spec-derived source material — plan will be code-driven only"

Output: Understanding of what the spec intends. Scenarios come in Step 2.5.

Step 2.5: Synthesize scenarios — atomic and compositional

From the spec source material (Step 2) and repo type (Step 0), synthesize scenarios at two levels:

Atomic conditions — individual testable checks derived from spec sections. One per acceptance criterion, failure mode row, shadow path, or state matrix cell. These are the building blocks.

Compositional user journeys — realistic multi-step user sessions that cross multiple stories, surfaces, and states. For each primary persona:

• What does the first 10 minutes of using this feature look like end-to-end?
• What multi-step workflows cross multiple stories? (e.g., filter → change status → verify filter still holds)
• What happens when action A succeeds, then action B fails — does recovery from B break A's state?
• What state transitions happen in real usage that aren't in the interaction state matrix?
• What happens when the user navigates away mid-operation and returns?
• What happens when two failures occur in sequence?

Compositional scenarios test the integration seams that atomic checks miss. Place them early in the scenario array.

Do not limit scenario count to the number of spec rows. The spec sections are source material, not a cardinality constraint. A spec with 3 acceptance criteria and 2 failure modes may produce 5 atomic scenarios and 4 compositional journeys.

Set source: "spec" on scenarios derived from SPEC.md content. Set source: "journey" on compositional scenarios that combine multiple spec elements into realistic user paths.

Output: Initial scenario list ready for code-grounding in Steps 3–5.

Step 3: Trace spec to code

For each spec-derived scenario, verify it is grounded in the actual codebase. This catches stale specs, renamed routes, and missing implementations.

Route verification (for scenarios with a route field):

• Glob/grep for the route in the codebase
• If found: record the file path in the scenario's enrichment
• If not found: mark as a gap (see gap types below)

Component verification (for UI scenarios):

• Search for components, pages, or views referenced by the scenario
• Verify they exist and are wired into the routing/rendering tree

Affordance check (for behavior scenarios):

• Search for the function, handler, or endpoint that implements the behavior described in the when clause
• Verify the expected behavior from the then clause is plausible given the code

Gap types:

Step 4: Trace code to spec

Analyze the diff to find code changes that are NOT covered by any existing scenario. This is the reverse of Step 3 — catching things the spec author missed.

Diff analysis:

1. Get the diff: git diff main...HEAD --stat for file list, git diff main...HEAD for full changes
2. If a PR number was provided: gh pr diff <number>

Map changed files to surfaces:

• Changed component → visual / ux-flow scenario needed?
• Changed API handler → error-state / edge-case scenario needed?
• Changed data model → integration / failure-mode scenario needed?
• Changed config / env → integration scenario needed?

Create scenarios for uncovered changes: For each changed surface without a covering scenario:

1. Create a new scenario with the next sequential QA-NNN id
2. Set source: "code" to distinguish from spec-derived scenarios
3. Write userOutcome first — what does the end user actually experience? Start from their perspective.
4. Write given/when/then based on the code's behavior, grounded in the user outcome
5. Synthesize verifies from the when/then clauses (what the test checks)
6. Set tracesTo to the most relevant user story ID as a string (e.g., "US-003"), or omit when the mapping is unclear
7. Set oracleType based on whether the expected behavior is deterministic (specified), relative (derived), or subjective (human)

Step 5: Code investigation

Go beyond the diff to discover testing concerns from code patterns and health signals.

Framework detection:

• Identify the framework and its testing conventions (e.g., Next.js → check for server components, API routes, middleware)
• Note framework-specific gotchas that affect QA (e.g., hydration mismatches, SSR vs CSR differences)

State management patterns:

• Identify how state is managed (Redux, Zustand, React context, server state, URL params)
• Each state management boundary is a potential source of sync bugs — flag untested transitions

Console / runtime signals:

• Check for console.error, console.warn patterns in changed code
• Check for error boundary components, try/catch blocks, error handlers
• Each is a potential failure scenario if not already covered

Code health hotspots:

• Files with high churn (many recent changes) touching the same feature
• Complex conditional logic (deeply nested if/else, switch statements)
• Type assertions or any types in the changed code
• TODO/FIXME/HACK comments in the changed code

Add new scenarios for uncovered concerns. Use the category emphasis from Step 0 for prioritization, but do not filter scenarios out.

Step 6: Coverage reality check

For each scenario, determine whether it is already verified by a real (non-mocked) test — not whether it *could* be a formal test. The question is not "could someone write a test for this?" but "does a test already exist that proves the actual user outcome with real dependencies?"

Mock detection (critical): Search for existing tests covering each scenario's behavior. For each test found, check whether it mocks the service boundary:

• Uses jest.mock(), vi.mock(), or manual mock files for the module under test
• Uses MSW, nock, VCR cassettes, or similar to intercept HTTP
• Uses stub/fake implementations of databases, queues, or external services
• Uses test doubles that replace the real dependency behavior

Coverage classification:

The default is: the scenario stays in the plan. Only mark a scenario formalizable and exclude it from QA execution when a real, non-mocked test already verifies the complete user outcome end-to-end.

Supervised mode: If many scenarios have only mocked test coverage, note this — it's a signal the test suite may need real integration tests. But this does not reduce the QA plan.

Headless mode: Document coverage findings in planMetadata.formalizableCount and proceed.

Step 7: Order scenarios

Arrange the scenario array so /qa executes the most critical paths first. Array position determines execution order — there are no priority tiers or importance labels. Every scenario is mandatory and receives equal execution effort.

Place first (critical path — failures here break the feature):

• Happy-path scenarios for core user journeys
• Compositional user journey scenarios (source: "journey")
• Error scenarios for data-loss or security-sensitive paths

Place next (important surfaces):

• Error states and failure modes for non-critical paths
• Edge cases for common user inputs
• Visual correctness for primary surfaces
• Integration scenarios with external dependencies

Place last (lower-traffic surfaces):

• Edge cases for uncommon inputs
• Cross-system scenarios for low-traffic surfaces
• Performance-related observations

This ordering is for execution sequencing only. It does not signal that later scenarios matter less or deserve less effort. /qa executes every scenario in the array and derives verdict severity from scenario properties (category, source), not from array position.

Step 8: Write qa-progress.json

Write the final enriched plan to tmp/ship/qa-progress.json. Create tmp/ship/ if it doesn't exist (mkdir -p tmp/ship).

qa-progress.json schema:

{
  "specPath": "specs/feature-name/SPEC.md",
  "prNumber": null,
  "planMetadata": {
    "repoType": "full-stack-web-app",
    "generatedAt": "2026-03-24T12:00:00Z",
    "totalScenarios": 12,
    "byCategory": { "ux-flow": 3, "error-state": 3, "edge-case": 2, "integration": 2, "visual": 1, "cross-system": 1 },
    "bySource": { "spec": 5, "journey": 3, "code": 4 },
    "formalizableCount": 2,
    "mockedOnlyCoverageCount": 3
  },
  "scenarios": [
    {
      "id": "QA-001",
      "category": "ux-flow",
      "name": "User can toggle task status from list",
      "userOutcome": "User sees the task status change to 'in_progress' instantly in the UI without needing to refresh the page",
      "verifies": "changing status to 'in_progress' via dropdown updates immediately without page refresh",
      "given": "A task exists with status 'pending' on the task list",
      "when": "User changes status to 'in_progress' via the dropdown",
      "then": "Status updates immediately, UI reflects the change without page refresh",
      "tracesTo": "US-003",
      "oracleType": "specified",
      "route": "/tasks",
      "source": "spec",
      "status": "planned",
      "verifiedVia": null,
      "notes": "",
      "evidence": [],  // populated by /qa: [{type: "assertion", check: "...", pass: true}, {type: "video", url: "..."}]
      "enrichment": {
        "codeLocation": "app/tasks/page.tsx:42",
        "gapType": null,
        "formalizable": false
      }
    }
  ]
}

Field definitions — top-level (compatible with /qa):

Field definitions — planMetadata (enrichment by /qa-plan):

Field definitions — scenario (compatible with /qa's required fields):

Field definitions — scenario enrichment (new fields beyond /qa's schema):

Skill scope boundaries

/qa-plan is the planner. /qa is the executor. The planner has no execution boundaries — it plans everything worth testing.

This skill does:

• Read spec.json, code, and diffs
• Create, enrich, and prioritize QA scenarios for every user outcome
• Write qa-progress.json
• Flag contradictions between spec and code
• Identify scenarios with only mocked test coverage

This skill does NOT:

• Execute any test scenario
• Start the application or dev server
• Modify application code
• Filter scenarios based on execution feasibility — that's /qa's job

Execution feasibility is /qa's problem, not /qa-plan's. If a scenario requires a real Stripe webhook, plan it. If a scenario requires browser automation, plan it. If a scenario requires Docker containers, plan it. /qa will attempt to execute everything locally (including spinning up Docker, using browser automation, writing scripts). Anything /qa genuinely cannot verify locally gets flagged as "blocked" with notes describing what was attempted and what a human needs to check — this flows to /pr as pending human verification. No scenario is silently dropped from the plan because it *might* be hard to execute.

Scenarios requiring browser automation, a running dev server, or Playwright are NOT a hard-to-execute class — they are the primary use case for /qa + /browser (/browser is always loadable; its Playwright engine runs headless Chromium by default). Never annotate a scenario with enrichment.gapType or notes suggesting it is likely-to-be-blocked for those reasons. /qa will load /browser and bootstrap the dev server as part of its normal workflow.

Edge case rules:

• If a spec scenario is clearly wrong (references a deleted feature), mark it stale but do not delete it — the spec author may need to update the spec
• If code introduces a security-sensitive change not in the spec, create a scenario and place it early in the array — safety overrides emphasis guidance
• If the diff is empty (no changes from main), produce a minimal plan with only spec-derived scenarios and note "no code changes detected"