QA API Testing and Contracts
Use this skill to turn an API schema into enforceable checks (lint, diff, contracts, and negative/security cases) and wire them into CI so breaking changes cannot ship silently.
Ask For Inputs
- API type and canonical schema artifact (OpenAPI 3.1, SDL, proto) and where it lives in-repo.
- Environments, auth method(s), and how to provision stable test identities/keys.
- Critical endpoints/operations and business flows (rank by risk and revenue impact).
- Data constraints (idempotency keys, pagination, ordering), rate limits, and error format (prefer RFC 7807
application/problem+jsonfor REST). - Versioning + deprecation policy, consumer inventory, and release cadence.
- Current test tooling/CI and what “blocking” means for your org.
Outputs (What to Produce)
- A minimal CI gate set (lint + breaking diff + contract suite) wired to PRs.
- A coverage map derived from the schema (critical operations first).
- A negative/security baseline aligned to OWASP API risks.
Quick Start
- Lint the schema (syntax + best-practice rules) and fix issues before writing tests.
- Add breaking-change checks against the base branch on every PR.
- Pick a contract strategy (CDC, schema-driven, or both) and run it in CI against an ephemeral environment.
- Add negative/security cases for auth, validation, and error handling.
- Make gates explicit (what blocks merge/release) and publish results.
Workflow
1) Establish Contract Artifacts (Source of Truth)
- REST: single OpenAPI 3.1 file or a compiled artifact; avoid drift across fragments.
- GraphQL: checked-in SDL (and federation/composition config if relevant).
- gRPC: checked-in
.proto+buf.yaml(or equivalent) with a stable module layout.
2) Validate the Schema (Fast, Deterministic)
- Run spec linting (Spectral / GraphQL Inspector / buf lint).
- Enforce a small, explicit ruleset (naming, descriptions, auth annotations, consistent error model).
3) Detect Breaking Changes (PR Gate)
- REST: OpenAPI diff with a breaking-change policy (remove/rename/type change/requiredness).
- GraphQL: schema diff with breaking checks (field removals, type changes, non-null tightening).
- gRPC:
buf breaking(do not reuse/renumber fields; avoid changing request/response shapes incompatibly).
4) Execute Contract Tests (CI Gate)
Choose one or combine:
- CDC (Pact): best when many independent consumers exist and behavior matters beyond schema.
- Schema-driven (Specmatic): best when schema is the contract and you want fast coverage across operations.
- Property-based (Schemathesis): best when you want systematic edge cases and server hardening.
5) Add Negative + Security Cases (Minimum Set)
- AuthN/AuthZ: missing/expired token (401), insufficient scope/role (403), tenant isolation.
- Validation: missing required fields, invalid types, boundary values, empty strings, large payloads.
- Error handling: stable error shape, safe messages, correct status codes, correlation IDs.
- Abuse & limits: rate limiting (429), pagination limits, idempotency replay, retry-safe semantics.
- For GraphQL, also validate operations checks (known/persisted queries) if you have an operation registry (GraphOS/Hive/etc.).
6) Define CI Quality Gates (Merge + Release)
- Pre-merge: schema lint + breaking-change diff (blocking).
- Pre-release: contract suite (blocking), plus smoke/functional tests for critical flows.
- Reporting: publish artifacts (diff report, contract verification, failing cases) and link in PR.
Quality Checks
- Fail fast: schema violations and breaking changes block merge.
- Determinism: isolate data, freeze time where needed, avoid shared mutable fixtures.
- Flake hygiene: separate network instability from contract failures; retry only for known-transient classes.
- Alignment: contracts reflect versioning/deprecation policy and consumer inventory.
- Scope control: keep load/resilience tests separate unless explicitly requested.
Use the Bundled Templates
- Coverage plan:
assets/api-test-plan.md - Release review:
assets/contract-change-checklist.md - Tooling map:
assets/schema-validation-matrix.md
AI Assistance (Use Carefully)
- Use AI to draft tests, suggest missing edge cases, and tighten matchers.
- Treat AI output as untrusted until verified against the schema and real behavior.
- Avoid uploading sensitive payloads; sanitize examples and logs.
- For a tool comparison and workflows, read
references/ai-contract-testing.md.
Read These When Needed
- Change safety and CDC patterns:
references/contract-testing-patterns.md - AI-assisted tooling and decision matrix:
references/ai-contract-testing.md - API versioning and backward compatibility:
references/api-versioning-strategies.md - Schema-driven and property-based testing:
references/schema-driven-testing.md - OWASP API security testing:
references/api-security-testing.md - Curated authoritative links:
data/sources.json
Related Skills
- Use dev-api-design for API design decisions.
- Use qa-testing-strategy for overall testing strategy.
- Use qa-resilience for chaos and reliability testing.
- Use software-security-appsec for API security review.
Fact-Checking
- Use web search/web fetch to verify current external facts, versions, pricing, deadlines, regulations, or platform behavior before final answers.
- Prefer primary sources; report source links and dates for volatile information.
- If web access is unavailable, state the limitation and mark guidance as unverified.