slop-test-detector（斜率测试探测器）

Slop Test Detector

Agents generate tests that compile, pass, and increase coverage — but catch zero bugs. This skill detects those patterns programmatically.

When to use: Auditing existing test files for quality. Validating generated tests before writing them to disk. Enforcing the // Defect: comment convention.

When not to use: Evaluating test *strategy* (what to test). Measuring code coverage. Reviewing production code quality.

Rationalizations (Do Not Skip)

What To Protect (Start Here)

Before auditing or generating tests, identify which slop patterns apply:

Assertion API Support

The detector recognizes node:assert, vitest/Jest expect(), and chai expect() patterns:

Chain modifiers .not, .resolves, .rejects and chai language chains (.to, .be, .have, .been, etc.) are handled. Multiline chains (matcher on next line) are supported. Commented-out expect() calls are detected. Chai property assertions (expect(x).to.be.true) are supported.

Included Utilities

import {
  analyzeTestFile,
  validateTestBlock,
  formatReport,
  formatReportJSON,
  getPreset,
  parseImports,
  parseTestFile,
} from './slop-detector.ts';

Configuration

Presets

Three built-in presets control which rules are active:

import { getPreset } from './slop-detector.ts';

const config = getPreset('balanced'); // default
const strict = getPreset('strict');   // all rules enforced

Custom Configuration

import { type SlopConfig, getPreset } from './slop-detector.ts';

const config: SlopConfig = {
  ...getPreset('balanced'),
  assertionEquivalents: ['assertLogEntry', 'assertNoLogsAbove'],
  scoreThreshold: 90,
};

• enabledRules: Set<SlopRule> — which rules produce findings
• assertionEquivalents: string[] — function names treated as assertions (prevents false empty_test_body)
• scoreThreshold: number — minimum passing score (for CI gating)

Suppression Comments

Suppress specific findings with a required reason:

// slop-ignore: tautological_assertion — intentional canary test for CI pipeline health
it("canary", () => {
  assert.ok(true);
});

• The — reason is required — suppressions without a reason are ignored
• Multiple rules: // slop-ignore: empty_test_body, truthiness_only — stub pending #1234
• Place on the line before it() or anywhere inside the test body

Core Workflow

Step 1: Audit Existing Files

import { readFileSync } from 'node:fs';

const source = readFileSync('my-module.spec.ts', 'utf-8');
const report = analyzeTestFile(source, 'my-module.spec.ts');
console.log(formatReport(report));

Step 2: Report Findings with Impact

When presenting findings, always include:

• Rule name — the specific slop pattern (e.g., tautological_assertion)
• Test name and line number — so the user can navigate to it
• Severity — must-fail (test is structurally broken) vs should-fail (quality concern)
• Why it matters — what production bug could slip through because of this pattern
• What clean tests should NOT be flagged — avoid false positives on well-written tests

If a test file has assertion-equivalent helpers (functions named assert*() or test*() that internally call assert.*), configure assertionEquivalents to prevent false empty_test_body findings.

Step 3: Validate During Generation

Before writing a generated test to disk, check for slop:

const findings = validateTestBlock(`
  // Defect: if the parser miscounts blocks then all downstream rules produce wrong results
  it("parses correctly", () => {
    assert.equal(result, expected);
  });
`);

if (findings.some(f => f.severity === 'must-fail')) {
  // Fix before writing
}

Step 4: Machine-Readable Output (CI)

const report = analyzeTestFile(source, filePath, getPreset('balanced'));
const json = formatReportJSON(report);
// Outputs structured JSON with filePath, score, summary, findings[]

Step 5: Interpret the Score

The score formula weights must-fail findings (1.0) higher than should-fail (0.3):

score = max(0, round(100 × (1 - weightedFindings / testCount)))

• 100: No slop detected
• 90-99: Minor should-fail findings (missing comments, no negative tests)
• Below 80: Significant quality issues — review must-fail findings first

Violation Rules

opt-in rules are only active in the strict preset. Use getPreset('strict') or add them to a custom enabledRules set.

Definition of Done

• analyzeTestFile() returns zero must-fail findings
• All should-fail findings are reviewed and either fixed or justified
• Score is 90 or above
• slop-detector.spec.ts itself passes all its own rules

Companion Skills

• observability-testing — Structured log assertions that the slop detector validates for completeness
• fault-injection-testing — Circuit breaker and retry tests where empty_test_body and no_negative_test rules are most likely to fire
• breaking-change-detector — Contract classification tests where duplicate_assertion_set detects copy-paste patterns
• model-based-testing — State machine transition matrix tests where no_input_variation catches redundant transition checks
• pairwise-test-coverage — Combinatorial test generation where truthiness_only and assert_return_type_only are common slop patterns

See patterns.md for all 12 patterns with before/after code examples.