Evaluation Criteria
A technology-agnostic skill for defining how deliverables should be evaluated.
Core Purpose
Establish consistent, objective criteria for:
- Assessing deliverable quality
- Scoring against requirements
- Making pass/fail determinations
- Providing constructive feedback
Evaluation Dimensions
Completeness
dimension: completeness
question: "Is everything required present?"
criteria:
functionality:
- "All specified functions implemented"
- "All user stories addressed"
- "All acceptance criteria met"
coverage:
- "All edge cases handled"
- "All error paths covered"
- "All integrations connected"
documentation:
- "Required documentation present"
- "API documentation complete"
- "Comments where needed"
scoring:
pass: "100% of must-have items present"
fail: "Any must-have item missing"Correctness
dimension: correctness
question: "Does it work correctly?"
criteria:
functional:
- "Business logic accurate"
- "Calculations correct"
- "Data transformations valid"
behavioral:
- "Expected outputs produced"
- "Error handling appropriate"
- "Edge cases handled correctly"
integration:
- "External systems connected"
- "Data flows correctly"
- "APIs respond correctly"
scoring:
pass: "All tests pass, no logic errors"
fail: "Any test failure or logic error"Quality
dimension: quality
question: "Is it well-built?"
criteria:
code_standards:
- "Follows project conventions"
- "No linting errors"
- "Consistent formatting"
architecture:
- "Follows design patterns"
- "Proper separation of concerns"
- "Dependencies managed correctly"
maintainability:
- "Code is readable"
- "Complexity manageable"
- "No code smells"
scoring:
pass: "No critical issues, minimal major issues"
fail: "Critical issues present"Security
dimension: security
question: "Is it secure?"
criteria:
vulnerabilities:
- "No SQL injection"
- "No XSS vulnerabilities"
- "No CSRF vulnerabilities"
authentication:
- "Proper auth checks"
- "Secure password handling"
- "Session management correct"
data_protection:
- "Sensitive data encrypted"
- "PII handled correctly"
- "Audit logging present"
scoring:
pass: "No security vulnerabilities found"
fail: "Any security vulnerability present"Performance
dimension: performance
question: "Does it perform well?"
criteria:
response_time:
- "API response < 200ms (95th percentile)"
- "Page load < 3 seconds"
resource_usage:
- "Memory usage reasonable"
- "CPU usage efficient"
- "No memory leaks"
scalability:
- "Handles expected load"
- "No N+1 queries"
- "Caching implemented where needed"
scoring:
pass: "Meets performance thresholds"
fail: "Below performance thresholds"Scoring System
Severity Classification
severity_levels:
critical:
definition: "Must fix before acceptance"
examples:
- "Security vulnerability"
- "Data corruption risk"
- "Core functionality broken"
impact: "Automatic FAIL"
major:
definition: "Should fix before acceptance"
examples:
- "Logic errors"
- "Missing tests"
- "Performance issues"
impact: "Accumulate to threshold"
minor:
definition: "Can fix later"
examples:
- "Style improvements"
- "Documentation gaps"
- "Minor optimizations"
impact: "Noted but not blocking"Pass/Fail Determination
pass_criteria:
all_of:
- "No critical issues"
- "Critical dimensions pass (completeness, correctness, security)"
threshold:
- "Major issues <= 3"
- "Overall score >= 80%"
fail_criteria:
any_of:
- "Any critical issue present"
- "Critical dimension fails"
- "Major issues > 5"
- "Overall score < 60%"Output Format
evaluation:
overall_verdict: PASS|FAIL
overall_score: <0-100>
dimensions:
completeness:
score: <0-100>
status: PASS|FAIL
issues: [...]
correctness:
score: <0-100>
status: PASS|FAIL
issues: [...]
quality:
score: <0-100>
status: PASS|FAIL
issues: [...]
security:
score: <0-100>
status: PASS|FAIL
issues: [...]
performance:
score: <0-100>
status: PASS|FAIL
issues: [...]
issue_summary:
critical: <count>
major: <count>
minor: <count>
feedback:
- "<actionable feedback item>"Integration
Used By Agents
primary_users:
- deliverable-evaluator: "Core evaluation methodology"
- quality-reviewer: "Quality assessment"