prowler-compliance（徘徊者合规性）

When to Use

Use this skill when:

• Creating a new compliance framework for any provider
• Syncing an existing framework with an upstream source of truth (CIS, FINOS CCC, CSA CCM, NIST, ENS, etc.)
• Adding requirements to existing frameworks
• Mapping checks to compliance controls
• Auditing existing check mappings as a cloud auditor (user asks "are these mappings correct?", "which checks apply to this requirement?", "review the mappings")
• Adding a new output formatter (new framework needs a table dispatcher + per-provider classes + CSV models)
• Fixing JSON bugs: duplicate IDs, empty Version, wrong Section, stale check refs, inconsistent FamilyName, padded tangential check mappings
• Registering a framework in the CLI table dispatcher or API export map
• Investigating why a finding/check isn't showing under the expected compliance framework in the UI
• Understanding compliance framework structures and attributes

Four-Layer Architecture (Mental Model)

Prowler compliance is a four-layer system hanging off one Pydantic model tree. Bugs usually happen where one layer doesn't match another, so know all four before touching anything.

Layer 1: SDK / Core Models — prowler/lib/check/

• compliance_models.py — Pydantic v1 model tree (from pydantic.v1 import). One *_Requirement_Attribute class per framework type + Generic_Compliance_Requirement_Attribute as fallback.
• Compliance_Requirement.Attributes: list[Union[...]] — Generic_Compliance_Requirement_Attribute MUST be LAST in the Union or every framework-specific attribute falls through to Generic (Pydantic v1 tries union members in order).
• compliance.py — runtime linker. get_check_compliance() builds the key as f"{Framework}-{Version}" only if Version is non-empty. An empty Version makes the key just "{Framework}" — this breaks downstream filters and tests that expect the versioned key.
• Compliance.get_bulk(provider) walks prowler/compliance/{provider}/ and parses every .json file. No central index — just directory scan.

Layer 2: JSON Frameworks — prowler/compliance/{provider}/

See "Compliance Framework Location" and "Framework-Specific Attribute Structures" sections below.

Layer 3: Output Formatters — prowler/lib/outputs/compliance/{framework}/

Every framework directory follows this exact convention — do not deviate:

{framework}/
├── __init__.py
├── {framework}.py            # ONLY get_{framework}_table() — NO function docstring
├── {framework}_{provider}.py # One class per provider (e.g., CCC_AWS, CCC_Azure, CCC_GCP)
└── models.py                 # One Pydantic v2 BaseModel per provider (CSV columns)

• {framework}.py holds the table dispatcher function get_{framework}_table(). It prints the pass/fail/muted summary table. Must NOT import Finding or ComplianceOutput — doing so creates a circular import with prowler/lib/outputs/compliance/compliance.py. Only imports: colorama, tabulate, prowler.config.config.orange_color.
• {framework}_{provider}.py holds a per-provider class like CCC_AWS(ComplianceOutput) with a transform() method that walks findings and emits rows. This file IS allowed to import Finding because it's not on the dispatcher import chain.
• models.py holds one Pydantic v2 BaseModel per provider. Field names become CSV column headers (public API — renaming breaks downstream consumers).
• Never collapse per-provider files into a unified parameterized class, even when DRY-tempting. Every framework in Prowler follows the per-provider file pattern and reviewers will reject the refactor. CSV columns differ per provider (AccountId/Region vs SubscriptionId/Location vs ProjectId/Location) — three classes is the convention.
• No function docstring on get_{framework}_table() — no other framework has one; stay consistent.
• Register in prowler/lib/outputs/compliance/compliance.py → display_compliance_table() with an elif compliance_framework.startswith("{framework}_"): branch. Import the table function at the top of the file.

Layer 4: API / UI

• API table dispatcher: api/src/backend/tasks/jobs/export.py → COMPLIANCE_CLASS_MAP keyed by provider. Uses startswith predicates: (lambda name: name.startswith("ccc_"), CCC_AWS). Never use exact match (name == "ccc_aws") — it's inconsistent and breaks versioning.
• API lazy loader: api/src/backend/api/compliance.py — LazyComplianceTemplate and LazyChecksMapping load compliance per provider on first access.
• UI mapper routing: ui/lib/compliance/compliance-mapper.ts routes framework names → per-framework mapper.
• UI per-framework mapper: ui/lib/compliance/{framework}.tsx flattens Requirements into a 3-level tree (Framework → Category → Control → Requirement) for the accordion view. Groups by Attributes[0].FamilyName and Attributes[0].Section.
• UI detail panel: ui/components/compliance/compliance-custom-details/{framework}-details.tsx.
• UI types: ui/types/compliance.ts — TypeScript mirrors of the attribute metadata.

The CLI Pipeline (end-to-end)

prowler aws --compliance ccc_aws
    ↓
Compliance.get_bulk("aws")  → parses prowler/compliance/aws/*.json
    ↓
update_checks_metadata_with_compliance()  → attaches compliance info to CheckMetadata
    ↓
execute_checks()  → runs checks, produces Finding objects
    ↓
get_check_compliance(finding, "aws", bulk_checks_metadata)
    → dict "{Framework}-{Version}" → [requirement_ids]
    ↓
CCC_AWS(findings, compliance).transform()  → per-provider class builds CSV rows
    ↓
batch_write_data_to_file()  → writes {output_filename}_ccc_aws.csv
    ↓
display_compliance_table() → get_ccc_table() → prints stdout summary

Compliance Framework Location

Frameworks are JSON files located in: prowler/compliance/{provider}/{framework_name}_{provider}.json

Supported Providers:

• aws - Amazon Web Services
• azure - Microsoft Azure
• gcp - Google Cloud Platform
• kubernetes - Kubernetes
• github - GitHub
• m365 - Microsoft 365
• alibabacloud - Alibaba Cloud
• cloudflare - Cloudflare
• oraclecloud - Oracle Cloud
• oci - Oracle Cloud Infrastructure
• nhn - NHN Cloud
• mongodbatlas - MongoDB Atlas
• iac - Infrastructure as Code
• llm - Large Language Models

Base Framework Structure

All compliance frameworks share this base structure:

{
  "Framework": "FRAMEWORK_NAME",
  "Name": "Full Framework Name with Version",
  "Version": "X.X",
  "Provider": "PROVIDER",
  "Description": "Framework description...",
  "Requirements": [
    {
      "Id": "requirement_id",
      "Description": "Requirement description",
      "Name": "Optional requirement name",
      "Attributes": [...],
      "Checks": ["check_name_1", "check_name_2"]
    }
  ]
}

Framework-Specific Attribute Structures

Each framework type has its own attribute model. Below are the exact structures used by Prowler:

CIS (Center for Internet Security)

Framework ID format: cis_{version}_{provider} (e.g., cis_5.0_aws)

{
  "Id": "1.1",
  "Description": "Maintain current contact details",
  "Checks": ["account_maintain_current_contact_details"],
  "Attributes": [
    {
      "Section": "1 Identity and Access Management",
      "SubSection": "Optional subsection",
      "Profile": "Level 1",
      "AssessmentStatus": "Automated",
      "Description": "Detailed attribute description",
      "RationaleStatement": "Why this control matters",
      "ImpactStatement": "Impact of implementing this control",
      "RemediationProcedure": "Steps to fix the issue",
      "AuditProcedure": "Steps to verify compliance",
      "AdditionalInformation": "Extra notes",
      "DefaultValue": "Default configuration value",
      "References": "https://docs.example.com/reference"
    }
  ]
}

Profile values: Level 1, Level 2, E3 Level 1, E3 Level 2, E5 Level 1, E5 Level 2 AssessmentStatus values: Automated, Manual

ISO 27001

Framework ID format: iso27001_{year}_{provider} (e.g., iso27001_2022_aws)

{
  "Id": "A.5.1",
  "Description": "Policies for information security should be defined...",
  "Name": "Policies for information security",
  "Checks": ["securityhub_enabled"],
  "Attributes": [
    {
      "Category": "A.5 Organizational controls",
      "Objetive_ID": "A.5.1",
      "Objetive_Name": "Policies for information security",
      "Check_Summary": "Summary of what is being checked"
    }
  ]
}

Note: Objetive_ID and Objetive_Name use this exact spelling (not "Objective").

ENS (Esquema Nacional de Seguridad - Spain)

Framework ID format: ens_rd2022_{provider} (e.g., ens_rd2022_aws)

{
  "Id": "op.acc.1.aws.iam.2",
  "Description": "Proveedor de identidad centralizado",
  "Checks": ["iam_check_saml_providers_sts"],
  "Attributes": [
    {
      "IdGrupoControl": "op.acc.1",
      "Marco": "operacional",
      "Categoria": "control de acceso",
      "DescripcionControl": "Detailed control description in Spanish",
      "Nivel": "alto",
      "Tipo": "requisito",
      "Dimensiones": ["trazabilidad", "autenticidad"],
      "ModoEjecucion": "automatico",
      "Dependencias": []
    }
  ]
}

Nivel values: opcional, bajo, medio, alto Tipo values: refuerzo, requisito, recomendacion, medida Dimensiones values: confidencialidad, integridad, trazabilidad, autenticidad, disponibilidad

MITRE ATT&CK

Framework ID format: mitre_attack_{provider} (e.g., mitre_attack_aws)

MITRE uses a different requirement structure:

{
  "Name": "Exploit Public-Facing Application",
  "Id": "T1190",
  "Tactics": ["Initial Access"],
  "SubTechniques": [],
  "Platforms": ["Containers", "IaaS", "Linux", "Network", "Windows", "macOS"],
  "Description": "Adversaries may attempt to exploit a weakness...",
  "TechniqueURL": "https://attack.mitre.org/techniques/T1190/",
  "Checks": ["guardduty_is_enabled", "inspector2_is_enabled"],
  "Attributes": [
    {
      "AWSService": "Amazon GuardDuty",
      "Category": "Detect",
      "Value": "Minimal",
      "Comment": "Explanation of how this service helps..."
    }
  ]
}

For Azure: Use AzureService instead of AWSService For GCP: Use GCPService instead of AWSService Category values: Detect, Protect, Respond Value values: Minimal, Partial, Significant

NIST 800-53

Framework ID format: nist_800_53_revision_{version}_{provider} (e.g., nist_800_53_revision_5_aws)

{
  "Id": "ac_2_1",
  "Name": "AC-2(1) Automated System Account Management",
  "Description": "Support the management of system accounts...",
  "Checks": ["iam_password_policy_minimum_length_14"],
  "Attributes": [
    {
      "ItemId": "ac_2_1",
      "Section": "Access Control (AC)",
      "SubSection": "Account Management (AC-2)",
      "SubGroup": "AC-2(3) Disable Accounts",
      "Service": "iam"
    }
  ]
}

Generic Compliance (Fallback)

For frameworks without specific attribute models:

{
  "Id": "requirement_id",
  "Description": "Requirement description",
  "Name": "Optional name",
  "Checks": ["check_name"],
  "Attributes": [
    {
      "ItemId": "item_id",
      "Section": "Section name",
      "SubSection": "Subsection name",
      "SubGroup": "Subgroup name",
      "Service": "service_name",
      "Type": "type"
    }
  ]
}

AWS Well-Architected Framework

Framework ID format: aws_well_architected_framework_{pillar}_pillar_aws

{
  "Id": "SEC01-BP01",
  "Description": "Establish common guardrails...",
  "Name": "Establish common guardrails",
  "Checks": ["account_part_of_organizations"],
  "Attributes": [
    {
      "Name": "Establish common guardrails",
      "WellArchitectedQuestionId": "securely-operate",
      "WellArchitectedPracticeId": "sec_securely_operate_multi_accounts",
      "Section": "Security",
      "SubSection": "Security foundations",
      "LevelOfRisk": "High",
      "AssessmentMethod": "Automated",
      "Description": "Detailed description",
      "ImplementationGuidanceUrl": "https://docs.aws.amazon.com/..."
    }
  ]
}

KISA ISMS-P (Korea)

Framework ID format: kisa_isms_p_{year}_{provider} (e.g., kisa_isms_p_2023_aws)

{
  "Id": "1.1.1",
  "Description": "Requirement description",
  "Name": "Requirement name",
  "Checks": ["check_name"],
  "Attributes": [
    {
      "Domain": "1. Management System",
      "Subdomain": "1.1 Management System Establishment",
      "Section": "1.1.1 Section Name",
      "AuditChecklist": ["Checklist item 1", "Checklist item 2"],
      "RelatedRegulations": ["Regulation 1"],
      "AuditEvidence": ["Evidence type 1"],
      "NonComplianceCases": ["Non-compliance example"]
    }
  ]
}

C5 (Germany Cloud Computing Compliance Criteria Catalogue)

Framework ID format: c5_{provider} (e.g., c5_aws)

{
  "Id": "BCM-01",
  "Description": "Requirement description",
  "Name": "Requirement name",
  "Checks": ["check_name"],
  "Attributes": [
    {
      "Section": "BCM Business Continuity Management",
      "SubSection": "BCM-01",
      "Type": "Basic Criteria",
      "AboutCriteria": "Description of criteria",
      "ComplementaryCriteria": "Additional criteria"
    }
  ]
}

CCC (Cloud Computing Compliance)

Framework ID format: ccc_{provider} (e.g., ccc_aws)

{
  "Id": "CCC.C01",
  "Description": "Requirement description",
  "Name": "Requirement name",
  "Checks": ["check_name"],
  "Attributes": [
    {
      "FamilyName": "Cryptography & Key Management",
      "FamilyDescription": "Family description",
      "Section": "CCC.C01",
      "SubSection": "Key Management",
      "SubSectionObjective": "Objective description",
      "Applicability": ["IaaS", "PaaS", "SaaS"],
      "Recommendation": "Recommended action",
      "SectionThreatMappings": [{"threat": "T1190"}],
      "SectionGuidelineMappings": [{"guideline": "NIST"}]
    }
  ]
}

Prowler ThreatScore

Framework ID format: prowler_threatscore_{provider} (e.g., prowler_threatscore_aws)

Prowler ThreatScore is a custom security scoring framework developed by Prowler that evaluates AWS account security based on four main pillars:

Scoring System:

• LevelOfRisk (1-5): Severity of the security issue

- 5 = Critical (e.g., root MFA, public S3 buckets) - 4 = High (e.g., user MFA, public EC2) - 3 = Medium (e.g., password policies, encryption) - 2 = Low - 1 = Informational

• Weight: Impact multiplier for score calculation

- 1000 = Critical controls (root security, public exposure) - 100 = High-impact controls (user authentication, monitoring) - 10 = Standard controls (password policies, encryption) - 1 = Low-impact controls (best practices)

{
  "Id": "1.1.1",
  "Description": "Ensure MFA is enabled for the 'root' user account",
  "Checks": ["iam_root_mfa_enabled"],
  "Attributes": [
    {
      "Title": "MFA enabled for 'root'",
      "Section": "1. IAM",
      "SubSection": "1.1 Authentication",
      "AttributeDescription": "The root user account holds the highest level of privileges within an AWS account. Enabling MFA enhances security by adding an additional layer of protection.",
      "AdditionalInformation": "Enabling MFA enhances console security by requiring the authenticating user to both possess a time-sensitive key-generating device and have knowledge of their credentials.",
      "LevelOfRisk": 5,
      "Weight": 1000
    }
  ]
}

Available for providers: AWS, Kubernetes, M365

Available Compliance Frameworks

AWS (41 frameworks)

Azure (15+ frameworks)

GCP (15+ frameworks)

Kubernetes (6 frameworks)

Other Providers

• GitHub: cis_1.0_github.json
• M365: cis_4.0_m365.json, iso27001_2022_m365.json
• NHN: iso27001_2022_nhn.json

Workflow A: Sync a Framework With an Upstream Catalog

Use when the framework is maintained upstream (CIS Benchmarks, FINOS CCC, CSA CCM, NIST, ENS, etc.) and Prowler needs to catch up.

Step 1 — Cache the upstream source

Download every upstream file to a local cache so subsequent iterations don't hit the network. For FINOS CCC:

mkdir -p /tmp/ccc_upstream
catalogs="core/ccc storage/object management/auditlog management/logging ..."
for p in $catalogs; do
  safe=$(echo "$p" | tr '/' '_')
  gh api "repos/finos/common-cloud-controls/contents/catalogs/$p/controls.yaml" \
    -H "Accept: application/vnd.github.raw" > "/tmp/ccc_upstream/${safe}.yaml"
done

Step 2 — Run the generic sync runner against a framework config

The sync tooling is split into three layers so adding a new framework only takes a YAML config (and optionally a new parser module for an unfamiliar upstream format):

skills/prowler-compliance/assets/
├── sync_framework.py          # generic runner — works for any framework
├── configs/
│   └── ccc.yaml               # per-framework config (canonical example)
└── parsers/
    ├── __init__.py
    └── finos_ccc.py           # parser module for FINOS CCC YAML

For frameworks that already have a config + parser (today: FINOS CCC), run:

python skills/prowler-compliance/assets/sync_framework.py \
       skills/prowler-compliance/assets/configs/ccc.yaml

The runner loads the config, validates it, dynamically imports the parser declared in parser.module, calls parser.parse_upstream(config) -> list[dict], then applies generic post-processing (id uniqueness safety net, FamilyName normalization, legacy check-mapping preservation) and writes the provider JSONs.

To add a new framework sync:

1. Write a config file at skills/prowler-compliance/assets/configs/{framework}.yaml. See configs/ccc.yaml as the canonical example. Required top-level sections:

- framework — name, display_name, version (never empty — empty Version silently breaks get_check_compliance() key construction, so the runner refuses to start), description_template (accepts {provider_display}, {provider_key}, {framework_name}, {framework_display}, {version} placeholders). - providers — list of {key, display} pairs, one per Prowler provider the framework targets. - output.path_template — supports {provider}, {framework}, {version} placeholders. Examples: "prowler/compliance/{provider}/ccc_{provider}.json" for unversioned file names, "prowler/compliance/{provider}/cis_{version}_{provider}.json" for versioned ones. - upstream.dir — local cache directory (populate via Step 1). - parser.module — name of the module under parsers/ to load (without .py). Everything else under parser. is opaque to the runner and passed to the parser as config. - post_processing.check_preservation.primary_key — top-level field name for the primary legacy-mapping lookup (almost always Id). - post_processing.check_preservation.fallback_keys — config-driven fallback keys for preserving check mappings when ids change. Each entry is a list of Attributes[0] field names composed into a tuple. Examples: - CCC: - [Section, Applicability] (because Applicability is a CCC-only attribute, verified in compliance_models.py:213). - CIS would use - [Section, Profile]. - NIST would use - [ItemId]. - List-valued fields (like Applicability) are automatically frozen to frozenset so the tuple is hashable. - post_processing.family_name_normalization (optional) — map of raw → canonical FamilyName values. The UI groups by Attributes[0].FamilyName exactly, so inconsistent upstream variants otherwise become separate tree branches.

1. Reuse an existing parser if the upstream format matches one (currently only finos_ccc exists). Otherwise, write a new parser at parsers/{name}.py implementing: def parse_upstream(config: dict) -> list[dict]: """Return Prowler-format requirements {Id, Description, Attributes: [...], Checks: []}. Ids MUST be unique in the returned list. The runner raises ValueError on duplicates — it does NOT silently renumber, because mutating a canonical upstream id (e.g. CIS '1.1.1' or NIST 'AC-2(1)') would be catastrophic. The parser owns all upstream-format quirks: foreign-prefix rewriting, genuine collision renumbering, shape handling. """ The parser reads its own settings from config['upstream'] and config['parser']. It does NOT load existing Prowler JSONs (the runner does that for check preservation) and does NOT write output (the runner does that too).

Gotchas the runner already handles for you (learned from the FINOS CCC v2025.10 sync — they're documented here so you don't re-discover them):

• Multiple upstream YAML shapes. Most FINOS CCC catalogs use control-families: [...], but storage/object uses a top-level controls: [...] with a family: "CCC.X.Y" reference id and no human-readable family name. A parser that only handles shape 1 silently drops the shape-2 catalog — this exact bug dropped ObjStor from Prowler for a full iteration. parsers/finos_ccc.py handles both shapes; if you write a new parser for a similar format, test with at least one file of each shape.
• Whitespace collapse. Upstream YAML multi-line block scalars (|) preserve newlines. Prowler stores descriptions single-line. Collapse with " ".join(value.split()) before emitting (see parsers/finos_ccc.py::clean()).
• Foreign-prefix AR id rewriting. Upstream sometimes aliases requirements across catalogs by keeping the original prefix (e.g., CCC.AuditLog.CN08.AR01 appears nested under CCC.Logging.CN03). Rewrite the foreign id to fit its parent control: CCC.Logging.CN03.AR01. This logic is parser-specific because the id structure varies per framework (CCC uses 3-dot depth; CIS uses numeric dots; NIST uses AC-2(1)).
• Genuine upstream collision renumbering. Sometimes upstream has a real typo where two different requirements share the same id (e.g., CCC.Core.CN14.AR02 defined twice for 30-day and 14-day backup variants). Renumber the second copy to the next free AR number (.AR03). The parser handles this; the runner asserts the final list has unique ids as a safety net.
• Existing check mapping preservation. The runner uses the primary_key + fallback_keys declared in config to look up the old Checks list for each requirement. For CCC this means primary index by Id plus fallback index by (Section, frozenset(Applicability)) — the fallback recovers mappings for requirements whose ids were rewritten or renumbered by the parser.
• FamilyName normalization. Configured via post_processing.family_name_normalization — no code changes needed to collapse upstream variants like "Logging & Monitoring" → "Logging and Monitoring".
• Populate Version. The runner refuses to start on empty framework.version — fail-fast replaces the silent bug where get_check_compliance() would build the key as just "{Framework}".

Step 3 — Validate before committing

from prowler.lib.check.compliance_models import Compliance
for prov in ['aws', 'azure', 'gcp']:
    c = Compliance.parse_file(f"prowler/compliance/{prov}/ccc_{prov}.json")
    print(f"{prov}: {len(c.Requirements)} reqs, version={c.Version}")

Any ValidationError means the Attribute fields don't match the *_Requirement_Attribute model. Either fix the JSON or extend the model in compliance_models.py (remember: Generic stays last).

Step 4 — Verify every check id exists

import json
from pathlib import Path
for prov in ['aws', 'azure', 'gcp']:
    existing = {p.stem.replace('.metadata','')
                for p in Path(f'prowler/providers/{prov}/services').rglob('*.metadata.json')}
    with open(f'prowler/compliance/{prov}/ccc_{prov}.json') as f:
        data = json.load(f)
    refs = {c for r in data['Requirements'] for c in r['Checks']}
    missing = refs - existing
    assert not missing, f"{prov} missing: {missing}"

A stale check id silently becomes dead weight — no finding will ever map to it. This pre-validation must run on every write; bake it into the generator script.

Step 5 — Add an attribute model if needed

Only if the framework has fields beyond Generic_Compliance_Requirement_Attribute. Add the class to prowler/lib/check/compliance_models.py and register it in Compliance_Requirement.Attributes: list[Union[...]]. Generic stays last.

Workflow B: Audit Check Mappings as a Cloud Auditor

Use when the user asks to review existing mappings ("are these correct?", "verify that the checks apply", "audit the CCC mappings"). This is the highest-value compliance task — it surfaces padded mappings with zero actual coverage and missing mappings for legitimate coverage.

The golden rule

A Prowler check's title/risk MUST literally describe what the requirement text says. "Related" is not enough. If no check actually addresses the requirement, leave Checks: [] (MANUAL) — honest MANUAL is worth more than padded coverage.

Audit process

Step 1 — Build a per-provider check inventory (cache in /tmp/):

import json
from pathlib import Path
for provider in ['aws', 'azure', 'gcp']:
    inv = {}
    for meta in Path(f'prowler/providers/{provider}/services').rglob('*.metadata.json'):
        with open(meta) as f:
            d = json.load(f)
        cid = d.get('CheckID') or meta.stem.replace('.metadata','')
        inv[cid] = {
            'service': d.get('ServiceName', ''),
            'title': d.get('CheckTitle', ''),
            'risk': d.get('Risk', ''),
            'description': d.get('Description', ''),
        }
    with open(f'/tmp/checks_{provider}.json', 'w') as f:
        json.dump(inv, f, indent=2)

Step 2 — Keyword/service query helper — see assets/query_checks.py:

python assets/query_checks.py aws encryption transit    # keyword AND-search
python assets/query_checks.py aws --service iam         # all iam checks
python assets/query_checks.py aws --id kms_cmk_rotation_enabled  # full metadata

Step 3 — Dump a framework section with current mappings — see assets/dump_section.py:

python assets/dump_section.py ccc "CCC.Core."      # all Core ARs across 3 providers
python assets/dump_section.py ccc "CCC.AuditLog."  # all AuditLog ARs

Step 4 — Encode explicit REPLACE decisions — see assets/audit_framework_template.py. Structure:

DECISIONS = {}

DECISIONS["CCC.Core.CN01.AR01"] = {
    "aws": [
        "cloudfront_distributions_https_enabled",
        "cloudfront_distributions_origin_traffic_encrypted",
        # ...
    ],
    "azure": [
        "storage_secure_transfer_required_is_enabled",
        "app_minimum_tls_version_12",
        # ...
    ],
    "gcp": [
        "cloudsql_instance_ssl_connections",
    ],
    # Missing provider key = leave the legacy mapping untouched
}

# Empty list = EXPLICITLY MANUAL (overwrites legacy)
DECISIONS["CCC.Core.CN01.AR07"] = {
    "aws": [],   # Prowler has no IANA port/protocol check
    "azure": [],
    "gcp": [],
}

REPLACE, not PATCH. Encoding every mapping as a full list (not add/remove delta) makes the audit reproducible and surfaces hidden assumptions from the legacy data.

Step 5 — Pre-validation. The audit script MUST validate every check id against the inventory and abort with stderr listing typos. Common typos caught during a real audit:

• fsx_file_system_encryption_at_rest_using_kms (doesn't exist)
• cosmosdb_account_encryption_at_rest_with_cmk (doesn't exist)
• sqlserver_geo_replication (doesn't exist)
• redshift_cluster_audit_logging (should be redshift_cluster_encrypted_at_rest)
• postgresql_flexible_server_require_secure_transport (should be postgresql_flexible_server_enforce_ssl_enabled)
• storage_secure_transfer_required_enabled (should be storage_secure_transfer_required_is_enabled)
• sqlserver_minimum_tls_version_12 (should be sqlserver_recommended_minimal_tls_version)

Step 6 — Apply + validate + test:

python /path/to/audit_script.py   # applies decisions, pre-validates
python -m pytest tests/lib/outputs/compliance/ tests/lib/check/ -q

Audit Reference Table: Requirement Text → Prowler Checks

Use this table to map CCC-style / NIST-style / ISO-style requirements to the checks that actually verify them. Built from a real audit of 172 CCC ARs × 3 providers.

What Prowler Does NOT Cover (accept MANUAL honestly)

Don't pad mappings for these — mark Checks: [] and move on:

• TLS 1.3 version specifically — Prowler verifies TLS is enforced, not always the exact version
• IANA port-protocol consistency — no check for "protocol running on its assigned port"
• mTLS on most Azure/GCP services — limited to App Service client certs on Azure, nothing on GCP
• Rate limiting on monitoring endpoints, load balancers, serverless invocations, vector ingestion
• Session cookie expiry (LB stickiness)
• HTTP header scrubbing (Server, X-Powered-By)
• Certificate transparency verification for imports
• Model version pinning, red teaming, AI quality review
• Vector embedding validation, dimensional constraints, ANN vs exact search
• Secret region replication (cross-region residency)
• Lifecycle cleanup policies on container registries
• Row-level / column-level security in data warehouses
• Deployment region restriction on Azure/GCP (AWS has organizations_scp_check_deny_regions, others don't)
• Cross-tenant alert silencing permissions
• Field-level masking in logs
• Managed view enforcement for database access
• Automatic MFA delete on all S3 buckets (only CloudTrail bucket variant exists for some frameworks — AWS has the generic s3_bucket_no_mfa_delete though)

Workflow C: Add a New Output Formatter

Use when a new framework needs its own CSV columns or terminal table. Follow the c5/csa/ens layout exactly:

mkdir -p prowler/lib/outputs/compliance/{framework}
touch prowler/lib/outputs/compliance/{framework}/__init__.py

Step 1 — Create {framework}.py (table dispatcher ONLY)

Copy from prowler/lib/outputs/compliance/c5/c5.py and change the function name + framework string. The diff between your file and c5.py should be just those two lines. No function docstring — other frameworks don't have one, stay consistent.

Step 2 — Create models.py

One Pydantic v2 BaseModel per provider. Field names become CSV column headers (public API — don't rename later without a migration).

from typing import Optional
from pydantic import BaseModel

class {Framework}_AWSModel(BaseModel):
    Provider: str
    Description: str
    AccountId: str
    Region: str
    AssessmentDate: str
    Requirements_Id: str
    Requirements_Description: str
    # ... provider-specific columns
    Status: str
    StatusExtended: str
    ResourceId: str
    ResourceName: str
    CheckId: str
    Muted: bool

Step 3 — Create {framework}_{provider}.py for each provider

Copy from prowler/lib/outputs/compliance/c5/c5_aws.py etc. Contains the {Framework}_AWS(ComplianceOutput) class with transform() that walks findings and emits model rows. This file IS allowed to import Finding.

Step 4 — Register everywhere

prowler/lib/outputs/compliance/compliance.py (CLI table dispatcher):

from prowler.lib.outputs.compliance.{framework}.{framework} import get_{framework}_table

def display_compliance_table(...):
    ...
    elif compliance_framework.startswith("{framework}_"):
        get_{framework}_table(findings, bulk_checks_metadata,
                              compliance_framework, output_filename,
                              output_directory, compliance_overview)

prowler/__main__.py (CLI output writer per provider): Add imports at the top:

from prowler.lib.outputs.compliance.{framework}.{framework}_aws import {Framework}_AWS
from prowler.lib.outputs.compliance.{framework}.{framework}_azure import {Framework}_Azure
from prowler.lib.outputs.compliance.{framework}.{framework}_gcp import {Framework}_GCP

Add provider-specific elif compliance_name.startswith("{framework}_"): branches that instantiate the class and call batch_write_data_to_file().

api/src/backend/tasks/jobs/export.py (API export dispatcher):

from prowler.lib.outputs.compliance.{framework}.{framework}_aws import {Framework}_AWS
# ... azure, gcp

COMPLIANCE_CLASS_MAP = {
    "aws": [
        # ...
        (lambda name: name.startswith("{framework}_"), {Framework}_AWS),
    ],
    # ... azure, gcp
}

Always use startswith, never name == "framework_aws". Exact match is a regression.

Step 5 — Add tests

Create tests/lib/outputs/compliance/{framework}/ with {framework}_aws_test.py, {framework}_azure_test.py, {framework}_gcp_test.py. See the test template in references/test_template.md.

Add fixtures to tests/lib/outputs/compliance/fixtures.py: one Compliance object per provider with 1 evaluated + 1 manual requirement to exercise both code paths in transform().

Circular import warning

The table dispatcher file ({framework}.py) MUST NOT import Finding (directly or transitively). The cycle is:

compliance.compliance imports get_{framework}_table
  → {framework}.py imports ComplianceOutput
  → compliance_output imports Finding
  → finding imports get_check_compliance from compliance.compliance
  → CIRCULAR

Keep {framework}.py bare — only colorama, tabulate, prowler.config.config. Put anything that imports Finding in the per-provider {framework}_{provider}.py files.

Conventions and Hard-Won Gotchas

These are lessons from the FINOS CCC v2025.10 sync + 172-AR audit pass (April 2026). Learn them once; save days of debugging.

1. Per-provider files are non-negotiable. Never collapse {framework}_aws.py, {framework}_azure.py, {framework}_gcp.py into a single parameterized class, no matter how DRY-tempting. Every other framework in the codebase follows the per-provider pattern and reviewers will reject the refactor. The CSV column names differ per provider — three classes is the convention.
2. {framework}.py has NO function docstring. Other frameworks don't have them. Don't add one to be "helpful".
3. Circular import protection: the table dispatcher file MUST NOT import Finding (directly or transitively). Split the code so {framework}.py only has get_{framework}_table() with bare imports, and {framework}_{provider}.py holds the class that needs Finding.
4. Generic_Compliance_Requirement_Attribute is the fallback — in the Compliance_Requirement.Attributes Union in compliance_models.py, Generic MUST be LAST because Pydantic v1 tries union members in order. Putting Generic first means every framework-specific attribute falls through to Generic and the specific model is never used.
5. Pydantic v1 imports. from pydantic.v1 import BaseModel in compliance_models.py — not v2. Mixing causes validation errors. Pydantic v2 is used in the CSV models (models.py) — that's fine because they're separate trees.
6. get_check_compliance() key format is f"{Framework}-{Version}" ONLY if Version is set. Empty Version → key is "{Framework}" (no version suffix). Tests that mock compliance dicts must match this exact format — when a framework ships with Version: "", downstream code and tests break silently.
7. CSV column names from models.py are public API. Don't rename a field without migrating downstream consumers — CSV headers change.
8. Upstream YAML multi-line scalars (| block scalars) preserve newlines. Collapse to single-line with " ".join(value.split()) before writing to JSON.
9. Upstream catalogs can use multiple shapes. FINOS CCC uses control-families: [...] in most catalogs but controls: [...] at the top level in storage/object. Any sync script must handle both or silently drop entire catalogs.
10. Foreign-prefix AR ids. Upstream sometimes "imports" requirements from one catalog into another by keeping the original id prefix (e.g., CCC.AuditLog.CN08.AR01 appearing under CCC.Logging.CN03). Prowler's compliance model requires unique ids within a catalog — rewrite the foreign id to fit the parent control: CCC.AuditLog.CN08.AR01 (inside CCC.Logging.CN03) → CCC.Logging.CN03.AR01.
11. Genuine upstream id collisions. Sometimes upstream has a real typo where two different requirements share the same id (e.g., CCC.Core.CN14.AR02 defined twice for 30-day and 14-day backup variants). Renumber the second copy to the next free AR number. Preserve check mappings by matching on (Section, frozenset(Applicability)) since the renumbered id won't match by id.
12. COMPLIANCE_CLASS_MAP in export.py uses startswith predicates for all modern frameworks. Exact match (name == "ccc_aws") is an anti-pattern — it was present for CCC until April 2026 and was the reason CCC couldn't have versioned variants.
13. Pre-validate every check id against the per-provider inventory before writing the JSON. A typo silently creates an unreferenced check that will fail when findings try to map to it. The audit script MUST abort with stderr listing typos, not swallow them.
14. REPLACE is better than PATCH for audit decisions. Encoding every mapping explicitly makes the audit reproducible and surfaces hidden assumptions from the legacy data. A PATCH system that adds/removes is too easy to forget.
15. When no check applies, MANUAL is correct. Do not pad mappings with tangential checks "just in case". Prowler's compliance reports are meant to be actionable — padding them with noise breaks that. Honest manual reqs can be mapped later when new checks land.
16. UI groups by Attributes[0].FamilyName and Attributes[0].Section. If FamilyName has inconsistent variants within the same JSON (e.g., "Logging & Monitoring" vs "Logging and Monitoring"), the UI renders them as separate categories. Section empty → the requirement falls into an orphan control with label "". Normalize before shipping.
17. Provider coverage is asymmetric. AWS has dense coverage (~586 checks across 80+ services): in-transit encryption, IAM, database encryption, backup. Azure (~167 checks) and GCP (~102 checks) are thinner especially for in-transit encryption, mTLS, and ML/AI. Accept the asymmetry in mappings — don't force GCP parity where Prowler genuinely can't verify.

Useful One-Liners

# Count requirements per service prefix (CCC, CIS sections, etc.)
jq -r '.Requirements[].Id | split(".")[1]' prowler/compliance/aws/ccc_aws.json | sort | uniq -c

# Find duplicate requirement IDs
jq -r '.Requirements[].Id' file.json | sort | uniq -d

# Count manual requirements (no checks)
jq '[.Requirements[] | select((.Checks | length) == 0)] | length' file.json

# List all unique check references in a framework
jq -r '.Requirements[].Checks[]' file.json | sort -u

# List all unique Sections (to spot inconsistency)
jq '[.Requirements[].Attributes[0].Section] | unique' file.json

# List all unique FamilyNames (to spot inconsistency)
jq '[.Requirements[].Attributes[0].FamilyName] | unique' file.json

# Diff requirement ids between two versions of the same framework
diff <(jq -r '.Requirements[].Id' a.json | sort) <(jq -r '.Requirements[].Id' b.json | sort)

# Find where a check id is used across all frameworks
grep -rl "my_check_name" prowler/compliance/

# Check if a Prowler check exists
find prowler/providers/aws/services -name "{check_id}.metadata.json"

# Validate a JSON with Pydantic
python -c "from prowler.lib.check.compliance_models import Compliance; print(Compliance.parse_file('prowler/compliance/aws/ccc_aws.json').Framework)"

Best Practices

1. Requirement IDs: Follow the original framework numbering exactly (e.g., "1.1", "A.5.1", "T1190", "ac_2_1")
2. Check Mapping: Map to existing checks when possible. Use Checks: [] for manual-only requirements — honest MANUAL beats padded coverage
3. Completeness: Include all framework requirements, even those without automated checks
4. Version Control: Include framework version in Name and Version fields. Never leave Version: "" — it breaks get_check_compliance() key format
5. File Naming: Use format {framework}_{version}_{provider}.json
6. Validation: Prowler validates JSON against Pydantic models at startup — invalid JSON will cause errors
7. Pre-validate check ids against the provider's *.metadata.json inventory before every commit
8. Normalize FamilyName and Section to avoid inconsistent UI tree branches
9. Register everywhere: SDK model (if needed) → compliance.py dispatcher → __main__.py CLI writer → export.py API map → UI mapper. Skipping any layer results in silent failures
10. Audit, don't pad: when reviewing mappings, apply the golden rule — the check's title/risk MUST literally describe what the requirement text says. Tangential relation doesn't count

Commands

# List available frameworks for a provider
prowler {provider} --list-compliance

# Run scan with specific compliance framework
prowler aws --compliance cis_5.0_aws

# Run scan with multiple frameworks
prowler aws --compliance cis_5.0_aws pci_4.0_aws

# Output compliance report in multiple formats
prowler aws --compliance cis_5.0_aws -M csv json html

Code References

Layer 1 — SDK / Core

• Compliance Models: prowler/lib/check/compliance_models.py (Pydantic v1 model tree)
• Compliance Processing / Linker: prowler/lib/check/compliance.py (get_check_compliance, update_checks_metadata_with_compliance)
• Check Utils: prowler/lib/check/utils.py (list_compliance_modules)

Layer 2 — JSON Catalogs

• Framework JSONs: prowler/compliance/{provider}/ (auto-discovered via directory walk)

Layer 3 — Output Formatters

• Per-framework folders: prowler/lib/outputs/compliance/{framework}/
• Shared base class: prowler/lib/outputs/compliance/compliance_output.py (ComplianceOutput + batch_write_data_to_file)
• CLI table dispatcher: prowler/lib/outputs/compliance/compliance.py (display_compliance_table)
• Finding model: prowler/lib/outputs/finding.py (do not import transitively from table dispatcher files — circular import)
• CLI writer: prowler/__main__.py (per-provider elif compliance_name.startswith(...) branches that instantiate per-provider classes)

Layer 4 — API / UI

• API lazy loader: api/src/backend/api/compliance.py (LazyComplianceTemplate, LazyChecksMapping)
• API export dispatcher: api/src/backend/tasks/jobs/export.py (COMPLIANCE_CLASS_MAP with startswith predicates)
• UI framework router: ui/lib/compliance/compliance-mapper.ts
• UI per-framework mapper: ui/lib/compliance/{framework}.tsx
• UI detail panel: ui/components/compliance/compliance-custom-details/{framework}-details.tsx
• UI types: ui/types/compliance.ts
• UI icon: ui/components/icons/compliance/{framework}.svg + registration in IconCompliance.tsx

Tests

• Output formatter tests: tests/lib/outputs/compliance/{framework}/{framework}_{provider}_test.py
• Shared fixtures: tests/lib/outputs/compliance/fixtures.py

Resources

• JSON Templates: See assets/ for framework JSON templates (cis, ens, iso27001, mitre_attack, prowler_threatscore, generic)
• Config-driven compliance sync (any upstream-backed framework):

- assets/sync_framework.py — generic runner. Loads a YAML config, dynamically imports the declared parser, applies generic post-processing (id uniqueness safety net, FamilyName normalization, legacy check-mapping preservation with config-driven fallback keys), and writes the provider JSONs with Pydantic post-validation. Framework-agnostic — works for any compliance framework. - assets/configs/ccc.yaml — canonical config example (FINOS CCC v2025.10). Copy and adapt for new frameworks. - assets/parsers/finos_ccc.py — FINOS CCC YAML parser. Handles both upstream shapes (control-families and top-level controls), foreign-prefix AR rewriting, and genuine collision renumbering. Exposes parse_upstream(config) -> list[dict]. - assets/parsers/ — add new parser modules here for unfamiliar upstream formats (NIST OSCAL JSON, MITRE STIX, CIS Benchmarks, etc.). Each parser is a {name}.py file implementing parse_upstream(config) -> list[dict] with guaranteed-unique ids.

• Reusable audit tooling (added April 2026 after the FINOS CCC v2025.10 sync):

- assets/audit_framework_template.py — explicit REPLACE decision ledger with pre-validation against the per-provider inventory. Drop-in template for auditing any framework. - assets/query_checks.py — keyword/service/id query helper over /tmp/checks_{provider}.json. - assets/dump_section.py — dumps every AR for a given id prefix across all 3 providers with current check mappings. - assets/build_inventory.py — generates /tmp/checks_{provider}.json from *.metadata.json files.

• Documentation: See references/compliance-docs.md for additional resources
• Related skill: prowler-compliance-review — PR review checklist and validator script for compliance framework PRs