grc-copilot

Automated evidence collection, control gap analysis, and evidence-grounded questionnaire drafting for SOC 2 and ISO 27001.

All data in this repository is synthetic. The fixtures/ directory contains a fabricated tenant. No employer, client, subscription, tenant, or customer data appears anywhere in this repo or its git history.

Most compliance work is not judgement, it is retrieval. Someone screenshots an MFA policy, pastes it into a folder, and eleven months later an auditor asks whether it is still true. The screenshot cannot answer that.

This is a working spine for the alternative: collect evidence from source-system APIs, sign and date it, map it to controls once, and reuse it for both frameworks and for customer questionnaires.

It is a proof of concept, not a product. It is deliberately small enough to read in one sitting.

flowchart LR
    A[Source systems<br/>Azure ARM, Graph,<br/>GitHub, GRC exports] -->|collect.py| B[(evidence.json<br/>sha256 + timestamp<br/>+ freshness policy)]
    B -->|analyze.py| C[Control gap report<br/>SOC 2 CC + ISO Annex A]
    B -->|answer.py| D[Questionnaire drafts<br/>with citations]
    M[mappings/*.yaml] --> C
    M --> D
    C -->|--fail-on high| E[CI gate]
    D --> F[Human review queue]

Collection is separated from assessment on purpose. You collect once and assess against as many frameworks as you have mappings for. Where a SOC 2 criterion and an Annex A control consume the same evidence type, that overlap is the argument for a unified control set instead of two parallel audit programmes.

No credentials, no API key, no cloud account. It runs against the synthetic tenant out of the box.

pip install -r requirements.txt

python -m grc_copilot.collect --out evidence.json
python -m grc_copilot.analyze --evidence evidence.json
python -m grc_copilot.answer --questionnaire examples/questionnaire.csv --offline

Against live Azure:

az login
python -m grc_copilot.collect --source azure --subscription <sub-id>

With an LLM for questionnaire drafting:

export ANTHROPIC_API_KEY=...
python -m grc_copilot.answer --questionnaire examples/questionnaire.csv

CONTROL GAP REPORT
========================================================================

TAMPERED:0  MISSING:0  PARTIAL:5  STALE:3  SATISFIED:15

SOC 2 (TSC 2017, rev. 2022)
---------------------------
  [SATISFIED] CC6.6    Boundary protections restrict external threats
  [STALE    ] CC6.1    Logical access security controls restrict access to
              -> Re-run collection for azure.privileged_role_review; evidence is past its retention window.
  [PARTIAL  ] CC6.7    Data in transit and at rest is protected
              -> Build or restore a collector for azure.tls_policy.
  ...

Audit readiness: 15/23 controls have current, verified evidence (65%).

Questionnaire output is a review queue, one row per item, carrying the draft, its confidence, the controls it was routed to, the exact evidence hashes it was grounded in, and the gaps that remain:

1. Freshness is a control status, not metadata. Every evidence type carries a max_age_days. Defender alerts expire in 30 days, an MFA conditional-access policy in 180. Required evidence past its window returns STALE, which is a finding, not a pass. Stale evidence is the failure mode manual collection hides: the screenshot is genuine, it is just from last year.

2. Absent collectors surface as gaps, never as passes. hr.onboarding_records and hr.offboarding_records have no automated collector here, so every control depending on them reports PARTIAL or MISSING. That is the correct output. A tool that quietly defaults an uninstrumented control to green is how automated compliance starts lying to the people relying on it.

3. The model cannot answer what the evidence does not support. answer.py injects only the routed, fresh, hash-verified evidence and instructs the model to return INSUFFICIENT_EVIDENCE rather than compose a plausible sentence. Where the required evidence set is incomplete, the deterministic path refuses before the model is called at all. A questionnaire response is a representation made to a customer; a fabricated one is a misrepresentation, and the liability does not move to the model.

4. Every claim is re-derivable. Each answer cites control ids and truncated sha256 digests of the artifacts behind it. A reviewer verifies rather than trusts. verify() re-hashes payloads on read, so an edited bundle reports TAMPERED instead of passing.

Stated because a POC that hides its edges is not useful to evaluate.

• Routing is keyword-based. Deterministic and auditable, but it mis-routes. A question about independent penetration testing routes to the vendor control on the word "third party". Embedding search is a two-line swap, at the cost of a reviewer being unable to explain a cosine score to an auditor. Both paths need the human review queue, so I kept the explainable one.
• SATISFIED means evidence exists and is current. It does not mean the control operates effectively. That is an assessor's judgement and the tool never claims it.
• The Azure backend covers seven collectors. Graph, backup vault, and TLS collectors are fixture-only. Adding a real one is a function plus a @register line.
• No evidence store. Bundles are files. Production needs versioned, append-only storage with retention.
• Mappings are paraphrased intent, not AICPA or ISO criteria verbatim, for licensing reasons.

grc_copilot/
  evidence.py   evidence record: hashing, provenance, freshness, tamper check
  controls.py   framework loader and questionnaire routing
  collect.py    pluggable collectors — fixtures and live Azure ARM
  analyze.py    status ladder, severity, remediation, CI gate
  answer.py     evidence-grounded drafting with citation and refusal
mappings/
  soc2_cc.yaml       11 Common Criteria
  iso27001_a.yaml    12 Annex A controls over the same evidence types
fixtures/azure/      synthetic tenant
examples/            10-item sample questionnaire

- run: python -m grc_copilot.collect --source azure --subscription ${{ secrets.AZURE_SUB }}
- run: python -m grc_copilot.analyze --evidence evidence.json --fail-on high

Control drift fails the build the same way a failing test does.

MIT.