Compliance evidence, gathered once a quarter

DevOps, SRE and securitySmall company preparing for its own audit

Every quarter our engineer spent a day collecting audit evidence: IAM users with MFA, backup settings, branch protection, log retention. The agent rebuilt the list of commands from scratch each time, and twice missed the same item.

How do we get an AI agent to collect audit evidence the same way every quarter?

Have the agent save the full procedure once with ArcRoom's complete_task: each control, the command that proves it and where the file goes. Next quarter, find_task returns it and the agent re-runs the same commands, checking first the controls saved with remember_failure last time. Evidence files stay on the laptop. ArcRoom stores only commands, paths and the procedure, and is not a compliance tool itself.

Also asked as: Our agent rebuilds the audit evidence checklist from scratch every quarter. How do we stop that? · How do I save a repeatable procedure for gathering compliance evidence with AI? · Can an AI agent remember which audit controls failed last time?

The work

The first quarter, the full procedure is saved with complete_task: each control, the command that proves it, where the file goes.

ArcRoom keeps
  • Reused answer
  • Saved answer
  • Saved decision
The next agent gets

find_task → reuse: "Q3 audit evidence" (Jul 4)

Step by step

  1. The first quarter, the full procedure is saved with complete_task: each control, the command that proves it, where the file goes.
  2. Next quarter, find_task returns it and the agent re-runs the same commands.
  3. Controls that failed are saved with remember_failure, so the next run checks them first.
  4. Evidence files stay on the laptop. ArcRoom stores only commands, paths and the procedure.

What the agent sees

find_task → reuse: "Q3 audit evidence" (Jul 4)
1 aws iam list-users + MFA check → evidence/iam-mfa.csv
2 gh api branch protection on main → evidence/branch.json
3 RDS backup retention ≥ 7 days → evidence/rds.txt
Failed last time: CloudTrail retention 30 days (needs 365)

Compared with today's tools

Compliance automation platform (e.g. Vanta or Drata)

Connect cloud and code accounts so the platform collects evidence continuously.

Setup
1–3 weeks
Ongoing
low once connected

ArcRoom

Every agent reads the team playbook before it works and writes to it when it finishes. Nothing to update by hand.

Setup
2 min (npx arcroom init)
Ongoing
None

A compliance platform is the better choice for a real audit programme; ArcRoom fits a small team doing evidence by hand with an agent.

Where it stops

ArcRoom helps gather evidence; it is not a compliance tool and is not SOC 2 certified itself.

Try it in your project

Works with the agents your team already uses, and any agent over MCP or the HTTP API.

More in devops, sre and security