Security review findings that stay found
DevOps, SRE and securityTeam running authorized security reviews of its own code
Every quarter we ask an agent to review our own app for security issues. Each review re-reported 'SSRF risk in the image proxy', which we had accepted with an allowlist, and missed checking whether last quarter's IDOR fix was still in place.
How do I stop an AI security review from re-reporting risks we already accepted?
Save accepted risks with arcroom decide and the reason, and have the agent save each finding with ArcRoom's complete_task, tied to the git hashes of the files it covered. The next review won't raise accepted risks again. When a fixed file changes, the saved finding goes stale and the agent re-checks that spot first. Keep finding text short, with no exploit steps or secrets.
Also asked as: Why does every AI security review flag the same accepted issues again? · How do I make an agent check that last quarter's security fixes are still in place? · Can an AI agent remember past security review findings between reviews?
Each finding is saved with complete_task, tied to the git hashes of the files it was about.
- Saved answer
- Decision
- Freshness check
- Off switch
find_task → 2 earlier findings for "security review api/"
Step by step
- Each finding is saved with complete_task, tied to the git hashes of the files it was about.
- Accepted risks are saved with arcroom decide and the reason, so the next review doesn't raise them again.
- When a fixed file changes, the saved finding goes stale and the agent re-checks that spot first.
- For the most sensitive repo, findings can stay out of the playbook entirely with arcroom off.
What the agent sees
find_task → 2 earlier findings for "security review api/"
• Accepted (Jul 10): image proxy SSRF, allowlist in proxy.ts. Don't re-raise.
• Fixed (Jul 12): IDOR on /invoices/:id. Stale: invoices.ts changed, re-check.Compared with today's tools
Issue tracker with security labels
Findings go into tickets, accepted risks get a 'wontfix' label, and the reviewer reads them before starting.
- Setup
- 30 min
- Ongoing
- export open and accepted findings for each review, ~20 min
ArcRoom
Every agent reads the team playbook before it works and writes to it when it finishes. Nothing to update by hand.
- Setup
- 2 min (npx arcroom init)
- Ongoing
- None
The tracker stays the record of ownership and fix dates; ArcRoom adds the link to file hashes, so a changed fix is re-checked automatically.
Where it stops
Finding text the agent saves leaves the laptop as saved memory. Keep details short, and never save exploit steps or secrets.
Try it in your project
Works with the agents your team already uses, and any agent over MCP or the HTTP API.