A prompt that gets better each week
Building agentsBuilder tuning the system prompt of a support-reply agent
I tune our reply agent's prompt with Claude Code: change a line, run 30 sample tickets, compare. After three weeks I had 40 versions, and I kept retrying wording I had already ruled out.
How do I keep track of which system prompt versions I already tried and ruled out?
Save each prompt version and its score with ArcRoom's complete_task, and wording that made things worse with remember_failure and why. Record the final choice with arcroom decide and a --why reason, so later sessions don't undo it. Next week, recall brings the kept version and the ruled-out ones before you start. ArcRoom doesn't run tests or score prompts; you still need your own eval.
Also asked as: I keep retrying prompt wording I already ruled out. How do I stop? · How do I keep a log of prompt changes and results while tuning an agent? · Can Claude Code remember which prompt versions did worse?
Each prompt version and its score is saved with complete_task.
- Saved answer
- Decision
- Saved decision
- Recall
$ arcroom decide "keep tone line v14: plain, no apologies unless at fault" --why "+9% on 30 tickets; v15 'warmer' read as fake"
Step by step
- Each prompt version and its score is saved with complete_task.
- Wording that made things worse is saved with remember_failure and why.
- The final choice is recorded with arcroom decide "…" --why …, so later sessions don't undo it.
- Next week, recall brings the kept version and the ruled-out ones before I start.
What the agent sees
$ arcroom decide "keep tone line v14: plain, no apologies unless at fault" --why "+9% on 30 tickets; v15 'warmer' read as fake"
Next session, recall: v14 kept · v15, v17 ruled outCompared with today's tools
A prompt version spreadsheet
Keep every version, score and note in a sheet and paste the best one into each session.
- Setup
- 30 min
- Ongoing
- Fill the sheet after each try, ~5 min
ArcRoom
Every agent reads the team playbook before it works and writes to it when it finishes. Nothing to update by hand.
- Setup
- 2 min (npx arcroom init)
- Ongoing
- None
A sheet is easy to share and sort; the agent doesn't read it on its own, so ruled-out wording comes back.
Where it stops
ArcRoom records what was tried and decided; it doesn't run the tests or score the prompts. You still need your own eval.
Try it in your project
Works with the agents your team already uses, and any agent over MCP or the HTTP API.