Dataset quirks every agent should know
Data, ML and researchData team with an old event table full of surprises
Our events table has traps: user_id 0 is load-test traffic, timestamps before March 2024 are in IST not UTC, and 'purchase' fires twice on the Android app. New analysts and their agents fell into each one, and two wrong numbers reached a weekly report.
How do I make every AI agent know the quirks in our old events table?
Add each quirk once as a rule on ArcRoom's Rules page, or let an agent propose it with propose_rule for a person to accept. Every agent at the desk gets the rules at the start of each session, in Claude Code, Codex or Cursor. When a quirk is fixed upstream, retire the rule on the Rules page and agents stop applying it.
Also asked as: New analysts' agents keep counting load-test traffic in our numbers. How do we stop it? · How do I teach AI agents the known traps in a dataset? · Can an AI agent remember that old timestamps are in a different time zone?
Each quirk is added once as a rule on the Rules page, or proposed by an agent and accepted by a person.
Session start · rules for analytics (3)
Step by step
- Each quirk is added once as a rule on the Rules page, or proposed by an agent and accepted by a person.
- Every agent at the desk gets the rules at the start of each session.
- When a quirk is fixed upstream, someone retires the rule so agents stop applying it.
What the agent sees
Session start · rules for analytics (3)
• events: user_id 0 is load-test traffic, exclude it
• events: ts before 2024-03-01 is IST, not UTC
• events: Android sends 'purchase' twice, dedupe on order_idCompared with today's tools
A data dictionary page + CLAUDE.md
List the quirks in a data dictionary and copy the important ones into CLAUDE.md.
- Setup
- 1 hour
- Ongoing
- keep both in sync, ~10 min per new quirk
ArcRoom
Every agent reads the team playbook before it works and writes to it when it finishes. Nothing to update by hand.
- Setup
- 2 min (npx arcroom init)
- Ongoing
- None
CLAUDE.md is read by some agents and not others; ArcRoom's rules reach every supported agent, and a person approves each change.
Where it stops
Rules go to every session at the desk, so keep them short. Someone has to retire a rule when its quirk is fixed.
Try it in your project
Works with the agents your team already uses, and any agent over MCP or the HTTP API.