Dataset quirks every agent should know

Data, ML and researchData team with an old event table full of surprises

Our events table has traps: user_id 0 is load-test traffic, timestamps before March 2024 are in IST not UTC, and 'purchase' fires twice on the Android app. New analysts and their agents fell into each one, and two wrong numbers reached a weekly report.

How do I make every AI agent know the quirks in our old events table?

Add each quirk once as a rule on ArcRoom's Rules page, or let an agent propose it with propose_rule for a person to accept. Every agent at the desk gets the rules at the start of each session, in Claude Code, Codex or Cursor. When a quirk is fixed upstream, retire the rule on the Rules page and agents stop applying it.

Also asked as: New analysts' agents keep counting load-test traffic in our numbers. How do we stop it? · How do I teach AI agents the known traps in a dataset? · Can an AI agent remember that old timestamps are in a different time zone?

The work

Each quirk is added once as a rule on the Rules page, or proposed by an agent and accepted by a person.

ArcRoom keeps
    The next agent gets

    Session start · rules for analytics (3)

    Step by step

    1. Each quirk is added once as a rule on the Rules page, or proposed by an agent and accepted by a person.
    2. Every agent at the desk gets the rules at the start of each session.
    3. When a quirk is fixed upstream, someone retires the rule so agents stop applying it.

    What the agent sees

    Session start · rules for analytics (3)
    • events: user_id 0 is load-test traffic, exclude it
    • events: ts before 2024-03-01 is IST, not UTC
    • events: Android sends 'purchase' twice, dedupe on order_id

    Compared with today's tools

    A data dictionary page + CLAUDE.md

    List the quirks in a data dictionary and copy the important ones into CLAUDE.md.

    Setup
    1 hour
    Ongoing
    keep both in sync, ~10 min per new quirk

    ArcRoom

    Every agent reads the team playbook before it works and writes to it when it finishes. Nothing to update by hand.

    Setup
    2 min (npx arcroom init)
    Ongoing
    None

    CLAUDE.md is read by some agents and not others; ArcRoom's rules reach every supported agent, and a person approves each change.

    Where it stops

    Rules go to every session at the desk, so keep them short. Someone has to retire a rule when its quirk is fixed.

    Try it in your project

    Works with the agents your team already uses, and any agent over MCP or the HTTP API.

    More in data, ml and research