dbt questions answered once

Data, ML and researchAnalytics engineers working in a dbt project with agents

Our dbt project has 340 models. Questions like 'why is fct_orders incremental?' or 'where does net_revenue get its refunds from?' came up every week from different people. Each time an agent traced the lineage again through a dozen SQL and YAML files.

How can our team stop agents from re-tracing the same dbt model lineage every week?

Have the agent check ArcRoom's find_task first, trace the lineage once, and save the answer with complete_task, tied to the git hashes of the model files it read. When a colleague's agent asks the same thing in other words, find_task returns it as a reuse match. If one of those models changed, the answer is marked stale and traced again. Warehouse changes are not detected.

Also asked as: Why does every agent trace dbt lineage from scratch for the same question? · How do I share answers about dbt models across analytics engineers? · Can an AI agent know when a saved dbt answer is out of date?

The work

The first time, the agent checks find_task, traces the lineage, and saves the answer with complete_task.

ArcRoom keeps
  • Reused answer
  • Saved answer
  • Freshness check
The next agent gets

find_task → match: reuse (0.88), saved by ana (9 days ago)

Step by step

  1. The first time, the agent checks find_task, traces the lineage, and saves the answer with complete_task.
  2. The answer is stored with the git hashes of the model files it read.
  3. When a colleague's agent asks the same thing in other words, find_task returns a reuse match.
  4. If one of those models changed, the answer is marked stale and the agent traces it again.

What the agent sees

find_task → match: reuse (0.88), saved by ana (9 days ago)
"net_revenue: gross from stg_payments minus refunds from stg_refunds, joined on payment_id, in int_revenue.sql"
files unchanged since saved

Compared with today's tools

dbt docs (generated lineage and descriptions)

Run dbt docs generate and have people read the lineage graph and model descriptions before asking.

Setup
1 hour
Ongoing
keep model descriptions current, ~10 min per model change

ArcRoom

Every agent reads the team playbook before it works and writes to it when it finishes. Nothing to update by hand.

Setup
2 min (npx arcroom init)
Ongoing
None

dbt docs are better for the lineage graph itself; ArcRoom keeps the agent's written answers to the 'why' questions and knows when they went stale.

Where it stops

Freshness covers files in git. A change in the warehouse itself, like a new upstream column, is not detected.

Try it in your project

Works with the agents your team already uses, and any agent over MCP or the HTTP API.

More in data, ml and research