Last year a client showed me a Slack message from their shiny new AI assistant: "Churn in the enterprise segment rose to 4.2% in May." Confident tone, one decimal place, delivered in 3 seconds. The actual figure was 2.9%. Nobody caught it for two weeks, because the number looked right.
That message is why every AI analytics agent we build at Satyadata starts from one design principle: it must be built to be checked, not believed. Not "more accurate". Not "smarter". Checkable.
An agent that is right 95% of the time but unverifiable is worse than useless - because the 5% arrives with the same confident tone as everything else. An agent that is verifiable can be wrong occasionally and still be trusted, the same way a junior analyst who shows their work can be trusted. Verifiability, not accuracy, is the product.
Rule 1: The model never generates a number
This is the non-negotiable one. In our agents, the language model is banned from producing figures out of its own weights. Every number in every answer must be the output of a SQL query that ran against the client's warehouse - a query that is logged, linked and inspectable by whoever received the answer.
The model's job is to write the query, interpret the result and explain it. It is a translator between business questions and the warehouse, never a source of facts. The 4.2%-churn Slack message could not have happened under this rule - not because the model got smarter, but because the architecture removed its ability to improvise.
Practically, every answer the agent sends carries its receipts:
Enterprise churn, May 2026: 2.9% (61 of 2,104 accounts)
source: fct_subscription_events, mart layer
query: [inspect] - 14 lines, ran 09:41 UTC
caveat: excludes accounts in migration (n=38)
A stakeholder who clicks "inspect" and reads 14 lines of SQL is not showing distrust. They are using the product as designed.
Rule 2: Take the guessing out of source selection
Left to their defaults, language models behave like the most dangerous kind of new hire: they commit to the first plausible table they find. Warehouse has a table called revenue_daily? Great, query it, answer delivered. That the finance team abandoned that table in 2024 is not in the model's training data.
So we take the guessing out of the process. Source selection follows a deterministic path wherever one exists: if a metric has a semantic-layer definition, that definition wins - no judgment call required. Where no rule can decide, the agent runs the checks a senior analyst would run before trusting a table: freshness, row counts, precedent in past analyses. Only then does it commit to a source. And if two candidate tables disagree beyond a small tolerance, the conflict gets surfaced rather than silently resolved.
This is the same structure behind the AI data analyst workflow I use myself: the accuracy gains never came from a better model, they came from encoding what an experienced analyst already knows to check. Deterministic where the answer is knowable, explicit context where it isn't - and the agent moves quickly through both, because it isn't guessing.
Design for verifiability, not for being right more often. A checkable agent that is occasionally wrong builds trust; an unverifiable one that is usually right destroys it.
Rule 3: Corrections become permanent memory
When an analyst tells the agent "no - active users excludes internal test accounts, we fixed this definition in March", that correction cannot live and die inside one chat session. It should be written to a persistent context file, reviewed by a human, and committed. From that day on, every session starts already knowing it.
This changes the economics of supervision. In month one, the team corrects the agent often. By month three, they are mostly correcting genuinely new things, because the old mistakes are structurally impossible to repeat. The agent is not learning in any mystical sense - it is accumulating a reviewed, versioned rulebook. Which, incidentally, is the metric-definitions documentation most teams never manage to write; we take the same "definitions as code" stance in our KPI framework work, and the agent simply inherits it.
Rule 4: The agent inherits the user's permissions
An AI analytics agent must never be a side door around your access model. If Maria from marketing cannot see salary tables, then the agent, when Maria asks it a question, cannot see salary tables either. Our agents run queries under the requesting user's credentials - same roles, same row-level security, same audit trail in the warehouse logs.
This sounds obvious and is routinely violated. The lazy implementation gives the agent one powerful service account, which means anyone who can type a question can effectively query anything. That is not an analytics tool; that is a data breach with a chat interface.
The five rules on one page
| Design rule | Failure it prevents |
|---|---|
| Model never generates a number | Hallucinated figures delivered with confidence |
| Every answer links to its query | Unverifiable claims, "trust me" analytics |
| Deterministic source selection | Committing to stale or wrong source tables |
| Corrections become permanent memory | Repeating known mistakes every session |
| Inherits user permissions | Chat interface becoming an access-control bypass |
An AI analytics agent at a fintech scale-up
A fintech scale-up came to us after pausing their first AI assistant pilot - the tool answered fast and wrong just often enough that the finance team banned it from anything touching reported numbers. Classic trust bankruptcy: one bad number in a leadership meeting and the whole initiative was radioactive.
We rebuilt it on the five rules over about eight weeks. The unglamorous majority of that time went into groundwork, not AI: consolidating metric definitions into the dbt semantic layer, wiring the permission pass-through, and seeding the memory file with the top 30 known pitfalls collected from the analysts ("settled volume is not the same as processed volume" and its 29 cousins).
The results were quieter than a demo and far more valuable. Within a quarter, the agent was handling roughly 60% of routine data questions - the "what was X last month, split by Y" traffic that used to eat a third of two analysts' time. Spot-check audits by the finance team ran monthly at first; every flagged discrepancy traced to a documented definition disagreement rather than a fabricated figure, which is exactly the failure mode you want. The most telling adoption signal: the CFO, the pilot's loudest skeptic, started using it - because for the first time he could click through and see where a number came from.
Turning that agent loose was only safe because the warehouse underneath was in order; if your models are a two-year-old jungle, start with an analytics engineering rescue before you point an agent at them.
Takeaway
Stop asking "how accurate is the agent?" and start asking "how would we catch it when it's wrong?". If the honest answer is "we wouldn't", you do not have a trust problem with AI - you have a design problem with your build.
Build agents that show their work, hesitate before committing, remember their corrections and respect your permissions. Trust follows architecture.
Need an analytics agent your CFO will actually use?
Satyadata designs and builds AI analytics agents on the verifiability rules above - grounded in your warehouse, your definitions and your permissions. Tell us where your last pilot stalled.
Start a conversation