Varra Assurance
Do not just trust the answer. See what supports it — and see, just as plainly, where nothing does yet.
Why this page exists
Almost everything an AI product claims about its own accuracy is unfalsifiable. A benchmark you cannot re-run, a percentage with no denominator, a promise that the system “reduces hallucinations”. This page is the opposite bet: describe exactly how Varra checks an answer, state how much ground those checks cover today, and name the part they do not cover, which is most of it, so the claim is specific enough to judge rather than take on trust.
Varra Assurance is a scoped mechanism in the infrastructure beneath Varra, not a guarantee. It checks stated conditions against evidence and records what passed. It does not decide what is allowed: verification does not grant permission, and an Assurance result is not an approval. This page is the evidence behind the name — the mechanism, the evidence it uses, what it covers, its limits, and where its synchronization boundaries are.
Two mechanisms
- Reference substitution — public in Varra. Where a verified table covers the exact figure an answer turns on, the table’s entry replaces the model’s text and the answer says where it came from. Described in full below.
- Deterministic run checks — in the engine, API only. A pure evaluator runs declared checks against supplied evidence, each piece of evidence pinned by identity, revision and digest, and records an immutable verdict: which checks passed, which failed, and for how long the result is valid. The record-only runtime step requires a current passing verdict. It has no interface in Varra. A digest shows the evidence has not changed since it was supplied; it does not show the evidence is true.
Reference-backed verification
Where Varra holds a checked reference value for the exact figure an answer turns on, that value replaces what the model wrote, caveat included, and the answer says where it came from. In the app it carries a BUILT-IN TABLE badge. The value is the reference’s, not the model’s, and the reader can see which one answered.
How the verification actually works
It is one mechanism, and it is worth stating precisely because a vague description of a safety property is worth nothing.
- The figure is made addressable. The engine asks the model to put the exact value an answer turns on in a structured field, rather than leaving it buried in prose. A number nobody can locate is a number nobody can check.
- The value is reduced to a canonical key. Different ways of writing the same value have to reach the same row, or the table silently stops matching and the answer looks identical either way.
- A match replaces the model's text. If the key names a row, the table's entry — caveat included, verbatim — replaces what the model wrote, and the answer is stamped with where it came from. In the app that shows as a BUILT-IN TABLE badge under the answer.
- A miss is logged rather than swallowed. A mode that has a table but stopped matching it would otherwise fail invisibly. That has happened before, which is why the log line exists.
Current coverage, stated plainly
This is the number that matters, so it is not buried:
Most Varra answers today are not independently verified. They are the model's own reasoning, and Varra says so rather than implying otherwise. Verified reference data is a narrow slice of a broad idea, and this page does not dress it up as more.
Those counts are Varra’s built-in reference data. What an account can add for itself is a separate thing, and it is the next section.
Your values, not just ours
Live in the product today: in any mode you can add your own verified reference values from the account menu. Yours take precedence over the built-in table, which takes precedence over whatever the model remembered. When your entry is what answered, the reply is stamped as coming from your account rather than from the built-in reference.
This is the smallest complete version of the whole argument — a model produces an answer, a trusted source overrides it, and the answer says which source won. A business that has checked a figure against its own equipment, its suppliers or the rules it works under knows something a general table cannot, and this is the one place Varra already lets it say so.
What it is not: it is account-level reference data, entered deliberately, one value at a time. It is not a system that has read your documents, learned your vendors, or worked out how your business runs — that is the company context layer, and it is planned rather than built. Your entries are not published here, so they widen what Varra will defer to for you without widening what a stranger can check.
Known limitations
- Coverage is thin. Thirteen of the sixteen modes have no verified reference data at all. In those, every figure is the model's own.
- The tables are conservative, not universal. Each entry carries the conditions it depends on, and none of them replaces the code or standard that governs the work.
- Verification runs on values, not on reasoning. A checked figure inside an otherwise unchecked argument is exactly that. Varra can tell you a figure came from a table; it cannot yet tell you the argument around it was sound.
- Nothing here verifies against your documents. Checking an answer against a contract, a schedule of values or a company policy needs the company context layer, which is planned rather than built.
From a verified value to a verified piece of work
Everything above verifies a value. A piece of work needs more than that: which documents were read, which number came out of which one, what could not be found, where two sources disagree, what a person changed, and what state the whole thing was approved in.
Most of that is not built. In the product there is no workflow provenance, no record of evidence assembled across documents, and no reviewer trail. The groundwork in the engine is the run verdict above, which records which evidence each check used, and the exact-action approval record, which records what a person approved. The verdict is API only, and so is the approval record, except for an email that an eligible account approves in Varra before it is sent. What a Varra user has today is the substitution described above and the account-level values in the section before it.
Verification is what execution waits on
Verification matters more, not less, once a system can act, and the engine keeps the two apart. The record-only runtime step will not advance a run without a current passing verdict. That verdict is still not permission: an action needs a separate, exact approval from its owner, and Relay acts only on that approval. In Varra, the one thing that executes is an approved plain-text email sent through Gmail, for eligible accounts; nothing else executes.
Company policy, risk scoring and organizational permissions are not built. Checking those before a consequential action is the direction, not the present.
Why a published value cannot drift
Any reference value this site publishes has to agree with the engine’s own source, varra/modes.py. A checker compares the two, and a page carrying a reference table that is not registered with the checker fails it, so a table cannot be published unchecked by forgetting a step. The site does not publish the reference tables themselves today; the check stays in place for when it does.
The point is not that the checker is clever. It is that a value here disagreeing with the engine is a build failure rather than something a reader has to catch.
Synchronization boundaries
The checker compares registered published tables with the engine only when such tables exist. With none published today, it checks that no page carries an unregistered table and makes no comparison of values against the engine. When it does compare, it proves one thing: the published page matches the engine. It does not prove the engine’s value is correct. Synchronized reference data is not automatically independently proven correct — the tables are conservative values gathered by hand and caveated, and they are as good as that work. Values an account adds for itself are that account’s own; Varra defers to them and does not verify them. An Assurance result is only ever as strong as the evidence it was given.
What comes next
Widening what can be checked — from values against reference tables, to arithmetic and consistency, to a claim against its source document — is part of the development direction. The order is deliberate: execution stays behind verification and approval, because a system that can act is only as safe as its ability to know when it should not.