A retrieval-augmented assistant has one promise: it answers from your documents, not from the open weights of a language model. Ask it something in scope and it retrieves the relevant passages, and the model writes an answer from them. The failure mode is quiet: the retrieval misses, or the model reaches past the passages, and a fluent, confident, unsupported sentence goes out with the same tone as a correct one. On a site that speaks for a company, that is the risk that matters.
Our own assistant already does the disciplined things — a similarity threshold that makes it refuse when nothing relevant is retrieved, the context framed as untrusted data, a system prompt that says answer only from the context. Those are necessary and not sufficient: they constrain the prompt, they don't verify the output. So we asked a separate, calibrated model to check the output, and we tested it on ourselves before trusting it.
The check: one calibrated question per answer
After the assistant drafts an answer, a System One model (TypeSafe's Jev) reads the retrieved context, the question, and the answer, and returns a calibrated probability for a single yes/no: is every claim in the answer supported by the context — no numbers, names, or facts that go beyond it?1 It doesn't rewrite anything or generate prose; it returns a number your code can gate on. That is the whole point of a decision model as a guardrail: the judgment is a typed value, not another paragraph you'd have to trust.
Testing it on ourselves
To measure it you need answers whose groundedness you already know. So we built a labelled set from our own site content: chunk the pages into passages, and for each one generate a faithful answer (grounded), the same answer with a single fabricated specific slipped in — a made-up number, name, or capability (a planted hallucination, the realistic failure), and the faithful answer paired with the wrong passage (a retrieval miss). 132 cases in all, a third of each.
The separation is not subtle. A grounded answer scores around 0.83; slip one fabricated detail into it and the score collapses to about 0.09; hand it a passage that doesn't match the question and it's 0.03. Across the set, the score ranks grounded above ungrounded at an AUC of 0.995 — and, unlike the credit and toxicity experiments we ran earlier, it was well-calibrated out of the box (ECE 0.07, no recalibration needed). Groundedness is a crisp entailment judgment; the model doesn't have to guess a base rate, it has to check a containment.
As a gate
A number this separable makes a clean gate: refuse (or flag) any answer scoring below a threshold. At a threshold of 0.3 the gate caught 97% of ungrounded answers — 93% of the subtle single-fact hallucinations and 100% of the wrong-context ones — while wrongly refusing 0% of the genuinely grounded answers. There is a wide, forgiving operating window between those two curves.
Shipping it without a new way to fail
A guardrail that can take down the thing it guards is not a guardrail. So the check is wired in fail-open: if the call errors, times out, has no key, or the feature is switched off, it returns “couldn't check” and the answer is served exactly as it is today. Enabling the guardrail cannot make the assistant less available than it was without it — the worst case is that it silently stops adding protection, which is precisely the behaviour you want when a dependency is down or out of credits. On the blocking endpoint an ungrounded answer is replaced with an honest “I can't confirm that from our sources”; on the streaming endpoint, where the words have already gone out, it appends a caveat.
It's also cheap enough not to think about: one small typed call per answer. The honest caveats are worth stating too — our planted hallucinations are synthetic and the corpus is one site's content, so the exact numbers are a controlled measurement, not a field benchmark. But the shape is clear enough to run in production behind a flag, and that is where it now is.
The general lesson is the one we keep finding: the interesting engineering in an AI product is at the boundary where the model's output becomes something a reader trusts. Making that boundary check itself — with a calibrated judgment your code can act on, that fails safe when it can't — is the difference between a demo and a system you'd put your name on. That is what we mean, at BIS, by decisions engineered.
Built in the open — with TypeSafe
This is an independent experiment on TypeSafe's System One model, Jev1 — not sponsored, and not affiliated. We ran it because a model trained for calibrated judgments is a genuinely new kind of building block, and the most honest way to test it was to point it at our own product. If you're on the TypeSafe team, or building on System One, we'd be glad to compare notes.