Can You Use the Citation Verifier, and Build Your Own?

In the cohort's evaluations a model asked to judge answers passed almost everything, and a short deterministic check caught the failures that mattered. That check is a small Python script that runs beside the judge. This report covers what it does, how it scores an answer, what you need to run it on your own content, and the failures it cannot see.

About the cohort. We ran a cohort of legal help website teams for 18 months, working together on common approaches to AI for legal help online. Where these reports say “the cohort,” that is who they mean.

In the cohort's first full evaluation, the LLM judge passed every answer. All eighty-eight cells scored a perfect one. The deterministic verifier was the entire source of differentiation. It caught hallucinated links, phone numbers attributed to the wrong organization, and citations to resources that do not serve the right state. None of those failures lowered the judge's confidence, and every one of them is the kind of error that sends a person to a dead end or a scam. A confident wrong citation sends a person somewhere that does not exist, and the model grading the model could not see it.

What the verifier is

The citation verifier is a deterministic Python check that runs on every AI answer. It looks for three kinds of factual claims and checks whether each one is real and correctly attributed: web links, organization names, and phone numbers. It runs alongside the LLM judge in promptfoo rather than in place of it. The judge scores the prose, the reasoning, and the tone. The verifier scores whether the answer's facts are true. Together they tell a fuller story than either tells alone.

It works against an allowlist: a list of the real URLs, domains, and organization names drawn from your own content index and service directory. For each link in an answer, the verifier sorts it into one of a few buckets: a resource in your knowledge base, a real resource not in it, a hallucinated address, or a broken link. A hallucinated or broken link fails the check. Organization names are checked against the directory, and phone numbers are checked for misattribution, the case where a real organization is paired with a number that is not its own.

The allowlist is one JSON file with four fields. urls holds the exact addresses in your content index. domains holds the hosts you publish from, so a link to a page the index has not caught yet still counts as yours. org_names holds the organizations in your service directory. org_phones maps each organization to the numbers that belong to it. A team that keeps a directory already holds all four, usually in a spreadsheet.

How it scores an answer

The verifier pulls every link and every phone number out of the answer with a pattern match, then sorts each link into one of four buckets. A link in your directory scores a full point. A link that resolves but is not in your directory scores seven tenths, because it is real and you have not vouched for it. A link that is not in your directory, when liveness is not being checked, scores half. A link that does not resolve, or that was invented, scores nothing.

The link scores are averaged, and then each phone number attributed to the wrong organization takes three tenths off the total. An answer that cites nothing at all scores a full point on links, which is worth knowing when you read the numbers: an empty answer is not a good answer, and the judge is the part that notices that.

The check passes when the score clears four tenths and no link is broken or invented. A single invented link fails the answer outright, whatever else it got right. That rule is deliberate, because the failure it catches is the one that sends a person to a dead end.

By default the verifier makes no network calls and checks links against your allowlist alone, which keeps a run fast and repeatable. Setting an environment variable turns on a liveness check, so each link is fetched and a dead link is separated from a link you simply have not indexed. The cohort ran the offline mode for scoring and the online mode when it wanted a list of stale links to fix.

What it cannot do

The verifier checks whether a cited thing exists and is attributed correctly. It does not check whether the legal advice is sound. That is the judge's job, and the human reviewer's. The verifier is only as good as the allowlist behind it: if your directory does not carry the right phone number, the verifier cannot catch a wrong one. It reduces dangerous hallucinations. It does not certify that an answer is correct.

Three limits are worth knowing before you read its numbers. Links and phone numbers are found by pattern matching, so a link written in prose without the protocol, or a number written in words, slips past it. Organization names are matched as plain text inside the answer, so a short name that is also an ordinary phrase will match more often than it should. The phone check compares the numbers in an answer against the numbers known for each organization the answer names, so an answer that names two organizations and gives one number can flag the second organization even though nothing is wrong. Read a flagged answer before you count it.

Can you use it

Yes. It is a short script, ready to pick up, and it runs in promptfoo, which is open source and forks without depending on anyone else to keep it running. You wire the verifier in as a second assertion next to the judge, so each answer gets two scores. Each evaluation round the cohort ran this way cost under twenty-five dollars.

How to build your own

The verifier is small enough that a team can build its own, and the steps are the same whether you pick up the cohort's version or write it fresh. Start by exporting your service directory to a CSV with a column for the web address, a column for the organization name, and, where you have it, a column for the phone number. The builder script takes that file and the names of those three columns and writes the allowlist, so nothing has to be reshaped by hand. Then add the verifier to your promptfoo configuration as a Python assertion sitting next to the judge rubric, point an environment variable at your allowlist, and run the evaluation. Each answer comes back with a prose score and a groundedness score side by side. The pattern to watch for is the dangerous one: a high judge score with a low verifier score, which is an answer that sounded convincing and cited something that does not exist.

The cohort's version is four small files: the verifier, the allowlist builder, a sample allowlist to run it against, and an example promptfoo configuration showing the judge and the verifier wired together. It is released under the MIT license. A team with engineering capacity can read the whole thing in an afternoon and adapt it to its own directory.

Write to legaldesignlab@law.stanford.edu for a copy while the public home is being set up.

Why this is worth doing

The verifier costs almost nothing to run and carries the signal that the model judge misses. Any team evaluating a brief-help tool, before a procurement decision or before a deployment, can run it over a set of real questions and see, in a number, how often the tool invents a citation. That number is the one a director should ask a vendor for, and a refusal to be measured on it is its own kind of answer.

The verifier sits in the Commons catalog of common infrastructure tools, alongside the other pieces a team should not have to build alone. The testing record shows what it caught across the cohort's rounds, and the measurement report covers the harness it runs inside.