Working draft. These reports are published while the work is still in progress, and we want comment on them. Write to legaldesignlab@law.stanford.edu.
Reports and Analysis

Legal Help Commons Reports

Foundation models are already answering the public’s legal questions. This is the work of getting them to do that well across the tasks people actually need, and the shared standards, evidence, and tools that let any state build on it instead of starting alone.

Reports

Reports to help legal teams, builders, funders, and evaluators with AI for legal help

Explore these reports to learn about AI strategy, development, and evaluation for legal help. Start with the first section or jump to the topic you need.

Start here

Planning and funding legal help AI

Justice professionals say where they are stuck on AI, these reports lay out the options for building and buying, and they make the case for investing in the shared layer.

Build, Buy, or Build Together?

The published article on how legal aid organizations are deciding to use AI, built on a survey of justice professionals. It reports what those teams weigh when they buy from a vendor, what they weigh when they build in-house, and how a commons model gives them shared infrastructure they still customize locally. Read the summary, the full text, or the PDF as it appeared in print.

What you’ll take away
  • What justice professionals weigh on the vendor path and on the in-house path, in their own words
  • What makes a commons model work, with an example already running in legal technology
  • Questions a leader can ask before the next vendor contract or in-house build begins
ArticleRead the article

How do you get a foundation model to do legal help well?

Owning or fine-tuning your own model does the least to change how it behaves. Grounding it in your content, writing the prompt, and what the model makers put inside all do more.

What you'll take away
  • What actually changes how a model behaves
  • Why owning or fine-tuning a model does the least for you
  • Where grounding and a safety wrapper fit for a small team
StrategyRead the report

What do we need national leadership on?

Some of this work outlasts any single state and does not belong to one. This report makes the case for national leadership on that shared layer. It says what the shared layer would own, why it has to stay neutral, and why funding it once costs less than fifty states each paying for it badly.

What you'll take away
  • The functions no single state can build alone
  • Why the standard, the tools, and the benchmark sit above any one state
  • The case to bring to a funder
LeadershipRead the report
Evaluation

Telling whether AI is any good

These reports show how to measure quality, what our measurements found, and what it would take to cover the whole task taxonomy.

How do you tell whether a legal AI tool is any good?

Set up promptfoo and a reusable protocol like the field has done, measure any tool against the tasks you care about, and gather your own evidence before you deploy.

What you'll take away
  • How to stand up your own evaluation with promptfoo, on the tasks you care about
  • The reusable protocol teams ran for under twenty-five dollars a round
  • Why deterministic checks carry the signal that an LLM judge misses
MethodRead the report

Does grounding the model in our own content make it better?

Across five rounds of testing, a verified list of your own resources lifted every model, while loading full articles helped only when they fit.

What you'll take away
  • What five rounds of testing showed about grounding in your own content
  • When a cheap grounded model beats a premium ungrounded one
  • When loading full articles helps, and when it backfires
EvidenceRead the report

When does grounding help a legal help answer, and when does it hurt?

The full record of the testing behind the grounding finding, across five rounds. It includes the round that came back a tie because the rubric could not see the difference, the round where the earlier result reversed on high-risk questions, and an account of how much weight each finding can carry.

What you’ll take away
  • Why a model judge alone shows a flat line where the differences are
  • When the full article text helps an answer and when it pulls a strong model off a better one
  • What each finding rests on, and what it would take to rely on it further
EvidenceRead the report

Can AI classify who a person is and what they need?

We ran 60 cases through Claude, Gemini, and GPT-4o, asking each to tag the legal issue, the audience, the jurisdiction, and the urgency from a person's own words. What made it work was putting our own category lists in the prompt.

What you'll take away
  • Putting your own category lists in the prompt is what makes intake classification work
  • The numbers across Claude, Gemini, and GPT-4o, and where each one breaks
  • How to run it as one component, with emergency detection and the not-a-legal-issue check kept separate
IntakeRead the report

How do we find out how good AI is across the task taxonomy?

Teams are measuring AI on legal help tasks and the results cannot be compared with each other. This sets out the choices, the partners, and the work to build a read the field can pool, and how the issue-area agendas and the task taxonomy fit together.

What you'll take away
  • What the field actually knows today, and why the measurements that exist cannot be pooled
  • How to build the read, whether through a central team, distributed contribution, or task stewards
  • Whether to do yearly check-ins, and how the issue and task slices fit together
CoverageRead the report
Tools

What you can use and build

Start with the inventory of what already exists, then get your own content ready for AI, then pick up the specific tools or rebuild them yourself.

What common tools did we build that every team can use?

You can pick up the shared vocabularies, the datasets to build and test with, and the working tools, each with an honest note on how ready it is.

What you'll take away
  • The vocabularies, datasets, and tools you can pick up now
  • An honest readiness note on each one
  • What to use instead of building it yourself
InventoryRead the report

How do you get your website and guides ready for AI?

This playbook turns your website, guides, and trainings into a knowledge base an AI can use safely. It covers gathering content and tiering it by authority, chunking it, building safety into the content itself, and growing from search to a knowledge graph.

What you'll take away
  • The journey from a content backlog to an AI-ready knowledge base
  • How to gather content by CMS export, tier it by authority, and chunk it for the question
  • How to build safety into the content itself, and when a knowledge graph is worth it
PlaybookRead the report

How do you work through each stage of getting content ready?

This is the step-by-step companion to the content playbook. It works through every stage in detail, giving the goal, the steps to take, the tools to use, a picture of what the output looks like, and the signs you have done it well.

What you'll take away
  • The whole journey from the playbook, worked through in detail
  • Every stage gives you the goal, the steps, the tools, and what the output looks like
  • The signs a stage is done, and the maintenance cycle that never ends
Full playbookRead the report

What does a legal help Q&A bot need to get right?

Many teams want to build one. This short note says what decides whether a Q&A bot answers safely, and it points to the reports that go deep on each part.

What you'll take away
  • The parts of a Q&A bot that outlast whichever model you start on
  • How to ground answers in verified content with citations
  • How to evaluate it before it ever reaches the public
Q&A noteRead the report

How can we automatically mask PII in documents?

We surveyed experts and built a labeled legal document set. This report says what counts as PII in legal records, how to judge whether a masking tool actually works, and how to test one without touching a real client's file.

What you'll take away
  • What counts as PII in legal records, in three tiers from an expert survey
  • The rubric for judging a masking tool, and the hard fails that override it
  • How to test a masker against a labeled document set without real client data
PIIRead the report

Can you use the citation verifier, and build your own?

A short deterministic check catches the hallucinated links and misattributed numbers an LLM judge waves through, and it costs almost nothing to run. This report shows how to run ours and how to build your own.

What you'll take away
  • What the citation verifier checks, and what it cannot
  • How to run it in your own evaluation alongside the model judge
  • How to build your own from an allowlist of your real resources
VerifierRead the report
Foundation

The shared groundwork and your next step

The tools rest on shared standards and on a way of running cohorts, and this is where a team decides what to do first.

Beyond these reports

The rest of our public work

The projects, standards, and tools the Commons has published, gathered here.