Start here
This document says what the system must do. It does not say how to build it. Two implementations can satisfy this list with completely different stacks, and that is the point.
Every function carries an identifier. The conformance test suite certifies against these identifiers. The class modules produce them as their output. The reference architecture and each jurisdiction profile map their components back to them.
The full explanation of this document type, including how it is versioned and published, sits on About functional agendas.
Scope boundary
The classifier's job ends when the code and the confidence are returned. What happens next belongs to other systems with their own agendas.
What the classifier does not do
- Decide what a person should do next once the issue is known. That is triage.
- Screen a person for eligibility for a service.
- Route a person to a specific organization. That is referral routing.
- Give legal advice or characterize the strength of a claim.
The reference taxonomy is LIST, maintained at taxonomy.legal. LIST runs five levels deep, and the current public export holds 1,124 codes: 20 at the top level, 207 at the second, 437 at the third, 331 at the fourth, and 133 at the fifth.
Depth matters for every requirement here, so three words are used consistently. Top-level category means the first level, one of the 20 codes ending in four zero pairs, such as HO for Housing. Specific code means the deepest code the input actually supports, which for a short input may be the second level and for a detailed narrative may be the fifth. Path means every code from the top-level category down to the specific code. A system that returns HO-01-05-00-00 also returns HO-01-00-00-00 and HO-00-00-00-00 as part of the path, with trailing zeros always written out.
Principles
The task areas group under five principles. Each principle names something the system has to be, and the task areas under it say how.
| Principle | What it means | Task areas |
|---|---|---|
| Accurate | The system places an input at the right point in the taxonomy, at the top-level category and at the specific code. | Input handling, taxonomy mapping, multiple issues in one input |
| Honest | The system reports how sure it is, says when nothing in the taxonomy fits, and asks a question when the input is too thin to place. | Confidence and uncertainty, clarifying questions |
| Safe | Urgency and danger in an input are never lost inside a routine code. | Urgency and safety signals |
| Reproducible | The same input against the same taxonomy version produces the same output, and any classification can be traced back and checked. | Output format and traceability |
| Interoperable | Output uses shared codes and declares its taxonomy version, so another organization's system can read it. | Taxonomy maintenance and versioning |
Function list
Choose the scope that matches the system you are building or evaluating, then filter to the levels you care about. The link, the downloads, and the print view all carry the current selection.
How to read a capability
Each capability states one thing the system does, using MUST, SHOULD, or MAY with the meanings given in BCP 14. The level shown beside it says whether that capability is required at the scope tier you selected.
Each capability also carries an evidence tier. Validated means tested and published on JusticeBench. Observed means seen working or failing in a real build, without a published measurement. Proposed means written from practitioner need and not yet tested.
Almost everything in this draft is Proposed. No accuracy evaluation of a LIST classifier has been run and published, which is also why every threshold in the companion conformance standard is blank.
Step one
What kind of deployment are you building or evaluating?
How to read the levels, and how to cite a function
| Level | Keyword | What it requires |
|---|---|---|
| Required | MUST |
The system does this in every deployment at the scope tier you selected. A deployment that misses a required function has a failure mode this field has already documented. |
| Critical | MUST |
A required function where failure harms a specific person or makes the output impossible to audit. Every critical function is enforced by a binary gate in the conformance standard, and a system that fails one does not deploy at any level. |
| Extended | SHOULD |
There may be good reasons to skip one of these, and the reasons should be written down. Most are what a mature deployment adds after its first year. |
| Aspirational | MAY |
Few systems do these yet. They mark the direction of travel, and they are the functions most likely to change in the next version. |
Identifiers
Each capability carries an identifier in the form ISC-<task area>.<number>, for example ISC-2.3. Identifiers do not move or get reused inside a version. A retired capability keeps its number and is marked retired rather than deleted. Cite an item with the version attached: ISC-2.3 (issue-classifier v0.1).
Scope tiers
A classifier declares which tier it is built for, and capabilities apply according to that tier. This agenda holds one catalog of capabilities, and each tier is a selection over that catalog. A capability marked as not applicable at a tier is one the tier does not ask for, such as path depth requirements at the category tier.
Evidence tiers
Every capability carries a tier. Validated means tested and published on JusticeBench. Observed means seen working or failing in a real build, without a published measurement. Proposed means written from practitioner need and not yet tested.
At version 0.1 almost everything here is Proposed. The Observed items come from an earlier Stanford build whose keyword-matching version returned codes that all existed in the taxonomy and were still wrong most of the time. That failure is the reason code validity and code accuracy are written as separate requirements with separate tests.
Open questions
These are unresolved and marked so nobody mistakes them for settled.
- Should the parent in a parent and specific code pair mean the top-level category, as written here, or the second level? The second level has 207 terms and is closer to how legal help websites organize their content. The top level has 20 and is closer to how funders and researchers count.
- Should ISC-2.3, the rule against descending below the depth the input supports, be Required or Critical? Over-specification is the failure most likely to send a person to the wrong page, and it is invisible to a scorer who only checks the top-level category.
- Should the jurisdiction-aware tier exist yet? No implementation does this today, and defining a tier nobody can meet may not be useful.
- Who annotates the test sets the conformance standard needs, and what does that cost?