Key takeaways
- Legal research tools produced incorrect information more than 17% of the time in Stanford benchmarks, and one more than 34%[1].
- Sanctions in Mata v. Avianca over fabricated citations remain the best-known warning[2].
- The duty to verify comes from competence: ABA guidance says AI cannot replace a lawyer's judgment and verification must fit the task[3][4].
- Verify every authority by reading it in full, and record who checked it.
What a hallucination looks like in legal work
A hallucination is output that sounds right and is wrong. In legal work it rarely looks like nonsense. It looks like a properly formatted citation to a case that does not exist, a real case cited for a proposition it does not support, a quotation that appears nowhere in the opinion, a pin cite to the wrong page, a statute section that was renumbered years ago, or a confident summary of a rule that was overturned. The danger is that each of these passes a quick read.
Hallucinations are not limited to research. Summaries of transcripts can invent a date. Intake notes can attribute a statement to the wrong person. Chronologies can merge two events. Any task where a tool generates text from other text can produce errors that look like facts.
How often it happens
Stanford RegLab and HAI researchers tested leading legal research tools on a benchmark of legal queries. Lexis+ AI and Ask Practical Law AI produced incorrect information more than 17% of the time, and Westlaw AI-Assisted Research hallucinated more than 34% of the time[1]. An earlier Stanford study found general-purpose chatbots hallucinated between 58% and 82% of the time on legal queries[5]. These are benchmark results, not a measure of your firm's experience, but they make one thing clear: purpose-built legal tools reduce the problem without removing it.
The sanction cases
Texas Professional Ethics Committee Opinion 705 cites Mata v. Avianca (S.D.N.Y. 2023), where lawyers were sanctioned for a brief citing non-existent opinions generated by an AI chatbot, as the most famous example of improper AI use[2]. Courts have since dealt with many similar filings. The pattern repeats: a lawyer under time pressure asks a tool for supporting authority, the tool supplies plausible citations, and nobody reads the cases before filing. The tool is rarely the whole story. The failure is the missing check.
Where the duty to verify comes from
No special AI rule is needed to create the duty. Comment 8 to ABA Model Rule 1.1 says that to maintain competence a lawyer should keep abreast of changes in the law and its practice, including the benefits and risks associated with relevant technology[4]. ABA Formal Opinion 512 applies that to generative AI: relying on a tool's output without an appropriate degree of independent verification or review could violate the duty of competence, and the tools cannot replace the judgment and experience lawyers need to advise clients or craft legal documents[3].
State guidance agrees. Florida Bar Opinion 24-1 says lawyers using generative AI must provide accurate and competent services[6], and Texas Opinion 705 addresses the same issues under the Texas rules[7]. Opinion 512 also makes the level of verification depend on the tool and the task, so a lawyer who has tested a tool on a narrow task may rely on lighter checks for that task, while research for a filing always needs full review[3].
What to check, and who checks it
| Output type | Check | Who | When |
|---|---|---|---|
| Case citation | Exists; read in full; supports the proposition; still good law | Drafting lawyer | Before the draft goes to the supervising lawyer |
| Quotation | Appears word for word at the cited page | Drafting lawyer or paralegal | Before review |
| Statute or rule | Current version, correct section, correct jurisdiction | Drafting lawyer | Before review |
| Summary of a document | Dates, amounts and names match the source | Person who requested the summary | Before it is relied on |
| Chronology | Each entry traced to a source page | Paralegal, then lawyer | Before use in a filing or letter |
| Intake or call notes | Facts match the recording or transcript | Intake reviewer | Same day |
A verification workflow for every authority
Confirm it exists
Pull the authority from an authoritative database or official source using the citation. If you cannot find it, treat it as fabricated.
Read it in full
Read the opinion, not the headnote or a summary. Many errors are real cases cited for something they do not say.
Match the proposition
Write one sentence stating what the authority holds and compare it with how the draft uses it.
Check quotations and pin cites
Find every quoted phrase at the cited page. Fix or remove any that do not match.
Check later history
Confirm the authority has not been reversed, overruled or superseded, and that a statute has not been amended.
Check jurisdiction and date
Make sure the authority binds or persuades in your court and fits the time period of your facts.
Record the check
Note who verified each authority and when, in the file or the draft's comments.
Hallucinations outside research
Firms that focus only on citations miss the quieter errors. A transcript summary that shifts a date by a day can change a limitation analysis. An intake summary that records "the other driver ran the light" when the caller said "I think the other driver ran the light" changes the strength of a claim. For these tasks, verify against the source material: the recording, the transcript or the document. Spot-check a sample each week even when output has been reliable, because tools change and error rates drift.
Firm-level controls
- A written rule that no authority suggested by an AI tool is cited until a lawyer has read it in full.
- A list of approved tools, with the tasks each may be used for.
- A short verification record for every filing that used AI assistance.
- Supervisors who ask "who checked the citations?" as a routine question in review.
- Realistic deadlines. Most fabricated citations reach courts when a draft was finished late at night.
- Training that shows staff real examples of plausible but fabricated citations.
If a fabricated citation has already been filed
Act quickly and in the open. Tell the supervising lawyer at once. Verify every other authority in the filing, because one error suggests others. Correct the record with the court promptly, considering your duty of candor to the tribunal, and tell the client what happened and what you have done. Notify your malpractice insurer if the policy requires it. Then fix the process gap that let it through, and write down what changed.
Build verification into the timetable
Verification takes time, and the time has to come from somewhere. If a brief is due at 5 p.m., the citation check cannot start at 4:30. Put a verification step into internal deadlines for every filing: the draft is complete a set number of hours before the filing deadline, and a named person signs off that every authority was read. For short deadlines, cut the argument rather than the check. A shorter brief with verified authority is always better than a longer one with a fabricated case.
Choose tools that make checking easier
- Prefer tools that link every citation to the source, so checking starts with one click rather than a search.
- Prefer tools that show which passage of a document a summary relied on.
- Avoid tools that cannot say where an answer came from.
- Test any new tool on a research question you already know the answer to before using it on live work.
- Re-test after major product updates, since behavior can change.
Example: a near miss in a motion response
Example (fictional): an associate at a five-lawyer firm uses a research assistant tool to find authority on a discovery issue at 9 p.m. The tool returns six cases. Following the firm's workflow, she pulls each from the official reporter. Four are real and on point, one is real but holds the opposite, and one does not exist. She removes both, adds a line to the verification record, and flags the fabricated case to the partner. The firm adds that example to its training session, because a fabricated case with a realistic citation is the most convincing lesson there is.
Teach staff the warning signs
- A case that perfectly supports your argument and that nobody in the firm has heard of.
- Citations to reporters or volumes that do not match the court or year.
- Quotations that read smoothly but cannot be found by searching the opinion.
- Summaries with round, confident statements where the source was hedged.
- Authority from the wrong jurisdiction presented as binding.
Frequently asked questions
Do legal research AI tools still hallucinate?
Yes. Stanford benchmarks found leading legal tools produced incorrect information more than 17% of the time, and one more than 34%[1].
Is there a rule that requires lawyers to verify AI output?
The duty comes from competence. ABA Formal Opinion 512 says relying on AI output without appropriate independent verification could violate Model Rule 1.1[3].
Can a lawyer rely on AI summaries without reading the source?
Opinion 512 suggests lighter checks may be reasonable once a tool has been tested on that task and found accurate[3]. For key facts and anything filed, check the source.
What should I do if I filed a fake citation?
Tell your supervisor, check every other authority, correct the record with the court promptly, inform the client and notify your insurer if required.
References
Stanford Institute for Human-Centered Artificial Intelligence (HAI) (2024). AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries (opens in a new tab) โ
Texas Center for Legal Ethics (Professional Ethics Committee for the State Bar of Texas) (2025). Opinion 705 (opens in a new tab) โ
American Bar Association, Standing Committee on Ethics and Professional Responsibility (2024). Formal Opinion 512: Generative Artificial Intelligence Tools (archived copy) (opens in a new tab) โ
American Bar Association (2025). Comment on Rule 1.1: Competence, ABA Model Rules of Professional Conduct (archived copy) (opens in a new tab) โ
Stanford Institute for Human-Centered Artificial Intelligence (HAI) (2024). AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries (opens in a new tab) โ
The Florida Bar (2024). Florida Bar Ethics Opinion 24-1 (opens in a new tab) โ
Texas Center for Legal Ethics (Professional Ethics Committee for the State Bar of Texas) (2025). Opinion 705 (opens in a new tab) โ