Legal Tech and AI

Checking AI Call Summaries for Accuracy

A review routine for AI-written call summaries at law firms: the errors to look for, a daily check, sampling rules, corrections and accuracy tracking.

AssistCall research teamPublished Reviewed by Faruque Amin, VP of Sales and Marketing6 min read

On this page
  1. Why call summaries need a routine
  2. Summaries are already mainstream
  3. Errors to look for
  4. A daily review routine
  5. The checklist for every summary
  6. How much to check, and when to sample
  7. Who reviews, and the supervision duty
  8. Correcting the record
  9. Recording and consent basics
  10. Track accuracy over time
  11. Autonomy raises the bar
  12. Signs a summary needs a full listen
  13. Keep the review sustainable
  14. Feed errors back into the script
  15. Example: catching a date error
  16. Frequently asked questions
  17. References
Legal Tech and AI: Checking AI Call Summaries for AccuracyArticle coverLEGAL TECH AND AIChecking AI CallSummaries for AccuracyAva for LawyersInsightsLegal Tech and AI: Checking AI Call Summaries for AccuracyArticle coverLEGAL TECH AND AIChecking AI CallSummaries forAccuracyAva for Lawyers

Key takeaways

  • Summaries are a mainstream use: 72% of legal professionals who use generative AI use it to summarize documents[1].
  • Check names, dates, contact details, urgency and anything promised against the recording before relying on a summary.
  • Start by checking every call, then move to sampling once error rates are known, as ABA Formal Opinion 512 allows for tested tools[2].
  • Correct the summary, keep the original recording and transcript, and track error types over time.

Why call summaries need a routine

AI call summaries are convenient. Instead of listening to a ten-minute recording, staff read a short note with the caller's name, the reason for calling and the next step. The risk is that the note becomes the record. If the summary misspells the opposing party, gets a date wrong or misses that the caller said the hearing is tomorrow, the error travels into the conflict check, the calendar and the callback.

Machine-written notes fail differently from human notes. A person who did not catch a name usually writes a question mark. A summary tool may write a plausible name with full confidence. The review routine below is designed around that difference.

Summaries are already mainstream

Among legal professionals who use generative AI, the top uses are document review (74%), legal research (73%) and document summarization (72%)[1]. Call summaries are the same kind of task applied to conversations. Even purpose-built legal tools make mistakes: Stanford researchers found leading legal research tools produced incorrect information more than 17% of the time on their benchmark[3]. Call summarization is a different task, but the lesson carries over: test before you trust.

Top generative AI uses among legal professionals who use it, 2025
Top generative AI uses among legal professionals who use it, 2025Column chart. Document review: 74%; Legal research: 73%; Document summarization: 72%.0%20%40%60%80%Document review: 74%74%Document reviewLegal research: 73%73%Legal researchDocument summarization: 72%72%Document summarizationTop generative AI uses among legal professionals who use it, 2025Column chart. Document review: 74%; Legal research: 73%; Document summarization: 72%.0%20%40%60%80%Document review: 74%74%DocumentreviewLegal research: 73%73%LegalresearchDocument summarization: 72%72%Documentsummarizat-ion

Source: Thomson Reuters Institute, 2025[1]

Show the data
ItemValue
Document review74%
Legal research73%
Document summarization72%

Errors to look for

The errors most likely to cause harm in intake.
Error typeWhat it looks likeHow to catch it
Name errorsA surname spelled phonetically, or two people merged into oneCompare against the recording; ask callers to spell names
Date errors"Next Tuesday" turned into the wrong calendar dateCheck every date against the recording and the call date
Contact errorsA digit dropped from a phone numberConfirm numbers by calling or texting back
Missing urgencyA hearing tomorrow buried at the end of the noteSearch the transcript for time words: today, tomorrow, deadline, court
Certainty drift"I think he ran the light" becomes "he ran the light"Read the transcript for hedges around key facts
Invented detailA fact that appears in the summary but not in the callTrace each key fact to a line in the transcript
Wrong speakerA statement by a relative attributed to the clientCheck who was speaking in the recording
PromisesThe summary misses that a callback time was offeredCheck the end of the call for commitments

A daily review routine

  1. Sort by risk

    Start with calls flagged urgent, calls about deadlines or court dates, and new inquiries. Routine messages come last.

  2. Check the fixed fields

    For each call, check name spellings, phone and email, practice area, other parties and any dates against the recording or transcript.

  3. Read for meaning

    Read the transcript around the key facts and confirm the summary did not strengthen, soften or invent anything.

  4. Check commitments

    Listen to the last minute of the call for anything promised: a callback, a time, a document.

  5. Correct and mark

    Correct the summary, mark it reviewed with your initials and the time, and note the error type.

  6. Act

    Run the conflict check, make the callback or escalate as needed, using the corrected record.

The checklist for every summary

  • Caller's full name, spelled correctly.
  • Best phone number and email, confirmed.
  • Practice area and a one-line description in the caller's own terms.
  • Names of opposing or other parties for the conflict check.
  • Every date mentioned, converted correctly to a calendar date.
  • Urgency: any court date, deadline, arrest, safety issue or time limit.
  • Whether the caller already has a lawyer.
  • What the caller was told and anything promised.

How much to check, and when to sample

ABA Formal Opinion 512 offers a practical model. It says the right amount of independent verification depends on the tool and the task, and gives the example of a lawyer who need not manually review every document a tool summarized if the lawyer previously tested the tool on a smaller subset and found the summaries accurate[2]. For call summaries, that suggests a staged approach:

  • First 30 days: check every summary against the recording or transcript, and log every error.
  • After 30 days: if errors are rare and minor, check every urgent and new-inquiry call fully, and a set sample of routine calls.
  • Always: check fully any call that will drive a deadline, a conflict decision or a call to a court or opponent.
  • After any product update or script change: go back to full checking for two weeks.

Who reviews, and the supervision duty

Name the role responsible for the daily review and a backup. A summary tool is not a person, but the people using and supervising it are covered by the firm's duties. ABA Model Rule 5.3 requires lawyers with managerial or direct supervisory authority to make reasonable efforts to ensure nonlawyer conduct is compatible with the lawyer's professional obligations[4]. In practice, that means a lawyer reviews the error log each month and decides whether the sampling rate and the tool are still appropriate.

Correcting the record

When a summary is wrong, correct the summary and mark the correction, but never edit the underlying recording or transcript. The original is the evidence of what was said; the summary is the firm's working note. If a correction changes something important, such as a date that drives a deadline or a name used in a conflict check, re-run the dependent step and note that you did.

Summaries usually depend on recordings. Federal law allows a party to a call to record it, or a recording where one party consents, unless the purpose is criminal or tortious, but some state laws are stricter[5]. Disclose recording at the start of every call, keep recordings only as long as your retention policy allows, and restrict who can listen to them.

Track accuracy over time

  • Share of summaries with any correction, by week.
  • Share with a material correction: a wrong name, date, number or missed urgency.
  • Most common error types, so you can fix scripts or ask callers to spell names.
  • Time spent reviewing per call, to check the routine stays sustainable.
  • Corrections found after the callback, which show the review missed something.

Autonomy raises the bar

The more a system does on its own, the more its records matter. California's State Bar names autonomously facilitating client intake as an example of agentic AI and says lawyer diligence and supervision must correspond to the system's level of access and autonomy[6]. If an AI system not only summarizes calls but also books consultations or routes urgent matters, its summaries are the record of decisions it made, and the review should check those decisions as well as the notes.

Signs a summary needs a full listen

  • The summary is much shorter than the call length would suggest.
  • It contains a deadline, court date, arrest or safety concern.
  • Names look phonetic or unusual, or two people share a surname.
  • The caller sounded upset, confused or was speaking through someone else.
  • The transcript shows long gaps, crosstalk or poor audio.
  • The summary uses firm, certain language about disputed facts.

Keep the review sustainable

A review routine only works if it fits into the day. Set a fixed time each morning, sort calls by risk so the important ones are checked first, and give the reviewer the recording and transcript side by side with the summary. If the review regularly takes longer than the time it saves, look at why: poor audio, a script that produces vague answers, or a tool that is not good enough for your calls. Fixing the source is usually better than adding more checking.

Feed errors back into the script

Most recurring summary errors start with the conversation, not the summary. If dates are often wrong, ask callers for calendar dates. If names are misspelled, ask callers to spell surnames. If urgency is missed, add a direct question about upcoming court dates or deadlines. Review the error log monthly and change one thing at a time, so you can see which change helped.

Example: catching a date error

Example (fictional): a caller to a family law firm says her custody hearing is "this coming Thursday". The call is on a Friday, and the summary records the hearing as the next day. During the morning review, the paralegal checks the date against the recording and the call date, corrects it to the following Thursday and marks it urgent. The error log shows three relative-date errors that month, so the firm changes its intake script to ask callers for the calendar date of any hearing.

Frequently asked questions

Are AI call summaries reliable enough to use without checking?

Not at first. Check every summary until you know the tool's error rate, then sample, and always fully check calls that drive deadlines or conflicts.

What are the most common errors in AI call summaries?

Misspelled names, wrong dates from relative phrases, dropped digits in phone numbers, missed urgency and facts stated more firmly than the caller said them.

Should I edit the transcript when it is wrong?

No. Correct and mark the summary, and keep the original recording and transcript unchanged.

Who should review call summaries?

A named staff member each day, with a lawyer reviewing the error log monthly as part of supervision[4].

References

  1. Thomson Reuters Institute (2025). 2025 GenAI report: Executive summary for legal professionals (opens in a new tab) โ†‘

  2. American Bar Association, Standing Committee on Ethics and Professional Responsibility (2024). Formal Opinion 512: Generative Artificial Intelligence Tools (archived copy) (opens in a new tab) โ†‘

  3. Stanford Institute for Human-Centered Artificial Intelligence (HAI) (2024). AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries (opens in a new tab) โ†‘

  4. American Bar Association (2026). Rule 5.3: Responsibilities Regarding Nonlawyer Assistance, ABA Model Rules of Professional Conduct (archived copy) (opens in a new tab) โ†‘

  5. Legal Information Institute, Cornell Law School (2025). 18 U.S. Code ยง 2511 - Interception and disclosure of wire, oral, or electronic communications prohibited (opens in a new tab) โ†‘

  6. The State Bar of California, Standing Committee on Professional Responsibility and Conduct (2026). Practical Guidance for the Use of Generative Artificial Intelligence in the Practice of Law (opens in a new tab) โ†‘

Book a demo

See Ava handle your calls.

Book a short demo. Weโ€™ll walk through your calls and workflow, and agree what Ava should take care of.

Book a demo