Key takeaways
- Pilot one line, one set of hours and a short list of call types; widen scope only after call reviews come back clean.
- Write the success measures before day one: answer rate, booked consultations, escalation accuracy and caller complaints.
- Bar guidance expects supervision that matches the tool's autonomy, and Florida requires AI chat tools to say they are not lawyers[1][2].
- Only a fifth of legal professionals measure generative AI return on investment, so a pilot with a baseline already puts you ahead[3].
Why pilot before you switch
A phone pilot is a controlled test with a decision at the end. It is not a soft launch. The difference matters because the risks at a law firm are specific: a caller who hears something that sounds like advice, a conflict that is not caught, a recorded call in a state that requires everyone's consent, or an urgent matter that sits in a queue overnight. A pilot lets you find those problems on a small slice of calls while staff still cover everything else.
The case for testing at all is easy to make. When Clio secret shoppers contacted 500 US law firms in 2024, only 40% answered the phone, down from 56% in 2019, and 48% neither answered nor called back[4]. Callers also screen: 80% of US adults say they generally do not answer calls from unknown numbers, so a missed inbound call is often hard to recover with a callback[5].
Source: Illinois Supreme Court Commission on Professionalism (2Civility), 2024[4]
Show the data
| Item | Value |
|---|---|
| 2019 | 56% |
| 2024 | 40% |
Choose a narrow scope
Scope is the main lever for controlling risk. Pick one option below and write it into the pilot charter so nobody quietly expands it in week two.
| Scope option | What the AI handles | Best when | Watch for |
|---|---|---|---|
| After-hours only | Calls from close of business to opening, plus weekends | Daytime staff answer well but evenings go to voicemail | Urgent matters such as arrests, protective orders and deadlines need a live escalation path |
| Overflow only | Calls that ring past a set count or arrive while every line is busy | Peaks after marketing pushes or on Monday mornings | Callers bounced between two systems; set one ring threshold and keep it |
| One practice area line | All calls to a single tracked number, such as the estate planning line | You want clean data on one intake script | Callers who dial the wrong line for another practice area |
| New callers only | Unknown numbers, with existing clients routed to staff | Your main risk is missed new inquiries | Existing clients calling from a new number |
Settle these with the vendor first
A pilot that stalls in week three usually stalls on a question nobody asked at the start. Get written answers to these before any live call:
- Can we change the script, urgent triggers and banned phrases ourselves, and how fast do changes take effect?
- Where do recordings, transcripts and summaries live, who at the vendor can see them, and can we set retention?
- Is our call content used to train shared models, and can that be switched off in the contract?
- How does a live transfer work, and what happens when nobody picks up?
- Can we export every call record from the pilot if we decide not to continue?
- What reports show answer times, outcomes and transfers without manual counting?
If the vendor cannot answer the data questions clearly, stop there. The supervising attorney needs those answers to judge confidentiality risk before any client information reaches the system.
Set the measures before the first call
Pull four weeks of baseline data from your phone system before the pilot starts. Without a baseline, every result becomes an argument. Thomson Reuters found that only 20% of legal professionals are measuring the return on investment of generative AI[3], so even a basic before-and-after comparison is more than most firms have.
- Answer rate: in-scope calls answered by a person or the AI within your target time, divided by all in-scope calls.
- Booked or requested consultations per hundred in-scope calls.
- Escalation accuracy: urgent calls that reached the right person within your target time, checked by reviewing every flagged call.
- Intake completeness: summaries that include name, callback number, matter type, other parties and how the caller heard of you.
- Caller complaints, early hang-ups and requests to speak to a person.
- Staff time spent fixing or redoing intake after the AI handled a call.
A six-week pilot plan
Week 0: charter and sign-off
Write a one-page charter covering scope, hours, call types, escalation contacts, success thresholds and the supervising attorney. California's bar guidance says diligence and supervision should match a system's access and autonomy, and names autonomous client intake as an example, so name a lawyer who owns the pilot[1].
Week 0: script and guardrails
Approve the greeting, the questions in order, the urgent-matter triggers and the phrases the AI must never use. Include a conflict prompt that collects the names of all parties and a question about whether the caller already has a lawyer.
Week 1: shadow mode
Run the AI on a test number or internal calls only. Staff play callers using the hardest scenarios from your own call history: angry callers, Spanish speakers, people who already have a lawyer, and people asking whether they have a case.
Weeks 2 and 3: live on a narrow scope
Turn on the chosen scope. Review every call summary daily and log each error by category: wrong routing, missed question, advice-like statement, wrong booking or tone.
Week 4: fix and retest
Change the script for every error category that occurred more than once, then rerun the shadow scenarios before going live again.
Weeks 5 and 6: measure and decide
Compare results with the baseline, read a random sample of calls, ask staff what changed in their day, and write the decision memo.
Greeting and disclosure wording
Callers should know early that they are speaking with an automated assistant. Florida Bar Opinion 24-1 says AI chatbots that talk with clients or third parties must include a disclaimer that the program is not a lawyer or an employee of the firm[2], and many firms apply the same standard to voice. Public attitudes point the same way: half of US adults say they are more concerned than excited about the growing use of AI in daily life[6].
Example greeting: "Thanks for calling Harlow and Price. I'm the firm's automated intake assistant, not a lawyer, and I can't give legal advice. I can take your details and book a consultation, or connect you with our team. Is this about a new matter or an existing one?"
If calls are recorded, put your recording notice before any questions about the matter, and check the consent rules for every state your callers are in, not only your own.
How to review calls during the pilot
- Read every summary for the first two weeks, then a random quarter of calls plus every flagged call.
- Listen to any call where the caller asked "should I" or "do I have a case" to confirm the AI declined to advise.
- Check that urgent triggers fired: arrests, court dates within days, injuries needing care, protective orders and deadlines the caller mentions.
- Compare each booked slot with the attorney calendar and the conflict check.
- Tag tone problems separately from accuracy problems; they need different fixes.
- Record the staff time spent on each correction, because that cost belongs in the decision.
Example: a five-lawyer injury firm
Example: a fictional five-lawyer personal injury firm in Ohio pilots AI answering on after-hours calls only. Its baseline shows about 140 after-hours calls a month, most reaching voicemail and roughly a third leaving a message. During the pilot the AI answers all of them, books 22 consultations and flags 9 calls as urgent, routing 7 of those correctly. The two misses were callers who described a hospital stay without using the word injury. The firm adds hospital, emergency room and ambulance to the urgent triggers, reruns the shadow tests and extends the pilot by two weeks before deciding.
Prepare staff before launch, not after
Thomson Reuters found that 64% of professionals had received no training on generative AI for their industry work[7]. In a phone pilot, staff need three skills: reading AI summaries critically, taking over a live call cleanly, and logging errors in the same categories every time. A short drill on handoffs before launch saves hours of cleanup later.
Tell staff what the pilot is for. If they think it is a step toward replacing them, they will either ignore errors or report every small one. Frame it as covering the calls nobody is answering today, and ask them to help write the urgent triggers, since they know the calls best.
Go, adjust or stop
| Result after six weeks | Decision | Next step |
|---|---|---|
| Answer rate and booked consultations up, no missed urgent calls, staff cleanup time falling | Go | Widen scope one step, for example from after-hours to overflow, and keep weekly reviews |
| Bookings up but urgent routing or intake completeness below threshold | Adjust | Fix the script, extend the pilot two to four weeks and retest the failed scenarios |
| An advice-like statement repeated after a fix, or complaints above baseline | Stop or pause | Return to staff coverage, document the failure and revisit the vendor and configuration |
| Results flat against baseline | Stop | The problem may be marketing or follow-up rather than answering |
If you are testing an SI Employee such as Ava, use the same charter: it answers new inquiries, follows the intake you approve, routes urgent calls and hands staff a summary, so each of those can be scored against the baseline.
For the wider return picture, see measuring the ROI of AI intake tools. Before going live, read what AI phone tools should never do, and if many of your callers speak Spanish, plan for AI on Spanish-language intake calls.
Frequently asked questions
How long should an AI phone pilot run?
Six weeks is a practical minimum: one week of shadow testing, two to three weeks live on a narrow scope, a week to fix and retest, and time to measure. Extend it if urgent routing is not yet reliable.
Should we tell callers they are talking to AI?
Yes. Say it in the greeting, along with the fact that the assistant is not a lawyer and cannot give legal advice. Florida already requires this kind of disclaimer for AI chatbots, and it is the safer default for voice.
Who should supervise the pilot?
A named attorney who reviews flagged calls and signs off on script changes, supported by the intake lead who reads the daily summaries.
What if the pilot fails?
Write down why, return to your previous coverage and keep the baseline data. A clear stop is a useful result because it shows whether the problem is answering, the script or the leads themselves.
References
The State Bar of California, Standing Committee on Professional Responsibility and Conduct (2026). Practical Guidance for the Use of Generative Artificial Intelligence in the Practice of Law (opens in a new tab) โ
The Florida Bar (2024). Florida Bar Ethics Opinion 24-1 (opens in a new tab) โ
Thomson Reuters Institute (2025). 2025 GenAI report: Executive summary for legal professionals (opens in a new tab) โ
Illinois Supreme Court Commission on Professionalism (2Civility) (2024). Phone Calls Ignored, Emails Unanswered: Fixing the First Impression Problem for Law Firms (opens in a new tab) โ
Pew Research Center (2020). Most Americans don't answer cellphone calls from unknown numbers (opens in a new tab) โ
Pew Research Center (2025). How Americans View AI and Its Impact on Human Abilities, Society (opens in a new tab) โ
Thomson Reuters (2025). Generative AI Adoption Nearly Doubles as Professional Services Reach Crossroads (opens in a new tab) โ