How do you bring an AI tool into a small practice safely?
Give the tool one narrow job, get six answers from the vendor in writing, and name the person who checks its work. Write down the number it should move, then check that number at 30 and 90 days.
By Jake Schwartz, CEO & Founder
Give the tool one narrow job, get six answers from the vendor in writing, and name the person who checks everything it produces. Before you turn it on, write down the one number it should move, then check that number at 30 and 90 days.
Most practices skipped those steps. In an August 2025 MGMA Stat poll, 71% of practice leaders said they use some AI in patient visits. In MGMA's January 2026 poll, 56% of medical group leaders said they had no AI governance or formal AI policy.
An independent practice has no IT department or compliance office to do the vetting, so the owner or manager runs this checklist. You are the institution.
What job should your first AI tool do?
Pick one job with a clear finish line. "Check codes before the claim goes out" is a job. "Help with documentation" is too vague to judge at 90 days.
MGMA suggests a short trial with one goal, tracked before you expand. Its August 2025 poll also found that practices using AI in 26% or more of visits were more likely to report less staff workload, so use the tool on every visit its one job fits.
Prefer a tool that points over one that acts. A read-only tool can be wrong, and a person still catches the mistake before it reaches a claim. A tool that writes codes or visit levels on its own turns one bad rule into hundreds of bad claims. In 2024, UCHealth agreed to pay $23 million to resolve allegations that an automatic rule coded certain emergency visits at the top level based on how often vital signs were checked. Here's where regulators draw the line on coding suggestions.
What should you ask an AI vendor before you sign?
The AMA asked 1,692 physicians what they need before adopting an AI tool, and their answers make a good vendor interview. In the AMA's 2026 physician survey, about 7 in 10 already used at least one AI tool at work. 88% said it's important that a tool's safety and accuracy be checked by a trusted party and monitored over time. 86% wanted their practice and their EHR vendor to assure data privacy. A place to report problems, coverage under standard malpractice insurance, and a tool that fits into the EHR were also near the top.
"Other physicians I know are using it" ranked near the bottom, at 44%. A colleague can tell you a tool is easy to use, and usually can't tell you where your patients' data goes. Physicians were also far more worried about privacy with AI tools their own organization didn't provide (71%) than with tools it did (42%).
Email the vendor these six questions and keep the replies.
| Ask the vendor | A good answer sounds like | A red flag |
|---|---|---|
| How did you measure accuracy, and who checked it? | How many real charts, who decided the right answers, and published scores that keep updating after launch | One impressive percentage with no method behind it |
| What leaves my computer, where does it go, and how long is it kept? | A list of the fields sent, where they're processed, which AI company handles them, how long they're kept, and exactly which patient details are removed first | "It's all secure," or "de-identified" when only the name and record number come out |
| Will you sign a business associate agreement (BAA), and when does it take effect? | Yes, with a copy you can read and a start date | The tool handles patient information and the vendor won't sign one |
| Can the tool change the chart or send a claim? | It reads and points, and a person makes every change | It applies codes or visit levels on its own |
| Does it flag both ways? | It shows codes the note doesn't support and levels set too high, as readily as anything missing | It only adds codes or raises levels |
| How do I report a wrong answer? | A named place to send it, a reply time, and fixes that reach every user | A feedback button nobody answers |
Each red flag has a real case behind it.
An accuracy number needs a method. In 2024, Texas settled with Pieces Technologies, a health AI company that advertised a "severe hallucination rate" below 1 per 100,000. The state said those metrics were likely inaccurate. The strongest answer is a test on your own charts. Here's how to run one with last quarter's denials.
"De-identified" has a legal meaning. HIPAA's Safe Harbor method requires removing 18 kinds of identifiers, including every part of a date except the year, phone numbers, street addresses and any other unique number. A good vendor tells you exactly what it removes and what can still slip through.
A BAA is the written assurance HIPAA requires before a vendor handles protected health information for you. Under 45 CFR 164.504(e), it spells out how the vendor may use the data and requires it to safeguard the data, report breaches and bind its subcontractors to the same terms. It says nothing about whether the tool is accurate, so ask both questions.
One-way review costs money. In March 2026, Aetna agreed to pay $117.7 million to resolve allegations that it used chart reviews to add diagnosis codes while ignoring results that showed codes were unsupported.
Liability comes up too. When the AMA asked about regulation, physicians ranked clear liability rules as the step that would most increase their trust in AI. A tool that only points keeps every decision with your clinician, and your malpractice carrier can tell you how your policy treats AI-assisted care.
Who should check the AI's work?
Name one person who reviews every output before it touches a note or a claim. For a coding tool, that's usually the biller or the clinician who signs. The tool never signs the note. Your clinician does.
Bring the clinicians in early, since 85% of physicians in the AMA survey want a say in bringing AI into their practice. Only 55% of solo-practice physicians had any AI training, compared with 76% in hospitals, so give each user 30 minutes on what the tool does, what it skips, and what to do when it's wrong.
Keep one shared log of bad output with the date, what the tool said and what was wrong, and leave patient names out. Send it to the vendor and expect answers. Also write down what staff may never paste into a free consumer chatbot. MGMA's governance guidance puts patient information and identifiers on that list.
What should you measure before you turn it on?
Record a baseline first. MGMA reports that limited tracking makes it hard for many practices "to establish a productivity baseline, let alone measure change." Pull three numbers from the last 90 days:
- Your denial rate and your top three denial reasons
- Average days from visit to payment
- Notes still unsigned at 6 p.m., or another after-hours charting number you already track
Pick the one this tool should move, and write it down with today's value and the date.
The headline number may differ from the one you feel. A 2026 JAMA study of 8,581 clinicians linked AI scribes to 16 fewer minutes of documentation for every 8 hours of scheduled patient time, while EHR time after hours didn't change significantly. A practice that bought a scribe to get its evenings back should track evenings. More on what the 2026 scribe research found.
What do you check at 30 and 90 days?
At 30 days, check who's using the tool, how often, and what shows up most in your bad-output log. Fix the setup or the training while the trial is young.
At 90 days, compare your number to the baseline. If it moved, keep the tool. If it didn't, change how you use it or cancel. Put the 90-day date on the calendar the day you sign.
Ask us the same six questions. Cortex Lens has one narrow job. It checks codes and the problem list in Practice Fusion before the claim goes out. It's read-only, its flags go both ways, and a person makes every change. Before a chart is checked, Lens removes the patient's name, record numbers, phone numbers, emails, addresses, Social Security numbers and written dates in the browser. Our security page covers how it handles data, and our benchmarks page shows how we measure accuracy.
Sources: AI use and workload: MGMA Stat, August 2025. AI governance and chatbot guidance: MGMA Stat, January 2026. Baselines: MGMA, "Are AI tools making clinicians more productive? It's complicated". Physician requirements, privacy concern and training: AMA, 2026 Physician Survey on Augmented Intelligence, March 2026, pages 2, 4-5 and 10-13 (fielded January 15 to February 2, 2026; n=1,692). Accuracy claims: Texas Attorney General, Pieces Technologies settlement, September 18, 2024. HIPAA: 45 CFR 164.502(e), 164.504(e) and 164.514(b). Settlements: U.S. Department of Justice, Aetna, March 11, 2026; U.S. Attorney's Office, District of Colorado, UCHealth, November 2024. Scribe study: JAMA, April 2026.