MedLegal AI Hallucination Transparency Report (Q2 2026)

By John Mahoney · Published May 28, 2026 · 14 min read

This is the kind of document plaintiff attorneys ask us for and almost no AI legal-tech vendor publishes. We are publishing it anyway, because the alternative — opaque AI tooling making evidentiary claims about your client's medical records — is incompatible with what plaintiff-side practice actually requires.

This report covers how MedLegal AI controls hallucinations in our medical records review and Daubert-workup tooling. It is intentionally honest about what we have measured, what we have not yet measured, and where the technology can still fail. If you are evaluating us against a defense-side competitor or a generic ChatGPT workflow, this is the document to read first.

This report is updated quarterly. The next update is scheduled for August 2026. If you find a hallucination in any MedLegal AI output, email [email protected] with the case ID and we will publish the analysis in the next quarterly update.

Why this report exists

In January 2024, Stanford's RegLab and Institute for Human-Centered AI published the most rigorous public study to date on LLM hallucination in legal practice. Their finding, after benchmarking GPT-4, PaLM 2, and LLaMA 2 across legal queries: general-purpose large language models hallucinate on between 58% and 82% of legal questions when used without retrieval-augmented grounding (Dahl, Magesh, Suzgun, Ho 2024).

A subsequent Stanford study in May 2024 tested two purpose-built legal AI products (Lexis+ AI and the then-Westlaw Thomson Reuters AI assistant) and found that even retrieval-augmented legal AI tools hallucinated on 17% to 33% of queries. The conclusion: "hallucination-free" claims by AI legal vendors at that time were not supported by independent evidence.

Two years later, the underlying models are dramatically better and the retrieval architectures have matured. But the core obligation has not changed: a plaintiff attorney using AI on medical records is making evidentiary representations to a court. If our tool fabricates a fact, the consequences are the attorney's. So the standard we hold ourselves to is not "low hallucination rate." The standard is "every factual claim in our output traces back to a citable page in the record you uploaded."

What we mean by "hallucination" in this report

The term "hallucination" gets used loosely. For purposes of this report we distinguish four failure modes and treat them very differently:

Most "AI is wrong" reports conflate these. This report does not. Each failure mode requires a different mitigation, and our architecture and audit process target them differently.

Our architecture: how citation grounding actually works

MedLegal AI is not a wrapper around ChatGPT. The records-review pipeline has four stages, each of which is designed to make fabrication structurally impossible (not just statistically rare):

Stage 1: Bates-anchored chunking

When a medical record PDF is uploaded, we extract text page-by-page and store each page with three identifiers: the source filename, the original page number, and the Bates number if one is stamped. Every later stage works on these chunks. The model never sees a "blob" of the record — it always sees explicitly numbered, addressable pages.

Stage 2: Retrieval, not memory

For each question (chronology, deviation flag, deposition outline question), the system retrieves the relevant pages from your record using semantic and keyword search. The model is then asked to answer the question using only those retrieved pages, with each retrieved page provided in-context with its Bates anchor.

This means the model is not free-associating from training data. It is summarizing what is in front of it. Fabrication in this stage requires the model to invent a fact about a page that is actually visible to it — a failure mode that is detectable.

Stage 3: Citation extraction and verification

The model is prompted to return every factual claim with an inline citation tag of the form [Bates 0001234] or [Smith.pdf p. 12]. Before the response is returned to the user, an automated post-processor checks every citation tag against the actual page content. Citations that do not resolve are flagged, not silently dropped.

Stage 4: Confidence-tiered presentation

The UI presents factual claims, interpretive conclusions, and inferred deviations in different visual tiers. A factual claim has a clickable citation that opens the source page in a viewer. An interpretive conclusion is labeled as such and requires user confirmation before it appears in any downstream artifact (chronology, deposition outline, expert prep).

What we measure internally

Our internal evaluation suite consists of three components:

Adversarial fact-injection tests

We maintain a private test corpus of medical record packets where we have hand-introduced specific facts the model should miss (e.g., a buried lab value in a 600-page record) and specific traps the model should not invent (e.g., a question about a medication that is not in the record). We re-run this corpus weekly. The pass criterion is that the model must not assert the trap facts as present. We are publishing our pass rate as part of this report and committing to update it quarterly.

Current results (May 2026 run, N=147 trap items):

The two failures both fell in the date-shift and provider-misattribution categories. Both are categorized as citation drift, not fabrication — the underlying facts existed in the record but were mis-attributed by one day and one provider, respectively. Both have been added to our active regression suite.

Citation-resolution audits

For every records-review run a customer executes, our system automatically samples 10% of inline citations and re-verifies them against the source pages. If a citation does not resolve to a page that supports the cited fact, it is logged for review. Across the rolling 30-day window ending May 27, 2026 (N = 12,400 sampled citations across 184 cases), the citation-resolution rate was 99.2%. The 0.8% of failures were investigated; 81% were citation drift (correct fact, wrong page), 19% were OCR-quality issues where the cited page exists but the text extraction was incomplete.

Customer-reported errors

Every customer-facing output has a "Flag error" button. When a user flags an item, we log it, investigate, and tag it as fabrication, drift, overreach, or omission. In the last 90 days we received 47 flags across 184 active cases. Distribution: 0 fabrication, 11 drift, 19 overreach, 17 omission. We publish the full triage spreadsheet (with case IDs redacted) to any customer who requests it.

Known limitations — what we have not solved

Handwritten records

Our OCR works well on typed records and structured EMR exports. Handwritten progress notes, particularly from older paper records, still have an OCR error rate around 8–14% per page depending on legibility. We surface OCR confidence at the page level, but a low-confidence transcription can still propagate into downstream summaries before a reviewer catches it.

Image-only PDFs and faxes

PDFs that are scans of scans of faxes have the worst quality in this workflow. We apply pre-processing (deskew, denoise, contrast normalization) but cannot recover information that is not in the image.

Clinical inference at scale

The model can identify a documented standard-of-care deviation (e.g., "ECG ordered 90 minutes after triage despite ACS rule-out protocol calling for ECG within 10 minutes") with high reliability when the deviation is explicit in the record. It is much less reliable when the deviation requires synthesis across multiple specialties or temporal reasoning across a long course of care. This is an active area of development; for now, the system flags such cases as "requires expert review" rather than asserting a deviation.

Jurisdiction-specific legal framing

Our Daubert-workup and deposition-outline tools generate skeletons that are grounded in the post-2023 amended FRE 702 framework. State-court applications under Frye, Mohan, or hybrid frameworks require attorney localization — the tool will produce a federal-shaped draft and a Frye-shaped draft on request but does not auto-detect the state-specific nuances.

Open call for independent audit

We will provide, at no cost and under NDA, our adversarial test corpus and citation-audit harness to any independent academic researcher, plaintiff-side technology committee, or AAJ section interested in running an external evaluation. If a reviewer finds a failure mode we have not documented, we will publish the finding in the next quarterly update and add it to our regression suite.

To request the audit packet, email [email protected] with the subject "Hallucination audit request" and a brief description of your institutional affiliation and intended scope. We commit to responding within 5 business days.

References

1. Dahl, M., Magesh, V., Suzgun, M., & Ho, D. E. (2024). Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models. Journal of Legal Analysis, 16(1).
2. Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C. D., & Ho, D. E. (2024). Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. Stanford RegLab Working Paper.
3. Federal Rule of Evidence 702 (as amended December 1, 2023).
4. MedLegal AI internal adversarial test corpus, May 2026 release notes (available on request under NDA).

Bottom line

If you want a vendor who will tell you they have zero hallucinations, we are not that vendor. If you want a vendor who will show you the architecture, publish the failure modes, and invite you to audit them, that is what this report is. Plaintiff practice deserves both honesty and competence from the AI it relies on. We are committed to delivering both, and to being publicly accountable when we fall short.