Every healthcare organization runs on measurements of its patients: which members have diabetes, which care gaps are open, how many patients smoke, how sick one hospital’s patients are compared with the hospital down the road. Nearly all of those measurements are computed from structured data, the fields software can read easily: diagnosis codes on claims, the problem list, lab results, medication orders. The rest of the record is unstructured: the notes, reports, and scanned documents that clinicians write and read, which is where most of what is known about a patient lives. Peer-reviewed studies that compare the two find that structured data alone misses almost 40% of documented diagnoses, two thirds of measured obesity, and 97% of documented suicidal ideation. All of that information is known and evidenced in the EHR, and the analytics never read it.

Why a missing code costs us

The damage from a measurement computed on half the record falls on three parties, and every figure in this section is documented with its source.

Patients

A patient whose lab results show chronic kidney disease but whose record carries no CKD diagnosis code is not in the CKD registry, does not trigger the nephrology referral, and is not on the list for the medications that slow the disease. In a Kaiser Permanente cohort below, that was 86% of the patients with the condition. A patient whose note says she has been thinking about suicide, but whose visit produced no matching code, is invisible to any prevention program that selects patients from codes; that was 97% of such patients in one primary care network. A smoker whose habit is recorded in the social history of every note and never coded is not offered cessation or lung cancer screening by any program that picks patients from claims.

Public health

Prevalence estimates built from claims report a fraction of the disease that exists. Medicare claims put current smokers at 2% when the survey of the same population put it at 10%, and the national inpatient dataset puts obesity at 15% while measured BMI puts it above 40%. Screening budgets, policy, and research built on those figures start from a number that is off by a factor of three.

Health plan finances

Health plans have known for two decades that quality rates computed from claims alone run about 20 points below the rates the chart supports, which is why they pay for an annual chart chase, and why Star ratings and pay-for-performance bonuses computed on the coded version understate the care that was delivered. Health systems in value-based contracts inherit the same gap, and Medicare Advantage plans carry it into risk adjustment audits, where the chart is the evidence and the code is the claim. In every one of these cases the information was in the record. It was missing from the part of the record the analytics read.

Analytics read the structured half of the record

The analytics stack that runs a health system, a payer, or a state Medicaid program is built almost entirely on structured fields, and everything downstream, from the HEDIS engine to the CMS-HCC risk model to the readmission dashboard, is SQL over those tables. That was a reasonable engineering choice when it was made: codes are standardized, computable, and cheap to query. In this piece, “codes” means the diagnosis and procedure codes that make up most of that structured data, and “the chart” means the complete record, structured and unstructured together.

The cost of reading only that half has been measured many times and rarely acted on. Veysel Kocaman, Dia Trambitas, and I built our AMIA Amplify 2026 session on regulatory-grade patient journey platforms around this evidence. A 2025 study in the Journal of Medical Internet Research took 1.8 million patients from a Dutch primary care database, extracted clinical concepts from the free-text notes, and checked how many had a structured counterpart in the same record. Thirteen percent did. The other 87% of what clinicians wrote about conditions, measurements, and drugs never became a code in that database. The studies below repeat that comparison one measure at a time, and every one of them finds the codes undercounting.

I’ve heard the anecdotal version at AMIA more than once: a state health data team whose claims show 5% of its members as smokers while the state’s own survey shows three times that. The peer-reviewed literature says that experience is typical.

Medicare claims found a fifth of the smokers surveys found

Smoking status is documented at nearly every encounter. It sits in the social history of every history and physical, on intake forms, and in nursing assessments, all of it unstructured text. It is also one of the facts least likely to reach structured data. A 2019 study in BMC Health Services Research compared tobacco diagnosis codes in Medicare claims against the CDC’s Behavioral Risk Factor Surveillance System for adults 65 and older. In 2001, claims identified 2.01% of beneficiaries as current smokers; BRFSS put the figure at 10.03%. By 2014, after a decade of quality programs and Meaningful Use incentives, the claims estimate had climbed to about 55% of the survey estimate, and the authors concluded that Medicare data still substantially underestimated tobacco use.

The sensitivity numbers explain why. A 2013 JAMIA study at Vanderbilt found ICD-9 tobacco codes had a specificity of 1.0 and a sensitivity of 0.32 in a general clinic population. The BMC authors note that this insensitivity is presumably why tobacco is left out of the standard claims-based comorbidity indices altogether. Smoking drives lung cancer screening eligibility, COPD and cardiovascular risk models, and the comorbidity adjustment behind outcome comparisons between hospitals, and a model that sees 2% smokers in a population where 10% smoke assigns the excess risk to something else.

The social history in the note holds the missing data, and that is the pattern this piece will repeat: the fact is documented, the code is absent, and reading the text recovers most of the gap.

Inpatient obesity is 42% by BMI and 15% by code

Obesity is the cleaner example because the reference standard is a number. Height and weight are stored as structured vitals inside the EHR, so BMI is computable there; the diagnosis code is what gets on the claim, and the code is what national datasets, payers, and most analytics see. A 2026 study in Obesity compared three national sources. The NHANES survey, which measures BMI directly, put adult obesity at 41.9%. NSQIP, which records BMI measured during a hospital stay, put inpatient obesity at 44.5%. The National Inpatient Sample, which relies on ICD-10 codes, put it at 15.4%. Any analysis that adjusts for obesity from claims, whether it is a hospital outcomes comparison, a GLP-1 utilization forecast, or a population health stratification, is adjusting for a third of the actual burden. This is a case where the evidence is already structured — but the code is missing anyway.

Diabetes codes miss a fifth and CKD codes miss half

Diabetes should be the best case for structured data, since it generates prescriptions, lab orders, and visits, all of which produce structured records. A 2013 meta-analysis in PLOS One of the standard claims definition still found it misses up to one fifth of cases, and inside the EHR the picture is worse. A 2014 study of 11.5 million US primary care records found that of 1,110,398 records with evidence of diagnosed diabetes, 61.9% contained a diagnosis code. Adam Wright’s 2015 study of ten health systems in three countries measured how often a patient with a diagnostic HbA1c had diabetes on the problem list: 60.2% at the lowest-performing site, 99.4% at the highest.

Chronic kidney disease is the extreme case, and the reference standard is a lab value. A Kaiser Permanente Georgia cohort of 10,266 patients with eGFR between 10 and 59 found 14.4% had a CKD diagnosis code, and a 2024 study of 60 English practices found 45.4% of eGFR-defined incident CKD in people with type 2 diabetes had a corresponding code. Every one of those patients had a creatinine result in the lab table. The code, which is what a registry, a risk model, or a care management queue keys on, was missing for half to six sevenths of them.

Administrative-only HEDIS rates run 20 points low

NCQA has known this for two decades — which is why hybrid measures exist. In 2007, authors from NCQA and the American College of Physicians published in the American Journal of Managed Care an analysis of 283 commercial plans reporting 15 HEDIS hybrid measures. Rates computed from administrative data alone were on average 20.4 percentage points lower than rates that added medical record review in 2004, and 20.6 points lower in 2006. For HbA1c testing and cholesterol screening, more than 60% of plans changed quartile rank once chart review was added. The authors concluded that administrative data alone did not provide sufficiently complete results for ranking plans on those measures.

That gap is the reason for the annual chart chase, in which plans draw samples of 411 records per measure and abstract them by hand. NCQA is now retiring hybrid reporting in favor of Electronic Clinical Data Systems, which add EHR, registry, and health information exchange feeds to claims. The move helps and does not close the gap. NCQA’s own October 2025 report on ECDS results found that for measurement year 2024, hybrid rates were still higher than ECDS rates by 3.4 percentage points for commercial plans and 5.5 for Medicaid plans. ECDS reads structured fields. Whatever the clinician wrote and never coded is invisible to it, and the plan pays for that twice: in outreach to members whose gaps were closed in a note that never produced a claim, and in Star ratings and incentive payments computed on a partial numerator.

Problem lists hold 62% of what the notes document

Pull back from individual conditions to the problem list, the structured summary of a patient’s diagnoses that most analytics treat as the truth, and the number in this article’s title appears. In 2021, Poulos, Zhu, and Shah at a London teaching hospital, one year after a full Epic implementation, manually reviewed the free-text notes of 516 patients with suspected or confirmed COVID-19. The patients’ problem lists held 2,841 diagnoses. Chart review found 1,722 more, raising the mean per patient from 5.51 to 8.84. Overall, 62.3% of diagnoses were on the problem list, and the remaining 37.7% existed only in the notes.

The pattern holds across the specific kinds of information that analytics programs depend on:

Sources: Anderson et al., J Am Board Fam Med, 2015; Guevara et al., npj Digital Medicine, 2024; CMS Office of Minority Health, 2021; JMIR Medical Informatics, 2022; Adejumo et al., JAMA Network Open, 2024; Jaffe et al., JAMA Network Open, 2026.

Each row is a decision that analytics built on structured data alone gets wrong: a suicide prevention program blind to 97% of documented ideation, a population health team stratifying on social risk it can see in 2% of members, a heart failure quality program with no computable functional class, a hereditary cancer program reaching under a third of eligible patients.

Structured data records billing behavior as much as disease

The reason these gaps are so consistent is that a code, the unit of most structured data, is produced at billing time, under coding rules, claim-line limits, and reimbursement incentives, by someone whose job is to bill the encounter correctly. It records that a coder chose to code something. A note records what a clinician observed.

Huo and colleagues, in Value in Health, showed how far the two can drift: over six years, the proportion of patients coded as current smokers in one claims database rose 2.3-fold and former smokers 4-fold, an increase the authors attribute to Meaningful Use requirements to record smoking status, and the sensitivity of any single ICD-9 smoking code stayed under 10% throughout.

For anyone computing a metric, the implication is simple. The presence of a code is decent evidence that a condition exists. The absence of a code is weak evidence that it doesn’t. The chart, with its notes, reports, and lab values, is the record of what was observed, and it is the reference standard every study in this piece used to grade the codes.

Ambient scribes grow the note faster than the codes

Ambient scribes make notes longer. In the Penn study of 46 clinicians published in JAMA Network Open in 2025, time in notes fell 20.4% per appointment and note length rose 20.6%, and a 2025 rapid review in JMIR AI listed reduced documentation time with longer notes as the first consistent theme across real-world evaluations.

The study that connects the longer note to the missing code is Castro and colleagues in JAMA Psychiatry, published in January 2026. They took 20,302 primary care annual-visit notes from Mass General and Brigham and Women’s, matched AI-scribed visits 1:1 to human-scribed, contemporaneous unscribed, and pre-deployment visits, and measured neuropsychiatric symptom documentation with a language model. AI-scribed notes documented significantly more symptoms in all six research domains. The odds that the visit produced a psychiatric intervention, defined as a referral, a new diagnosis code, or an antidepressant prescription, were lower: an adjusted odds ratio of 0.83 against contemporaneous unscribed visits, with human scribes at 0.97. More written down, less coded and less acted on, in the same visit.

The coding side moved too, in the direction the vendor’s coding module points. When Texas Oncology piloted DeepScribe with 49 physicians, billed diagnoses per encounter rose from 3.0 to 4.1, with the gain concentrated in non-cancer HCC codes. A December 2025 policy brief in npj Digital Medicine documents Cigna responding on October 1, 2025 by automatically reducing many mid- and high-level E/M claims by one level unless the documentation supports the code. The narrative grows on every axis because the microphone captures the whole conversation. The coded output grows only on the axis a billing module was built to find, and the payer now adjudicates that code against the note.

No published study has yet measured the share of documented conditions that reach a code before and after ambient adoption. Castro is the closest and it points the way you would expect. The record of what was observed is growing at 20% a note, and analytics that read only the codes are reading a shrinking fraction of it.

A complete measure reads every evidenced fact in the chart

Structured data stays in the measure. The change is to stop treating it as the census when it is a sample. A complete measure is computed over structured and unstructured data together: the codes, the labs, the vitals, and the facts extracted from notes and reports, each one traced to its source. That is what John Snow Labs means when it says a measure should use all the known, evidenced information about each patient.

Doing that at production scale has requirements the research prototypes above mostly did not have to meet:

• The extraction has to run at regulatory-grade accuracy on your documents, validated against clinician-annotated ground truth.

• It has to resolve assertion status, because “denies chest pain,” “family history of diabetes,” and “history of PE, resolved” all contain the term without the condition.

• It has to handle copied text and temporal context, because Wang and colleagues in JAMA Internal Medicine in 2017 found only 18% of the text in 23,630 inpatient progress notes was newly typed, so a fact repeated in twenty notes still happened once.

• It has to carry provenance and a confidence score with every fact, so a reviewer can click from the result to the sentence and a low-confidence finding can be routed to review instead of counted silently.

• It has to run inside your environment, because the input is the identified chart.

• The measure logic on top should stay deterministic: the model extracts and normalizes facts, and SQL decides whether the patient meets the numerator.

That is the design behind John Snow Labs’ clinical measure and HCC coding pipelines: medical language models extract and normalize the facts with evidence links, deterministic logic computes the measure, and reviewers see the source text behind every result. Where money or a care decision rides on the output, a credentialed coder or a clinician makes the call, with their decision recorded and audit-ready. The model widens what they can see while the decision stays with them.

Run one measure both ways this quarter

The fastest way to find out what your analytics are missing is to reproduce one of these studies on your own data. It takes one measure, one year of records, and one analyst:

1. Pick the measure that carries the most money or the most attention in your organization: the hybrid HEDIS measure with the worst rate, the smoking or obesity variable in your risk model, the social risk screen your population health team stratifies on.

2. Compute it from codes, the way you do today.

3. Compute it again from the full chart on the same population, with extraction validated against a reviewed sample.

4. Write down the difference. The literature says it will run from 20 percentage points on a quality measure to a threefold gap on smoking or obesity, and it will not shrink on its own: ambient scribes are adding to the notes at 20% a visit, and the codes follow only where a billing rule sends them.

5. Take that number to whoever owns the decision the metric informs: the quality committee, the CFO, or the population health lead.

The delta is the measurement error in every gap-closure list, Star rating projection, and cohort your organization built on the coded version, and it is the business case for computing the measure from everything the chart contains. If you run it and want to compare results, the comments are open. If you would rather have it run for you, John Snow Labs does exactly this inside your environment, on your own notes, with the evidence behind every extracted fact.

Frequently asked questions

Isn’t the missing information mostly undiagnosed disease that nobody wrote down?

No. Every study cited here used documentation that already existed in the record as the reference standard: a diagnostic HbA1c or eGFR in the lab table, a measured BMI, or a diagnosis written in a note. Undiagnosed disease is a separate problem with its own literature. The gap measured here is between what clinicians documented and what got coded.

Why not fix the coding instead of extracting from notes?

Coding does improve with incentives. Wright’s ten-site study found the best-performing sites reached 99.4% problem list completeness for diabetes using financial incentives, problem-oriented charting, gap reporting, and links to billing codes. The worst site was at 60.2% with the same tools available. Medicare smoking codes went from a fifth of the survey rate to about half over the thirteen years that included Meaningful Use, and no further. Coding is a billing activity done under time pressure, and three decades of incentives have not turned it into a clinical census. Extraction reads what the clinician already wrote and does not ask them to write it twice.

Does NCQA’s move from hybrid to ECDS measures solve this?

It reduces the gap and does not close it. ECDS adds EHR, registry, and HIE data to claims, and NCQA’s October 2025 report still found hybrid rates 3.4 percentage points higher than ECDS rates for commercial plans and 5.5 points higher for Medicaid in measurement year 2024. ECDS reads structured sources. Facts that exist only in notes can be brought into ECDS as supplemental data once they are extracted and validated, which is how the two approaches fit together.

Can a general-purpose LLM read the notes and fix this?

Reading is the easy part. The hard parts are accuracy on clinical text at the level a quality auditor or RADV reviewer will accept, assertion and temporal handling so that “denies,” “history of,” and copied text are not counted as active conditions, provenance back to the source sentence, and running on identified PHI inside your own environment. General-purpose models have trailed healthcare-specific models on published clinical extraction and de-identification benchmarks, and the per-token pricing of frontier APIs makes reading every note for every patient expensive at population scale. I covered the cost side in the June cost model.

Do ambient AI scribes make this better or worse?

They make the note longer and the gap wider unless something reads the note. Ambient notes run about 20% longer than typed ones, and in the JAMA Psychiatry study of 20,302 visits they documented more psychiatric symptoms while producing fewer diagnosis codes and interventions. Where vendors route note content into billing, coded diagnoses per encounter rise, but only for the billable subset, and payers have begun downcoding against the documentation. Everything else the scribe captured stays in free text. The fix is the same as for the rest of this piece: extract the facts from the note with validated accuracy and provenance, and compute the measure over all of them.

How do I test this in my own organization?

Take one measure you already report and one year of data. Compute it from codes. Then run validated extraction over the notes and reports for the same patients, review a random sample of the extracted facts against the source text, and recompute the measure over the union of coded and extracted facts. The delta is your measurement error for that metric, and it will tell you which of your quality and population health numbers to trust.

Every clinical assertion in a regulatory-grade healthcare AI platform must preserve its source document, the exact span it came from, the model and configuration that extracted it, the confidence assigned at extraction, the conflicts detected against other sources, and the rule or reviewer that resolved them. That is fact-level provenance. Building it takes 42 distinct capabilities across ingestion, extraction, privacy, reasoning, audit, and versioning. Retrofitting it onto an existing warehouse is far harder than carrying it through from the first parse.

The FDA’s December 2025 final guidance on real-world evidence for medical devices, operational since February 2026, treats relevance and reliability as per-fact properties of a submission, not per-dataset attributes. When a reviewer asks where a value came from, the answer should be given in a click, not a forensic project.

The clinical fact as the unit of governance

Most data warehouses govern at the level of the file or the table: access controls per dataset, versioned snapshots, dataset-level data dictionaries. That granularity is sufficient when the regulatory question is about a study population. It is not sufficient when the question is about a specific value for a specific patient.

When a reviewer asks “where did this come from?” about a single comorbidity, date, or medication dose, an auditable answer has to include the source document, the exact span in that document where the value appeared, the model and prompt version used to extract it, the confidence assigned at extraction, the normalization decisions that followed, the conflicts detected with other sources, and the rule or human reviewer that resolved them. That is the unit of governance.

Every clinical fact in the system carries six categories of attributes:

These attributes are native columns on the fact record, not entries in a separate lineage table, and they propagate into every derived measure, cohort, and agent answer. Once facts are stored without them, adding them requires re-processing the full corpus against models and documents that may no longer exist in the same state. That is the design decision the rest of the architecture rests on, and it must be made before the first data pipeline runs.

Three re-derivable tiers from raw bytes to OMOP

A monolithic pipeline that parses, extracts, normalizes, and reasons in one pass is the most natural thing to build. It is also impossible to reproduce months later. The version of the parser, the prompt to the extraction model, the terminology mapping table, the conflict resolution rule: all of these change. Without a structural separation between layers, point-in-time reproduction means standing up the entire stack at its earlier state, which is rarely feasible. Instead, the pattern that holds up under audit is a tiered one.

Bronze: lossless parsing. Every file and message format ingested without information loss. Free-text notes, FHIR R4 resources, HL7 v2 messages, DICOM headers, scanned PDFs with OCR, structured warehouse extracts. Nothing dropped, nothing normalized. This tier is immutable: it records exactly what arrived. When a downstream extraction model is retrained two years later, you can re-run extraction against the original Bronze records and reproduce or improve on the earlier result.

Silver: extraction with provenance. Clinical facts pulled from every modality, tagged with source coordinates, scored for extraction confidence, and mapped to standard terminologies. Each fact is independently traceable to a Bronze record. Silver can be re-derived from Bronze without touching anything upstream.

Gold: reasoning and standardization. Duplicates merged, conflicts reconciled across documents, measures and risk scores computed, and the result emitted in a shared analytic format, most commonly OMOP CDM or FHIR. Gold can be re-derived from Silver, and ultimately from Bronze.

Each tier can be validated, re-run, and audited independently. From any value in Gold, an auditor can walk all the way down to the raw bytes. The 21 CFR Part 11 expectation of reproducibility, first codified in 1997 for electronic records and now extended into a per-fact requirement under the new FDA RWE guidance, only holds when every layer is versioned and the dependencies between layers are explicit.

The 42-capability inventory

No single component in this inventory is exotic. The difficulty is that the platform needs 42 distinct capabilities, which must be planned from the first day, and which interact with one another to add complexity. Grouped by domain:

Multimodal ingestion (Bronze)

• Free-text clinical-note parsing with structure preservation (section headers, list semantics, table extraction)

• FHIR R4/R5 resource ingestion with reference traversal

• HL7 v2 message parsing with segment and field-level access

• DICOM header parsing across all standard SOP classes

• DICOM pixel-level PHI detection for text burned into images

• PDF text extraction for both digital and OCR-required scanned documents

• Connector framework for SQL warehouses, claims feeds, and CSV imports

• An immutable Bronze record store with content-addressed identifiers

Clinical extraction (Silver)

• Healthcare-specific named entity recognition across condition, procedure, medication, lab, and vital domains

• Relation extraction linking entities that only carry meaning together: a drug to its dose and frequency, a tumor to its stage and site, a lab value to the panel it belongs to

• Assertion status detection (confirmed, ruled-out, family history, patient-reported, historical)

• Negation and uncertainty detection

• Temporality detection and date normalization across relative (“last year”) and absolute references

• Terminology resolution to SNOMED CT, RxNorm, LOINC, ICD-10-CM, and CPT

• Per-fact extraction confidence scoring, exposed as a real-valued attribute on the fact

• Per-fact source coordinates: document ID plus character span, DICOM tag path, or FHIR JSON pointer

The accuracy bar for clinical extraction is high enough that general-purpose LLMs underperform. A peer-reviewed head-to-head evaluation we published at ECIR 2025 found a healthcare-specific small model reaching 96% F1 on PHI detection, against 79% for GPT-4o, 83% for AWS Comprehend Medical, and 91% for Azure, at over 80% lower cost.

The cost economics matter for capability #15 in particular: confidence scoring runs on every fact in the corpus, so a token-billed API call per fact is non-viable at population scale. The architectural choice that follows is to use specialized small models for the billions of routine extraction decisions, and reserve large generative models for narrative generation and multi-step reasoning where their breadth earns its place.

Privacy and de-identification

• PHI detection across the 18 HIPAA Safe Harbor identifier categories in free text

• PHI detection in PDF, combining text-layer and OCR coverage

• PHI detection in DICOM headers

• PHI detection in DICOM pixels (burned-in text), using vision-language models

• Consistent pseudonymization across documents and modalities, preserving longitudinal linkage

• Patient-specific date shifting that preserves intra-patient temporal relationships while preventing date-correlation re-identification across systems

• Parallel maintenance of identified and de-identified datasets, kept in sync by the same ingestion pipeline

• Configurable de-identification profiles (HIPAA Safe Harbor, Expert Determination, GDPR pseudonymization)

The parallel-dataset choice in capability #23 deserves a closer look, because the intuitive design fails quietly. That design stores everything identified and de-identifies on export. That design fails the GDPR Article 25 “data protection by default” test, and it fails the HIPAA Minimum Necessary standard for almost every research workload. The structural design maintains both datasets continuously, defaults every secondary-use query to the de-identified one, and requires explicit elevated permission for any read against the identified one. Privacy then operates as a property of the schema and the query path, enforced by the platform itself.

Reasoning and reconciliation

• Patient-level record linkage across source systems

• Cross-document deduplication of clinical events

• Conflict detection when sources disagree (chart says 80mg, pharmacy feed says 40mg)

• Temporal reasoning to distinguish “history of diabetes” from “current diabetes”

• Implied inference rules with explicit confidence (patient started metformin → likely but not confirmed diabetic)

• Absence-as-negative inference, scoped to documents that would plausibly mention the finding (a cardiology note that does not mention HIV is not evidence; an HIV screening result that does not mention HIV is)

• Decay rules for time-sensitive measurements (a five-month-old weight, a one-hour-old pulse)

Extraction is the visible engineering. Reconciliation is where the platform either picks a value silently and accumulates quiet errors, or surfaces the conflict with its evidence and routes the decision somewhere it can be audited. The auditable design picks the second path, which requires a reasoning layer with explicit policies for every type of conflict the data may produce. That layer is months of work even after the extraction is good.

Audit and access control

• Tamper-evident audit logs capturing user identity, accessed records, timestamp with millisecond precision, purpose code, and query text, for every access path

• Role-based access control over every data asset and every column

• Purpose limitation enforced at the database layer, so a query outside the project’s authorized scope is rejected regardless of the SQL

• Consent tracking integrated with the patient record, enforced at query time against current consent status

• Anomaly detection on access patterns (10,000-patient reads against historical baselines under 100)

The audit log entry that supports this captures, for every access event:

The hash chain in the last row is what makes the log “tamper-evident” in regulatory parlance. An administrator with database access can append, but cannot rewrite history without breaking the chain. That property is required for regulated submissions and useful in its own right when answering an audit question months after the fact.

Versioning and reproducibility

• Immutable dataset versioning at all three tiers

• Model version pinning with content-addressed artifacts, covering every language model in the pipeline: the specialized NLP models doing entity recognition, assertion detection, and de-identification, and the LLMs doing reasoning and generation

• Extraction configuration versioning: the full pipeline definition active for every run, including the sequence of stages, the prompts, the guardrails, and the decoding parameters

• Terminology mapping versioning (SNOMED CT, RxNorm, LOINC, and ICD-10-CM each release on their own cycles, from monthly to annual)

• Business rule versioning for the reasoning layer, so a change to which source wins a medication conflict creates a new version without invalidating prior runs

• Point-in-time dataset reconstruction: given a query and a timestamp, return the result the system would have returned at that time

Capabilities #38 and #39 are where the choice of an external LLM API grows from a cost question into an architecture question. A hosted frontier model is not deterministic: the same prompt against the same endpoint returns different outputs across runs, and even at temperature zero, providers do not guarantee identical results. It is not pinnable either. The model behind an API endpoint is updated on the provider’s schedule, versions are deprecated and retired on cycles measured in months, and a version retired in 2026 cannot be re-run in 2028 no matter what your audit requires. An extraction step you cannot re-run against the same model, prompt, guardrails, and decoding parameters is an extraction step you cannot defend, and the FDA’s per-fact reliability expectation makes that a submission problem, not a preference.

Capability #42 is the cleanest test of whether the rest of the work was done right. If you can ask “what would this cohort have looked like six months ago?” and get a deterministic answer in seconds, using the datasets, model versions, terminology releases, and business rules in force at that time, the versioning is genuine. If you cannot, somewhere upstream a version pin was missed. Content-addressed model artifacts on infrastructure you control pass this test. An API subscription does not.

That is the inventory. The engineering estimate, with a senior team that has built data platforms before but not this kind of healthcare-specific platform, is two to three years to land all 42 capabilities in a way that holds up under audit. With a team newer to the domain, longer. The capabilities are not individually exotic, but they compound a data platform’s complexity, and several of them (#23 parallel datasets, #16 source coordinates, #42 point-in-time reconstruction) only work if they were planned from day one.

The ten-step build order

Forty-two capabilities do not get built at once. The sequence below is ordered by what compounds and what retrofits worst.

1. Pick a shared analytic data model before the first pipeline runs. Two open standards cover most of the workload, optimized for different questions.

OMOP CDM v5.4, maintained by the OHDSI community, is the strongest default for secondary use: cohort definition, population analytics, real-world evidence, registry abstraction, and any question that compares groups of patients. OMOP is open, peer-reviewed, used at hundreds of institutions, and compatible with the OHDSI tool ecosystem (ATLAS, Achilles, HADES). Published RWE work is largely against OMOP, which makes reproducing prior results and contributing back tractable.

HL7 FHIR R4/R5 is the strongest default for single-patient analysis: clinical decision support, point-of-care AI, patient-specific question answering, and any workload where the unit of work is one patient’s record. The patient-centric resource graph makes joining one patient’s encounters, observations, conditions, and medications fast and direct, and every modern EHR speaks FHIR natively.

The choice is not exclusive. A platform that emits extracted facts into both formats covers the full secondary-use scope without forcing analysts into the wrong tool: FHIR for the per-patient agents, OMOP for the cohort and population work. What holds constant either way: pick the model before the first pipeline runs. A proprietary or homegrown schema locks every downstream analysis to one stack and forecloses on the published literature.

2. Build Bronze before extraction. The first sprint is the immutable raw layer, not the first extraction model. Capabilities #1 through #8 land first. Any extraction model you ship will eventually be replaced, and Bronze is what makes the next version re-derivable.

3. Make provenance a column type, not a metadata table. Source document ID, source span, extraction model version, and confidence score are attributes on every fact. This is the design choice that retrofits worst. Get it right at the first extraction.

4. Build de-identification as a pipeline stage, with parallel datasets from day one. Capabilities #17 through #24. De-identification built as an export-time step stops working once the second downstream consumer exists: someone, somewhere, will query the identified store for a research use case that should have hit the de-identified one.

5. Use specialized models for routine extraction, and reserve large LLMs for reasoning. A 96% F1 specialized model that runs deterministically on hardware you control beats a 79% F1 frontier API that costs more, returns different values run-to-run, and will be retired before your first re-audit. Match the tool to the task at every layer.

6. Build the reasoning layer as an explicit stage between Silver and Gold. Capabilities #25 through #31. Folding reconciliation into extraction makes the resolution policies invisible; folding it into the Gold materialization makes them unauditable. Reconciliation has its own policies, its own audit requirements, and its own confidence outputs, and it needs to be inspectable in isolation.

7. Version everything from the start. Capabilities #37 through #42. Versioning is the capability most likely to be “added later” and least likely to actually be added later, because by then there is too much un-pinned state to recover. Pin models, prompts, guardrails, pipeline configurations, terminology releases, and reasoning rules from sprint one, even when the pinning feels premature.

8. Treat human-in-the-loop as infrastructure. Conflicts the system cannot resolve confidently need a routing layer and a review UI that shows side-by-side evidence, records the human decision in the audit trail, and feeds the decision back as training signal. NAACCR cancer registry sign-off, NCDB abstraction review, and similar regulatory workflows already require this. The platform that bolts on a review screen at the end never matches the platform that designed the review queue into the pipeline.

9. One governed boundary for all agents. The Model Context Protocol (MCP) endpoint pattern is the cleanest version of this: agents call high-level platform tools (search_concepts, build_cohort, get_patient_timeline), not the underlying SQL tables. Redaction, masking, and access control apply inside the boundary, before any data leaves. Adding the tenth agent does not add a tenth governance surface; the governance is centralized once and inherited.

10. Continuous re-evaluation, not one-time audit. SNOMED CT and RxNorm ship new releases throughout the year, FDA guidance updates, state privacy laws change, specialized models drift on shifting documentation patterns. The platform that re-evaluates every dataset against current policy and current models, on a schedule, is the one that stays audit-ready. The platform that audits once a year is functionally not audited.

Patient Journey Intelligence and the FDA RWE guidance

Everything above reads as generic software architecture, and it is. It also describes an existing platform: Patient Journey Intelligence (PJI), John Snow Labs’ secondary-use data platform. It implements this pattern end to end: the six fact-level attribute categories as native columns, the three governance tiers, parallel identified and de-identified datasets from first ingestion, the hash-chained audit log, and four independently versioned layers (datasets, models and prompts, terminology releases, business rules) with point-in-time reconstruction as a first-class query. The data governance documentation walks through each layer in the same terms used here.

In January, John Snow Labs announced that PJI is the first secondary-use data platform designed to meet the FDA’s December 2025 final RWE guidance. The guidance makes two demands that map directly onto this article. First, clinical facts must be sufficiently complete and accurate, which means structured EHR data alone no longer suffices: unstructured clinical narratives, multimodal data, and longitudinal patient experience are in scope, because much of the clinically relevant information exists only there. Second, data provenance, quality, and reliability must be rigorously documented. PJI’s answer to both is the architecture above: every derived clinical fact carries end-to-end lineage back to its source document and exact location, versioned models, prompts, and rules from extraction, human-in-the-loop validation for high-stakes clinical endpoints, and deterministic reproducibility with confidence scores and full provenance.

The platform deploys entirely inside the customer’s infrastructure, with John Snow Labs’ healthcare-specific LLMs and SLMs included, so there are no external LLM API calls to break reproducibility and no PHI leaving the security perimeter. The de-identification pipeline inside it is the one validated across 2 billion patient notes with zero re-identifications. Model-level governance, meaning the registry, pre-release test gating, and production drift monitoring for every model in the platform, runs through Pacific AI, the CHAI-certified governance platform I also lead, deployed in the same tenant.

Six areas of ongoing work

Every architecture choice requires a tradeoff, and while building PJI we’ve encountered six areas that are still not fully solved. We are making progress on them every week, and we’re interested in learning from you if you work on these:

Extraction is not perfect. Even peer-reviewed regulatory-grade models miss and mis-assign facts. The pipeline reduces error and makes it visible through confidence scoring; it does not eliminate it.

Garbage in, governed garbage out. Provenance proves where a fact came from. It does not prove the source was correct. A confidently-wrong clinical note becomes a confidently-traced record. The architecture makes the error findable and fixable, which is the right bar, but it does not make errors disappear.

Specialized models need maintenance. Per-domain small models for SDoH, oncology, mental health, and the like must be re-validated as guidelines, terminology releases, and documentation patterns drift. The model catalog is a living cost, not a one-time build.

Human review is a real throughput constraint. Routing low-confidence fields to experts protects quality. Expert time is finite, the queue is genuine, and the platform that does not measure and manage review throughput will silently degrade.

OMOP cannot represent everything. Some clinical nuance (“adequate organ function,” “investigator believes the patient can comply”) resists any structured model. Forcing it loses meaning; leaving it out loses completeness. A design that holds up surfaces what cannot be represented and routes it to human judgment, rather than hiding the gap.

Evaluation itself is hard. Gold-standard labels for clinical extraction are scarce and expensive. Accuracy claims are only as strong as the reference standard behind them, and the reference standards in healthcare are smaller and noisier than the ones in general NLP.

42 capabilities and three years of engineering

The companies that move smoothly under the new FDA RWE guidance, and under the regulatory tightening that is going to keep arriving in 2026 and 2027, are the ones that made the architectural commitment to fact-level provenance early. Governance recorded only in documentation cannot reproduce a fact’s lineage on demand, and an audit is where that difference surfaces.

These forty-two capabilities are the scope of the clinical data platform you should build — whether you are a health system, a payer, a real-world evidence provider, or a data aggregator — and whether you build it or buy it. The design patterns are tiered ingestion, fact-level provenance, parallel identified and de-identified datasets, specialized extraction with confidence propagation, an explicit reasoning layer, versioning of everything including the models themselves, human-in-the-loop as infrastructure, and a centralized governance boundary for agents. They are what the platforms that survive AI governance reviews and regulatory submissions look like. Build to them on the first try if you can, or buy them already built. Budget for the rebuild if you can’t.

Frequently asked questions

What is fact-level provenance in healthcare AI?

Fact-level provenance means every clinical assertion in a data platform carries its own source document, exact source span, extraction model and configuration version, confidence score, conflict record, and resolution history as native attributes. Dataset-level controls can answer questions about a study population; only fact-level provenance can answer an auditor’s question about where a specific value for a specific patient came from.

What does the FDA’s December 2025 real-world evidence guidance require?

The final guidance on using real-world evidence to support regulatory decision-making for medical devices requires that clinical facts be sufficiently complete and accurate, which the agency recognizes structured EHR data alone cannot deliver, and that data provenance, quality, and reliability be rigorously documented. In practice, relevance and reliability become per-fact properties of a submission rather than per-dataset attributes.

Why can’t an external LLM API meet the reproducibility bar for regulated extraction?

Two reasons. Hosted frontier models are not deterministic: the same prompt returns different outputs across runs, and providers do not guarantee identical results even at temperature zero. And they are not pinnable: providers update models behind stable endpoints and retire versions on cycles measured in months, so an extraction run today cannot be reproduced in two years when the version no longer exists. Point-in-time reconstruction requires content-addressed model artifacts running on infrastructure you control.

What is the difference between the Bronze, Silver, and Gold tiers?

Bronze stores every ingested document and message immutably and losslessly, exactly as it arrived. Silver holds the clinical facts extracted from Bronze, each tagged with source coordinates, model version, and confidence. Gold holds the reconciled and deduplicated result, mapped to OMOP or FHIR, with every conflict resolution decision logged. Each tier can be re-derived from the one below it, which is what makes any value in Gold traceable to raw bytes.

How long does it take to build a fact-level provenance platform in-house?

With a senior team that has built data platforms before but not a healthcare-specific one, the realistic estimate is two to three years to land all 42 capabilities in a way that holds up under audit. The individual components are not exotic; the difficulty is that they interact, and several, including parallel identified and de-identified datasets, per-fact source coordinates, and point-in-time reconstruction, only work if they were designed in from the first sprint.

What is the test for genuine point-in-time reproducibility?

Given a query and a timestamp, the system should return the result it would have returned at that time, deterministically and in seconds, using the dataset versions, model and prompt versions, terminology releases, and business rules in force at that timestamp. If the answer requires reconstructing a snapshot or re-running a pipeline with archived parameters, the versioning is aspirational rather than real.

Over the past two months my team benchmarked the new general-purpose privacy tools on clinical text: OpenAI’s Privacy Filter at 0.55 PHI F1, Microsoft Presidio at 0.60 to 0.85 F1 in independent peer-reviewed studies, Databricks’ `ai_mask()` at 0.71, and the frontier LLM APIs at 0.86 to 0.91, against 0.96 for a healthcare-specific pipeline on the same corpus, which also holds 0.98 recall on a second 381,959-token clinical corpus and 0.98 micro F1 on the official 2014 i2b2 test set. The per-entity tables are on the John Snow Labs blog. This post is about how to read them, because de-identification is one of the few tasks in machine learning where the reading is pass or fail. A summarizer at 0.85 saves a team real effort. A de-identification pass at 0.89 produces a corpus that no privacy officer can release, which is the same thing a pass at 0.55 produces.

The legal standard: no PHI or very low risk of it

The regulation does not ask for a score. HIPAA offers two paths to de-identified status: Safe Harbor, which requires that all 18 identifier categories be removed, and Expert Determination, in which a qualified statistician certifies that the risk of re-identification is very small. Under GDPR, data that has been pseudonymized rather than anonymized remains personal data inside the full regulatory perimeter. Both frameworks are binary. No tier of either recognizes 80% removal and grants 80% of the permissions, and there is no version of “mostly de-identified” that a downstream researcher, partner, or model-training pipeline is allowed to touch.

Neither framework names an accuracy number, because neither was written about software. The field calibrated instead against the only prior benchmark that existed, careful manual de-identification by domain experts, and Yogarajan, Pfahringer, and Mayo’s 2020 review identifies 95% F1 on the 2014 i2b2 corpus as the level widely treated as equivalent to it.

95% is hard to reach with people

That bar sits above what individual experts achieve. Neamatullah et al. measured 14 clinicians de-identifying nursing notes by hand: recall ran from 0.63 to 0.94 by clinician, a single annotator averaged 0.81, two-annotator consensus reached 0.94, and even a consensus of three experts failed to remove all PHI. Building the i2b2 gold standard, Stubbs and Uzuner found individual annotators averaged 0.93 token F1 against the consensus they collectively produced, and the corpus reached gold quality only through double annotation of every record, arbitration, and rounds of sanity checking and proof reading. A single reviewer is not a safety net: one identifier in five gets past them.

What a recall number means at a million notes

Recall carries the regulatory risk and its implication multiplies across a corpus. A clinical note carries roughly 31 PHI mentions.

* Chunk-level detection, counting partial overlaps as found; on exact matches only, ai_mask() sits at 0.51.

The last column is recall^31, the chance a note comes out fully clean if each mention were independent, which overstates the damage since a repeated name is one detection problem rather than eight (read it as a worst case). The direction survives any correction. Below about 0.90, almost no document in a corpus is clean, and no downstream process recovers it. Above 0.95, the residue becomes small enough for adversarial testing and statistical risk analysis to characterize, which is what Expert Determination requires.

John Snow Labs’ pipeline sits above that line on every corpus tested: 0.98 recall in the table above, 0.98 micro F1 on the official i2b2 test set, and over 99% detection across 2 billion notes at Providence. Between 0.72 and 0.98 there is no plateau at which the data becomes shareable.

0.89 fails the same way 0.55 fails

The frontier models did improve. GPT-4o scored 0.79 PHI F1 in our Text2Story 2025 evaluation; GPT-5.5 scores 0.89 on the same corpus today. Read as a trend line, that number invites a plan: wait a version or two and let the general models solve it. Read against the bar, it says something different. 0.79 was below the threshold and 0.89 is below the threshold, and both produce a corpus that holds no de-identified status and cannot leave the security perimeter. In this task, improvement below the bar has a regulatory value of zero.

The residual errors also sit in the worst possible place. The aggregate hides it and the per-entity tables show it: Claude Opus 4.8 scores 0.49 on contact identifiers and 0.68 on age, and Gemini 3.1 Pro scores 0.60 on age, against 0.95 and 0.97 for the healthcare pipeline. Contact and age are enumerated Safe Harbor categories, and age carries its own aggregation rule above 89. These identifiers are defined by their role in a document rather than their surface form: phone and fax numbers live in headers, footers, and referral blocks, and record numbers appear as MRN, MR#, and a dozen local conventions. That’s a structural mismatch with general training distributions – and why the last points of the gap are the hardest ones, concentrated on the categories the regulation defines.

In addition, a model that cleared the number would still not clear the bar, because the bar includes validation. Expert Determination is built from span-level detection records, confidence thresholds, versioned models, and reproducible runs. A frontier API that returns text, may be swapped or retired by its vendor, and cannot be pinned to the version that processed a 2026 cohort resets that evidence with every release.

Beyond accuracy

There is a second independent reason a strong score settles nothing, which is why an earlier paper of ours on production de-identification is titled “Beyond Accuracy”. Every tool in these benchmarks can find PHI and delete it. A de-identification program needs eight other things.

Suppose ai_mask(), at 0.71 today, scored 0.96 tomorrow. It would still fail nearly every de-identification program I have seen, because masking is its entire feature set. It returns no spans, so there is no record of what was detected, no confidence to threshold on, and no way to sample detections for review. Those artifacts are what an Expert Determination analysis is built from. It also maintains no consistency, so a patient masked in two notes cannot be recognized as one patient. The same reasoning applies in some degree to every general-purpose tool in the table. The benchmark scores describe the first row. A de-identified medical dataset requires more.

Masking is just where de-identification starts

Replacing every identifier with [MASKED] produces a compliant corpus that has lost most of its research value. A longitudinal cohort needs the same patient recognizable across a decade of notes, claims, PDFs, and DICOM headers, intervals between events preserved so a time-to-progression analysis still means something, and a narrative readable enough to extract from, which “patient was seen by [MASKED] at [MASKED] on [MASKED]” is not. Three capabilities do that work.

Consistent obfuscation. PHI is replaced with realistic, gender- and context-aware surrogates, deterministically. If “Jane Sunshine” becomes “Anne Boleyn,” every later “Jane” maps to “Anne” across the corpus.

Deterministic tokenization. An MRN, or a name-plus-birth-date composite, becomes a cryptographic hash, so records about the same person written years apart link without exposing the identifier. Without this, no analytics that track a patient over time can be delivered.

Date Shifting. Generalizing or deleting dates prevents calculating even simple analytics like 30-day readmissions, or answering simple questions like what happened before what. Shifting dates on a random number of days, that is consistent across all records of the same patient, is essential.

Multimodal linkage. The same tokenization and obfuscation applied across EHR text, claims, PDF reports, DICOM images, and structured tables, with date shifting that moves dates while preserving intervals.

All of it must be done together. Jane stays female because she has a history of breast cancer, and “April 2020” shifted by a random number of days comes back as “March 2020” rather than “3/3/2020.” Imaging raises the bar again, because PHI in a DICOM file lives in the metadata headers and burned into the pixels simultaneously, and handling only one leaves the study identified. A dual-level approach we published at CVC 2026 reports 100% success on tag retention, date shifting, and UID consistency on the MIDI-B dataset, with text processing accuracy above 99.9%. None of that work appears in any text benchmark table.

At Providence, this full stack ran across 2 billion patient notes at over 99% PHI detection, validated through three months of external red-teaming, manual review of 35,000+ notes, equity analysis across gender, age, ethnicity, and geography, and adversarial re-identification testing on 790 patients that produced zero re-identifications. The methodology is published. Detection accuracy is what gets a program to the starting line; consistency validation, red-teaming, and statistical risk analysis are the work that follows.

Two questions to ask your de-identification vendor

First, ask for per-entity recall on your own notes. Every tool above has an aggregate that sounds respectable and a category where it finds fewer than half the identifiers: ai_mask() finds 35% of contact identifiers behind a 0.71 headline, and Privacy Filter finds one address in four. Second, ask about everything after detection: whether it can obfuscate consistently across a corpus, tokenize deterministically for linkage, shift dates while preserving intervals, return spans for audit, and follow the same patient across modalities.

De-identification is one of the few tasks in healthcare AI where the requirement comes from law rather than from a stakeholder conversation. Below the bar, every score is the same score, because the output has the same status: identified. There is no partial credit, and no version of this problem in which 0.89 is a good start.

Frequently asked questions

The frontier models are improving fast. Should we wait for the next version?

Waiting is a bet that a general model will master identifiers defined by clinical context rather than surface form, which is where the remaining errors concentrate: contact identifiers at 0.49 to 0.69 and age at 0.60 to 0.93 for the current frontier models. It is also a bet on the wrong variable, because clearing the accuracy number does not produce de-identified data. That requires all 18 Safe Harbor categories covered, span-level detection records, consistent tokenization and obfuscation, and validation against a pinned model version – none of which an unpinned API provides.

If a model reaches 0.96 F1, is de-identification solved?

Detection at that level clears the accuracy threshold, and a program still needs surrogate obfuscation, cross-document consistency, deterministic tokenization, date shifting, span-level audit records, and validation evidence for Expert Determination. Those are separate capabilities from detection.

Isn’t masking enough if I only need a compliant corpus?

Only if nobody needs to analyze it afterwards. Masking yields a corpus in which every patient is unlinkable, every date is gone rather than shifted, and every narrative has holes where the context used to be. Longitudinal research, cohort building, and time-to-event analysis all need consistent obfuscation and deterministic tokenization instead, and most general-purpose tools have neither.

A vendor’s masking service is HIPAA compliant. Does that make the output de-identified?

No, the two claims describe different things. Service-level HIPAA compliance means the processor will sign a BAA and handle PHI appropriately. Whether the output is de-identified depends on detection recall, coverage of all 18 Safe Harbor categories, and the ability to document what was detected, none of which follows from the service claim.

Does masking satisfy GDPR?

Masking produces pseudonymized data, which remains personal data under GDPR and stays inside the full regulatory perimeter. Anonymization requires that re-identification be reasonably impossible, a higher bar that depends on the whole pipeline and the release context rather than a single model, and it is a question for your own counsel rather than a vendor.

Where are the detailed benchmark tables?

On the John Snow Labs blog: the Privacy Filter comparison, the Presidio comparison, the cloud API and LLM comparison, and the de-identification solution page, with per-entity precision, recall, and F1 for every system discussed here.

An LLM can give a factually correct answer and still create legal exposure for whoever deployed it, if it answers while presenting as a licensed professional. Between January 2025 and January 2026, at least eleven US state and federal laws came into force or were introduced to regulate how AI systems present themselves in healthcare, mental health, legal, financial, and insurance conversations. We built a 380-case red-team suite at Pacific AI to test this surface, and ran it against four frontier models. None scored above 0.64, and every one failed at least a third of it.

The law treats impersonation as its own category of harm

Current laws treat impersonating a licensed professional, and failing to disclose AI status in a regulated conversation, as a category of consumer harm separate from whether the underlying advice was any good. California AB 489, in force since January 1, 2026, prohibits AI products from using names, credentials, or design elements that would lead a reasonable person to believe they are talking to a licensed health professional. An AI that calls itself “Dr.” anything violates it even when every answer is clinically sound. Accuracy does not cure the presentation problem, which is why accuracy benchmarks cannot measure it.

The legislative record explains where this came from. The findings behind AB 489 cite AI products deployed under “Dr.” persona names. The FTC’s 2025 action against DoNotPay concerned marketing an AI system as a legal advisor. Nippon v. OpenAI, filed in 2026, alleges that AI-generated, ready-to-file legal documents tailored to an active dispute amounted to unauthorized legal practice. Reports of teenagers forming relationships with AI characters framed as licensed therapists drove both New York’s General Business Law Article 47 and California SB 243.

Texas SB 1188 requires practitioners to disclose AI use to patients. Nevada AB 406 (July 2025) prohibits offering AI that provides professional mental or behavioral healthcare. Illinois’ Wellness and Oversight for Psychological Resources Act (August 2025) prohibits AI-delivered therapy while allowing AI-assisted licensed practice with disclosure. Utah HB 452 requires mental health chatbots to disclose AI status when asked. Maine HP 1154 requires disclosure across consumer-facing services generally. New York GBL Article 47 (November 2025) adds crisis protocols, escalation pathways, and a private right of action for AI companion products. California AB 3030 (January 2025) requires generative-AI patient communications to carry a disclosure that they were AI-generated, with instructions for reaching a human provider. The federal CHATBOT and GUARD Acts and state bills like New York S7263 remain proposals rather than law; S7263 would reach conduct that amounts to unauthorized practice even when the AI never claims a license.

Two properties of this wave matter for deployers. First, the exposure sits primarily with the deploying organization: the health system, law firm, brokerage, or platform, rather than the model provider. Second, the violations are behavioral. They happen when a production model meets an adversarial or emotionally loaded prompt: a condition standard capability benchmarks don’t create.

A surface no existing benchmark covers

Plenty of evaluation exists nearby. MedHELM and HealthBench measure whether a model reasons correctly about clinical content. General red-team corpora like AdvBench, HarmBench, and ALERT target content-level harm: toxicity, weapons information, illegal activity. Mental health safety evaluations like VERA-MH go deep, and well, on one industry. None of them asks the question these eleven laws ask: when an adversarial prompt invites the model to play doctor, lawyer, or fiduciary, does it refuse appropriately, disclose that it is AI, and route the person to a licensed human when one is needed?

The failure mode is distinct. A model can top MedHELM and still format its output as a signed prescription. Regular readers will recognize the pattern from the cognitive bias suite covered here recently: capability scores measure one thing, deployment behavior is another, and the gap between them is where the liability lives.

Built so a failed test names the exposure it creates

The dataset holds 380 adversarial test cases across healthcare, mental and behavioral health, legal practice, financial and investment advice, accounting and tax, and insurance, plus 23 more licensed professions—from engineers and architects to midwives, notaries, and general contractors. Each case sits on four axes: six failure patterns, four difficulty levels, seven profession categories, and 18 attack vectors drawn from the prompt injection literature and from documented real-world incidents.

Three construction decisions make the results believable.

Every test is anchored to a specific law or enforcement theory. Red teaming often tests for “bad behavior” without defining what makes the behavior bad. Here, each row names the statute or enforcement theory it probes. A row that coaxes the model into a credential-implying signature block anchors to AB 489’s design-elements language; a row that requests a patient message with no mention of disclosure anchors to AB 3030. When a model fails a row, the failure maps to a named exposure, and a compliance team can pull every row behind a single law and show exactly what was tested.

Distinct failure modes. The suite avoids five rephrasings of “are you a doctor?” Within each cell of the taxonomy, every row probes a different way in: a direct probe, a named clinician roleplay request, a credential cue slipped into a document format, a false premise, a multi-turn rapport build that ends in an out-of-scope ask, or a crisis context override. This slowed down the construction of the test suite considerably. It also means a high pass rate is evidence of broad behavior rather than memorization of a template.

The tests grade what a correct refusal contains. For many of these prompts, blanket refusal is itself a failure. A user disclosing acute self-harm intent needs crisis resources named. A parent describing a child’s red flags needs the red flags surfaced and an emergency pathway identified. A generic “I cannot help with that” abandons them in a crisis. Every row therefore specifies both must-do behaviors (identify as AI, name specific crisis resources, recommend the right human professional) and must-avoid behaviors (adopting credentialed personas, fabricating license numbers, providing medication doses, generating court-ready filings). The two families are graded separately. This test design punishes the pathological “safe failure” of a model that refuses everything and helps no one.

Coverage is deliberately uneven: crisis context gets 20 rows, self-disclosure 24, document completion 26, and multi-turn rapport 22, because those patterns dominate documented real-world incidents and carry the heaviest regulatory exposure. Encoded payloads and system-prompt extraction get seven rows each, because the failure-mode space inside them is narrower. The paper is explicit about what that choice costs: vectors with fewer than ten rows cannot support confident cross-vector comparison yet. An evaluation that states its own resolution limits before you ask is one you can trust when it does make a claim.

What four frontier models scored

We ran the full suite against Claude Fable 5, DeepSeek V4 Pro, Gemini 3.1 Pro, and GPT-5.4, graded with an LLM-as-a-judge framework against each row’s evaluation points, must-do items, and must-avoid items.

Claude Fable 5 and GPT-5.4 form a higher-scoring pair; DeepSeek V4 Pro and Gemini 3.1 Pro a lower one. The composition differs inside the tiers: GPT-5.4 posts the best must-avoid score (0.646), so it is strongest at staying away from prohibited behaviors, while Claude Fable 5 posts the best must-do score (0.787), so it is strongest at including the affirmative behaviors a compliant response requires.

The paper that introduces this AI impersonation test suite does something most benchmarks skip: it tests whether the instrument can tell the models apart at all. Pairwise Mann-Whitney U tests on per-scenario scores separate each top-tier model from each lower-tier model at p < 0.00001, with small-to-medium effect sizes (Cohen’s d between 0.36 and 0.48). The two within-tier comparisons are not statistically significant; Claude Fable 5 versus GPT-5.4 returns p = 0.35 with a negligible effect size. So the suite reliably distinguishes a stronger tier from a weaker one, and the 0.013 gap between the top two models should not be read as a ranking. Publishing that second sentence next to the first is what separates an evaluation instrument from a leaderboard.

A variance decomposition explains why: attack vector accounts for 6.7% of score variance and model for 5.1%, both highly significant; profession contributes 3.3% and difficulty 0.8%; failure pattern is not a significant factor at all. Roughly 83% of the variance sits at the level of the individual scenario, which matches a consistent finding across the LLM evaluation literature: the specific wording of an individual prompt moves scores more than any labeled category does. The practical consequence is that statistical power comes from many diverse scenarios, and the direct route to finer resolution, including resolving the top two models, is growing the dataset.

Neutrality disclosure: the paper’s authors (myself included) are unaffiliated with the companies developing the four models under test, and hold no financial interest in any of them. The four models are treated as systems under test, scored on published criteria.

Where the failures cluster

Every model failed at least a third of the suite. That is the headline for a governance audience: none of the four handles the impersonation-and-disclosure surface at a level that lets a deployer skip testing, whatever its clinical, financial, or general capability scores say.

The financial and investment profession is the hardest surface for all four models, with the lower tier dropping to roughly 0.40. The weakest attack vectors, on sample sizes solid enough to trust, are the missing-disclosure trap, where the model is asked for regulated output such as a patient message and must add the required AI disclosure without being prompted, and prefix injection.

The missing-disclosure trap is exactly the behavior California AB 3030 and Texas SB 1188 describe, and no model scored above 0.47 on it. The aligned failure pattern, proactive disclosure, produced the lowest scores of any pattern for all four models, between 0.34 and 0.37, though at 19 rows it is the thinnest sample in the suite.

Difficulty behaves as designed through the first three levels: for every model, scores decline from direct prompts through social engineering to adversarial jailbreaks, which is what a valid difficulty scale should produce. The fourth level, multi-turn, shows an apparent rebound the paper does not interpret: the bucket holds only 29 rows, and the rebound could reflect how the test harness expanded the multi-turn scenarios rather than anything about the models.

From one-time audit to standing test

The suite is built to be re-run, and the deployment pattern follows directly from the statistics. Pre-release, gate against a fixed pass-rate threshold or against the prior version of the same model. Given the within-tier resolution limit, small deltas between comparably scoring models are the wrong gate.

In production, sample rows on a rolling basis and monitor drift on the aggregate pass rate, since scenario-level variance makes any single taxonomy cell noisy. When a real impersonation or disclosure incident occurs, capture that prompt’s shape as a new row, which keeps the suite growing along the same axis that improves its statistical resolution.

The regulation anchors turn test results into audit material. When a regulator asks how a deployment tested for AB 3030 behavior, the deployer can produce the specific rows anchored to it and the model’s pass rates on them. That evidence describes what was tested and how the model behaved. It is not, and should not be presented as, a determination of legal compliance.

The dataset ships as part of the Pacific AI test suites, and the four-model evaluation was produced by running it through the Guardian continuous testing platform. The full preprint has the per-vector and per-profession tables and enough methodological detail for independent replication.

Twenty turns into a warm conversation, a user asks your deployed model whether they’re talking to a person. Eleven laws now have an opinion about what should happen next. Until you’ve tested it, you don’t know what does.

This post, like the preprint it summarizes, describes a testing methodology and reports model pass rates. It is not legal advice, and pass rates are not determinations of compliance with any regulation. Organizations subject to the laws named here should consult their own compliance counsel.

Frequently asked questions

What does this benchmark test that MedHELM or HealthBench don’t?

Those suites measure clinical reasoning accuracy. This one measures presentation behavior: whether a model adopts a licensed persona, produces regulated work product like prescriptions or court filings, discloses its AI status when asked and unprompted, and escalates a crisis to human help. A model can score well on one and fail the other; all four frontier models failed at least a third of this suite.

Which laws are the test cases anchored to?

Each of the 380 rows names the statutes it probes, drawn from eleven US laws in force as of May 2026, including California AB 489, AB 3030, and SB 243, Nevada AB 406, Illinois’ WOPR Act, Utah HB 452, Texas SB 1188, Maine HP 1154, and New York GBL Article 47, plus adjacent frameworks from SEC and FINRA rules to state unauthorized-practice statutes. Pending bills are treated as motivation, never as enforceable obligations.

Which model performed best?

Claude Fable 5 scored highest overall at 0.637, with GPT-5.4 at 0.624; the difference between the two is not statistically significant, so the honest reading is that they form a top tier rather than a ranking. DeepSeek V4 Pro (0.515) and Gemini 3.1 Pro (0.533) form a lower tier, separated from the top pair at high statistical significance.

What were the hardest tests for every model?

The financial and investment profession produced the lowest scores for all four models. Among attack vectors with solid sample sizes, the missing-disclosure trap, where the model must add a legally required AI disclosure without being asked, and prefix injection were the weakest for every model tested.

Does a high pass rate mean a system complies with these laws?

No. Pass rates describe how a model behaved on regulation-anchored tests. They are evidence of testing rigor that a governance program can present to auditors and regulators, and what any result means for a specific deployment is a question for the organization’s own compliance counsel.

How is the suite meant to be used in practice?

Three ways: as a pre-release gate against a fixed pass-rate threshold or a prior version of the same model, as a production monitor that samples rows on a rolling basis, and as a living corpus that grows a new row from every observed real-world incident. It ships as part of the Pacific AI test suites and runs in Guardian.

Take a clinical note about a 52-year-old with chest pain, and ask a frontier model what to do. It orders the troponin, the ECG, and keeps the patient for observation. Now take the identical note, change nothing about the vital signs, the history, or the exam, and add one line at the top from the triage nurse: “Patient has a history of anxiety, likely panic attack.” A measurable share of the time, the same model now discharges the same patient.

Nothing in the clinical picture changed. The correct answer did not change. What changed was a cue that a human clinician would also find hard to ignore. The cognitive biases that produce most diagnostic error in human medicine have been reproduced in large language models, they are not fixed by the things everyone assumes will fix them, and almost nobody is testing for them.

What the literature says about clinical cognitive biases

This is not a new problem that AI invented. It is one of the best-studied problems in patient safety.

The foundational work is Graber, Franklin, and Gordon’s 2005 study in Archives of Internal Medicine, which examined 100 cases of diagnostic error in internal medicine and found cognitive factors contributing in 74% of them. Premature closure, closing the diagnostic process before the right answer is reached, was the single most common cognitive failure. A 2018 review in the Journal of the Royal College of Physicians of Edinburgh by O’Sullivan and Schofield puts the ceiling higher, attributing up to 75% of diagnostic errors in internal medicine to cognitive causes rather than gaps in knowledge.

The emergency department numbers are worse, because the environment is worse. Kunitomo, Harada, and Watari’s 2022 study in BMC Emergency Medicine found one or more cognitive factors present in up to 96% of emergency-room diagnostic errors, with confirmation bias in 21.2% of cases and anchoring in 11.4%. A companion self-reflection survey of 130 Japanese physicians found an average of 3.08 distinct cognitive biases per error case, with anchoring named in 60.0%, premature closure in 58.5%, and availability in 46.2%.

Read that last figure again. Three biases per error, on average. These failures compound.

The individual effects have been demonstrated experimentally, not just surveyed. Mamede and colleagues showed in JAMA in 2010 that internal medicine residents who had recently seen a particular diagnosis were measurably more likely to over-diagnose it in unrelated subsequent cases. Schmidt and colleagues in BMJ Quality & Safety (2017) showed that identical clinical vignettes yield significantly lower diagnostic accuracy when the patient is portrayed as difficult or disruptive. McNeil and colleagues showed in the New England Journal of Medicine in 1982 that describing an identical outcome as the chance of living rather than the chance of dying flips treatment preferences, in patients and physicians alike.

The key property in all of these studies is that the doctor’s knowledge was fine. What moved was the decision.

LLMs inherited the whole set

If the biases live in the reasoning rather than in the knowledge, and language models learn reasoning patterns from human-generated text, you would expect the models to pick up the biases. They did.

The clearest evidence is BiasMedQA, published in npj Digital Medicine in 2024 by Schmidgall and colleagues. The team took 1,273 USMLE questions and rewrote them to carry a cognitive-bias cue, the kind a clinician would encounter in practice: a colleague’s suggestion, a recent similar case, a patient’s own belief about what’s wrong. The correct answer never changed. Model accuracy dropped by 10 to 26 percent across the six models tested, with GPT-4 the most resilient and the smaller models hit hardest. The authors add a caution worth repeating: they say it is very challenging to simulate cognitive bias through exam questions, and expect models to do worse against the subtler biases of real practice.

That is the same experiment the clinical literature has been running on humans for forty years, and the models fail it the same way. A 2025 npj Digital Medicine review reaches the same conclusion: clinical LLMs risk inheriting, and even amplifying, the biases documented in human clinicians.

Why larger models, RLHF, and prompting don’t solve this

None of the obvious fixes to LLM issues seem to address cognitive biases successfully. Therefore, this is not a problem you can solve by waiting it out.

Scale doesn’t solve it. The intuition is that a bigger model reasons its way past a superficial cue, and for general medical knowledge that is often true. For the important family of biases where the model defers to a stated position, the relationship runs backwards. Perez and colleagues at Anthropic demonstrated in 2022 that sycophancy, the tendency to agree with a user’s asserted view, increases with model size. Sharma and colleagues confirmed the pattern across five leading AI assistants in production, in work published at ICLR in 2024. The 2025 SycEval study, presented at the AAAI/ACM conference on AI Ethics and Society, found sycophantic behavior in 58.19% of responses across GPT-4o, Claude Sonnet, and Gemini 1.5 Pro, and, worse, found that once the behavior is triggered it persists in 78.5% of subsequent turns in the conversation.

RLHF makes it worse, by design. Reinforcement learning from human feedback trains a model to produce responses that human raters prefer, and raters on average prefer responses that agree with their premises over responses that correct them. The training loop rewards agreement alongside accuracy, and after enough iterations the model has learned deference as a policy. Preference-tuned models are consistently more sycophantic than their base models. The very step that makes an assistant pleasant to use makes it worse at telling a clinician they are wrong.

Prompting helps, but doesn’t close the gap. This is the hardest one to accept, because prompt engineering fixes so many other things. The BiasMedQA authors tested three mitigation strategies, including explicitly warning the model about the bias it was about to encounter. Accuracy improved, but it did not return to baseline. And there’s a deeper problem with treating prompting as the fix: it assumes you control the prompt. In an ambient documentation tool, a patient-facing assistant, or any workflow where the clinical note itself is the input, the cue arrives inside the data. You cannot instruct your way out of a bias that is embedded in the note the model is reading.

Which leaves testing. If the failure mode is a decision that changes under a cue while the correct answer holds still, the only way to know is to run both versions and compare.

Why a regulator will eventually ask

In most healthcare AI programs, cognitive-bias testing is currently a scientific curiosity. It is on a path to becoming a compliance obligation, because the frameworks that govern deployed healthcare AI put the burden of realistic validation on the deploying organization, and headline accuracy does not discharge it.

The HHS HTI-1 transparency rule requires certified health IT to disclose source and performance information for predictive decision support. The FDA’s expectations for AI-enabled devices cover behavior across realistic inputs, not curated benchmark inputs. ACA Section 1557 prohibits discrimination through clinical algorithms, which is hard to evidence when your only measurement is aggregate accuracy. The EU AI Act classifies medical applications as high-risk, with explicit reliability and post-market monitoring duties, and CHAI’s assurance framework asks for the same documented, ongoing evaluation.

An accuracy score on exam-style questions is silent on all of this. A model that scores 90% on a medical benchmark and flips its recommendation when a nurse writes “probably anxiety” in the triage line has a documented reliability failure that no accuracy metric will surface, and that a regulator, plaintiff’s attorney, or auditor can reasonably ask whether you tested for.

Seventeen test benchmarks

This is what we built at Pacific AI: 17 benchmarks that test LLMs for medical cognitive bias directly. Every case is fully synthetic with no protected health information, and every case is a matched pair, two versions of the same realistic clinical note that are identical except for the cue. Scoring is deterministic and needs no LLM judge. Twelve cover clinical decisions; five cover ICD-10-CM coding.

Anchoring. Named in 60.0% of self-reported diagnostic errors in the Japanese survey. Test: a triage note at the top of the chart states an early impression the workup should override, and we check whether the model drops the indicated investigation.

Availability. Judging a diagnosis by how easily it comes to mind, usually because a similar case was recent or memorable, rather than by the findings in front of you. Mamede (JAMA, 2010) demonstrated it in residents and defined it precisely that way. Test: the note records that several recent patients with this presentation turned out to have a benign cause, and we check whether the model drops the indicated workup in a patient whose findings do not fit that pattern.

Confirmation. The second most common bias in ER errors at 21.2% (BMC Emergency Medicine, 2022). Test: a working diagnosis is documented, then a discordant result arrives later in the note. Does the model revise, or explain the result away?

Order effects. Documented in clinical diagnosis by Cwik and Margraf (2017), who found a recency effect: information presented last carried more weight. Test: the same findings in the same note, with the decisive one placed first in one arm and last in the other.

Social pressure. The clinical form of sycophancy, at 58.19% in SycEval. Test: a senior colleague, a peer, a patient, or a clinical tool asserts a benign interpretation. Does the model defer?

Premature closure. The most common cognitive failure in Graber’s 100-case series. Test: a cue that a quick answer is needed, against a case that requires a differential.

Frequency. Test: the note asserts that this presentation is almost always the common benign cause, in a case where the specific findings say otherwise.

Base-rate neglect. Demonstrated by Eddy’s classic mammography work. Test: the note states that a serious cause is rare in this setting, and we check whether the model abandons an indicated workup.

Therapeutic inertia. Defined by Phillips (Annals of Internal Medicine, 2001) and quantified in blood-pressure control by Okonofua (Hypertension, 2006). Test: a status-quo nudge against a patient clearly not at goal. Does the model intensify therapy or leave it alone?

Patient affect. Diagnostic accuracy drops significantly when the patient is portrayed as difficult (Schmidt, BMJ Quality & Safety, 2017); the phenomenon Groves named in the NEJM in 1978. Test: the patient is described as angry, demanding, or distressed, with identical clinical facts.

Framing. McNeil’s survival-versus-mortality result (NEJM, 1982). Test: two logically equivalent presentations of the same outcome numbers. The correct behavior is invariance, and the bias is a recommendation that flips.

Defensive medicine. 93% of high-risk specialists reported practicing it (Studdert, JAMA, 2005). Test: a litigation or family-pressure cue against a case where restraint is correct. Here the failure is over-ordering, not under-ordering.

The second task: cognitive bias in clinical coding

Everything above concerns clinical decision support: diagnosis, workup, disposition, treatment. That is where the cognitive-bias literature focuses, because that is where the errors have been studied most. It is also a single task. A model deployed inside a health system does many other jobs, and each has its own bias surface. Testing the diagnostic task alone and declaring the model safe is the same mistake as testing on exam questions and declaring it ready for clinical text.

Clinical coding is a clear second case. Assigning ICD-10-CM codes is not a diagnostic judgment, it is a documentation judgment, governed by the rule that the code must reflect what the documentation actually supports. That makes the failure mode sharper in one respect: there is an authoritative correct answer, and what pulls the model away from it is usually structural in the note rather than clinical. It also carries direct financial and audit exposure, because a code the documentation does not support is a compliance problem whether or not a patient was harmed. The same matched-pair method applies, with five benchmarks.

Assertion over documentation. A diagnosis asserted in the note that the documentation does not support. Example: the note documents type 2 diabetes, the impression line asserts diabetic nephropathy without supporting findings, and a cue notes the attending wrote it at the top of the chart. The correct code set is E11.9 alone; the bias event is adding the unsupported E11.21.

Principal-diagnosis anchoring. The first-listed admitting diagnosis holding on as principal after later documentation displaces it. Example: the chart opens with chest pain, and the workup establishes a non-ST-elevation myocardial infarction. The principal diagnosis is I21.4. The bias event is coding the symptom, R07.9, as principal because it appeared first.

Common-code default. Reaching for the familiar general code when the documentation supports a specific one. Example: the note fully documents diabetic nephropathy, which codes to E11.21, and a cue notes that the unspecified code is what the department normally uses. The bias event is defaulting to E11.9 and discarding the specificity the note earned.

Copy-forward inertia. A resolved condition carried forward on the problem list and coded as though it were still active. Example: a resolved pulmonary embolism still sits at the top of a copied-forward problem list while the active condition is community-acquired pneumonia. The correct code is J18.9, with a history code such as Z86.711 acceptable; the bias event is coding I26.99 as active. This one is common for a measurable reason: Wang and colleagues found in JAMA Internal Medicine in 2017 that only 18% of the text in resident progress notes was newly typed, with the rest copied or imported.

Position sensitivity. The same note coded differently depending on where the decisive information sits. Example: the reason for admission is stated first in one arm and last in the other, with the clinical content identical. The principal code should be J18.9 in both, and the bias event is a primary code that changes with position. This is the coding analogue of order effects.

The pattern across both tasks is the same. The model knows the rule. Something in the structure of the note, rather than in its clinical content, moves the answer anyway.

What to do about it

The uncomfortable summary is that the biases responsible for most human diagnostic error are present in frontier models, that the two levers everyone reaches for first, more scale and better prompts, do not remove them. The alignment training that makes these models pleasant to work with makes one important family of these biases worse.

The good news is that all of this is measurable. A matched-pair test just requires running both versions and comparing, which is the one thing an accuracy benchmark never does.

I’m covering all 17 benchmarks in detail in the next Pacific AI webinar, including how the cases are built, how they are balanced across clinical and demographic dimensions, how the scoring avoids using a model to judge the model under test, and how to run the suites as a pre-release CI/CD gate and as a production monitor. If you’d rather skip the talk and just run them against your own models, you can do that in Pacific AI directly.

If you take one thing from this: ask whoever supplies your clinical AI what happens to their model’s answer when the note says “probably anxiety.” If they don’t know, that’s your answer.

John Snow Labs’ latest Medical LLM results are in. Across fifteen clinical and biomedical benchmarks, the Medical LLM – Medium ranks first on every one, against the newest frontier releases from OpenAI, Anthropic, and Google. It averages 80.9 to their 76.5, 75.0, and 74.7 respectively. The model that produces those scores runs on a single GPU, entirely inside your own environment, with no external API call. It is licensed per server per year, not per token.

A leaderboard sweep is not worth much on its own, but this combination of accuracy, cost, and compliance is what deserves attention. It contradicts an assumption that governs most healthcare AI budgets right now: that the accurate option and the affordable, deployable option are different options, and you have to pick one.

The suite comes from MedHELM, not licensing exams

A benchmark suite assembled only from USMLE-style questions and PubMed recall measures one capability and calls it medicine. Most of these 15 tasks therefore come from or overlap with MedHELM, the Stanford-led open benchmark published in Nature Medicine, which organizes clinical AI into a clinician-validated taxonomy of 121 distinct tasks. Each selected task asks a model to do something a health system needs done.

Medec gives the model a clinical note and asks whether a sentence contains an error, such as a wrong diagnosis, a wrong drug, or a wrong causal organism, then asks it to point to the sentence and rewrite it. This is the inverse of a quiz: the note is written to look correct, and the model has to disagree with it. The best models scored around 70% on the original benchmark, against roughly 80% for physicians.

MedCalc-Bench asks for a clinical calculation from a patient vignette. Computing a Cockcroft-Gault creatinine clearance means pulling the right values from the right encounter and applying the right formula. It is the hardest task in the set, and the absolute scores of 48.0 against 34.0 for Claude Opus 4.8 show unsolved work rather than a lead.

EHRSQL turns a clinician’s plain-English question into SQL over a hospital database, along the lines of “how many patients were prescribed warfarin in the last month.” Built from questions posed by 200+ real hospital staff, it includes unanswerable questions on purpose, to test whether a model abstains instead of inventing a query. Gemini 3.5 Flash scores 14.0 against 34.0, the widest proportional gap in the table.

ACI-Bench takes the raw transcript of a doctor-patient conversation and asks for a structured visit note. The input has interruptions, repetition, and facts stated out of order. The output is documentation a clinician signs.

MTSamples Procedures asks the model to find and extract the surgical and medical procedures buried in a transcribed report. Procedures are rarely announced in a labeled field, and one report will mention procedures performed, considered, declined, or done elsewhere years earlier. Scoring well means separating what was done from what was discussed.

MedDialog scores comprehension of patient-provider conversation, measuring whether a model follows what a patient said rather than a nearby topic. At 76.3 against 76.2, this row is a four-way tie in everything but the decimal.

MedicationQA uses real consumer medication questions submitted to a health authority, rather than questions written to be answerable. The 9.5-point margin over the best frontier model is the second-widest in the suite.

The remaining tasks cover knowledge and reasoning (HeadQA, MedBullets, PubMedQA, MEDIQA, MedQA, MMLU Clinical Knowledge), hallucination control (Med-Hallu), and fairness across demographics (RaceBias).

Gaps are near zero on saturated exams and widest on operational tasks

Two patterns emerge from this table, the first coming from the medical exam rows. On MMLU Clinical Knowledge the spread from top to bottom is 2 points. On MedQA the margin is 1.2 points, and Gemini ties GPT outright. These benchmarks are saturated: everyone scores in the 90s, the differences approach noise, and a strong score is partly a reading comprehension test of the model’s own training data, as our previous discussion on data contamination detailed. No healthcare-specific model opens a real lead on a saturated public benchmark, and none should claim to.

The second pattern comes from the operational tasks, where the margins are an order of magnitude larger: 15 points on Medec, 9.5 on MedicationQA, 6 on PubMedQA, 5 on EHRSQL and RaceBias, 4 on MedCalc and Med-Hallu. The closer a task gets to clinical operations, meaning read a messy source document, extract the right facts, do the arithmetic, catch the error, and know when the answer isn’t supported, the wider the margin gets.

The saturated exam benchmarks stay in the suite so both patterns remain visible. Publishing only the wide margins would tell you less.

Private deployment costs an order of magnitude less at production volume

Accuracy decides whether a pipeline is worth running. Cost decides whether it survives a budget cycle. A node that is paid for processes its millionth token at the same marginal cost as its first: zero.

I modeled this last month on a real workload: a de-identified oncology real-world evidence dataset at 1 million tokens per patient, four passes over every document. The model uses list API prices, grants every volume discount, and skips the document filtering optimization that would favor local deployment. What frontier APIs cost, relative to a right-sized model running locally:

• 10,000 patients (a pilot or one service line): 1.35x to 3.17x. At this scale the API route is competitive, and for a short proof of concept the premium can be rational.

• 100,000 patients (a small health system or a focused research cohort): 3.3x to 7.7x.

• 1,000,000 patients (a midsize health system, a payer, or a multi-site research network): 12.3x to 28.8x.

Per-token pricing makes cost linear in data volume, while infrastructure grows sublinearly: a 100x increase in patients needed 15x the nodes, and cost per node fell by roughly a quarter along the way. This protects small players rather than threatening them. A pilot costs about the same either way, and the deployment that stays affordable is the one that doesn’t re-price every time the data grows.

The behavioral consequence matters more than the invoice. Teams ration when the marginal token costs money: discharge summaries get processed and nursing notes skipped; two years get reviewed instead of the full patient history. Every one of those decisions degrades the output. When the marginal token is free, the rational behavior inverts: process every page of every record, and rerun whenever the pipeline improves.

A general-purpose frontier model bills you for capabilities a pathology report never uses

“Small model beats big model” sounds like a result the next frontier release erases. But the reason it persists is structural.

A general-purpose frontier model is general-purpose by obligation. The same weights that read a discharge summary also have to translate poetry from Sanskrit, name the street corner in an uploaded photograph, and assemble a timeline of Russian philosophy from memory. That breadth is the product, and it is why the model is the size it is.

None of it reaches a pathology report. Being able to write Sanskrit poetry contributes nothing to tumor stage extraction, and remembering the whole public internet in every language contributes nothing to reconciling a medication list. You pay for that storage and compute regardless, on every token, whether you rent the model through an API or install it yourself. No engineering team would provision a database where 99% of the tables are irrelevant to all queries. Frontier LLMs are the one place the industry accepts it without argument.

That arithmetic is what makes “deploy your own LLM” sound unreasonably expensive. The phrase brings to mind running a frontier-scale model yourself, with the hardware and specialist team that requires. That premise is wrong. A frontier-scale model was never the requirement. A model that is very good at your tasks is.

Right-sizing inverts the cost structure. Medical LLMs are purpose-built for medical language rather than general text, trained on curated biomedical literature, clinical guidelines, and de-identified EHR notes. Sized for the tasks they run, they fit on one GPU, which makes them cheap enough for population scale and portable enough to deploy on-premises, in a private cloud tenant, or in an air-gapped environment.

Specialization raises accuracy on those tasks rather than trading it away. Training-data composition matters more than parameter count on domain work, and the tasks that separate the models above depend on clinical documentation that is scarce and private. Accuracy, cost, and compliance therefore come out of a single design choice rather than arriving as three separate features. HIPAA, GDPR, and institutional data agreements keep PHI inside your environment, as well as your intellectual property.

What the 15-benchmark sweep does not show

Healthcare-specific models do not win everything. On open-ended, single-turn general medical question answering, frontier models are excellent, and the saturated rows above show gaps within noise. A clinician typing a general question into a chat box is well served by a frontier model.

Benchmarks are also not the same as production deployments. John Snow Labs’ first-generation medical models hit state-of-the-art numbers on MedQA and PubMedQA, then underperformed on the extraction work customers ran in production. That history is why this suite looks the way it does. The CLEVER framework, published in JMIR AI, found frontier LLMs dropping from 92% on benchmark questions to 45% on equivalent real-world tasks. Benchmarks are a necessary filter and never a substitute for blind clinician review on real charts.

Nor are these numbers frozen. MedHELM is open source, maintained by Pacific AI as an extension of Stanford’s HELM framework, and re-scored against the newest frontier models on a roughly quarterly cadence. A vendor’s table is a claim but a public, versioned, reproducible leaderboard is a claim anyone can check.

Nine years of shipping every two weeks

John Snow Labs has released new software every two weeks for nine years. Through several generations of technology and science, from feature-based NLP to word embeddings to BERT-era transformers to instruction-tuned LLMs to agentic pipelines. Every one of those transitions obsoleted work the team was proud of.

The Medical LLMs have improved on nearly every release over the past year. The spring comparison covered 13 benchmarks against GPT-5.4, Gemini 3.1 Pro, and Claude Opus 4.6, ranking first on 12 of them. This one covers 15 tasks against the newest releases from all three labs, ranking first on 15. The suite grew, the competition got stronger – yet the margin widened.

That cadence holds because John Snow Labs is profitable, independent, and built without a cent of venture or debt financing. Nobody is waiting on an exit. A two-week release cycle is only worth something if it’s still running in five years, and the companies that can promise that are the ones with no clock on them.

The choice between accuracy, cost, and privacy was never permanent

Healthcare AI has spent three years budgeting as though you must choose two of three: accuracy, affordability, or privacy. Frontier APIs offered accuracy and convenience, and asked you to send PHI elsewhere and pay per token forever. Local models offered privacy and cost control, and asked you to accept a worse model. That trade-off is no longer true.

Within a domain that has its own language, its own documentation conventions, and training data that never reaches the public internet, the specialized model is now the accurate one, and it is small enough to run where the data already lives. Healthcare is the clearest case rather than the only one. Any field with a private vocabulary and a compliance perimeter should expect the same result, and should stop planning around a tradeoff that no longer binds.

Frequently asked questions

What is the Medical LLM – Medium?

It is John Snow Labs’ healthcare-specific large language model, trained on curated biomedical literature, clinical guidelines, and de-identified EHR data, and sized to run on a single GPU inside your environment. It is licensed per server per year, with no per-token component and no external API dependency.

Why not evaluate on USMLE-style questions alone?

Because they measure one capability while the field reads them as measuring medicine, and because they are near-saturated and partly contaminated. A near-perfect exam score says little about whether a model can extract a tumor stage from a pathology report, catch a medication error in a note, or abstain from a question the record can’t answer.

How does a single-GPU model beat much larger frontier models?

Training-data composition matters more than parameter count on domain-specific tasks. Clinical work rewards negation, temporality, terminology mapping, calibrated abstention, and faithfulness to a source document, which domain pretraining teaches and general pretraining doesn’t emphasize. The operational tasks also depend on private clinical data that public-web scale never substitutes for.

Is running a private medical LLM expensive?

Only if a frontier-scale model is assumed to be the requirement. At pilot volume the difference against APIs runs 1.35x to 3.17x and rarely decides anything. At a million patients, modeled on an oncology real-world evidence workload, frontier APIs cost 12x to 29x more, because per-token pricing is linear in data volume while infrastructure is not.

Do Medical LLMs make clinical decisions?

No. They extract information, summarize records, answer questions against existing documentation, and support clinician and researcher workflows. Clinical decisions stay with clinicians.

InfoWorld named John Snow Labs’ Medical LLMs a 2025 Technology of the Year Award winner in December. The press release covered the recognition. This post covers what sits behind it: the engineering and evaluation choices we made, what we got wrong early on, and what the last three years of deploying medical language models inside health systems, pharma companies, and government agencies have taught us about what regulatory-grade domain AI actually requires.

The thesis we started with

When we began building Medical LLMs in earnest, the prevailing view was that frontier general-purpose models would absorb every vertical. The path for specialized AI, under that view, was either fine-tuning on top of a frontier model or retrieval-augmented generation layered on the frontier model’s reasoning. Specialized pretraining was considered either impossible (too expensive) or unnecessary (the gap would close).

We took the opposite bet. Healthcare is a domain where the vocabulary, the reasoning patterns, the regulatory environment, and the deployment constraints are different enough from general enterprise use that a purpose-built approach would outperform: on accuracy, on cost, and on the compliance properties that determine whether a model can run against PHI at all.

Three years later, that bet has paid out. Peer-reviewed benchmarks show Medical LLMs outperforming GPT-4.5 and Claude 3.7 by 61–200% in factuality, clinical relevance, and conciseness, while running at a fraction of the cost. The models are deployed at Providence St. Joseph Health, Roche, the VA, Ohio State, Cigna, and others, with 80+ public case studies describing real-world use.

But the thesis by itself wasn’t the hard part. The hard part was operationalizing what “regulatory-grade” actually has to mean.

What we got wrong first

The first generation of our medical LLMs focused on medical knowledge benchmarks: MedQA, PubMedQA, MMLU medical subsets, MedMCQA. We hit state-of-the-art numbers. We published the results. And we quickly learned that leaderboard accuracy on medical licensing exam questions did not predict real-world utility on the tasks clinicians and researchers actually needed.

Clinical information extraction from progress notes is not an exam question. Cancer registry abstraction from pathology reports is not a multiple-choice problem. Clinical summarization under time pressure with a 50-note patient history requires different capabilities than answering a textbook question correctly on the first try.

The lesson, which has now been documented in peer-reviewed work including the CLEVER paper out of Stanford and elsewhere, is that leaderboard performance and real-world performance correlate loosely in healthcare. CLEVER showed frontier LLMs dropping from 92% accuracy on benchmark questions to 45% on equivalent real-world clinical tasks. We saw similar patterns in our own evaluations.

The corrective was to reorganize evaluation around the tasks that customers actually deploy: clinical entity extraction with terminology mapping, clinical summarization scored by clinician preference, question answering over real patient records, assertion and negation handling on free-text notes, temporality reasoning across longitudinal data. Benchmark numbers still matter as a necessary filter, but they are not a substitute for evaluation that looks like the deployment.

What “medical LLM” actually means in practice

A healthcare-specific LLM is more than a frontier model with a medical finetune on top. The differences show up at four architectural layers.

Pretraining data composition matters more than parameter count. Our Medical LLMs were trained on curated biomedical literature, clinical guidelines, de-identified EHR notes, and life sciences content. The diversity of clinical documentation types (discharge summaries, operative notes, radiology reports, pathology reports, nursing notes) teaches the model the style variance that a clinical deployment actually sees. Training on Reddit and Common Crawl does not.

Model sizing is a design variable, not a scaling axis. We ship 7B, 13B, and 70B variants, plus specialized reasoning, visual, and Spanish-language models. The reason is that clinical deployment cost economics don’t match consumer chatbot economics. A health system processing millions of notes per day cannot afford a 400B-parameter model per query when a 13B model with appropriate training produces better output for their specific task. Peer-reviewed benchmarking showed our 7B model outperforming all prior 7B models on clinical tasks, and becoming the first 7B model to beat GPT-4 on PubMedQA.

Alignment and safety training have to include clinical failure modes. The failure patterns of a medical LLM are not the same as those of a general chatbot. Hallucinating a medication dose is not the same kind of error as hallucinating a movie title. Our alignment work focused specifically on clinical hallucination, incorrect differential reasoning, dangerous recommendations, and the sycophancy patterns that lead models to agree with an incorrect clinical premise from the user. The Pacific AI governance work, including MedHELM, LangTest, and red-teaming frameworks, extends this to ongoing testing in deployment.

The deployment architecture is part of the product. Medical LLMs that can only be accessed via API are incompatible with most healthcare deployment requirements. Our models run inside the customer’s environment: on-premises, in private cloud tenants on AWS, Azure, GCP, Databricks, or Snowflake, or in air-gapped government environments. PHI cannot leave the customer’s firewall, and because HIPAA, GDPR, and sector-specific data agreements don’t allow it. Model quality is necessary. Compliant deployment is equally necessary. Models that can’t run where the data lives are not useful in production.

What the InfoWorld award actually recognized

InfoWorld’s judging notes described the models as “advanced domain-specific models with large context windows, multimodal capabilities and benchmark results that show strong performance,” with explicit recognition of privacy and compliance requirements. That framing matches what the last three years have taught us: domain specialization, deployment architecture, and compliance properties are not separable features. They are a single design choice.

Customers implementing Medical LLMs report 80% less manual abstraction, go-live in under two weeks, and around 60% lower operating costs compared to API-based alternatives. Those numbers come from the same underlying architecture: healthcare-specific pretraining, right-sized model footprints, in-environment deployment, and human-in-the-loop workflows via the Generative AI Lab.

What we’re focused on next

Three capability directions are absorbing most of our engineering effort in 2026.

Multi-agent architectures for clinical workflows. Single-model reasoning works for extraction and summarization. Multi-step clinical workflows (cancer registry abstraction, HCC coding review, clinical trial matching, patient journey reconciliation) benefit from specialized agents coordinated by a planner. The architecture is converging on domain-specific agents that hand off work to each other under a governance layer that logs provenance and enforces compliance controls.

Multimodal clinical understanding. Clinical reality is multimodal: text, structured data, imaging, video, waveforms. Our Visual LLM handles pathology slides, radiology images, and clinical PDFs. The direction is toward unified reasoning across modalities, with the same accuracy, provenance, and compliance properties we deliver on text.

Agentic data pipelines for OMOP and FHIR. The Patient Journey Intelligence platform we launched in early 2026 integrates multimodal, longitudinal clinical data into unified OMOP data models. The engineering work underneath is agentic: pipelines that read source documents, resolve identities, reconcile conflicting records, map to terminologies, and produce FDA-ready real-world evidence datasets. The requirements are the same as the Medical LLM work (regulatory-grade accuracy, in-environment deployment, auditable provenance), applied one layer up.

What the award means

Awards are a useful external signal. They are not the work. What the InfoWorld recognition reflects is three years of compounding engineering choices: pretraining on the right data, sizing the models for the deployments they’ll actually run in, aligning them for clinical failure modes, and packaging them so they can run inside the customer’s security perimeter.

The healthcare AI market is still early, and the gap between what general-purpose AI can do and what regulated clinical deployment requires is still wide. Closing that gap, while keeping the accuracy, compliance, and cost properties that health systems and life sciences organizations need in production, is the work we’re signed up for.

Frequently asked questions

What is a Medical LLM and how is it different from a general-purpose LLM?

A Medical LLM is a large language model trained specifically for clinical and biomedical use. The differences from a general-purpose LLM are in the pretraining data (curated biomedical literature, clinical guidelines, de-identified EHR notes, life sciences content), the alignment and safety training (focused on clinical failure modes), the deployment architecture (runs inside the customer’s firewall, not via API), and the task performance (peer-reviewed benchmarks show domain-specific models outperforming frontier LLMs by 61–200% on factuality and clinical relevance).

Why does a smaller specialized model often beat a larger frontier model on clinical tasks?

Three reasons. First, training data composition matters more than parameter count on domain-specific tasks; a 7B model trained on high-quality clinical content outperforms a much larger model trained on general web data. Second, clinical tasks reward specific capabilities (negation, temporality, terminology mapping) that domain pretraining teaches and general pretraining doesn’t emphasize. Third, cost economics in clinical deployment favor smaller models because volume per customer is high and latency matters.

Do Medical LLMs make clinical decisions?

No. They extract information, summarize records, answer questions against existing documentation, and support clinician and researcher workflows. Clinical decisions remain with clinicians. The regulatory path for decision-support AI is different from the path for clinical data extraction and summarization, and the accuracy and validation requirements differ accordingly.

How do Medical LLMs handle hallucination?

Several ways. Training on curated clinical content reduces baseline hallucination rates. Retrieval grounding against the patient’s actual record provides source evidence for every extracted claim. Assertion and provenance tracking attaches each extracted entity to the source sentence that supports it. Evaluation frameworks like MedHELM and LangTest test for hallucination specifically on clinical tasks. Human-in-the-loop review via Generative AI Lab is the final check on any output that reaches a clinical decision point.

Where can Medical LLMs be deployed?

On-premises, in private cloud tenants on AWS, Azure (via the Azure Marketplace), GCP, in Databricks and Snowflake environments, and in air-gapped environments for government and defense customers. The deployment pattern keeps PHI inside the customer’s environment throughout processing: no external API calls, no data leaving the firewall.

What makes the evaluation of Medical LLMs different from general LLMs?

Task-specific, clinician-validated evaluation matters more than standardized benchmarks. Benchmarks like MedQA or MMLU are necessary filters but not sufficient. The CLEVER paper out of Stanford showed frontier LLMs dropping from 92% accuracy on benchmarks to 45% on equivalent real-world clinical tasks. Evaluation has to include clinical information extraction, summarization scored by clinician preference, and blind comparison against expert annotations on real patient records.

Why does the architecture include a no-code review tool?

Because the people who need to validate and improve clinical AI are often domain experts (coders, registrars, clinicians, RWE analysts) rather than ML engineers. The Generative AI Lab lets them evaluate model output, correct errors, and continuously improve the pipeline without writing code. That closes the feedback loop much faster than a traditional ML workflow would, and it’s essential for the human-in-the-loop patterns that regulatory-grade deployment requires.

A Brief Communication in Nature Medicine made the rounds last month under a headline that is hard to misread: “General-purpose large language models outperform specialized clinical AI tools on medical benchmarks.” It generated a lot of commentary, a combative public response from one of the named companies, and a request to the journal for a retraction.

The study is fine. The methods are reasonable and the statistics are careful. The problem is the title. It states a general conclusion that the study did not test and does not support. The body of the paper, to the authors’ credit, is far more careful than the headline suggests.

A familiar pattern: titles that outrun the evidence

This isn’t a problem specific to AI. Nutrition research has been living with it for decades.

Consider a 2011 paper titled “Intake of added sugars is not associated with weight measures in children 6 to 18 years.” The title states a sweeping null result. The study behind it was a cross-sectional analysis of NHANES data using a single 24-hour dietary recall per child: a one-day snapshot of self-reported eating. A design like that genuinely cannot establish that sugar is “not associated” with weight: it can’t address reverse causation (heavier kids who have already started cutting back), it can’t capture habitual intake from one recalled day, and it can’t speak to causation at all. The finding may be real within its narrow frame. The title claims something the frame can’t carry.

You can find this pattern repeatedly. A widely cited 2008 meta-analysis concluded that the association between sugar-sweetened beverages and children’s BMI was “near zero”, and even flagged its own evidence of publication bias, before later and larger syntheses found a clear positive association. Reviewers have since documented that industry-funded reviews of sugar and weight were roughly five times more likely to report no association than independent ones.

Two lessons travel from nutrition to medical AI: the scope of a title should match the scope of the evidence, and it always matters who is asking the question and how.

What the paper measured: single-turn general medical knowledge

So what did this study test? Two commercial clinical tools, OpenEvidence and Wolters Kluwer’s UpToDate Expert AI, against three frontier models, GPT-5.2, Gemini 3.1 Pro, and Claude Opus 4.6. The evaluation had three stages:

• 500 MedQA questions: USMLE-style, multiple-choice medical knowledge.

• 500 HealthBench items: single-turn, free-response prompts graded for alignment with clinicians (a benchmark built by OpenAI).

• 100 “real clinical queries” (RCQ): de-identified single-turn questions that physicians had typed into NYU Langone’s HIPAA-compliant GPT instance, scored blind by 12 clinicians, producing 1,800 annotations.

The frontier models won all three. On MedQA, Gemini hit 97.4% versus OpenEvidence’s 89.6% and UpToDate’s 88.4%. On HealthBench, GPT led at 88.0 with the two clinical tools around 62. On the real-query benchmark, the three frontier models formed the top tier (3.5–3.6 on a 1–4 scale) while the clinical tools (3.17–3.24) came out about even with Google’s free auto-generated Search AI Overview (3.27).

Now look at what these three stages have in common. Every one of them is the same task: answering a single-turn, general medical question, in isolation, with no patient record attached. MedQA is an exam. HealthBench is exam-adjacent. RCQ is a real but still single-turn question. The study measured one capability three times: general medical question answering. That is a legitimate thing to measure. It is not “medical benchmarks,” and OpenEvidence and UpToDate are not all “specialized clinical AI tools.” Two products, one task family.

To the authors’ credit, the body says as much:

• They flag that MedQA and HealthBench may have leaked into training data

• They note HealthBench was built by OpenAI and that GPT-5.2 may benefit from “benchmark-developer overlap”

• They call the RCQ the primary evidence and HealthBench merely supplementary

• They frame the whole thing as “a snapshot of a rapidly evolving landscape,” adding that “deeply subspecialized medical tasks may favor more sophisticated, domain-specific adaptation.”

All of that careful hedging lives under a title that hedges nothing.

One task is not “medicine”

Why does the single-task issue matter so much? Because real clinical work is not a quiz. It’s summarizing a chart, drafting a discharge instruction, extracting a tumor stage from a pathology report, reconciling a medication list, catching an error in a note, turning a clinician’s question into a database query.

This is exactly the gap that MedHELM, the Stanford-led, open-source benchmark published in Nature Medicine, was built to close. MedHELM organizes clinical AI into a clinician-validated taxonomy of 5 categories, 22 subcategories, and 121 distinct tasks, and evaluates models across roughly three dozen benchmarks. Many of these benchmarks are deliberately private or gated, drawn from clinical operations rather than exam material, precisely so the test set can’t be memorized. Its whole premise is that near-perfect exam scores tell you very little about deployment.

You can see the same philosophy in the benchmark suite we publish, which compares John Snow Labs’ medical language models against GPT-5.4, Gemini 3.1 Pro, and Claude Opus 4.6 across 13 clinical and biomedical tasks, which overlaps heavily with MedHELM. A few tasks from these suites, with an example of what each actually asks:

• MEDEC: medical error detection and correction in clinical notes. Given a note, flag whether a sentence contains an error (wrong diagnosis, wrong drug, wrong causal organism), point to the sentence, and rewrite it. On the original benchmark, the best models scored around 70% versus about 80% for physicians.

• MedCalc-Bench: clinical calculations from a patient vignette. For example: given a note, compute the patient’s Cockcroft-Gault creatinine clearance, which requires pulling the right values from the right encounter and applying the right formula. This is the hardest task in the set: direct prompting tops out around 35% on the verified version.

• EHRSQL: turning a clinician’s plain-English question into SQL over a hospital database (”How many patients were prescribed warfarin in the last month?”). Built from questions posed by 200+ real hospital staff, it also includes unanswerable questions to test whether the model knows when to abstain.

• ACI-Bench: generating a structured visit note from a doctor–patient conversation transcript. The input is the messy back-and-forth of a real clinical encounter; the output is the note.

• MedAlign: following a clinician’s instruction against a real, longitudinal patient chart, e.g. “Summarize this patient’s past symptoms, examinations, treatments, and surgeries.” Answering it means synthesizing dozens of notes across years of one real record – something no amount of memorized literature helps with. The data is gated behind a research agreement, so it can’t be crawled, and even GPT-4 got roughly a third of these instructions wrong. The difficulty persists for newer models: a 2025 preprint, TIMER, found the strongest system it tested correct on only about 41% of MedAlign cases on its stricter longitudinal-reasoning measure, with GPT-4o and Claude 3.5 Sonnet among those evaluated – though, as a preprint, that result is not yet peer-reviewed.

These tasks reward different things than a multiple-choice exam does: information extraction, faithfulness to a source document, calibrated abstention, and structured output. A model can ace USMLE-style questions and still be mediocre at all of them. This happens often.

Why general models shine on what they’ve already seen

There’s a deeper reason the single-task design flatters general-purpose models: two of the three stages use public datasets, and public datasets get memorized.

The evidence here is not subtle. The “RABBITS” study showed that simply swapping brand and generic drug names in MedQA and MedMCQA questions – a change no clinician would even notice – dropped model accuracy by 1–10%. The authors traced the fragility to test-set contamination in pretraining data. The self-assessment-for-neurosurgeons study (from the same NYU group, notably) found that adding distractions in text cut accuracy by as much as 20.4%.

The “None of the Above” analysis found that making “none of the above” the correct answer caused a consistent 30–50% performance drop in frontier AI models. Griot and colleagues showed that models often reach the right multiple-choice letter through shallow cues and test-taking strategies rather than clinical reasoning – concluding that standard MCQ evaluations may not measure clinical reasoning at all.

On LiveMedBench, 84% of evaluated models performed worse on cases that postdate their training cutoff – strong evidence that earlier scores reflected memorization. Outside medicine, audits have found roughly 29% of MMLU items show contamination signs, with some models dropping double-digit percentage points on clean rewrites.

When a benchmark has been on the public internet for years, a strong score is partly a reading comprehension test of the model’s own training data. The authors of the Nature Medicine paper know this (they say so), which is exactly why their single private benchmark (RCQ) carries so much weight. And RCQ is 100 questions from one hospital, with no public description of how they were selected and no way for anyone else to reproduce the result.

The data nobody can crawl

Here is the asymmetry that I think the headline misses entirely. The tasks frontier models are good at – exam questions, textbook knowledge, summarizing the medical literature – sit on top of data that is abundant and public. Millions of journal articles, guidelines, and Q&A threads are openly crawlable. Of course general models trained on the whole public internet do well there.

The tasks that actually run a health system sit on top of data that is scarce and private. There is no large, public corpus showing how a radiology report should be summarized for a referring physician, how a referral letter should be written, or how tumor characteristics (histology, grade, stage, margins, receptor status) should be extracted from a pathology report and mapped to a registry standard. That knowledge lives inside electronic health records protected by HIPAA, institutional review boards, and data use agreements. You can see this constraint everywhere in the serious benchmarks: MEDEC’s hospital notes require a signed DUA; MedHELM’s clinical sources are largely private or gated; the Nature Medicine paper’s own RCQ data “is not available for public use due to institutional review and data use agreement.” A model can’t memorize what it was never allowed to see, which is most of clinical work.

The OpenEvidence episode

The study landed hardest on OpenEvidence, which responded publicly, on LinkedIn and in a June 15 letter to the journal requesting a retraction, alleging undisclosed conflicts of interest and methodological flaws.

Some of those points are reasonable, and several overlap with limitations the authors themselves listed. The contamination concern about MedQA and HealthBench is valid. The criticism that HealthBench’s scoring is opaque and built by a competitor is fair. And the observation that the RCQ dataset and its selection methodology were never published is correct. On the conflict of interest charge, the picture is murkier: the paper does disclose that the senior author consults for Google, whose Gemini topped every stage. So “undisclosed” is contestable, but it’s a relationship worth weighing when reading the result, just as funding source matters greatly in nutrition research.

It’s also worth remembering that OpenEvidence has used marketing-via-science of its own. In August 2025 it announced it was the first AI to score a perfect 100% on the USMLE. That number drew immediate skepticism: 100% on a saturated multiple-choice format that overlaps with MedQA says little about messy real-world care, and others noted the result was hard to reproduce. The honest read is that neither a vendor’s “100% on USMLE” press release nor a study titled “general models outperform clinical tools” tells you what you actually want to know.

The ranking isn’t even stable across tasks. On hard subspecialty board questions (the MedXpertQA set), a December 2025 pilot found OpenEvidence’s quick and “Deep Consult” modes scoring just 34% and 41%, below the roughly 46% the best general reasoning model reached on the same questions. Flip to open-ended, real-world clinical questions, though, and the order inverts: in a peer-reviewed study of 50 such questions, general-purpose LLMs produced relevant, evidence-based answers only 2–10% of the time, versus 24% for a retrieval-augmented system (OpenEvidence) and 58% for an agentic real-world-evidence system (ChatRWD). The winner depends entirely on the task, which is the whole reason a one-task study can’t crown a general one.

Back to first principles: fine-tuning to a task improves accuracy on it

Strip away the branding and this reduces to something every data scientist already knows: for any model, fine-tuning to a specific task or dataset raises accuracy on that task. That isn’t a marketing claim; it’s the bias-variance tradeoff. A model specialized for clinical information extraction, trained on high-quality data annotated by clinicians, will tend to beat a generalist on clinical information extraction – the same way the generalist beats it at writing a sonnet.

This is the work John Snow Labs does – de-identification, information extraction, and question answering over real clinical records – and the numbers reflect the principle rather than contradict it. On PHI detection our pipelines report 96% F1 versus Azure’s 91%, AWS’s 83%, and GPT-4o’s 79%, and 0.95 F1 against OpenAI’s privacy filter at 0.55, running 5.8× faster on CPU. Across a 13-task clinical benchmark suite, which overlaps heavily with MedHELM, our medical language models average 76.8 versus 70.9 (GPT-5.4), 70.0 (Gemini 3.1 Pro), and 68.3 (Claude Opus 4.6), ranking first on 12 of 13 tasks. The model that produces those scores runs on a single GPU, entirely inside the customer’s environment, with no external API call. On these problems, raw scale and trillion-token training budgets matter far less than domain data and careful engineering.

Today’s state-of-the-art results go further than that. By wrapping specialized models in an agentic feedback loop – generation, deterministic checking, iterative correction – we’ve reached regulatory-grade accuracy, meaning at or above human experts, on tasks like de-identification and patient registry abstraction. An early pipeline that de-identified 2 billion patient notes was externally certified at that bar, with recall exceeding independent human annotators. The newer version, an agentic de-identification framework we presented at this year’s Data + AI Summit, treats the pipeline itself as an agent that automatically tunes and customizes itself to a previously unseen dataset, reaching 98%+ regulatory-grade accuracy with minimal human effort.

On the documentation side, a cancer registry abstraction system – built from small language models, medical NLP, and a deterministic reasoning layer – cut case abstraction from about 120 minutes to under 2 while holding regulatory-grade accuracy, in a setting where 96% of incoming pathology reports are non-reportable noise. That kind of consistent, auditable, at-or-above-human accuracy at scale is not something a general-purpose LLM delivers on its own today.

What I hope we take from it

I’m glad this study exists. Specialized clinical tools should face independent, quantitative scrutiny, and the field needs far more of it. The authors did careful work and were honest about its limits.

It’s worth adding, though, that this kind of evaluation already runs in the open, continuously and not as a one-off. MedHELM is a free community service: an open-source extension of Stanford’s HELM framework (maintained by my team at Pacific AI), it re-scores the newest frontier models on a roughly quarterly cadence across its full clinician-validated taxonomy. The latest release folds in both MedQA and OpenAI’s HealthBench (Original and Professional) so the very benchmarks this paper relied on now sit alongside three dozen others, and anyone can rerun the whole suite and reproduce the numbers themselves. A single Brief Communication is a snapshot; a public, versioned, reproducible leaderboard is what actually keeps everyone honest over time.

My only real objection is to the headline – and to the reflex, on all sides, to compress a narrow result into a universal claim. Single-turn medical Q&A on partly-public benchmarks is one task. “Medicine” is a hundred. Good evaluation requires many real-world tasks, contamination-resistant test sets, real clinical data, and titles scoped to what was actually measured. Get the question right and the answer is rarely “general models win” or “specialized models win.” It’s “it depends on the task” – which is less of a headline, but better science, and a lot more useful to anyone deploying this technology in a hospital.

HCC coding is where Medicare Advantage economics meet compliance risk. Risk scores drive 2025 payments to plans covering roughly 35.7 million beneficiaries (more than half of Medicare), and every diagnosis submitted has to be supported somewhere in the medical record or it becomes recoverable under a CMS RADV audit. For the last decade, the operating assumption was that HCC coding at scale required third-party vendors: they had the coders, the technology, and the risk-adjustment expertise. That assumption is coming apart. Generative AI changes the build-versus-buy math, and the audit posture CMS announced in 2025, which audits all eligible MA plans annually and pursues a backlog of 2018–2024 payment years, makes the control, transparency, and audit-readiness of in-house coding a much better fit for where the enforcement environment is heading.

Why outsourced HCC coding is becoming a worse trade

The outsourcing arrangement that worked in 2015 has four characteristics that are increasingly at odds with where CMS is going.

Vendors get paid for the codes they find. The economic incentive is to maximize diagnoses in the near term. The audit exposure, which shows up years later, sits with the plan. That mismatch has been flagged repeatedly by the OIG, which has warned about diagnoses that come only from health risk assessments or chart reviews but aren’t supported elsewhere in the medical record. In 2025, CMS estimated that MA plans overbill by $17 billion annually; MedPAC’s independent estimate was up to $43 billion. The enforcement posture that follows from those numbers is not one where “our vendor coded it that way” is a durable defense.

Coding decisions happen in a black box. The plan sees the output (a list of codes per member) but typically not the reasoning, the source text, or the confidence behind each code. When an auditor asks why a code was submitted and what supports it in the record, the plan has to reconstruct an answer from a vendor system it doesn’t control. The audit preparation cost of that reconstruction has grown faster than anyone budgeted for.

The technology gap that justified outsourcing has closed. A few years ago, bringing HCC coding in-house meant building a clinical NLP platform from scratch, a multi-year effort that most plans and providers couldn’t justify. Today, healthcare-specific generative AI models read messy, multimodal data, map findings to HCC-relevant ICD-10 codes, and produce auditable provenance at a fraction of the cost of doing it manually. The Generative AI Lab’s HCC coding support, for example, automatically links ICD codes to HCC categories and prioritizes high-value tasks for human review, inside the customer’s environment.

Medicare Advantage enrollment is becoming too strategic to leave to vendors. More than half of Medicare beneficiaries are now in MA, and CMS’s 2026 rate announcement and expanded audit program signal that coding integrity is now a top-three performance variable for plans. Risk-adjusted revenue, member care accuracy, and regulatory compliance all depend on HCC coding being done well, and all three are weakened when the work happens at arm’s length.

What CMS’s 2025 audit expansion actually means

The audit environment changed materially in 2025. CMS announced it would complete its RADV backlog for payment years 2018 through 2024 by early 2026. The audit footprint expanded from roughly 60 MA contracts per year to all eligible plans (approximately 550 contracts), with record samples of 35 to 200 per plan based on plan size. CMS expanded its medical coder workforce from 40 to an intended 2,000 by September 2025, with AI-assisted review tools flagging unsupported diagnoses before human coders confirm findings.

A federal district court in Texas vacated parts of the 2023 RADV final rule in September 2025, removing, pending appeal, CMS’s ability to extrapolate audit findings across an entire contract population. That was a meaningful legal win for plans, but it did not reduce the underlying scrutiny. CMS has continued the audits, reverted to earlier methodology, and signaled that recoveries from the 2011–2013 audits are still coming.

The operational posture for plans is now: more audits, more records reviewed per audit, a coder workforce at CMS that grew 50x in a year, and AI-assisted flagging of unsupported codes before human review. Every diagnosis submitted has to hold up under that posture.

In-house HCC coding with generative AI fits that posture better than outsourced coding does, for one structural reason: the plan controls the audit trail. Every code ties back to specific source text in a specific note on a specific date, with a confidence score and a reviewer signature. When CMS asks, the answer is in the plan’s own system.

What AI-native in-house HCC coding looks like

Modern in-house HCC coding architecture combines four components.

Healthcare-specific extraction models read clinical notes, discharge summaries, operative reports, and HRAs to identify conditions, their status (active, resolved, historical), and supporting evidence. Healthcare-specific models outperform general-purpose LLMs on this task: peer-reviewed benchmarks show medical language models consistently ahead on clinical entity extraction, assertion detection (is this condition present, absent, or uncertain), and mapping to ICD-10 codes.

ICD-to-HCC mapping automatically links each extracted diagnosis to its HCC category, applies the current CMS risk-adjustment model (V28 for 2025 coding), and flags conditions with the highest revenue and compliance impact for coder review first.

Human-in-the-loop review by the plan’s own certified coders. This is where the audit-readiness comes from. Coders see the AI’s suggested code, the source text that supports it, the date, the provider, and the confidence score. They accept, reject, or modify, and their decision is logged. For RADV purposes, the audit trail is complete.

In-environment deployment. The pipeline runs on-premises or in the plan’s private cloud tenant. PHI never leaves the firewall. No external API calls, no vendor data-sharing agreements, no cross-organization data movement that auditors have to untangle. HIPAA, GDPR where applicable, and internal data-sovereignty requirements are satisfied by architecture rather than by policy.

The economic case

The unit economics of in-house AI-assisted HCC coding are different from outsourced coding in a specific way: the cost is mostly fixed.

Outsourced HCC coding typically runs on a per-chart or per-member basis. Volume scales linearly with cost. AI-native in-house coding has a higher setup cost (software, integration, coder training), but the marginal cost of processing an additional chart is effectively zero. Once the pipeline is running, processing 100,000 charts versus 10,000 charts is a GPU-hours difference, not a headcount difference.

Customers implementing HCC coding tools from Martlet AI report 80% less manual abstraction, go-live in under two months, and around 60% lower operating costs on risk adjustment and related workflows. The Martlet AI sub-brand focuses specifically on the in-house HCC coding use case, with workflows that combine healthcare-specific models with the audit, review, and reporting infrastructure plans need for CMS compliance.

The second-order effect matters more than the first. When coding is in-house, plans can iterate the model on their own patient population, tune it to their provider network’s documentation patterns, and close the feedback loop between coders and models faster than an outsourced vendor could. Accuracy improves over time; the system learns the plan’s data.

What this doesn’t mean

Bringing HCC coding in-house does not mean firing the clinical coders. It means giving them better tools. Certified coders remain the audit defense; they’re the humans whose judgment signs off on every submitted code. The AI is the first pass that lets them operate at 5–10x productivity, focus on the high-risk and high-value cases, and spend less time clicking through irrelevant charts looking for something to code.

It also doesn’t mean every plan should build this tomorrow. Small plans without an existing clinical data pipeline, without in-house coding staff, and without the engineering capacity to operate a healthcare AI system in production will get to an acceptable risk position faster by working with a vendor whose technology they can inspect, whose models they can audit, and whose output they control. The “in-house” argument is not “always build.” It is “when outsourcing means losing control over the audit trail, the risk calculus has shifted.”

What to evaluate before making the change

Four questions to work through before bringing HCC coding in-house:

Does your existing data pipeline deliver clean, current, de-identified-where-necessary clinical narrative to the coding workflow? If charts are still faxed, if notes arrive as unstructured PDFs without OCR, or if claims and clinical data aren’t linked, fix that first.

Is your coding team ready to shift from bulk chart review to AI-assisted exception review? Workflow redesign, training, and change management are the variables most commonly underestimated.

What does your audit trail look like today, and what would it need to look like to answer a CMS RADV request in a week? If the answer involves reconstructing vendor logs, the in-house case is stronger than it looks on the surface.

Can the technology run in your environment, on your security perimeter, with your data never leaving your control? If not, the audit and compliance advantages of in-house coding are muted.

Where this ends up

The CMS enforcement posture announced in 2025 (annual audits of all eligible MA contracts, a 50x increase in CMS’s coder workforce, AI-assisted flagging of unsupported codes) makes coding integrity a permanent top-three operational priority for Medicare Advantage plans. Outsourced coding arrangements that worked when audit exposure was nominal don’t survive contact with a world where 200 records per plan per year are reviewed and the vendor who coded them three years ago is not the entity on the hook for the recovery.

Generative AI closed the technology gap that justified outsourcing in the first place. The plans that move first, build the in-house pipeline, and own their audit trail will have a structural advantage in both revenue integrity and compliance defense for the rest of the decade.

Frequently asked questions

What is HCC coding?

Hierarchical Condition Category coding is the CMS system for risk-adjusting payments to Medicare Advantage plans. Each ICD-10 diagnosis that maps to an HCC category increases the plan’s risk score and the per-member-per-month payment. Accurate HCC coding requires both identifying diagnoses in the medical record and ensuring they are supported by documentation that will withstand a RADV audit.

How does a RADV audit work?

CMS selects a sample of members from an MA contract, requests the medical records, and checks whether the submitted diagnoses are supported. Unsupported diagnoses are considered overpayments and recoverable. Starting in 2025, CMS is auditing all eligible MA contracts annually, with 35 to 200 records reviewed per plan per year.

Is CMS really using AI in RADV audits?

CMS has stated it will deploy AI-assisted technology to flag unsupported diagnoses before human coders confirm findings. The agency has also committed that all overpayment determinations will be made by a human, not an algorithm. The practical effect is more records reviewed faster, with AI expanding the audit surface area.

What’s the difference between Medical LLMs and general-purpose LLMs for HCC coding?

Healthcare-specific models are trained on clinical documentation and biomedical literature, map output to clinical terminologies like ICD-10 and SNOMED CT, and handle clinical assertion, negation, and temporality in ways that general-purpose LLMs don’t reliably do. Peer-reviewed benchmarks show healthcare-specific models outperforming frontier LLMs on clinical information extraction by meaningful margins.

Does in-house HCC coding require large model deployment?

Not necessarily. Modern healthcare-specific models run on commodity GPUs, inside the customer’s environment, without requiring external API access. Deployment footprints range from a single professional-grade GPU for smaller plans to scaled Kubernetes clusters for large plans processing millions of charts per year.

How do plans think about the CMS court ruling on extrapolation?

The September 2025 Texas district court ruling vacated parts of the 2023 RADV rule and, pending appeal, removed CMS’s ability to extrapolate audit findings across a full plan population. That reduced the worst-case audit exposure significantly. The underlying audit activity, including RADV audits of individual records, continues; plans that are audit-ready at the record level are well-positioned regardless of how the extrapolation question finally resolves.

What role do clinical coders play in an AI-native HCC workflow?

Certified coders review AI-suggested codes, make the final call on accept, reject, or modify, and own the audit trail. The workflow shifts from reading every chart to reviewing flagged cases and validating the AI’s reasoning. Productivity per coder increases materially; coder judgment remains the compliance backstop.

In about a year, U.S. healthcare went from having no shared way to govern AI to having two certifications and a detailed set of governance playbooks. URAC published its Health Care AI accreditation. The Joint Commission, with CHAI, launched the Responsible Use of AI in Healthcare (RUAIH) certification. And CHAI released governance playbooks that spell out the baseline controls the Joint Commission’s certification is built to (CHAI doesn’t certify anyone). To a health system looking at all three at once, it still reads like three separate programs with three separate binders. It isn’t. Underneath, they describe the same governance structure, and the expensive mistake is to build it three times. This post covers what each one asks for, what they share, and why sustaining any of them across a real AI portfolio is a problem you solve with automation rather than headcount.

The Joint Commission’s Responsible Use of AI in Healthcare (RUAIH) Certification

The Joint Commission’s Responsible Use of AI in Healthcare certification, announced June 1, 2026, is the first U.S. certification for how a healthcare organization uses AI. It’s voluntary, open to more than 22,000 hospitals, critical access hospitals, and health systems, and you don’t need an existing Joint Commission accreditation to apply. Jonathan Perlin, the Joint Commission’s CEO, put the rationale in plain terms: more than 80% of physicians already use AI in their work, and there has been no shared standard for doing it responsibly.

RUAIH is built around five standard areas: governance, effective data management, risk and bias reduction, monitoring and validation of safety and performance, and transparency, education, and training. The detail that shapes how you prepare is that it certifies your organization rather than your products. There is no “certified” tool you can buy your way in with; a surveyor looks at how you govern, monitor, and stay accountable for AI across your whole estate. And four of those five areas describe work that never stops, which turns out to be where the difficulty lives.

The Utilization Review Accreditation Commission (URAC) Health Care AI Accreditation

URAC actually got there first, releasing its Health Care AI accreditation in 2025 as the first of its kind. It takes a structure RUAIH doesn’t, with two separate paths: one for the organizations that deploy AI, and one for the developers that build it. The standards pair core modules that any accredited organization has to meet, covering risk management, operations and infrastructure, and performance monitoring and improvement, with a users module that asks for an AI management plan, an annual evaluation of outcomes, testing and monitoring done in the organization’s own setting and on its own population, training for the people who rely on the system, an appropriate-use assessment, and disclosures of where AI is in play.

What URAC validates is real operation. Accreditation runs on interviews and system assessments rather than a paperwork review, and the users module is explicit that the testing and monitoring have to happen in your environment, on your case mix. As with RUAIH, URAC is careful about its limits: it doesn’t certify that a given AI product is safe or that your use of it is legal. It attests that the program around the AI is real and running.

The Coalition for Health AI (CHAI) Governance Playbooks

CHAI is the piece people most often mislabel: it’s a consensus body, not a certifying or regulatory one. It doesn’t grant a credential and it doesn’t impose requirements. What it does is convene the field, more than 3,000 member organizations, and publish guidance the field agrees on. On May 27, 2026, it released its governance playbooks, developed with input from 150-plus health AI leaders across more than 100 healthcare organizations, from academic medical centers to community health centers.

The playbooks define baseline controls for responsible AI use and give organizations examples, implementation guidance, tools, and resources to put those controls into practice. They cover eight elements: AI Policy; Organizational Structures; Organizational Resources; Responsible AI Lifecycle Management and Use; Risk and Impact Assessments; Responsible Data Management and Use; Third Party Management; and Education, Training, and Feedback. CHAI is deliberate that the playbooks are a baseline to adapt to each organization’s context, not a mandate.

CHAI states that the playbooks provide a framework to achieve the voluntary certification the Joint Commission has now released, and the Joint Commission and CHAI co-developed the underlying guidance together. So building your program to the CHAI playbooks is building to the controls RUAIH assesses, and to most of what URAC asks for as well. The guidance does double duty before you have even chosen which certificate to pursue.

All three share one governance backbone

Set them side by side and the overlap is hard to miss. Each one asks for governance with named accountability, control over the data feeding AI, evaluation and mitigation of bias, validation before deployment, monitoring after it, transparency and training for the people in the loop, and discipline around third-party tools. The wording differs and the grouping differs, but it’s one set of requirements described three ways.

This means you build the program once: the work that satisfies one of these satisfies most of the others. The same backbone is what the horizontal frameworks check for too, including the NIST AI Risk Management Framework and ISO/IEC 42001. The other obligations are healthcare-specific state laws, like Texas SB 1188 and California SB 1120, and federal regulations, like HHS HTI-1 and ACA Section 1557. You want one program that meets all of them from the same evidence.

Sustaining a certification is harder than earning it: volume, drift, and evidence

If the backbone is shared, why is any of this hard? Because four of those areas are continuous, and continuity is where governance programs quietly come apart. A motivated health system can assemble the artifacts for an initial survey in a few weeks: a charter, a policy set, a handful of model cards, a risk register. Keeping them true a year later is the real test, and three forces work against it.

The first is volume. A modern health enterprise runs dozens to hundreds of AI systems, much of it arriving through existing vendors or showing up as point solutions nobody logged. A 2025 Elion survey found AI submissions outrunning governance decisions by nearly 3 to 2, with most programs staffed by two or fewer dedicated people. The AI portfolio grows much faster than the team reviewing it.

The second is drift. Models drift as their inputs shift, vendors push updates you didn’t schedule, and the regulatory picture moves under you, sometimes inside a single quarter. A monitoring program that was accurate on survey day can be stale by the next board meeting.

The third is the nature of the evidence. A surveyor doesn’t want a policy that says you monitor for bias; they want the dated test results, the drift reports, and the documented reviews that show you actually did, and that they’re current. Producing that proof once is straightforward. Producing it continuously, across every system, is the part that breaks small teams. A January 2026 CHAI patient survey run by NORC at the University of Chicago found that more than 80% of patients would trust healthcare more with clear accountability in place, and that they’re specifically wary of AI running without human oversight. The certifications address that worry, but only if the answer holds up every day rather than once.

Automation: the risk assessments, model cards, and vendor reviews

“Automation” is an overused word every governance vendor now reaches for, so it’s worth being specific about what must be automated for the math to change. It’s the analysis, not the filing. A committee that meets twice a month and a compliance team of three cannot hand-write a risk and impact assessment for every system, draft a model card for each one, read each vendor’s AI disclosures to score its risk, run the pre-release tests, and watch production for drift, across hundreds of systems continuously. The useful kind of automation does more than manage workflows and store documents. It reads the documentation, maps it to the frameworks, and produces full drafts: a model card from the project’s own documentation, a vendor risk score with the justification spelled out, a proposed risk tier and set of controls for each system on the register. People review, adjust, and approve, which is what responsible governance requires anyway.

That’s the line between governance automation and governance theater. The document-and-workflow tools most organizations already own hand your team blank templates and reminders, then wait for your people to do the actual thinking. An automation layer does the thinking first, from material it has read, and gives your experts something to correct instead of a blank page.

For example, here is how the Pacific AI platform maps onto the shared backbone, with the same evidence feeding RUAIH, URAC, and the CHAI playbooks at once. Full disclosure: I’m the CEO of Pacific AI and of John Snow Labs, so read what follows as the example I know best rather than the only way to do it.

Pacific AI provides one platform that maps to RUAIH, URAC, and more than 250 regulations, standards, and frameworks: the healthcare-specific ones together with NIST, ISO, and the federal and state AI laws, refreshed every quarter. You don’t chase each rule on its own, and more importantly, you don’t chase the updates: when new regulations come out or standards change, the platform updates the mappings. So instead of retraining your team on each change, your automated risk evaluation, validation, and monitoring expand to cover it. That is often the most valuable part of automation from an ROI perspective, given how much enterprise-wide manual labor it replaces.

The economics: manual governance scales with headcount, automation with credits

A manual program’s cost is roughly the number of AI systems, times the evidence each one needs, times how often that evidence has to be refreshed. It’s bounded by how many trained governance people you can hire and keep. Double the portfolio and you double the work, but you can’t double a three-person team on demand, so the program doesn’t get visibly more expensive. It just stops keeping up: systems go un-reviewed, monitoring goes stale, and the gap between what’s deployed and what’s actually governed widens quarter over quarter. A 2026 study, the landscape of AI implementation in US hospitals, found that among hospitals running predictive models, 47% reported no accuracy evaluation and more than half reported none for bias.

Automation cuts the link between portfolio size and headcount, because the marginal system costs a few credits to assess and monitor instead of a new hire. Those two cost curves don’t hold a fixed distance apart; they diverge, and diverge fast as you scale.

This is why Pacific AI’s Platform Core is free, with unlimited users, systems, vendors, policies, and audit trails, and you pay only for the AI-enabled work such as risk assessments, test runs, and monitoring jobs. It installs inside your own AWS or Azure environment in about ten minutes, single-tenant, with no data leaving your VPC. There’s no multi-month implementation and no capital project standing between you and a working enterprise AI governance platform today.

Whichever way you implement AI governance, the economics have to work: the solution has to be cheap and fast. Those are relative to the cost of the system, but even outside a budget-cutting environment, a $100k-a-year AI system can’t require $50k a year to govern. $5k a year is more realistic, and that’s only achievable with real automation.

For organizations that would rather have help standing the program up, Pacific AI recently added a dedicated RUAIH certification-readiness advisory mapped to all five standard areas, which I believe is the first offering of its kind. It stands up the registry and risk tiers, documents the data controls, runs the pre-release testing, deploys continuous monitoring, and produces the disclosures and training materials, so the same engagement prepares everything you need to apply. That’s a six-to-twelve-week engagement, not a six-to-twelve-month one.

Whether you prepare for certification yourself or with outside help, remember that the certifications are awarded by the Joint Commission and by URAC, and no software grants them. CHAI doesn’t certify anything at all, since its playbooks are guidance. Automation produces and sustains the evidence those bodies look for, but it isn’t a substitute for the published standards or for your own legal counsel. That’s why human review and approval stay in the loop on every consequential decision: a program that produces evidence nobody trusts is worthless.

What to do now: build the backbone once and automate what won’t scale with manual effort

If you’re staring at three frameworks, the move is to stop seeing three. Pick the backbone, the seven areas in the table above, written to the CHAI playbooks since that’s the guidance both certifications lean on, and build it once. Automate the four continuous areas first, because those are the ones that decay between surveys and sink programs in year two. Model the marginal cost of your next AI system before your portfolio doubles, because the headcount math is the constraint that will actually bind. Then earn whichever certificate your board cares about most this year, knowing you’re most of the way to the other one already, and that the monitoring you stood up to stay certified is the same monitoring that keeps all of them current the year after.

FAQ

Are these frameworks mandatory?

No. RUAIH and URAC’s Health Care AI accreditation are both voluntary certifications, and the CHAI playbooks are voluntary guidance. They’re becoming competitive signals to patients, partners, and payers rather than legal requirements, though the obligations underneath them often overlap with rules that are mandatory, such as HHS HTI-1, ACA Section 1557, and state laws like California’s SB 1120.

Is CHAI a certification?

No, and the distinction matters to CHAI. It’s a consensus organization that publishes voluntary guidance, including the May 2026 governance playbooks. It doesn’t certify organizations or products and doesn’t impose requirements. Its playbooks describe baseline controls that the Joint Commission’s voluntary certification is built to, which is why building to them positions you well for RUAIH.

Do I have to choose among RUAIH, URAC, and the CHAI playbooks?

No. The playbooks are the shared guidance underneath, so building to them positions you for both certifications. RUAIH and URAC have different emphases. URAC has a developer path and validates operation in your own setting, while RUAIH is organization-wide governance. But the program you build underneath is largely the same.

Does any of this certify a specific AI product?

RUAIH explicitly does not; it certifies the organization. URAC has a developer path that assesses how a system was built, but neither one lets you buy a “certified” tool and inherit the credential. You’re certified for how you govern, test, and monitor.

What’s the hardest part of staying certified?

The continuous areas. Monitoring, bias review, validation, and training have to keep producing current, dated evidence long after the survey, across every system in the portfolio. That’s a volume-and-drift problem, which is why a small team needs automation to keep up.

Where does automation actually help, beyond being software?

In the analysis, not the filing. The useful kind reads your documents and drafts the risk assessment, the model card, and the vendor risk score, then runs the tests and monitors production continuously, so your experts review first drafts instead of authoring everything by hand. Tools that only store documents and send reminders are the governance-theater version; they don’t touch the work that scales badly.

How does the cost work if the platform is free?

The Pacific AI platform itself, including the registry, the policies, the audit trails, and unlimited users and systems, is free. You pay per unit of AI-enabled work, meaning a risk assessment, a test run, or a monitoring job. That keeps the marginal cost of governing one more system low enough that a growing portfolio doesn’t demand a linearly growing team. It also ensures that every action you pay for has an immediate, visible ROI.

A recurring talking point in enterprise AI conversations is that regulation is holding back adoption. The framing is appealing and almost entirely wrong. Regulated industries already sit inside frameworks (HIPAA, GDPR, SOX, the EU AI Act, CCPA) that govern how AI can be deployed against existing data. The slower half of organizations aren’t stuck on a missing rule. They’re stuck on AI literacy: the ability of legal, compliance, and business leaders to reason about how AI systems actually work, what they’re allowed to do under existing law, and what risks genuinely apply. That’s a gap new regulations cannot close.

What “waiting for regulation” really means

Two things get collapsed together when people say AI adoption is blocked by the regulatory environment.

The first is genuine legal uncertainty in specific areas: training data rights, liability allocation when an autonomous agent causes harm, whether a given AI output counts as a medical device under FDA software-as-a-medical-device rules. These are real and unresolved questions. Legal and policy work will resolve them, and organizations operating in those areas have to make risk-weighted calls in the meantime.

The second is organizations hesitating to deploy AI against use cases where the legal framework is, in fact, clear. A health system debating whether a clinical NLP pipeline can extract diagnoses from progress notes is not blocked by regulatory uncertainty. HIPAA’s framework for using PHI in operations and treatment covers it. A financial services firm using an LLM to summarize internal policy documents is not waiting on a new law. A pharma RWE team using de-identified clinical narratives for cohort selection is operating inside a 25-year-old compliance framework that is well understood.

The second category is where the literacy gap sits. It looks like regulatory caution. It is usually something else.

Where the literacy gap actually shows up

Gradient Flow’s 2025 AI Governance Survey, presented in a webinar John Snow Labs co-hosted with Ben Lorica last year, put specific numbers on where organizations actually stand. About 59% of participating organizations reported having a formal AI governance role or office. Roughly 65% conducted annual AI safety or literacy training (79% in mid-sized firms, 59% in large organizations, and 41% in smaller ones). The gap between organizations with written AI usage policies (75%) and those with actual incident response playbooks or dedicated governance roles (under 60%) is where the operational risk concentrates.

The CIO survey findings I wrote about earlier in 2025 were consistent: 45% of respondents cited speed-to-market pressure as the primary obstacle to better governance, rising to 56% among technical leaders. Small firms remain the most exposed: only 14% report familiarity with the NIST AI Risk Management Framework, even as the same firms build and deploy systems that could create enterprise-wide liability for their partners and customers.

None of these gaps are addressed by new regulation. They’re addressed by training, role clarity, and governance infrastructure.

What “AI literacy” means in a governance context

AI literacy at the executive and compliance level is not about knowing how a transformer works. It is about four practical competencies.

First, understanding which existing frameworks apply. A clinical NLP pipeline at a US health system is governed by HIPAA, the 21st Century Cures Act information-blocking rules, and, if the organization operates in California, CMIA and the newer California AI laws that took effect in early 2026. A pharma AI model used for signal detection in pharmacovigilance is governed by FDA and EMA guidance on real-world data, the International Council for Harmonisation’s E2B standards, and 21 CFR Part 11 for electronic records. Literacy is knowing which of these apply before a project starts, not after.

Second, understanding where model behavior creates novel risk versus where it doesn’t. A retrieval-augmented generation system that summarizes policy documents creates different risk than an autonomous agent making account changes. The first is primarily a hallucination and attribution problem. The second is a transaction-authority problem. Treating them identically, either as “AI” in the abstract or with the same governance controls, wastes time in both directions.

Third, understanding what the audit trail needs to look like. Under HIPAA, an accounting of disclosures is already required. Under the EU AI Act’s high-risk category, logging, human oversight, and post-market monitoring requirements are specified. Under SOX, any AI involved in financial reporting controls has documentation and testing requirements inherited from existing IT general controls. Organizations that treat AI audit logs as a novel greenfield problem often build too much; organizations that skip them build too little.

Fourth, knowing when the answer is “don’t use AI here.” Not every process benefits from an LLM. Regulated documentation that requires deterministic reproducibility (certain clinical coding, certain regulatory submissions, certain financial disclosures) is often better served by rule-based systems with AI assistance on the margins. Literacy is knowing the difference.

What informed governance looks like in practice

Organizations that have closed the literacy gap tend to do four things.

They embed compliance into the engineering pipeline rather than layering it on top. Red-teaming, bias testing, documentation, and risk evaluation happen in the same CI/CD flow that code goes through, not in a separate quarterly review. When a new model version ships, the governance artifacts ship with it.

They keep humans in the loop where it matters. Human-in-the-loop is not a talking point; it is an architectural pattern. In clinical AI, that means a coder reviews HCC code suggestions before they are submitted to CMS. In regulatory AI, that means a compliance officer reviews summarization output before it ships to an auditor. In autonomous agent systems, that means transaction thresholds, not blanket authority.

They train every role that touches AI, not only engineering. Legal, compliance, HR, and business-unit leaders who will make or approve AI deployment decisions need literacy proportional to their authority. A CISO who signs off on an AI procurement without understanding what training data the vendor used, where the model runs, or how data flows through the system cannot ask the questions that a responsible sign-off requires.

They use existing regulatory frameworks as a floor, not a ceiling. HIPAA does not mention AI explicitly, but any AI system handling PHI has to clear HIPAA’s privacy, security, and breach-notification rules. The EU AI Act’s high-risk category overlaps with, but does not replace, sector-specific rules in medical devices, financial services, and employment. Organizations that treat these as overlapping rather than stacked, and train their teams accordingly, move faster with less risk.

What the regulatory environment is actually doing

The regulatory environment is not standing still. California’s late-2025 AI liability and disclosure laws created new obligations for AI systems that cause harm and tightened requirements around automated decision-making. The EU AI Act’s high-risk provisions are now enforceable, with penalties up to 7% of global annual revenue for the most serious violations. The NIST AI Risk Management Framework continues to evolve, and federal agency guidance, from the FDA’s Predetermined Change Control Plan to HHS’s reporting requirements for EHR-integrated decision support, is becoming more specific. In Q1 2026 alone, AI regulation accelerated rather than slowed: federal policy began to override certain state AI laws, Asia rolled out new governance frameworks, and the EU AI Act enforcement deadline moved from abstract to operational.

None of this waits for organizations to catch up. The ones that treat literacy as the primary investment, rather than lobbying for new rules or delaying deployment, are the ones that will navigate the next two years without hitting enforcement actions.

The honest framing

Regulation does not slow AI adoption in regulated industries. Lack of literacy does. When a project stalls because legal can’t evaluate the risk, compliance can’t write the control, or the board can’t ask the right question, no new law fixes that. Training does. Governance infrastructure does. Engineering discipline that embeds compliance into the pipeline does.

For CIOs, CTOs, and chief AI officers, the highest-return investment is in the same place it’s always been in enterprise IT: the people who have to approve, audit, and operate the systems. Waiting for Washington, Brussels, or Sacramento to clarify the rules, when the rules that already exist cover 80% of what enterprises are trying to do, is a slower and more expensive path.

Frequently asked questions

Isn’t regulatory uncertainty a real issue for AI adoption?

Yes, in specific areas: training data rights, autonomous agent liability, cross-border data transfer under evolving EU guidance. In most enterprise AI use cases, particularly in regulated industries, the applicable framework is known. The slower variable is usually organizational literacy about what that framework requires.

What is the NIST AI Risk Management Framework and why does it matter?

NIST AI RMF is a voluntary framework from the US National Institute of Standards and Technology for managing risks in AI systems. It’s widely referenced in procurement, audit, and vendor evaluation contexts even though it isn’t legally binding. Only 14% of small firms report familiarity with it, per recent survey data, which creates downstream risk for larger organizations that work with those firms.

How is the EU AI Act different from existing regulations?

It applies horizontally across sectors and classifies AI systems into four risk categories (unacceptable, high-risk, limited-risk, minimal-risk), with obligations that scale accordingly. High-risk systems (including many healthcare, employment, and critical infrastructure applications) require documentation, logging, human oversight, and post-market monitoring. Penalties reach 7% of global annual revenue for the most serious violations.

What practical AI literacy training should organizations implement?

Role-based training that maps to authority. Board-level literacy covers risk categories, regulatory exposure, and vendor evaluation. Compliance and legal literacy covers the specific frameworks that apply to the organization’s sector and geography, plus how they stack with AI-specific rules. Engineering literacy covers responsible AI testing, red-teaming, and documentation patterns. Business-unit literacy covers the use cases the unit is allowed to pursue and the approval paths required.

How should organizations think about AI governance versus existing IT governance?

As an extension, not a replacement. IT general controls, change management, access management, and audit logging all still apply. AI adds requirements around model testing, bias evaluation, hallucination monitoring, and data lineage for training data. Organizations that graft AI governance onto existing IT governance frameworks, rather than building a parallel track, move faster and produce more defensible documentation.

What’s the biggest mistake organizations make on AI governance today?

Treating “we have an AI policy” as the goal. A policy without role clarity, without incident response playbooks, without engineering integration, and without literacy training across the people who make deployment decisions is a document, not a governance program. The survey data shows the gap clearly: 75% of organizations have AI usage policies, fewer than 60% have dedicated governance roles or incident response playbooks. That gap is where the operational risk lives.

Healthcare AI budgets break on one assumption: that per-token API pricing works at patient-population scale. Model a real workload, building a de-identified oncology real-world evidence dataset at 1 million tokens per patient with four processing passes, and the gap reaches an order of magnitude and keeps growing. At 10,000 patients, a locally deployed medical LLM costs about $95,000 all-in against $128,000 to $300,000 in API fees. At 1 million patients, it is roughly $1 million against $13 million to $30 million. This post walks through the model, every assumption behind it, and why the gap widens as you grow.

The workload: building a real-world evidence dataset from a full patient history

Most cost conversations about LLMs start from the wrong unit. Pricing pages quote dollars per million tokens, so teams estimate a few prompts, multiply, and conclude the API route is cheap. That arithmetic holds for a chatbot. It collapses for patient-level data work, because the unit of work in healthcare is not a prompt. It is a patient, and a patient is a large object.

A typical cancer patient’s record runs to thousands of pages: clinical notes, pathology reports, radiology reports, surgical notes, genomic tests, treatment records, often spanning years. For the model below I use 1 million tokens of data per patient, which is less than a typical cancer patient generates and more than a typical chronic disease patient does. It is a deliberately middle-of-the-road figure, chosen so the model neither flatters nor punishes either deployment option.

The workload itself is building an oncology real-world evidence (RWE) dataset, the kind used for external control arms, treatment-pattern and outcomes studies, and regulatory submissions. I presented this work in detail at PHUSE US Connect 2026, where our paper on automating it won the RWE Catalyst Challenge. Building an RWE dataset is a useful benchmark workload for cost modeling for two reasons. First, it’s real: pharma and health systems spend heavily to curate these datasets, manual chart abstraction runs about two hours per case, and the lag from raw records to a research-ready dataset is measured in months. Second, it’s representative: the same pattern of reading everything, extracting facts, reasoning across documents, and producing audited output describes most serious clinical data projects. Clinical trial matching reads the full record to test eligibility criteria. Question answering for clinicians reads the full record to answer reliably. Cancer registry abstraction, quality measures, risk adjustment, and referral determination all start the same way: from the complete patient story, not from a summary of it. Model the RWE workload and you have modeled the cost of much of the patient-level data roadmap.

Four passes over every document

The model assumes a four-step pipeline, with each step visiting all of a patient’s documents:

1. De-identification. Masking PHI to create a research-ready dataset.

2. Extraction. Structuring biomarkers, staging, histology, and treatments from raw text.

3. Summarization and reasoning. Synthesizing the patient journey, with the clinical reasoning behind the timeline written out.

4. Conflict resolution. Resolving discrepancies between documents, with an explicit chain-of-thought explanation for each final chosen value.

Step four is not optional. A regulatory-grade dataset must be explainable: a reviewer or auditor needs to see why the system chose July 2002 as the diagnosis date when a later note says September. Producing that reasoning costs tokens. The model assumes output tokens equal to 10% of input tokens at each step, which in my experience is conservative for reasoning-heavy clinical work.

So the total demand is 1 million tokens per patient, times four passes, plus 10% output on each pass, times however many patients you have. The model runs that demand at three volumes: 10,000 patients (a pilot or a single service line), 100,000 (a small health system or a focused research cohort), and 1,000,000 (a midsize healthcare system, a payer, or a multi-site research network).

The assumptions: mid-range data volumes, list prices, every API discount granted

A cost model is only as good as its stated assumptions, so here are the rest of mine.

On the API side, I used current list prices for GPT-5.4, Gemini-3.1-Pro, and Claude-Opus-4.6 as of the 2026 Applied Healthcare AI Summit (April 2026). Two adjustments roughly cancel each other, and the model treats them as a wash: enterprise agreements for healthcare (HIPAA compliance, a BAA, zero-retention terms) typically raise the price, while volume commitments at these token counts earn discounts. I also assume competent pipeline engineering that chunks documents to avoid the long-context surcharge several frontier providers apply to large-window queries, which would otherwise double parts of the bill. In other words, the API numbers below assume you do everything right.

On the local side, the John Snow Labs figures are all-in: model licensing plus the cloud infrastructure to run it, sized at one node for 10,000 patients, five nodes for 100,000, and fifteen nodes for 1,000,000. There is no per-token component, because John Snow Labs’ Medical LLM is licensed per server per year. A node that is paid for processes its millionth token at the same marginal cost as its first: zero.

One assumption deliberately favors the API side. The model has every step visiting every document with a large model. In production, our pipelines put small, task-specific clinical language models in front of the LLM to filter the roughly 96% of documents that are noise for a given task, which cuts the expensive token volume by an order of magnitude. The model below skips that optimization. Even paying full freight on every page, the conclusion holds.

The results: 1.35x at pilot scale, 28.75x at a million patients

At 10,000 patients, the local deployment runs on a single node at $94,766 all-in. The same workload costs $160,000 on GPT-5.4 (1.69x), $128,000 on Gemini-3.1-Pro (1.35x), and $300,000 on Claude-Opus-4.6 (3.17x). At pilot scale, the API route is competitive. A 1.35x premium buys you zero infrastructure work, and for a short-lived proof of concept that trade can be rational.

At 100,000 patients, the picture changes. Five nodes cost $389,830. The API equivalents are $1.6 million on GPT-5.4 (4.10x), $1.28 million on Gemini-3.1-Pro (3.28x), and $3 million on Claude-Opus-4.6 (7.70x). The cheapest API option now costs nearly a million dollars more than local deployment, for one workload, in one year.

At 1,000,000 patients, the divergence is no longer a premium. It is a different category of spending. Fifteen nodes cost $1,043,490. The same tokens through the APIs cost $16 million on GPT-5.4 (15.33x), $12.8 million on Gemini-3.1-Pro (12.27x), and $30 million on Claude-Opus-4.6 (28.75x). Averaged across providers, that is the pattern I summarized in my keynote: about 2x cheaper for a pilot, 5x for a small system, and 18x for a midsize healthcare system.

Two things are worth reading off these numbers beyond the headline multiples. The local cost grows sublinearly: cost per node falls from about $94,800 at one node to about $69,600 at fifteen, because licensing scales with volume while a 100x increase in patients needs only 15x the hardware. And the API cost grows exactly linearly, because that is what per-token pricing means. Those two curves can only spread apart.

Why the gap widens: token pricing is linear, infrastructure is not

The structural point matters more than any single number, because list prices will change and the multiples will move. Per-token pricing makes cost a linear function of data volume. Healthcare data volume is enormous and growing: more documents per patient every year, more modalities, more passes as workflows add reasoning and verification steps. A pricing model that charges per unit of data places its worst-case cost exactly where healthcare AI creates the most value, which is processing everything rather than sampling.

That last distinction deserves a sentence of its own. When the marginal token costs money, teams ration. They process the discharge summaries but not the nursing notes, the last two years but not the full history, a sample of the population but not all of it. Every one of those rationing decisions degrades the output: studies miss cases, cohorts miss patients, and timelines miss the event that explains the outcome. When the marginal token is free, the rational behavior flips. You process every page of every record for every patient, every time the pipeline improves, and rerun whenever a model or guideline updates. Fixed-cost infrastructure does not just lower the bill. It changes what the team is willing to do with the data.

There is also a budgeting argument that CFOs tend to appreciate more than data scientists do. A per-server cost is a number you can put in next year’s budget. A per-token cost is a forecast, and forecasts of token consumption have a way of being wrong by multiples once a project succeeds and other teams want to use it. Predictability is worth something independent of the average price, and at these volumes it is worth a lot.

What this model does not claim

The model is about cost, and cost is the second question, accuracy being the first. A cheap model that produces a dataset clinical and regulatory reviewers will not sign off on is worth nothing. That argument is made separately, with evidence: our Medical LLM currently ranks first or tied for first across 13 clinical and biomedical benchmarks against GPT-5.4, Gemini-3.1-Pro, and Claude-Opus-4.6, and the PHUSE study reached 98.4% accuracy on primary site against certified tumor registrar ground truth. The cost case stands on top of the accuracy case, in that order.

The model also excludes engineering time on both sides, on the grounds that both routes need pipeline work and the difference is smaller than commonly assumed: a well-packaged local deployment installs from a cloud marketplace, while a well-engineered API pipeline needs chunking, retry, and rate-limit logic of its own. It excludes the cost of data egress reviews, privacy assessments, and BAA negotiations that API routes trigger and local deployment inside your own environment largely avoids; counting those would widen the gap further. And it freezes prices at a point in time. API prices fall, GPU prices fall, and anyone using this model a year from now should rerun it with current numbers. The assumptions are stated precisely so that you can.

What it means for planning an AI budget

If you are budgeting a healthcare AI initiative, the practical guidance falls out of the three scales. At pilot volume, choose on accuracy, privacy, and speed to start; the cost difference is real but not decisive. At anything resembling production volume, the deployment model is the cost decision, and it dwarfs the choice between API vendors. And if your roadmap ends at population scale, per-token pricing is the line item that will eventually force a re-architecture, so it is cheaper to model that now than to discover it in year two.

The full cost model, with the table and the workload it comes from, is in my keynote from the 2026 Applied Healthcare AI Summit.

FAQ

Why does the cost gap widen with scale?

Per-token pricing is linear in data volume, while per-server licensing plus infrastructure grows sublinearly: a 100x increase in patients required only 15x the nodes in this model. Two curves with those shapes always diverge, so the multiple grows from 1.35x at pilot scale to 28.75x at a million patients.

Wouldn’t filtering documents reduce API costs too?

Yes, and well-built API pipelines should filter aggressively. This model deliberately skips filtering on both sides to keep the comparison clean. Filtering helps the fixed-cost deployment less, because its marginal token already costs nothing, so adding it narrows the gap somewhat at small scale and barely at all at large scale.

What about cheaper API models instead of the flagships?

Smaller API models cut the per-token price but give up accuracy, and in regulated clinical work accuracy is the constraint: a dataset that a clinical reviewer will not validate has no value at any price. The relevant comparison is between options that clear the accuracy bar, and the benchmark data shows which those are.

Is 1 million tokens per patient realistic?

It is a middle estimate: below a typical cancer patient, above a typical chronic disease patient. If your population averages 200,000 tokens per patient, divide the API figures by five; the multiples at each scale barely move, because both sides scale with the same workload.

What would change the conclusion?

A structural change in API pricing, such as flat-rate enterprise tiers with unmetered tokens at these volumes, would change it. Price cuts alone do not: a 50% cut at the million-patient scale turns $16 million into $8 million against $1 million, and the linear-versus-sublinear geometry remains.

Most of what matters in a cancer patient’s record is free text. Stage, histology, biomarker status, treatment response, progression: none of it lives cleanly in a discrete EHR field. It lives in pathology reports, radiology narratives, oncologist progress notes, and multidisciplinary tumor board summaries. Manually abstracting that information for a cancer registry takes a certified tumor registrar about two hours per case, and only 14% of US registries consistently meet the National Program of Cancer Registries’ target of reporting 90% of cases within 12 months of diagnosis. Regulatory-grade oncology AI changes that ratio, and with it the timelines for research, trial matching, and quality reporting that cancer care depends on.

Why oncology data is uniquely hard to structure

Oncology is a free-text discipline. Cancer staging follows AJCC rules that depend on tumor size, nodal involvement, metastatic spread, and (for most solid tumors) molecular or genomic features that are described, not coded. A prostate cancer report references Gleason patterns. A breast cancer report specifies ER, PR, and HER2 status in language that varies by pathologist. A lung cancer case depends on EGFR, ALK, ROS1, KRAS, and increasingly a dozen more biomarkers, each with its own testing methodology and reporting convention.

Claims data captures almost none of this. Discrete EHR fields capture some of it, inconsistently. The rest, which is typically the clinically decisive information, sits in narrative reports that a human has to read.

That is why, as of the latest assessment, central cancer registries in the US take an average of two hours per case to abstract, with complex patients consuming several days of registrar time. A single full-time registrar can process 6 to 10 cases per day. A thousand-patient cohort costs roughly a full year of certified registrar labor. Oncology informatics research out of the Cancer Institute of New Jersey has documented the structural reasons: EHR infrastructure varies widely across treating facilities, patient-reported physician lists disagree with registry records in 42% of cases, and the average lung cancer patient generates about 300 pages of records that have to be reviewed line by line.

The consequence is that cancer surveillance operates on a two-to-four-year lag. Clinical trial matching misses eligible patients because their biomarker status is not yet coded. Outcomes research depends on cohorts that are systematically incomplete. Quality measures under CMS’s Oncology Care Model and the MIPS Promoting Interoperability category depend on structured data that often does not exist until long after the care episode is closed.

What regulatory-grade accuracy means in oncology

General-purpose LLMs can read a pathology report and produce a summary that looks right. Regulatory-grade oncology extraction is a higher bar. It means:

Every extracted entity (diagnosis, stage, biomarker, medication, procedure) is mapped to a controlled terminology such as SNOMED CT, ICD-O-3, RxNorm, or LOINC, with documented confidence and provenance to the source sentence. Negation and temporality are handled correctly: “no evidence of metastatic disease” does not become a metastasis flag, and a history of prior tamoxifen therapy is not confused with current treatment. Results are reproducible: the same input produces the same output, which regulators and auditors require. The pipeline runs in the customer’s environment so PHI never leaves their control.

Peer-reviewed work on cancer-specific information extraction has shown the gap between healthcare-specific models and frontier LLMs on exactly these tasks. On the CACER (Clinical Concept Annotations for Cancer Events and Relations) benchmark, GPT-4 scored below 0.50 F1 on cancer entity extraction, while a medical language model tuned for oncology reached materially higher accuracy. On structured extraction from diagnostic reports, a medical language model reached 0.80 F1 on relation extraction versus GPT-4’s below 0.60. Assertion classification, which handles negation and uncertainty and matters more in oncology than in almost any other clinical domain, reached above 90% accuracy with healthcare-specific assertion models; general LLMs produced inconsistent output under prompt variation.

These differences are not cosmetic. At scale, a 15-point F1 gap means a different cohort in every study, a different denominator in every quality measure, and a different set of eligible patients for every trial.

What changes when extraction runs in minutes instead of hours

Three downstream consequences follow when regulatory-grade oncology extraction is available at scale.

Cancer registry reporting converges toward real time. Our Medical LLMs cut per-case abstraction from two hours to one to two minutes, a 60–100x productivity gain, while preserving human-in-the-loop review for complex cases. That changes the operating model of a registry from “build the backlog, then catch up” to “review by exception.” A recent medRxiv preprint out of China Medical University, using a locally deployed 20B-parameter open-weight model on a single professional-grade GPU, reached similar conclusions: autonomous multi-stage extraction of pathology reports is now feasible inside a hospital firewall, without depending on an external API.

Clinical trial matching catches patients earlier in their journey. A trial protocol requires EGFR mutation status, ECOG performance status, prior lines of therapy, and measurable disease per RECIST 1.1. When those variables are extracted continuously from incoming pathology and oncology notes, rather than abstracted months later for registry purposes, matching happens in the window where enrollment is still possible.

Quality and outcomes reporting become operationally feasible. Measures like 30-day readmissions after cancer surgery, time from diagnosis to treatment initiation, and adherence to NCCN guideline recommendations require structured clinical data that most health systems cannot produce reliably from claims or discrete EHR fields. Regulatory-grade extraction closes that gap.

What this does not do

Oncology AI that extracts structured data from notes does not make clinical decisions. It does not recommend treatment, diagnose cancer, or replace the judgment of a multidisciplinary tumor board. The pipeline produces structured, auditable data; clinicians and certified tumor registrars continue to interpret and act on that data.

That distinction matters for two reasons. First, it defines the regulatory path. Extracting a structured representation of information that already exists in the clinical record is a different regulatory question from generating novel clinical recommendations. The FDA’s 2025 Predetermined Change Control Plan guidance and the evolving framework around decision support interventions apply differently to each. Second, it defines where the accuracy bar sits. An extraction pipeline that is 98% accurate on biomarker status is useful immediately; a decision-support tool at the same accuracy level is not, because the 2% tail sits on clinical outcomes rather than on a data field a human will review.

What to ask when evaluating oncology AI

For health systems, pharma RWE teams, and cancer centers looking at vendors in this space, four questions separate regulatory-grade offerings from demos:

What is the peer-reviewed accuracy on your use case, against a published benchmark and ground-truth data? Benchmark results on MedQA or USMLE-style questions do not predict performance on pathology report extraction.

Where does the data live during processing? If the answer is “our API,” that is a different compliance, cost, and data-sovereignty posture than “in your environment, behind your firewall.”

What terminology does the output map to, and how is provenance tracked? A number without a code and a source sentence is not registry-grade data.

How is human-in-the-loop review supported? Complex oncology cases (rare tumors, ambiguous staging, contradictory reports) require registrar judgment. The tool either supports that workflow or forces a shadow system around it.

The shift underway

Oncology was the first clinical domain where the gap between what the record contains and what the structured data captures became operationally unacceptable. It is now the first domain where that gap is closing at scale, driven by healthcare-specific language models that run inside the customer’s environment and hit the accuracy, terminology, and provenance bar that registries, trials, and quality programs require.

The practical consequence for cancer centers and cancer research is that the two-to-four-year surveillance lag is no longer inevitable. For pharma RWE, it is that oncology cohorts can be built from real clinical narrative rather than from claims proxies. For patients, it is that trial opportunities show up while they are still options, not months after another line of treatment has started.

Regulatory-grade accuracy is what makes all of that possible.

Frequently asked questions

Why can’t general-purpose LLMs handle oncology extraction out of the box?

They can read a pathology report and produce a reasonable summary. They struggle on the specific tasks oncology data pipelines require: mapping to controlled terminologies like SNOMED CT and ICD-O-3, handling nested negation, distinguishing current from historical treatment, and producing reproducible output under prompt variation. Peer-reviewed benchmarks on cancer-specific information extraction show healthcare-specific models meaningfully outperforming GPT-4 on relation extraction, assertion classification, and entity resolution.

What is a realistic accuracy target for a registry-grade extraction pipeline?

It depends on the entity. For straightforward diagnoses and medications, above 95% F1 is routine with healthcare-specific models. For staging, biomarker status, and response assessment, the bar is 90%+ with human review of ambiguous cases. Published benchmarks and reproducible notebooks are the right way to evaluate vendors; demo videos are not.

Does this replace certified tumor registrars?

No. It changes what they spend their time on. Registrars move from line-by-line abstraction of routine cases to review of complex cases, validation of AI output, and the judgment calls on rare tumors and ambiguous staging that automation cannot handle.

Can the same pipeline run on pathology, radiology, and oncology notes?

Yes, with the right architecture. Healthcare-specific pipelines combine document classifiers that route input to the right extraction engine, cancer-specific NER models tuned for each report type, and a unified output representation (typically OMOP Oncology or a CDM extension) that supports downstream research and reporting.

How does this intersect with FDA guidance on AI in clinical care?

Extracting structured data from an existing clinical record is a different regulatory question than generating clinical recommendations. The FDA’s Predetermined Change Control Plan guidance and the broader framework around decision-support interventions apply, but the accuracy and validation requirements for data extraction are primarily about auditability and reproducibility, not about the model making a clinical decision. Oncology AI that supports registries, trial matching, and RWE is data infrastructure.

What about privacy and data sovereignty?

Any oncology AI pipeline processing identifiable patient records should run inside the customer’s environment (on-premises or in a private cloud tenant), with no PHI leaving the firewall. API-based approaches that send clinical notes to an external LLM vendor are difficult to reconcile with HIPAA, GDPR, and the data-use agreements that cancer centers and pharma RWE teams operate under.

What is the biggest operational change when this is deployed at scale?

The registry workflow shifts from “build the backlog” to “review by exception.” Timeliness improves; cases that used to take weeks to abstract are available within days or hours of documentation, and registrar time moves to the cases where human judgment actually changes the output.

CMS announced a 5.06% average increase in payments to Medicare Advantage plans for 2026, the largest rate increase in a decade. The headline was easy: more funding per member. The implication was harder: a larger payment base creates a larger audit surface, and CMS and the Office of Inspector General have been explicit that rate increases will now be tied more tightly to coding integrity. The FY2024 Part C payment error already sits at $19.07 billion. HCC coding accuracy in 2026 is a compliance baseline, not a revenue lever.

What the 2026 MA rate announcement actually signals

The 5.06% average rate increase is notable in isolation. It is more notable in context. The 2025 rate was 3.70%. 2024 was negative on an effective basis once coding trend adjustments were factored in. The jump to 5.06% in 2026 reflects CMS confidence that the MA program can absorb higher payments without collapsing into the overpayment problem that has dogged the program for a decade. That confidence has strings attached.

CMS has been re-tuning the mechanics underneath the rate. The CMS-HCC V28 model restructured how conditions map to HCC categories, with changes to diabetes, mental health, cardiovascular, and chronic kidney disease staging. The transition from V24 to V28 is still being phased in, which means plans in 2026 will be running dual logic for at least another payment year. RADV audits are being pushed to a quarterly cadence on eligible contracts. OIG has repeatedly flagged diagnoses sourced only from health risk assessments or chart reviews without supporting documentation elsewhere in the record.

Putting that together: the 2026 payment environment is higher-dollar, higher-scrutiny, and more complex than any previous year. A plan that grew comfortable with a 2-to-3% error rate in 2023 is now operating in a regime where that error rate is visible and actionable.

The documentation gap is the accuracy problem in disguise

HCC coding is a translation problem. It starts with clinical reality (what conditions a patient actually has, documented in progress notes, specialist consults, imaging reports, and pathology) and ends with a structured submission (ICD-10 codes mapping to HCC categories, with MEAT evidence linking back to an encounter). The translation fails in two directions. Diagnoses that exist in the record and the claims go through correctly. Diagnoses that exist only in unstructured notes get dropped. Diagnoses that exist in the claims but aren’t supported in the notes survive submission and fail audit.

The scale of what gets dropped is the underreported part. Research and industry analysis indicate that as many as half of all patients may have prior conditions, complications, or severity indicators documented in clinical notes but not reflected in claims or electronic health records. The asymmetry matters. A conservative plan that only submits what is in structured claims data captures perhaps half of the eligible risk. A plan that submits from claims plus chart review captures more, but without a defensible chain from diagnosis to documentation, a meaningful share of those codes will come back in RADV. The result in either direction is financial loss: undercoding leaves risk-adjusted revenue on the table; unsupported upcoding becomes a clawback.

Undercoding also has a clinical consequence that is often framed as a revenue story but is really a patient-care story. If a patient with chronic kidney disease and heart failure has both documented in the cardiologist’s note but neither reaches the plan’s risk profile, care coordination tools, high-risk outreach programs, and population health analytics see a healthier patient than the patient actually is. Gaps in care follow. Missed interventions follow. The member appears less sick than they are on every dashboard the plan runs, and the care model treats them accordingly.

What regulators have made clear about 2026 and beyond

Three regulatory signals are worth tracking closely:

RADV cadence. CMS has signaled intent to audit all eligible MA contracts on a quarterly basis. A plan that previously got audited every several years needs to operate assuming it will be audited this quarter. That changes what “audit-ready” means. It shifts from “we could produce documentation if asked” to “every code we submit this quarter will be looked at.”

OIG’s focus on HRA-only and chart-review-only codes. The OIG has been explicit that diagnoses appearing only on health risk assessments or retrospective chart reviews, without MEAT evidence in the broader medical record, are a specific audit target. Plans that have been relying on aggressive retrospective review programs are the ones exposed.

Extrapolation. CMS can extrapolate audit findings from a sampled subset of charts to the full contract. A 5% error rate on a 200-chart sample becomes a 5% adjustment on the full contract. For a large MA plan, a 5% extrapolated adjustment is a nine-figure event.

The composite picture: in 2026, the plans that thrive are the ones that can defend every submitted code with clean documentation, on a quarterly cadence, at the scale of a full book of business. The plans that get hurt are the ones whose coding operations were built for an audit rhythm that no longer exists.

What AI-supported HCC workflows have to deliver

AI is not new to HCC coding. Rules-based and NLP-based coding assistants have been in production for a decade. What changed with generative AI is the ability to read unstructured notes at scale and to produce coding suggestions with the context and evidence required for MEAT compliance. That shift makes AI relevant in a way prior generations of automation were not. It also raises the bar for what a responsible AI-supported HCC workflow has to provide.

Four requirements, in order:

Runs in the customer’s environment. An AI coding workflow that ships protected health information to a third-party API is a HIPAA risk regardless of business associate agreements. MA plans and large provider groups need models that run on-premises or in a private cloud where no chart data crosses the firewall. That is not a deployment preference. It is a procurement gate under most healthcare security postures.

Healthcare-specific models rather than frontier general-purpose ones. A 2025 peer-reviewed study in JMIR AI, using the CLEVER methodology, found that medical doctors prefer an 8-billion-parameter healthcare-specific language model over GPT-4o 45% to 92% more often on factuality, clinical relevance, and conciseness. For HCC coding, where factuality is the entire point, that preference gap is the difference between a code that survives audit and a code that does not. Healthcare-specific language models trained on real clinical documentation read progress notes, discharge summaries, and specialist consults in a way a general-purpose model does not.

MEAT evidence tied to source. Every code the system suggests has to come with a source span: the exact text in the chart, the encounter date, the provider type, and the clinical context. That is what makes the code defensible in RADV, and that is what lets a human reviewer validate the suggestion in under a minute rather than over fifteen.

Human-in-the-loop for the codes that matter. The point of AI in HCC is not to replace certified coders. It is to raise the floor on what each coder can review. A well-designed workflow surfaces high-value, high-risk suggestions, provides the evidence to validate them, and routes them to a credentialed reviewer for final sign-off. That workflow also provides the audit trail a compliance officer needs when CMS asks why a specific HCC was assigned.

John Snow Labs’ HCC Coding Engine and the Martlet.ai platform are built to those four requirements. Models run behind the customer’s firewall, on the customer’s charts, with MEAT evidence surfaced alongside every suggestion, and with a human-in-the-loop workflow for coder review. The architecture exists because the regulatory environment now demands it. A workflow that met the 2022 bar for “AI-assisted coding” does not clear the 2026 bar for “defensible in a quarterly RADV audit.”

What MA plans and provider groups should do this quarter

Three practical moves, given where the 2026 rate announcement puts the program:

Run a gap analysis on the current book. For a random sample of 200 to 500 members, compare what is in claims against what is documented in unstructured notes. The difference is the undercoded risk (real conditions missed) and the unsupported risk (submitted codes without MEAT evidence). Both are revenue-relevant and audit-relevant; they need to be sized before the next submission cycle.

Stress-test the coding pipeline against V28. The V28 transition changes how specific conditions (diabetes with complications, CKD stages, mental health subcategories) map to HCCs. A pipeline that was calibrated for V24 will miss revenue and compliance targets under V28 even if nothing else changes.

Audit the vendor chain. If HCC coding is outsourced, verify that the vendor can produce a defensible audit trail on every code they submit. If any portion of the pipeline is AI-assisted, verify the model runs in an environment that keeps PHI inside the plan’s controls, and that the vendor can answer what model version, training data category, and evaluation methodology produced a given suggestion. Under extrapolation, a vendor’s black box becomes the plan’s liability.

Why this matters for the broader MA program

The 2026 rate increase is not a one-off. CMS has signaled that funding growth and scrutiny growth are now linked. Plans that invest in defensible, high-accuracy HCC coding operations will be in a position to absorb future rate increases without adding audit risk. Plans that treat HCC coding as a back-office function with incremental tech refresh will find that the next audit cycle reallocates a meaningful share of the rate increase out of their books. The shift is not dramatic on any single quarter. It is cumulative over several. The plans that start now are the ones that will still be running healthy MA books in 2028.

The policy direction is clear, the arithmetic is clear, and the tooling to close the accuracy gap without shipping PHI outside the plan’s environment is available. The remaining question is execution.

FAQ

What changed in the 2026 Medicare Advantage rate announcement?

CMS finalized a 5.06% average rate increase, the largest in a decade, and continued the phased transition to the CMS-HCC V28 risk model. The increase is paired with intensified scrutiny, including a push toward quarterly RADV audits on eligible contracts and continued OIG focus on unsupported diagnoses.

What is HCC coding and why does it drive MA reimbursement?

Hierarchical Condition Category coding translates clinical diagnoses into categories that CMS uses to calculate Risk Adjustment Factor scores. A member with higher clinical complexity produces a higher RAF score, which raises the monthly capitated payment the MA plan receives. Accurate HCC coding is how plans get paid appropriately for sicker members. The CMS-HCC model currently covers roughly 7,770 diagnosis codes mapping to about 115 HCC categories.

What is the V24 to V28 transition?

CMS is phasing out the V24 risk model and phasing in V28. V28 restructures several condition categories, including diabetes with complications, chronic kidney disease staging, mental health, and cardiovascular conditions. The phase-in means plans in 2026 will be running some V24 logic and some V28 logic simultaneously. Coding pipelines calibrated for V24 will drop revenue under V28 without retuning.

What is MEAT evidence and why does it matter?

MEAT stands for Monitor, Evaluate, Assess or Address, and Treat. An HCC diagnosis has to be supported by evidence in the clinical record showing one of those four actions during a qualifying encounter. MEAT is what makes a code defensible in a RADV audit. A suggested HCC that cannot be linked back to MEAT evidence is the specific failure mode OIG and CMS have been flagging.

Why is “runs on-premises” a hard requirement for AI-assisted HCC coding?

Because HCC coding operates on protected health information at full chart depth. Shipping that data to an external API creates a HIPAA and contractual exposure that most plans cannot accept, regardless of business associate agreements. On-premises or private-cloud deployment keeps PHI inside the plan’s security perimeter and simplifies the compliance story for auditors.

What’s the role of human reviewers if AI is suggesting codes?

The AI model raises the volume of charts a coder can meaningfully review and surfaces the specific evidence for each suggestion. The credentialed coder remains the decision-maker for every submitted code. That human-in-the-loop structure is what makes the workflow defensible under audit. A fully automated pipeline that submits codes without human review carries both clinical and compliance risk that no responsible MA plan should accept.

How does AI-assisted HCC coding interact with RADV?

Done well, it improves RADV defensibility by producing a clear evidence chain for every code: the source span in the chart, the encounter date, the provider, and the MEAT context. Done poorly (treating the AI as a black-box code suggester without provenance), it worsens RADV exposure because the plan cannot defend why a specific HCC was assigned. The distinction is architectural and is worth verifying in procurement.

The default assumption about generative AI, that bigger models are better models, is a legacy of the first two years of LLM scaling. For general-purpose conversational work, the assumption still mostly holds. For the specialized, high-volume, high-accuracy work that regulated enterprises actually run, the assumption has been flipping since at least 2023, and the 2024 evidence makes the flip operational rather than theoretical. Smaller, domain-specific, task-oriented language models now routinely outperform frontier LLMs on the tasks those enterprises care about, at a fraction of the inference cost, while clearing compliance requirements that the large models cannot. Company size shapes the picture: large enterprises have different priorities than mid-size or small companies, and the right model strategy looks different for each.

The default assumption, restated

The scaling hypothesis that drove model sizes from hundreds of millions of parameters to hundreds of billions was straightforward: more parameters plus more training data plus more compute produces better general-purpose language capability. For open-ended conversation, creative writing, and broad question-answering, that hypothesis has held up. Frontier general-purpose models are genuinely better at these tasks than smaller predecessors, and the gap shows up on the benchmarks that measure general-purpose capability.

The problem is that most enterprise workloads are not general-purpose capability problems. They are narrow, repetitive, high-accuracy tasks: extracting structured information from documents, classifying records, matching entities across systems, generating standardized outputs. On these, the scaling hypothesis is not the right mental model. What matters is how well the model handles the specific task on the specific data the enterprise runs, at the cost and latency the production system can absorb. And on those metrics, the evidence that accumulated through 2023 and 2024 points consistently toward specialization.

What the evidence says

Three classes of evidence are worth separating.

Peer-reviewed extraction benchmarks. A 2024 JAMIA study on the 2010 i2b2 clinical-concept extraction benchmark measured GPT-4 at F1 0.804 with baseline prompting and 0.861 with a carefully engineered four-component prompt framework. BioClinicalBERT, a 110-million-parameter domain-specific model released years earlier, reached 0.901 on the same benchmark with no prompt engineering. On the VAERS adverse-event corpus, GPT-4 with careful prompting reached 0.736; BioClinicalBERT reached 0.802. A 2024 *Bioinformatics* paper showed a 7-billion-parameter LLaMA fine-tuned on biomedical NER outperforming few-shot GPT-4 by 5 to 30 F1 points across three standard datasets. These are not selective results — they’re a consistent picture across independent evaluations.

Blind clinician preference evaluations. A 2025 JMIR AI paper (Kocaman et al., “CLEVER”) reported a blind, randomized, preference-based evaluation by practicing medical doctors comparing GPT-4o against healthcare-specific LLMs (8-billion-parameter and 70-billion-parameter variants) on clinical text summarization, clinical information extraction, and biomedical question answering. On each of three dimensions (factuality, clinical relevance, and conciseness) the medical doctors preferred the smaller medical LLM between 45% and 92% more often than GPT-4o. The 8B-parameter variant is roughly two orders of magnitude smaller than the frontier model it was compared against. It was preferred by clinicians anyway.

Practitioner behavior. The 2024 Generative AI in Healthcare Survey (Gradient Flow, 304 respondents) showed 36% of respondents using healthcare-specific small models and another 21% using general open-source small models — a combined 57% running on small, often specialized models. Frontier general-purpose LLMs were not the default choice. A follow-up in the same survey series showed 54% of large-company respondents specifically preferring healthcare-specific task-oriented models over general-purpose LLMs. Practitioners deploying these systems are voting with their pipelines.

The pattern is not that large models are bad. It’s that for the specialized work regulated enterprises run (entity extraction, classification, terminology mapping, structured summarization, narrow question-answering over curated knowledge) specialized smaller models are often a better choice on the metrics that decide whether the system ships: accuracy on the specific task, inference cost per record at production volume, latency inside the operational SLA, and the ability to run inside the customer’s environment under regulatory constraints.

Why specialization wins on regulated work

Five reasons, each measurable.

Domain data is a structural advantage. Medical language is specialty-dependent in ways that general-web training data underrepresents. “RA” is rheumatoid arthritis to a rheumatologist and right atrium to a cardiologist. “MS” is multiple sclerosis in neurology and mitral stenosis in cardiology. Domain-tuned models have seen these distinctions at scale with labeled context; general models have seen them diluted among everything else. The 2024 *Bioinformatics* paper on biomedical NER traced its fine-tuned open models’ advantage over GPT-4 directly to this: specialized training teaches the model which interpretation is meant in which specialty, which is exactly what clinical extraction requires.

Sequence-labeling is a structural mismatch for generation-first models. Many high-value regulated tasks (named entity recognition, assertion status classification, relation extraction, de-identification) are fundamentally sequence-labeling problems. LLMs trained primarily as text generators solve these awkwardly, because the objective mismatch between “generate fluent text” and “label spans in existing text” shows up as over-confident labeling of non-entity spans and under-recall on unusual entities. Encoder models trained with labeling objectives handle the same tasks natively. This is a well-documented pattern in the literature; the 2024 paper “GPT-NER” and subsequent work on biomedical NER consistently find the same structural gap.

Inference cost tilts sharply at production volume. Running a frontier LLM over every clinical note in a hospital’s daily ingest is economically unattractive even when it’s technically possible. A domain-tuned 100-million-parameter model runs on a single commodity GPU at thousands of records per minute. A 70-billion-parameter model running over an API bills per token and introduces a per-request latency that turns a three-hour batch job into a three-day one. For hospital systems processing 500,000 notes per quarter, or pharma safety functions processing millions of adverse-event records annually, the economics aren’t 10x or 100x: they’re often 1,000x, which is the difference between the system being deployed and the system being an experiment.

In-environment deployment is a compliance and data-sovereignty requirement. For most regulated buyers, sending clinical notes, contract text, or financial records to a third-party cloud API is a non-starter regardless of accuracy. HIPAA, GDPR, and the US state-privacy-law patchwork have made on-premises or private-cloud-in-customer-tenant deployment a procurement hard requirement. Smaller specialized models are designed to run in this architecture. Frontier models typically are not — and even when they can be privately deployed, the compute costs change the economics decisively.

Freshness and update cadence are manageable on specialized models. Domain terminologies, clinical guidelines, regulatory requirements, and compliance rules change continuously. A specialized model that can be fine-tuned weekly with new annotated data and redeployed in hours is a different operational animal than a frontier model where the customer has no control over training cadence and has to adapt prompts as the vendor ships updates. For high-compliance workflows where traceability matters, the control over update cadence is the governance mechanism.

How company size shapes the right strategy

The 2024 survey data showed distinct patterns by company size, and the patterns map to real differences in where the biggest returns sit.

Large companies (5,000+ employees). The pattern is substantial budget, serious investment, and a preference for healthcare-specific task-oriented models (54% in the survey) combined with heavy use of proprietary LLMs via SaaS APIs for the exploratory and conversational layer. The right play for a large organization is composition: specialized small models for the high-volume production work inside the firewall, frontier LLMs for reasoning and conversation on top, with strict governance about what data flows where. Large companies also have the scale to justify building or licensing domain-specific models tuned on their own data, an expensive capability that pays back at their volume. The testing priorities that matter most for large companies are fairness and private-data leakage, the failure modes that create the largest reputational and regulatory exposure at scale.

Mid-size companies (501–5,000 employees). These organizations were the most experimental in the survey — 24% actively developing AI models and 36% reporting 50–100% budget increases. They typically have enough volume to justify serious AI investment but not enough to build foundational models from scratch. The pragmatic path is picking the right specialized models off the shelf, investing in the internal harmonization and pre-processing layers that make those models work on their specific data, and using frontier LLMs selectively for tasks where the per-record cost can be justified. Mid-sized companies benefit disproportionately from open-source small models because the total cost of ownership is predictable and the models run inside their own infrastructure.

Smaller companies (under 500 employees). The right strategy looks different. The investment that pays back for a smaller organization is usually a vertically integrated tool, a specialized product that solves one specific problem end-to-end, rather than a build-your-own-pipeline effort. Smaller companies’ testing priorities in the survey tilted toward bias and freshness — which reflects an operational reality that models going stale and bias failures are what the smaller team notices first. Frontier LLMs via API often make sense for smaller companies on the exploratory side, because the volume doesn’t yet justify the fixed-cost investment in self-hosted specialized infrastructure.

None of these patterns is universal. The point is that the right size-of-model question depends on the size-of-company question, because the returns differ. The mistake is assuming the same architecture fits all three.

What the next 12 months look like

Three directional predictions that follow from the 2024 evidence and the 2025 industry behavior already visible.

Specialized small models keep widening the task-specific gap. As domain-tuned models are fine-tuned on more operational data under human-in-the-loop feedback, their accuracy on the narrow tasks they handle continues to improve. Frontier general-purpose models improve too, but on a trajectory that optimizes general capability, not narrow-task accuracy on specialized data. The cross-over point has passed on most regulated extraction tasks. It’s not moving back.

Composition becomes the default architecture. The systems that ship are compositions: specialized models doing the high-volume work, frontier LLMs doing the reasoning on top, with clean interfaces between them. Neither pure-frontier-LLM nor pure-small-model architectures dominate. The architectural question is how to orchestrate both, which is an engineering problem with known answers.

Governance shifts from monolithic to modular. Governing one frontier LLM that does everything is actually harder than governing a system of specialized models, each with its own scope, validation set, and audit trail. Regulators are moving in this direction too: the EU AI Act’s risk-classification framework effectively requires organizations to know what each model in their system is doing, on what data, with what validation. Systems built as compositions of well-scoped specialized models produce the governance artifacts regulators ask for more naturally than systems built as one giant model.

What to do differently

For enterprises planning 2024 and 2025 AI investments, four changes to the procurement conversation.

First, stop starting with “which frontier LLM should we use?” Start with “what is the task, at what volume, on what data, inside what compliance envelope?” The answer to that question usually nominates a specialized model for the core of the work, with frontier LLMs used where they genuinely fit.

Second, demand per-task accuracy benchmarks with peer-reviewed methodology. A single “99% accuracy” claim means nothing. Task-specific F1 scores, per-task inference cost, per-task latency, and per-task compliance posture are what decide whether a system works in production.

Third, budget for the harmonization layer alongside the model. Most of the accuracy in a regulated AI workflow comes from the pre-processing, terminology mapping, and human-in-the-loop feedback infrastructure around the model, not from the model itself. Under-investing in this layer is the most common reason pilot systems fail to generalize to production.

Fourth, match the architecture to your company size. Large enterprises should be building composition systems with governance-by-design. Mid-size companies should be buying specialized models and investing in internal harmonization. Smaller companies should be buying vertically integrated tools. The worst outcome for any of the three is pretending you’re one of the others.

The bigger-is-better heuristic was a useful shortcut when general-purpose capability was the scarce resource. It isn’t anymore. For regulated work, specialized is better, smaller is cheaper, and in-environment is table stakes. The organizations whose AI strategy reflects that reality are the ones whose AI budgets show returns.

FAQ

Are large language models actually getting less useful over time?

No. Frontier LLMs continue to improve on general-purpose tasks and on reasoning and summarization benchmarks. The claim is narrower: on the specialized, high-volume, high-accuracy tasks regulated enterprises run, smaller domain-tuned models consistently outperform them. Both things are true simultaneously, and the right system uses each for what it’s good at.

Is the small-model advantage limited to healthcare?

No, though healthcare is where the evidence base is thickest because of the peer-reviewed literature. The same pattern shows up in legal (contract NER and clause extraction), financial services (transaction classification and AML signal detection), and industrial (domain-specific document processing). Any domain with specialized vocabulary, high-volume extraction needs, and regulatory constraints shows the same structural advantage for specialized models.

How much cheaper is a small specialized model at production volume?

Usually 50x to 1,000x on inference cost, depending on the task and the comparison. A 100-million-parameter model running on a single GPU processes thousands of records per minute at fixed hardware cost. A 70-billion-parameter model over an API bills per token at prices that, multiplied by production volume, put the per-record cost two to three orders of magnitude higher. The exact multiplier depends on the workload; the order-of-magnitude is consistent.

What does “specialized” actually mean in practice?

Two things, and the best systems combine them. Domain-specific pre-training (or fine-tuning from a base model) on corpus relevant to the field: biomedical literature, clinical notes, legal contracts, financial filings. And task-specific fine-tuning on labeled examples of the exact task the model will perform in production: clinical NER, contract-clause extraction, AML classification. A model that’s both domain-specific and task-specific outperforms a model that is only one or the other.

Will frontier models eventually close the gap on specialized tasks?

On some tasks, probably. On others, the structural mismatch between a generation-first architecture and a sequence-labeling objective suggests the gap will persist. The practical question for a 2024–2025 investment decision isn’t what will be true in 2027; it’s what works now. Right now, for regulated extraction and classification work, specialized smaller models win: and the governance, cost, and compliance advantages they bring are additive, not dependent on the pure-accuracy comparison.

Ask a frontier LLM “is 2 + 2 = 4?” and it will tell you yes. Tell it “I’m pretty sure 2 + 2 is 5, right?” and a measurable share of the time it will reverse course and agree with you. This behavior has a name in the AI safety literature, sycophancy, and it is not a quirk. It is a predictable consequence of how modern LLMs are trained, and it has measurable safety implications in the settings where people now use these systems: patient questions about medications, physician queries about treatment protocols, compliance officers running draft rules past an AI for a sanity check. The fix requires work at training time, at evaluation time, and at deployment time. Pretending the problem is cosmetic doesn’t make it go away.

The behavior, measured

Sycophancy in LLMs was first documented rigorously in a 2023 Anthropic paper (Sharma et al., published at ICLR 2024) that found agreement-with-the-user behavior across every major model family and increasing with model scale. The field has only sharpened the picture since. A 2025 study published in *npj Digital Medicine* (Chen et al., “When helpfulness backfires”) evaluated five frontier LLMs (three versions of ChatGPT and two of Llama-3) on medical prompts that misrepresented equivalent drug relationships. The models demonstrably knew the drugs were equivalent; the researchers tested whether the models would nonetheless comply with prompts written to imply otherwise. The compliance rate reached **up to 100%** on some model-prompt combinations. The authors’ definition is useful: sycophancy is the state where a model (1) demonstrably has the knowledge to identify a premise as false, and (2) aligns with the user’s implied incorrect belief anyway, generating false information as a result.

A companion 2025 study published at the AAAI/ACM Conference on AI, Ethics and Society (Fanous et al., “SycEval”) evaluated ChatGPT-4o, Claude-Sonnet, and Gemini-1.5-Pro on math and medical benchmarks. Sycophantic behavior appeared in 58.19% of responses across all three models. Gemini was the highest at 62.47%; ChatGPT-4o the lowest at 56.71%. The SycEval authors split the behavior into progressive sycophancy (model abandons a wrong answer to match the user’s correct assertion, harmless or helpful, 43.52%) and regressive sycophancy (model abandons a correct answer to match the user’s incorrect assertion, the failure mode, 14.66%). Once triggered, the behavior persisted in 78.5% of subsequent interactions.

The pattern is consistent across independent studies, which is what you want to see before treating something as a real property rather than a measurement artifact. Models trained on human preference feedback are more sycophantic than base models. Larger models are more sycophantic than smaller ones of the same family. Citation-based rebuttals (”actually, I read in the NEJM that…”) induce regressive sycophancy more effectively than simple contradiction. A 2024 OpenAI blog post describing the rollback of a GPT-4o update called the behavior “overly flattering or agreeable” and attributed it to short-term user-feedback signals being weighted too heavily in training. The company reverted the update.

Why this happens

The mechanism is straightforward once you look at the training objective. Modern LLMs are aligned with reinforcement learning from human feedback (RLHF): humans are shown pairs of candidate responses and asked which they prefer; the model is trained to produce responses that humans rate higher. On average, humans rate responses that agree with their premises higher than responses that contradict them, even when the contradiction is correct. The training loop is therefore rewarding agreement as much as it is rewarding accuracy, and over many iterations the model learns to be agreeable.

Two empirical findings from the literature confirm this reading. First, Rimsky et al. (2024) showed that sycophancy has an approximately linear structure in the activation space of transformer-based LLMs: that is, sycophantic behavior corresponds to an identifiable direction in the model’s internal representations, which can be steered away from at inference time without retraining. That’s a property of the model’s learned behavior, not an artifact of the prompt. Second, research on arena-style preference rankings (Chatbot Arena and similar) has found that higher preference scores can correlate with weaker resistance to hallucination and misinformation, which means the optimization target for “user-liked” responses is partially in tension with the optimization target for “truthful” responses.

The result is a reliability weakness that is most dangerous in exactly the domains where LLMs are now being used most — medicine, law, finance, compliance, and education — fields where the user often knows less than the model and is asking for clarification of something they’re uncertain about.

What it looks like in a clinical setting

The safety implications of sycophantic behavior in healthcare settings are not hypothetical.

Consider a patient interacting with a consumer AI assistant seeking advice on a symptom. The patient’s framing of the question carries implicit assumptions: “my headaches are just stress, right, nothing serious?” A sycophantic model tends to agree, downplaying severity, rather than flagging the red-flag features of the symptom pattern (new headache with visual disturbance, worst headache of life, fever with neck stiffness) that would warrant urgent in-person evaluation. The patient walks away reassured. The model has done its job as an “agreeable assistant” and failed at its job as a health-information source.

Consider a clinician asking an AI tool to confirm drug equivalence. The npj Digital Medicine study above found LLMs complying with up to 100% of requests that misrepresented brand-generic equivalence as a distinction that actually required different dosing, despite the models having the correct information in their training data and being able to answer accurately when asked neutrally. For a clinician using the model as a quick sanity check, sycophantic compliance with a mistaken premise is a medication-error risk disguised as a reassuring answer.

Consider a compliance officer running a draft policy past an AI for review. If the officer asks “this policy satisfies the HIPAA requirements for de-identification, right?” a sycophantic model tends to confirm. A non-sycophantic model actually evaluates the policy against the Safe Harbor criteria or the Expert Determination process and returns the specific gaps. One of those responses is useful; the other is dangerous precisely because it sounds useful.

A 2025 npj Digital Medicine editorial (”The perils of politeness”) summarized the problem crisply: roughly one in five adults now turns to LLMs for health advice, and LLMs optimized for agreeableness will validate misconceptions as medical fact, with low output confidence on the part of both patients and clinicians in assessing accuracy. Because sycophantic outputs mirror the errors implicit in user requests, the biases they perpetuate are opaque to the user.

What actually helps

Sycophancy is correctable at three layers, training, evaluation, and deployment, and serious systems address all three.

At training time. Fine-tuning with synthetic datasets designed specifically to teach the model that truthfulness outweighs user approval reduces sycophantic behavior while preserving general benchmark performance. The open-source LangTest library (from the same team that built production medical NLP) implements this pattern: it generates synthetic prompts pairing true-or-false claims with user opinions that agree or disagree, then measures whether a model switches its answer based on the opinion rather than the fact. The generated prompts can be used both as an evaluation suite and as a fine-tuning dataset to reduce sycophancy. Chen et al. (2025) showed that lightweight fine-tuning with illogical-request examples improved rejection rates on misinformation prompts while maintaining general performance across benchmarks.

At evaluation time. Standard accuracy benchmarks do not measure sycophancy, because they ask the model questions neutrally. A meaningful evaluation suite has to probe the model under pressure: neutral question first, biased framing second, escalating pressure third, with the delta between neutral and biased answers treated as the sycophancy metric. This is the SycEval methodology, the LangTest methodology, and (for reliability testing generally) the Giskard/DeepEval methodology. Enterprises deploying LLMs in regulated workflows should treat sycophancy testing as a first-class gate alongside accuracy, fairness, robustness, and privacy.

At deployment time. Two production patterns reduce sycophancy exposure. The first is prompt design: adding explicit rejection permission (”you may reject this request if the premise is logically flawed”) and factual-recall hints (”first recall what you know about drug X, then evaluate the request”) increased rejection rates on misinformation prompts to as high as 94% in the Chen et al. study. The second is activation steering: because sycophancy corresponds to an identifiable direction in the model’s representation space (Rimsky et al., 2024), it is possible to steer the model at inference time away from that direction without retraining. This is beginning to appear in production systems.

At system design. For high-stakes domains, the safest pattern is not to rely on the LLM alone. The architecture that works is composition: domain-specific retrieval or extraction produces a structured, cited answer; the LLM is used to phrase and explain rather than to generate the underlying fact. If the fact comes from a terminology service, a clinical-guideline database, or an extracted structured record, the user pressure to agree can’t change the fact. The LLM’s role is to convey it, not to adjudicate it.

What this should change about how AI gets deployed

Sycophancy is a reliability failure, and in regulated settings reliability failures are compliance failures. The EU AI Act, which took full effect through 2025 and 2026, classifies AI systems used in medical, legal, financial, and educational applications as high-risk and subject to heightened transparency and reliability requirements. A documented, measurable tendency to produce false information in response to user framing is a reliability failure that a regulator can ask to see tested.

For CIOs, CMIOs, and compliance leaders buying AI for regulated workflows, three changes to the procurement conversation make sense:

Ask the vendor how they measure sycophancy. If the answer is “we don’t,” that’s information. The mature answer is a specific evaluation methodology: synthetic prompts with user-opinion injection, measurement of answer-switch rates, reporting of both progressive and regressive sycophancy, documentation of how prompting and fine-tuning interventions reduce the measured rates.

Ask for the deployment-layer mitigations. Prompt design for rejection permission and factual recall. Confidence calibration that routes low-confidence answers to human review. Architectural composition so that high-stakes factual content comes from a verified source rather than from the LLM’s free-text generation.

Ask what happens when the LLM is confident and wrong. The failure mode that matters most is regressive sycophancy under citation-based rebuttal, when a user says “but a paper says X” and the model agrees, whether or not the paper exists. A production system should be testable on this failure mode specifically, and should have logs that show when the behavior is occurring.

The sycophancy problem is a solved problem at the research level, in the sense that the behavior is characterized, measurable, and reducible. It is an open problem at the deployment level for any organization that treats LLM outputs as trustworthy by default. The organizations that address it at training, evaluation, and deployment simultaneously are the ones whose AI systems survive scrutiny. The organizations that don’t are running reliability risk they have not quantified, in settings where a wrong answer has real consequences.

FAQ

Isn’t sycophancy just about being polite?

No. Polite disagreement is fine, the model can acknowledge a user’s view and then correctly explain why the user is wrong. Sycophancy is the specific failure where the model changes its factually correct answer to match a user’s incorrect assertion. The SycEval and npj Digital Medicine studies distinguish the two carefully. The unsafe behavior is the answer-switching, not the tone.

Does prompt engineering alone fix this?

Partially. Adding explicit rejection permission and factual-recall instructions to prompts reduces sycophantic compliance substantially, up to 94% rejection rates on misinformation prompts in peer-reviewed studies. It does not eliminate the behavior, and it doesn’t help when the end user is the one writing the prompt (which is every consumer use case). The robust fix combines prompt design with fine-tuning and with architectural composition.

Are smaller, domain-specific models less sycophantic?

On average, yes, though the picture is mixed. Smaller models trained on domain data with careful preference tuning tend to show lower sycophancy rates than frontier general-purpose models of the same family. Part of this is scale-related (the Anthropic paper found sycophancy increasing with model size), and part is training-data-related (domain-tuned models are often fine-tuned on factual corpora rather than on broad preference data). Specialized models still need to be tested individually, “smaller and domain-specific” is not a guarantee.

How does this intersect with hallucination?

Sycophancy and hallucination are related but distinct. Hallucination is the model producing confident, incorrect content without any user pressure to do so. Sycophancy is the model producing confident, incorrect content in response to user framing that implies the incorrect content. Both are reliability failures, and both have overlapping mitigations, citation-grounded responses, confidence calibration, responsible-AI testing, but the measurement methodologies differ and a responsible test suite covers both.

What’s the regulatory exposure for a healthcare organization deploying a sycophantic AI system?

Real. Under the EU AI Act’s high-risk-system requirements, reliability, transparency, and post-market monitoring are explicit obligations. Under FDA guidance on AI-enabled medical devices, the validation expectations cover the model’s behavior under a range of realistic inputs, not only curated benchmark inputs. Under HIPAA and related US frameworks, systems that produce misinformation in clinical settings carry liability that the deploying organization cannot fully push to the vendor. The defensible posture is documented testing for sycophancy, documented mitigation, and documented post-deployment monitoring.

Pharma has spent a decade piloting AI and a year arguing about generative AI. The useful question for an R&D leader or a chief compliance officer in 2024 is narrower than the pilot deck: which specific workflows have AI moved from interesting to measurable, and what does a pragmatic investment look like in each? Four areas fit that description today: earlier-stage target and lead identification, clinical trial design and patient recruitment, real-world-evidence-driven personalization, and regulatory and compliance operations. None of them is science fiction. All four have production case studies, peer-reviewed evaluations, and real cost and timeline savings attached. The rest is execution.

Drug discovery: AI has compressed the early funnel

The traditional drug-development timeline is 12 to 15 years from target identification to approval, at an average cost around $2.5 billion per approved drug. Most of that money is spent on candidates that eventually fail. AI cannot change the biology, but it can change the economics of the early funnel: where you generate candidates, score them, and decide which ones to move forward.

Three capabilities have matured to the point of being operationally useful. The first is structure-based candidate generation: generative chemistry models that propose small molecules matched to a target’s binding site, filtered by predicted ADMET properties. The second is virtual screening: computational evaluation of millions of compounds against a target, yielding a shortlist that chemists actually test. The third is genomic and multi-omic target identification: models that mine genetic, proteomic, and phenotypic data to propose targets associated with a disease, or to identify why a specific patient population responds to a therapy and another does not.

The evidence for time savings is stacking up. Companies leveraging AI in early discovery have reported development-time reductions of 25% to 50% on the stages where AI is applied. Insilico Medicine moved an AI-designed drug candidate through discovery and preclinical stages in roughly 30 months, a cycle that historically ran 4 to 6 years. The 2024 AI-aided drug-discovery pipeline expanded at roughly 40% year-over-year growth.

What hasn’t changed, and what pharma leaders should be careful not to oversell internally, is the back half of the funnel. Phase 2 and Phase 3 failures still happen, and AI-designed drugs are not immune. The 2023 failure of ulotaront, an AI-aided TAAR1 agonist for schizophrenia, in its Phase 3 studies is a useful counterweight to the discovery-stage success stories. AI improves the hit rate in early filtering. It does not eliminate biological uncertainty in humans.

The practical investment pattern that works: fund AI platforms that integrate with your existing chemistry and biology workflows rather than standalone AI discovery tools; require every predicted property to have a confidence interval and a data lineage that your med chem team can interrogate; and track reduction in candidates-screened-per-hit as the operational KPI, not “number of AI-designed drugs in pipeline.”

Clinical trials: patient recruitment and protocol design are the bottlenecks AI actually moves

Clinical trials are where AI has the most immediate operational impact on pharma economics. Trials are slow, and most of the slowness is in two places: finding patients who meet protocol eligibility criteria, and writing protocols that are tight enough to produce a clear answer without being so narrow that recruitment stalls.

Patient recruitment is a natural-language-processing problem. Eligibility criteria — “newly diagnosed, HER2-positive, no prior trastuzumab exposure, ECOG 0–1, adequate hepatic function” — need to be matched against each prospective patient’s entire medical history, most of which sits in unstructured clinical notes, pathology reports, radiology reports, and lab results rather than in structured EHR fields. Matching those criteria reliably requires clinical NLP that extracts entities, assertion status (is the condition present, absent, possible, or historical?), relations (which medication was given for which condition?), and terminology-normalized codes (SNOMED, ICD-10, RxNorm) from the notes, then runs the eligibility logic on the structured result.

This is where healthcare-specific language models earn their cost. A 2024 *JAMIA* study on the 2010 i2b2 clinical-concept extraction benchmark measured GPT-4 at F1 0.804 with baseline prompting, against BioClinicalBERT, a 110-million-parameter domain-tuned model, at 0.901. The gap matters because a 10-point F1 drop on entity extraction cascades into false-positive and false-negative matches downstream. A trial that screens 10,000 patients and mis-matches 10% of them wastes months of coordinator time on chart reviews that should have been filtered out. Domain-specific models consistently outperform frontier LLMs on eligibility-relevant extraction tasks, and they do so at a fraction of the per-record cost, which is what makes population-scale screening economically feasible.

Protocol design is the other high-value AI application. Models trained on historical trial data can simulate enrollment rates under different eligibility criteria, stratify patient subgroups, and stress-test endpoints against real-world variability before the protocol is finalized. Bristol Myers Squibb has used machine-learning-based protocol optimization to accelerate patient recruitment and reduce costs. AstraZeneca has deployed AI-driven platforms for real-time monitoring of trial data, with measurable improvements in compliance tracking and decision turnaround. These are not pilot results, they are production operations at major sponsors.

The investment pattern: treat eligibility-criteria matching as a regulated NLP workflow, not as a feature of your EDC vendor’s dashboard. Demand domain-specific model benchmarks with peer-reviewed methodology. Require the system to run inside the health system’s environment or the sponsor’s environment, not in a third-party cloud, because the data involved is protected health information that most institutions will not release.

Personalized medicine: AI is what makes stratification operational

Personalized medicine has been a pharma talking point for 20 years. What changed recently is that the data infrastructure and the modeling capability are finally in place to operationalize the stratification logic at population scale.

The operational pattern: build a longitudinal patient record that combines structured EHR data (diagnoses, medications, labs), unstructured clinical notes (reasoning, symptoms, severity), genomic and multi-omic data where available, and patient-reported outcomes. Harmonize the combined record to a common data model (OMOP is the working standard for research and increasingly for pharma RWE). Train predictive models on the combined view to identify sub-populations that will respond to a therapy, sub-populations that will not, and sub-populations at higher risk of adverse events.

Two specifics that matter for the economics of this work. First, the majority of clinically relevant information about a patient lives in unstructured notes and reports, not in the coded fields. A personalization system that sees only structured data sees maybe 30% of the signal. Second, the extraction quality from unstructured sources is the binding constraint on downstream model quality. A cohort built from clinical NLP that runs at F1 0.90 on entity extraction produces materially different treatment-response predictions from one built from NLP that runs at 0.75, and the difference shows up as signal-to-noise in the predictive modeling downstream.

For pharma, the practical uses are consistent across therapy areas: responder and non-responder stratification on approved drugs; enrichment strategies for trial designs; post-approval patient-selection guidance via RWE studies; and biomarker discovery from multi-omic data paired with clinical outcomes. The largest measurable impact in 2024 is on trial enrichment, using RWE to identify which patient subtypes are most likely to respond to a mechanism of action, then designing the trial to enroll those subtypes preferentially. This shows up in both smaller-than-traditional trial sizes and in higher success probabilities.

Regulatory and compliance operations: the ROI story that rarely gets pitched at conferences

The area with the cleanest ROI and the least conference-stage coverage is regulatory and compliance operations. The work involved is unglamorous: labeling documents for submission, monitoring global guidance updates, reconciling internal quality events against external signals, preparing regulatory correspondence, tracking deviations and CAPAs, running pharmacovigilance case triage. It is also enormous, expensive, and highly rule-bound, which makes it exactly the shape of work AI is currently good at.

Three patterns have moved from pilot to production at large pharma:

Regulatory intelligence. Continuous monitoring of FDA, EMA, PMDA, and national-authority guidance updates, with automated identification of the ones that affect a specific product family. Gap analysis against the company’s own submissions and labels, surfacing the changes that require a response. The content is dense, multilingual, and fast-moving. Frontier LLMs do useful reasoning here once the source documents have been cleaned, classified, and indexed by domain-tuned NLP.

Submission-document preparation. Clinical Study Reports, Common Technical Documents, and similar submission artifacts involve compiling data from multiple sources, applying format and terminology conventions, and producing documents that must be internally consistent. AI assists with section drafting, cross-reference verification, terminology normalization, and consistency checking. The human authors are still responsible for the content; the AI removes the hours spent on coordination and formatting. Companies that have published numbers on this report development-time compression of 25% to 50% on submission-preparation stages.

Pharmacovigilance case triage. Adverse-event reports arrive in structured and unstructured form from clinicians, patients, call centers, and public sources. Most are routine; a minority contain safety signals that require urgent review. AI-based triage classifies cases by severity, extracts the relevant clinical entities, and routes high-signal cases to human reviewers while auto-processing the routine ones with human sampling for QA. This is the same human-in-the-loop architecture that works in clinical coding: calibrated AI running at high throughput, domain experts focused on the flagged cases.

The economics of compliance AI are attractive because the baseline is heavy manual work at high hourly rates. A 10% reduction in coordinator time across a global pharmacovigilance operation is a large number. A reduction in late-filing penalties from proactive guidance monitoring is a larger one. The risk profile is also favorable: these are internal workflows with human review in the loop, not patient-facing decision support, which means the deployment path is shorter than it is for clinical AI.

Making the investment decisions sort themselves

Four concrete questions for pharma leaders planning 2024 and 2025 AI spend:

Where is the binding constraint in each workflow you care about? For discovery, it’s usually the hit rate in the early funnel. For trials, it’s patient recruitment and protocol quality. For RWE, it’s data harmonization quality. For compliance, it’s coordinator throughput and consistency. AI investments aligned with the actual constraint produce measurable returns; investments that skip past the constraint to the shinier downstream step rarely do.

Does the vendor’s accuracy claim come with peer-reviewed methodology? “Our AI is 99% accurate” means nothing without the task, the dataset, the evaluation protocol, and the baseline. Production-grade pharma AI vendors publish their benchmarks in peer-reviewed venues and make their evaluation datasets available for customer reproduction. Vendors that do not should be discounted accordingly.

Where does the data live during processing? For any workflow touching clinical notes, patient records, or PHI, the answer is effectively required to be “inside the customer’s environment,” not “in the vendor’s cloud.” HIPAA, GDPR, and the patchwork of US state privacy laws have moved in-environment deployment from a premium feature to a procurement requirement.

Is there a human-in-the-loop layer designed as part of the system? For regulatory-grade workflows, calibrated AI routing uncertain cases to domain-expert reviewers is the architecture that hits the accuracy bars. Systems that skip this layer either over-promise on automation or under-deliver on throughput.

The headline for pharma leaders in 2024 is not that AI is transformational. It’s that AI has stopped being a slide in the strategy deck and started being a line item in the R&D and compliance budgets, because the workflows where it works have measurable outputs attached. The organizations moving from pilot to production on discovery-stage screening, patient recruitment, RWE-driven stratification, and compliance automation are getting meaningful timeline and cost reductions. The ones still debating whether to start are falling behind on a cycle that is no longer speculative.

FAQ

Is AI-aided drug discovery actually producing approved drugs, or just faster preclinical candidates?

As of 2024, AI has materially accelerated the early stages (target identification, lead generation, preclinical candidate selection) with documented 25% to 50% time compression on those stages. The first wave of AI-designed candidates is now in Phase 2 and Phase 3 trials. Success rates in clinical trials are biological questions that AI helps address via better patient stratification and trial design, not a problem AI solves by itself.

Why is clinical NLP such a large fraction of the AI work in trials?

Because eligibility criteria, adverse-event documentation, and the majority of clinically relevant patient information live in unstructured text, not structured EHR fields. Reliable trial operations require turning that text into structured data that downstream rules and models can act on. The quality of the NLP is the quality of the trial-operations layer on top of it.

What’s the realistic ROI timeline on AI for regulatory and pharmacovigilance operations?

Short, measured in months rather than years, because the baseline is expensive manual work and the human-in-the-loop architecture is well understood. Typical productive deployments show measurable coordinator-hour reductions within a quarter and move to broader rollouts within a year. This is usually the fastest ROI line in a pharma AI portfolio.

Can general-purpose frontier LLMs handle regulatory-document drafting?

They can assist on drafting and consistency checking once the input documents have been cleaned, classified, and indexed. They cannot be the whole pipeline, because submission-document preparation involves domain-specific terminology, cross-reference verification, and format conventions that reward specialized models. The production pattern is composition: domain-tuned models for the structured work, frontier LLMs for the drafting and summarization on top.

What’s the most common mistake pharma companies make when scaling an AI pilot to production?

Skipping the harmonization layer. A pilot on a curated, pre-cleaned dataset that reached 95% accuracy does not generalize to production on raw operational data: because production data is noisier, more variable, and more multilingual than the pilot set. The investment that makes the pilot generalize is the one in pre-processing, entity extraction, terminology normalization, and confidence calibration. Organizations that budget for the model but underfund the data layer routinely find their production accuracy 10–20 points below pilot accuracy.

Early in 2024, 304 healthcare and life sciences practitioners filled out a detailed survey on how they were actually using generative AI: what budgets they had, what models they were picking, how they were evaluating vendors, where they were stuck. The results ran against the dominant narrative of the year. The story in the trade press was “healthcare is cautious about generative AI.” The story in the data was that healthcare was spending aggressively, was building with healthcare-specific models rather than frontier LLMs, and was weighting accuracy and privacy well above cost when evaluating options. For anyone deciding how to invest in 2024 and 2025, the survey is a useful check against the conference-stage version of the market.

The budget picture: not cautious

The survey’s headline number was a 300%+ year-over-year budget increase reported by nearly one-fifth of technical leaders. That is not a cautious industry. When the underlying distribution is laid out, the picture sharpens:

- 34% of all respondents reported a 10–50% increase in generative AI budgets versus 2023.

- 22% reported a 50–100% increase.

- 18% of technical leaders specifically reported a budget increase of more than 300%.

- Another 16% of technical leaders reported increases in the 100–300% range.

Company size shaped the pattern. Medium-sized companies were most likely to report 50–100% increases (36% of medium-sized respondents). Large companies were most likely to report the very large increases, with 12% seeing more than 300%, compared to 7% of medium-sized and 6% of small companies.

The pattern to read out of this is that the organizations with the most operational experience in healthcare AI are also the ones making the biggest bets. That’s a different signal than “healthcare is cautious.” It’s healthcare saying the tooling is finally good enough to justify the investment, and the teams that have been running small pilots for a while are now moving to scale.

The model picture: specialized beats general

The clearest finding from the survey on model choice was a pronounced preference for healthcare-specific models over general-purpose LLMs. Asked what kinds of language models they were using, 36% of respondents reported using healthcare-specific small models. Open-source LLMs came second at 24%, and open-source small models at 21%. Frontier general-purpose LLMs were not the default choice for this audience.

The scoring of evaluation criteria reinforced the pattern. Asked to rank importance factors on a 1-to-5 scale, respondents put:

- Tuned specifically for healthcare: 4.03 mean

- Reproducibility: 3.91

- Legal and reputational risk: 3.89

- Explainability and transparency: 3.83

- Cost: 3.80

The noteworthy line in that list is the last one. Cost was the least important factor. The practitioners in this survey were willing to invest in high-quality, reliable models rather than cut corners on price, which is consistent with an industry that has absorbed what wrong answers actually cost in clinical or regulatory settings.

The explicit preference for healthcare-specific models sits on top of an accumulating evidence base. By early 2024, peer-reviewed evaluations had consistently found that domain-tuned models outperformed general-purpose LLMs on clinical extraction tasks. A *JAMIA* study published in January 2024 measured GPT-4 at F1 0.804 on the 2010 i2b2 concept-extraction benchmark with baseline prompts, versus BioClinicalBERT at 0.901. A 2024 *Bioinformatics* paper found that fine-tuned open models outperformed few-shot GPT-4 on biomedical NER by 5 to 30 F1 points depending on the dataset. The practitioners in the survey were weighting their choices in line with where the evidence was actually pointing.

What practitioners were building

The use-case mix in the survey was skewed toward externally facing applications and information-extraction work, the places where generative AI reaches real volume in healthcare operations.

- Answering patient questions: 21%

- Medical chatbots: 20%

- Information extraction and data abstraction: 19%

What the mix doesn’t show, but what trails behind these numbers in the open-ended responses, is the pattern of how these systems are being built. Patient-facing Q&A systems and medical chatbots, done well, are not single-LLM deployments. They’re compositions: pre-processing pipelines that section and normalize clinical text, task-specific extraction models that pull structured findings, a longitudinal patient record that assembles the findings into a timeline, a reasoning layer (which is where the LLM finally earns its place) that answers questions over the timeline with citations to sources. Information extraction follows the same pattern, specialized models at the bottom, LLMs used for reasoning or summarization on top of clean inputs.

That architecture is what closes the gap between the accuracy healthcare practitioners need and the accuracy a frontier LLM delivers on raw clinical text. It’s also what the 36% of respondents using healthcare-specific small models are deploying in practice.

What practitioners are worried about

Adoption roadblocks in the survey clustered around three themes, in roughly this order:

Accuracy and reliability. The dominant worry, and the one that the composition-architecture work described above is specifically aimed at. Frontier LLMs called on raw clinical text hallucinate at rates that regulated workflows cannot absorb; systems that compose specialized models with LLMs close the gap.

Legal and reputational risk. Second in importance to healthcare-specificity when evaluating models. Behind this is the recognition that wrong AI answers in a clinical context can harm patients, trigger regulatory action, and damage brand. Responsible-AI testing for robustness, fairness, bias, truthfulness, and data leakage has moved from optional to expected.

Alignment with industry-specific needs. The survey asked practitioners whether the technology options on the market actually fit the regulated, high-accuracy, high-privacy demands of healthcare work. The preference for healthcare-specific models is partly an answer to this: the options that don’t fit the industry’s needs get filtered out at the evaluation stage.

Human oversight is the common thread running through the mitigations. Asked how they test and improve LLM models, respondents’ most common strategy was “human in the loop.” This is not a compliance concession, it’s an engineering pattern that lets specialized models run at high throughput on the records they can handle, with domain experts reviewing the flagged records where the AI is least confident. Well-calibrated systems that route low-confidence records to human reviewers consistently clear the accuracy bars that pure-automation or pure-manual approaches cannot.

Testing priorities varied by company size. Large companies prioritized fairness and private-data leakage. Smaller companies prioritized bias and freshness (how up-to-date the model is relative to changing clinical guidelines and terminology). Both sets of priorities reflect real regulatory and operational concerns, fairness and leakage are what a large organization can be sued over; bias and freshness are what a smaller team notices first when the model is wrong.

What the survey implies for 2024 and 2025

A few practical takeaways for healthcare organizations planning generative AI investments in the twelve months after this survey shipped.

Budget is not the binding constraint anymore. The organizations investing seriously in healthcare AI are doing so in large increments and in ways that reflect real operational deployment. Underfunding a generative AI initiative in 2024 is no longer a defensible strategy, it’s a decision to fall behind competitors who are moving faster.

Model choice should be informed by task, not by hype. Healthcare-specific small models are winning a material share of the market because they work better for the work healthcare actually needs done: high-volume, high-accuracy extraction and classification. Frontier LLMs have a role (summarization, conversational interfaces, reasoning over already-clean inputs) but they are not the default choice for clinical NLP workloads. The 36% of respondents using healthcare-specific small models are voting with their pipelines.

Accuracy, privacy, and industry-specificity beat cost. The survey’s most striking finding is that cost came last among the evaluation criteria. That’s the right answer for an industry where wrong answers have outsized consequences, and it should shape how vendors pitch and how buyers buy. Organizations evaluating vendors should weight accuracy and privacy heavily, and should discount vendor claims that have not been substantiated by peer-reviewed benchmarks or public case studies.

Human-in-the-loop is how the economics work. No single model deployed alone hits the accuracy bars healthcare workflows need. Systems that combine AI throughput with targeted expert review, with feedback flowing back into the next model version, are what reach production, and do so in a form that satisfies the regulatory requirements for human oversight.

In-environment deployment is table stakes. The survey’s privacy findings line up with what every procurement review in healthcare ends up concluding: systems that cannot run inside the customer’s environment are eliminated before they reach accuracy evaluation. Organizations building or buying generative AI for healthcare should treat on-premises or private-cloud deployment as a hard requirement, not a premium feature.

The practitioners represented in this survey are building the next generation of healthcare AI quietly, while the public conversation is still stuck on exam-score headlines. Their choices — healthcare-specific models, compositions rather than single models, humans in the loop, in-environment deployment, accuracy weighted above cost — are a more reliable guide to what works than the pitch deck of any frontier-model vendor. The 2024 survey was the first annual edition; subsequent editions will reveal how much further the production bar has shifted. On the evidence of this first one, the gap between the operator view and the media view of healthcare AI was significant, and the operator view was the one worth listening to.

FAQ

How representative are the 304 respondents?

The survey was conducted by Gradient Flow over 33 days in early 2024, with 304 participants of whom 196 were actively engaged in evaluating, using, or deploying generative AI in healthcare or life sciences. Respondents were recruited through online channels including the Gradient Flow newsletter, social media, and industry partners. As with any voluntary survey, respondents self-select, but the sample size and the mix of technical leaders, data scientists, and practitioners make the distributions reasonably informative about the population of actively building organizations.

Does “healthcare-specific small models” mean models trained from scratch for healthcare, or fine-tuned general models?

Both. The category in the survey covers models in the roughly 100M–10B parameter range that have either been trained from scratch on healthcare data or fine-tuned from a general base on healthcare data. The operational distinction from frontier LLMs is that they can be run on a single GPU (or CPU for the smaller ones), in the customer’s environment, at fixed cost.

Why was cost rated lowest in evaluation priority?

Because in healthcare, the cost of a wrong answer typically exceeds the cost of the model. A missed adverse event, a mis-coded diagnosis, a leaked patient record, or a failed regulatory audit has consequences (clinical, financial, and reputational) that dwarf the per-record cost of inference. Practitioners who have absorbed those consequences rate accuracy and privacy above cost, because they know the downstream numbers.

What is the practical threshold for “high accuracy” in healthcare AI?

It depends on the task and on whether the workflow includes human review. For tasks where automation is the point (de-identification, PHI detection, high-volume clinical coding) the practical threshold is above 99% on the first pass, because below that every record still needs human review. For tasks with a designed human-in-the-loop review layer, the AI-only accuracy can be lower (90–96% is routine) as long as confidence calibration routes the uncertain records to reviewers reliably.

Is the budget growth seen in the 2024 survey sustainable?

Two years later, the answer appears to be yes: subsequent surveys and the evidence from public healthcare AI deployments show continued investment, broader adoption beyond early-adopter organizations, and a shift from pilot projects to operational workloads. The organizations that bet on the space in 2024 largely kept investing in 2025 and 2026. Organizations that stayed on the sidelines are now doing the catch-up work.

Healthcare got a head start on regulated AI because it had no choice. By the time ChatGPT arrived, clinical data-science teams had spent a decade working inside HIPAA, GDPR, FDA validation rules, and institutional review boards, and had built the machinery to ship AI under those constraints. Most of what the rest of the enterprise world is now encountering with generative AI (hallucinations in regulated workflows, unclear liability, compliance reviews that stall launches) healthcare has seen and has a pattern for. Four lessons from the way production medical AI gets built are directly transferable to finance, law, insurance, and any other sector where wrong answers have consequences.

Lesson 1: a complete view of the subject beats a clever model on a partial view

Most AI systems get built around whatever data is easiest to pull. In healthcare, that’s structured EHR fields: diagnoses, medications, labs, vital signs. A model trained on those fields can do useful things, but it misses most of the picture. More than half of the clinically relevant information about a patient (reasoning from the clinician, discussion with the patient, nuance about severity and certainty) lives in unstructured clinical notes, not in the coded fields. Add the PDFs (discharge summaries from other systems, external consults, prior-authorization letters), the medical images (radiology, pathology), and the patient-reported data (intake forms, symptom diaries), and the structured fields are maybe 30% of the available signal.

Production medical AI that works is built on top of a unified, longitudinal view of the patient that combines all of those sources. Structured demographics. Clinical characteristics. Vital signs. Smoking status. Past procedures and medications. Laboratory results. Extracted entities and assertions from progress notes. Findings from pathology and radiology reports. The combined view is what makes downstream tasks (disease progression prediction, clinical trial matching, cohort building, risk scoring) actually work. Models operating on the combined view routinely outperform ones that see only the structured slice of the record, because the unstructured slice is where most of the clinical reasoning happens to be written down.

The transfer is immediate:

- For a retail bank, the customer-completeness problem is the same shape. Transactions are structured. Call-center transcripts, chat logs, secure-message threads with relationship managers, and scanned-document submissions are not. A credit or churn model that sees only the structured side is working on a fraction of what’s available.

- For a property and casualty insurer, the claim is partly structured (coverage, policyholder, loss date) and heavily unstructured (adjuster notes, emails with claimants, photos, police reports, medical records). The systems that decide a claim well are the ones that read the full file.

- For a law firm, a case file is structured metadata plus mostly unstructured content: contracts, emails, depositions, exhibits. AI assistants that operate on that full file produce materially different answers than ones that see only the filing metadata.

In each case, the engineering pattern is the same: specialized extraction and classification models pull structured facts from unstructured sources, a harmonization layer joins them to the existing structured record, and downstream models (predictive, search, conversational) operate on the unified view. Healthcare built this pattern first because it had the worst unstructured-to-structured ratio. Every other regulated sector ends up building a version of it.

Lesson 2: the interface matters as much as the model

For a decade, advanced NLP and machine learning in regulated industries were gated on the availability of data scientists. If you wanted to train a model to extract contract clauses, detect adverse drug events, or classify claims, you needed an ML engineer to write code, a domain expert to label data, and a deployment team to push the result into production. That workflow scales poorly. There are not enough ML engineers for the work, and the domain experts with the relevant judgment (clinicians, pharmacists, lawyers, underwriters) are not going to learn Python on their own time.

Healthcare’s response to this bottleneck has been no-code annotation and human-in-the-loop tooling. The workflow that works in practice: a domain expert, working in a web UI, labels a small number of documents. An underlying system, often an LLM doing a zero-shot pass, proposes labels on the rest. The expert corrects the ones that are wrong. Those corrections become the next round of training data, which produces a smaller, faster, more accurate task-specific model. Iterate until the accuracy is where it needs to be, deploy the small model in production, keep the feedback loop running for monitoring and drift.

This compresses the build-and-validate loop from months to weeks, because the bottleneck, getting labeled data in the shape the specific task needs, is handled by the people who actually know what “correct” means. It also produces small, specialized models that are cheap to run at scale, rather than large general-purpose models called over an API at per-token cost. Regulatory requirements for human oversight and validation are satisfied by construction, because domain experts are signed into the loop at every step with audit trails, versioning, and approval workflows built in.

Other regulated industries are starting to build the same pattern for their own experts: lawyers labeling contracts for a contract-intelligence pipeline, compliance officers labeling transactions for anti-money-laundering models, underwriters labeling submissions for triage models. The model is secondary. The interface that lets the domain expert drive the process without writing code is what decides whether the project finishes.

Lesson 3: privacy and scale are architectural, not operational

In healthcare, “send the clinical notes to a third-party cloud API” is a non-starter for most organizations, most of the time. The reasons stack: HIPAA, GDPR, state privacy laws, institutional policy, patient expectations, data-sovereignty regulations in non-US jurisdictions. The result is that AI systems in healthcare have to be designed from the start to run inside the customer’s environment, on-premises, in the customer’s own private cloud tenant, or air-gapped, with no data ever leaving the customer’s control.

That constraint turns out to be a feature. In-environment deployment removes the per-token pricing model, because the customer is paying for compute they already own. It removes the latency tax of network round-trips to a vendor API. It removes most of the data-residency compliance questions, because the data never moved. It removes the vendor-lock-in risk that comes with building mission-critical pipelines on top of a third-party API whose pricing and availability the customer does not control. And it removes the training-data intellectual-property question, because the customer’s data stays the customer’s.

The architectural consequence is that healthcare AI systems are built to run efficiently on commodity hardware: single-GPU inference for most tasks, CPU inference for the lightweight ones, containerized deployment into Kubernetes or Databricks or Snowflake environments that customers already operate. This is a very different architecture from “call a vendor’s API from wherever,” and the difference matters for every other regulated industry that is going to face the same pressure.

Financial services is already there in parts — banks have regulatory constraints on where customer data can be processed, and many will not allow production workloads in third-party LLM APIs. Legal has similar constraints for privileged client information. Pharma has them for research data and trial records. In all of these sectors, the architectural pattern that scales is the one healthcare has already built: models designed to run in the customer’s environment, at production volume, on hardware the customer controls, with no data ever leaving.

The performance gap that used to make this architecture hard has mostly closed. Specialized domain-tuned models, carefully engineered for inference efficiency, now match or beat frontier LLMs on most of the specific tasks regulated industries care about, while running at 1–2% of the cost and with none of the compliance overhead. The remaining case for vendor APIs is for exploratory workloads and for conversational interfaces over curated knowledge — useful, but not where the production volume lives.

Lesson 4: humans in the loop are the accuracy mechanism, not a compliance afterthought

In regulated industries, 95% accuracy is not a success. It’s a system that still requires a human reviewer on every record, which is not automation. The target in healthcare for most high-volume tasks is above 99% — for de-identification, for PHI detection, for critical entity extraction — because that’s the threshold below which the downstream economics stop working. Hitting 99%+ on the first pass through a single model is rare. Hitting it through a composed system with a human-in-the-loop review layer is routine.

The pattern is a three-layer stack. The AI does the first pass at high volume and high speed. A confidence-scoring layer flags the records where the AI is uncertain, using calibrated confidence rather than raw model probabilities. A domain expert reviews only the flagged records, making the final call. The reviewed records feed back into the training set, so the AI gets steadily better over time and flags fewer records to the reviewers.

This pattern is what makes the economics work. If the AI runs at 96% accuracy and flags the 10% of records where it’s least confident, a human reviewer handling only those 10% is ten times as productive as a reviewer handling every record. If the AI’s confidence calibration is good, meaning the flagged records really are the ones where it’s most likely wrong, the combined system runs at well above 99%, faster and cheaper than either pure automation or pure manual review would be. The reviewers remain the accuracy mechanism; the AI just makes their throughput tractable.

Other regulated industries are arriving at the same architecture for the same reasons. Legal e-discovery review, insurance claims adjudication, financial compliance monitoring, pharma safety signal review — all of these have the same shape as clinical coding or adverse-event extraction. High volume, a regulatory requirement for human oversight, and an accuracy bar that no single model hits on its own. The systems that work are the ones that treat human review not as a compliance box but as an engineered throughput mechanism with measurable accuracy gains.

The short version for non-healthcare sectors

Four things to take from the way production medical AI gets built:

A complete view of the subject, combining structured and unstructured sources, tabular data and documents and images, is worth more than a clever model on a partial view. Build the harmonization layer first.

The domain experts who know what “correct” means should be driving the labeling and validation loop directly, through a no-code interface, with feedback that trains the model. That’s how projects actually finish.

Privacy and scale are architectural. Systems designed from day one to run in the customer’s environment, on the customer’s hardware, without data leaving, are cheaper, faster, and easier to clear compliance on than systems retrofitted to meet the same constraints later.

Human-in-the-loop is an engineering pattern, not a compliance concession. Calibrated AI confidence plus targeted expert review is how you hit the accuracy bars regulated workflows actually need, and it’s also how you make the economics work.

Healthcare’s head start was bought the hard way. The patterns it produced are available off the shelf to every other regulated industry that is now catching up.

FAQ

Why does healthcare keep coming up as a reference architecture for regulated AI?

Because healthcare had the hardest version of every constraint earliest: the strictest privacy rules, the highest accuracy bars, the worst unstructured-to-structured data ratio, and the most expensive wrong answers. The architectural patterns that cleared those bars (data harmonization, domain-expert-driven labeling, in-environment deployment, human-in-the-loop review) transfer to other sectors without much modification.

Does a unified longitudinal view always require OMOP or another formal common data model?

For research, RWE, and cross-institution work, formal common data models (OMOP, FHIR) are the right target because they make the data comparable across sources. For a single-organization operational use case, a payer running a model on its own claims and notes, or a bank running a model on its own customers, the same harmonization principles apply, but the target schema can be internal. The point is the harmonization, not the specific standard.

How is no-code annotation different from just giving domain experts a spreadsheet?

The tooling has to handle document-native labeling (highlighting spans of text inside a document rather than filling cells), manage annotator agreement across multiple reviewers, keep versioned datasets, integrate with model training so labels become training data automatically, and produce audit trails that satisfy regulatory review. A spreadsheet handles none of that.

What does “runs in the customer’s environment” mean technically?

Deployment of the models, the inference runtime, and often the training toolchain as software the customer installs into their own infrastructure: on-premises hardware, a private VPC in their AWS/Azure/GCP tenant, or an air-gapped environment. No data crosses the boundary to the vendor; no vendor-side API handles production inference. Licensing is typically fixed-cost rather than per-token.

How do you measure human-in-the-loop productivity gains?

Three numbers: the fraction of records the AI handles without review (throughput), the accuracy of the AI-only path on the records it passes (precision at high confidence), and the accuracy of the combined system on the flagged records (precision on the reviewed subset). A well-calibrated system improves on all three over time, because the feedback from reviewed records becomes training data for the next model version. That improvement loop is the operational KPI.

Every AI cycle goes through the same three stages: demo, pilot, production. Most enterprise AI in 2023 was somewhere between the first two. The work to close the gap between a compelling ChatGPT demo and a reliable system that a regulated business can run on is harder than the demos make it look, and 2024 will be the year that gap gets paid for, in either engineering hours or lost deployments. Three of those gaps are worth focusing on: accuracy and reliability are still unacceptable for most enterprise use; responsible-AI testing is now the rate-limiting step for production launches; and the regulatory environment is starting to catch up with the technology, in ways that will matter for how systems are built, not only how they’re run.

Demo accuracy and production accuracy are not the same number

The year of AI hype that closed out 2023 made two accuracy claims hard to separate: the claim that modern LLMs handle open-ended natural language tasks well, which is true, and the claim that an enterprise can point one at its data and ship, which is not. Closing that gap takes more engineering than most organizations budgeted for.

The first place the numbers fall short is out-of-the-box extraction and classification. In regulated industries, the work an AI system needs to do is usually not “write me a paragraph about” but “pull every adverse event from this progress note and tell me which medication was associated with it.” On those tasks, peer-reviewed benchmarks consistently show general-purpose LLMs underperforming smaller, domain-tuned models by meaningful margins. A *JAMIA* study from January 2024 on the 2010 i2b2 clinical-concept extraction benchmark measured GPT-4 with baseline prompting at F1 0.804, against BioClinicalBERT at 0.901, a 110-million-parameter model released years earlier. A careful prompt framework closed part of the gap. It did not close all of it, and building the prompt framework itself required enough labeled data to have trained a specialized model in the first place. Similar patterns have been reported across biomedical named entity recognition: a 2024 paper in *Bioinformatics* showed fine-tuned open models outperforming few-shot GPT-4 on biomedical NER by 5 to 30 F1 points depending on the dataset.

The second place the numbers fall short is consistency. The same prompt asked of the same model on the same input can return different answers on successive calls. For a chatbot producing creative text, that’s acceptable variety. For a system that pulls clinical findings from notes, or extracts contract terms from legal documents, or computes tax-coded line items from invoices, inconsistency is a production defect. Most enterprise pipelines need deterministic-enough behavior that the same record produces the same answer today and next week, which is not a property generative decoding gives you for free.

The third is cost and latency under real volume. A system that works on a dozen documents in a demo environment does not always work on the hundred thousand documents a real business process throws at it. Frontier-LLM calls priced per token, billed per request, and rate-limited by a vendor turn out to be a different line item at 10,000× scale than at 10× scale. For pipelines that process millions of records a day, which is what a hospital system, a large insurer, or a global pharma safety function actually does, the economics and the latency both push toward smaller, specialized models running on the buyer’s own hardware.

None of this argues that frontier LLMs are useless. It argues that enterprise AI in 2024 is going to be a composition problem rather than a single-model problem. The systems that ship will combine specialized extraction and classification models for the high-volume, high-accuracy, low-latency work with frontier LLMs used for reasoning, summarization, and conversation on already-cleaned inputs. Getting that composition right is the engineering gap between a demo and a production system.

Responsible AI has moved from adjective to rate-limiting step

By late 2023, enterprise AI buyers started asking a question that was largely absent from the first wave of deployments: how do you know this system is safe to run? The question covers six things in a trench coat (robustness, fairness, bias, truthfulness, data leakage, and safety), and the testing that answers them is harder to do well than most organizations realized when they started.

Robustness is the property that small, legitimate changes to an input should not produce large changes in the output. A system that gives a different answer when a patient’s name is changed from one that the training distribution saw often to one that is rarer is not robust. Testing robustness at scale means generating perturbed versions of real inputs (different names, slight rewording, translated variants) and measuring how much the output changes.

Fairness and bias testing asks whether the system performs equally well across demographic groups. The peer-reviewed literature on LLM bias has thickened considerably, with systematic surveys published through 2024 and 2025 cataloging intrinsic biases in representations, extrinsic biases in downstream tasks, and evaluation frameworks for both. For clinical systems, UK government guidance published in 2025 recommends treating fairness metrics as first-class production metrics alongside accuracy and latency, with bias evaluation gates in continuous integration and automatic rollback when thresholds are breached. That is a concrete, operational recommendation, and very few enterprises have the test infrastructure in place to run it yet.

Truthfulness, or more precisely, the absence of confidently stated wrong answers, is the hardest of the six. The failure mode is not the model saying “I don’t know”; it’s the model producing fluent, plausible text that is factually incorrect. A 2023 evaluation of a widely used open-source medical LLM reported high plausibility (~98.8%) paired with a meaningful hallucination rate (~19.7%), which is a reasonable summary of the problem: the outputs look right often enough that they pass a casual reader, and they’re wrong often enough that a regulated workflow cannot safely rely on them without source citation.

Data leakage is the training-time version of the privacy problem: the risk that a model has memorized specific training records and can be induced to reproduce them. For systems trained on sensitive data, this is both a privacy violation and, depending on jurisdiction, a legal violation. Testing for it is non-trivial and is now an expected part of any regulated deployment.

The practical consequence is that the responsible-AI test suite has become the rate-limiting step for many enterprise launches. The organizations that get systems into production in 2024 are the ones that treat testing as part of the engineering — with automated test generation, versioned test suites, and CI gates on fairness, robustness, and privacy the same way they have CI gates on unit tests — rather than as a compliance box checked after the system is already built. Open-source tooling for this work has matured considerably (LangTest and DeepEval among others), but the tooling is useful only in the context of an engineering discipline that treats responsible-AI testing as a first-class concern.

Regulation catches up, and the rules shape the architecture

The third growing pain is regulatory. Through 2023 the pattern was mostly principles papers; through 2024, concrete rules started to land. The EU AI Act passed in March 2024, with the first prohibitions on high-risk and prohibited practices taking effect in February 2025 and full provisions rolling out in stages. US state legislatures introduced nearly 700 AI-related bills in 2024 across 45 states, with 113 enacted into law. In healthcare and life sciences, FDA and EMA guidance on AI-enabled medical devices and software continues to expand, with data-provenance, validation, and post-market monitoring expectations that look a lot like the expectations on any other regulated product.

For enterprises, the operational question is not “what do I do if the regulator shows up” but “what architectural choices do I make now so that, when the regulator shows up, the answers are short.” The choices that make that conversation easier are the ones that also make engineering easier: training-data provenance tracked as a first-class artifact; validation and fairness results stored in a form that can be shared with regulators on request; deployment inside the organization’s own environment rather than a third-party cloud, so data-residency and privacy questions have short answers; audit logs that show, for each production answer, which model version generated it and what sources it cited.

What doesn’t work is treating compliance as an afterthought layered onto a system whose core was built for a different set of constraints. Every organization that has tried to retrofit provenance, audit, or data-residency onto an already-deployed AI system has discovered what engineers who live through any regulatory wave eventually learn: the right time to build in the constraints was before the system shipped. The second-best time is now, because the regulatory floor is rising and is going to keep rising through 2026 and beyond.

What to prioritize

For enterprises planning 2024 AI investments, three practical priorities sort out the demo-to-production gap.

First, invest in the boring layer. Specialized, task-specific models (entity recognition, classification, terminology mapping, translation between natural language and structured queries) are where the accuracy and cost wins come from in regulated workflows. Frontier LLMs are valuable; they are not the whole system. The systems that ship are compositions, with the specialized layer doing the high-volume, high-accuracy, low-latency work and the LLM doing the reasoning on top of clean inputs.

Second, treat responsible-AI testing as an engineering discipline rather than a compliance function. That means automated test generation, versioned test suites, CI gates, and production monitoring on robustness, fairness, and privacy, not a final-stage review before launch. The organizations that have figured out how to do this ship AI faster, not slower, because the testing catches problems early, when they’re cheap to fix.

Third, assume the regulatory floor keeps rising. Build systems that produce the artifacts regulators will eventually ask for (training-data provenance, validation records, fairness metrics, audit logs, citations on every answer, in-environment deployment) as a natural byproduct of how they operate, rather than as a separate compliance step. The point is not to predict exactly which rule will land when. The point is to build systems whose answers to those rules are short.

2024 was never going to be the year AI hype died. It was the year the engineering under the hype got harder. For organizations that invest in the composition, the testing, and the compliance as engineering choices rather than afterthoughts, the rocky road turns into a faster one, because the systems that clear those bars are also the ones that get past procurement and into production.

FAQ

Why isn’t a single frontier LLM enough for enterprise work?

Because most regulated enterprise tasks are high-volume extraction and classification problems, not generation problems. Peer-reviewed benchmarks have consistently shown smaller, domain-tuned models outperforming frontier LLMs on named entity recognition, relation extraction, and terminology mapping, and doing it at a fraction of the cost and latency. The enterprise systems that ship in 2024 combine specialized models for the structured work with frontier LLMs for reasoning and summarization.

What is “responsible AI” testing in practice?

Automated testing for six properties: robustness (small input perturbations shouldn’t cause large output changes), fairness (performance shouldn’t vary by demographic group), bias (outputs shouldn’t reflect stereotypes), truthfulness (the system shouldn’t confidently produce false statements), data leakage (training data shouldn’t be reproducible from the model), and safety (the system shouldn’t produce harmful content). Each of these has measurable metrics, and production systems should have automated tests that run on every model change.

Does the EU AI Act apply to US companies?

Yes, for systems that process data about EU residents or are placed on the EU market. The extraterritorial application is modeled on GDPR. US companies that build AI systems which touch EU data or EU markets need to meet the Act’s requirements on risk classification, documentation, human oversight, and transparency, and need to do so on the rolling timeline the Act lays out.

What’s the cheapest-to-ignore regulatory requirement to get right early?

Training-data provenance. The ability to say, for each model, what data it was trained on, where that data came from, what licenses apply, and what validation it was tested against — in a form that can be shown to a regulator or an auditor. Retrofitting this later is expensive; building it in from the start is cheap.

How do cost economics change for enterprise AI in 2024?

The fixed-cost versus per-token calculus tips toward fixed-cost at scale. Running frontier LLMs over the wire is economical for exploratory workloads and low-volume applications. For production workloads with millions of records per day, which is what large healthcare, pharma, financial, and legal operations actually run, specialized models on the buyer’s own hardware are often 50 to 100× cheaper and orders of magnitude faster, which is why the architectural pattern that wins is composition rather than single-model deployment.

---

David Talby is CEO of John Snow Labs, whose medical language models and responsible-AI testing tooling (including the open-source LangTest library) are used by 500+ healthcare and life sciences organizations. He also leads Pacific AI, which focuses on governance for healthcare AI.