How we know it isn't making things up

The real problem with an AI-written clinical report

A language model can write a drug interaction that does not exist with the same confidence it writes one that does. It can cite an invented study. It can state a kidney function value it never calculated. All of it reads exactly as well as the correct version. That is why Iatra's verification is not handed to another model: it is written in code, where a rule is either met or not.

23 checks per report, written in code

Not a model grading another model. Deterministic rules that run over the final text of every report. Seven of them hold it back. Some of them:

• Holds the reportThe report cannot cite what we did not give itEvery published reference is compared against the source registry handed to the model. If one appears that is not there, the report is blocked. This is the net against the wholly invented citation.
• Holds the reportThe emergency number belongs to the patient's countryA report telling someone in Peru to call 112 is blocked. There it is 106.
• Holds the reportNo live vaccine in an immunosuppressed personIf the report recommends a live-virus vaccine and the patient's medication suppresses their immune system, it is blocked.
• Holds the reportThe report does not cut off mid-wayWe check that it closes where it should, and that the generator did not fall into a loop repeating text until it ran out. Both are blocked separately, because one is the symptom and the other the cause.
• Holds the reportThe tone cannot contradict the alertA report that opens reassuringly and carries an urgent alert further down is blocked. That inconsistency is what makes someone skip the part that matters.
• Flags itIt does not deny data that was measuredIf the report says potassium or creatinine is missing and the lab results contain them, it is flagged.
• Flags itIt does not propose a drug the person already takesThe alternatives table is cross-checked against current medication.
• Flags itNo study dated in the futureA lab test already done cannot be dated three months from now. When it is, the date was invented.
• Flags itThe European regulator is not presented as the local oneCiting the Spanish agency to a Colombian patient without saying it is a European reference presents it as their regulator, and it is not.
• Flags itThe register does not change mid-documentThe report addresses the patient the same way from start to finish. If it drifts into formal address, it is flagged for the reviewer.

Whatever fails is written down next to the report, with its name and its reason. Shown here are 10 of the 23; 5 of these hold the report.

When a physician steps in, and what is required of them

Not every report carries a physician's signature, and saying so is part of the answer. When one of those checks raises a clinical alert, the report is held and does not go out until a physician reviews and signs it with their name and license number. When none is raised, the report is delivered marked as an informational document, without individual medical review, and that is printed on the document itself.

How we measure whether the system gets worse

There is a bank of 40 synthetic patient cases, each with the clinical expectation annotated in advance by profile: geriatric, multimorbidity, post-cardiovascular event, heart failure, chronic kidney disease, mental health, chronic pain and oncology. An evaluation runs over them with a rubric and a regression threshold: if a new version drops below a safety floor, it does not ship.

With two clarifications worth stating plainly: the graders are AI agents playing specialist roles, not real physicians, and the bank is run manually, not on every change to the system.

Where the data is processed

Clinical inference runs in the European Union. This is not a statement of intent: an automated test reads the real deployment configuration, not the code default, and fails if any region is not European. Reading the code alone gave an answer that could be untrue.

What is kept from every report

• The full text of every model call that produced it: the three generators, the two critics and the final synthesis.
• The result of the deterministic checks, with the name of the one that failed.
• The source registry handed to the model, and which sources it ended up citing.
• The automatic score and its verdict.
• The machine's draft and the assessment the physician wrote, kept separate.

If someone asks why a particular report said what it said, the answer is not a reconstruction: it is stored.