What an AI system must show before it’s relied on, and what happens when it’s wrong
A framework for deciding when an AI system may be used, who can challenge its decisions, and how errors get corrected.
The products are built to it: NextConsensus keeps track of what the evidence currently says, Ambit of what an agent has actually been approved to do, and Refract of what has changed underneath. Each one writes down something AI systems in hospitals and similar settings usually leave in people’s heads.
The method
Every AI system in a setting where an error can hurt someone makes claims it does not state: that a guideline is current, a drug protocol is safe, or a patient needs immediate prioritization.
A claim can be checked when you can see what backs it, when you can challenge it, and when a correction reaches the systems that relied on it.
Most AI governance work starts with the operating model — who approves what, under what conditions, for how long. This framework starts earlier: how do you know what you know, and how is an error repaired?
NextConsensus freezes each forecast before the outcome is known. Refract keeps source changes verifiable, and Fast Harm, Slow Repair applies the same discipline to recovery evaluation.
Ambit builds the operating model into the software agents run on: what an agent is approved to do is kept apart from what it can reach, every change it makes carries its own undo, results are checked before anyone trusts them, and each tool has a fixed scope.
Ethotechnics publishes ten proposed standards, each numbered and versioned. Every one maps itself to NIST AI RMF, ISO/IEC 42001 and the EU AI Act in a section of its own — the mapping is mine, not an endorsement by those bodies. No outside institution has adopted one yet, so they remain proposals. The one the projects on this site share is STD-07, the Revisable Delegation Record, which has a conformance checker.
What the standards bind
The twelve lawsEach law is a rule the standards enforce: capability does not imply authority, authority decays unless renewed, every delegation creates a correction obligation, and nine more.
The one rule underneathOne sentence the laws come down to: no system should gain the power to act on people faster than the institution gains the ability to inspect, challenge, change, and survive its decisions.
Core axiomsThe five principles the laws come from, and why each is owed to people rather than offered to them.
TheoryEssays on why the laws hold. Motivation only: no standard cites one as a requirement, and a reader can adopt a clause without the argument.
Principles
What I require from the teams I lead. The beliefs these implement live on /about/; each principle links to the work it came out of.
01Human override is a product feature.
Override, escalation, and return-for-review decide whether a system that can hurt someone can be deployed safely.
Recurs in two deployments
AndwiseAutomated clause analysis surfaced and explained each issue. A named reviewer checked every analysis before a physician could act on it.
EthotechnicsThe stages of approval written down as an open standard: who may override a system, and what they have to record when they do.
02Provenance beats confidence scores.
Knowing where a recommendation came from and what evidence backed it matters more than a confidence score nobody can audit.
Recurs in two deployments
RefractEvery change replayed into a verifiable event carrying its provenance, with the judgment left to the caller.
NextConsensusA public ledger of forecasts registered and frozen before the outcome is known, scored against the record afterwards.
03The decision the AI feeds into matters more than the AI’s output.
When an automated system causes harm, ask who owned the decision the model fed into, not only whether the model was wrong.
Recurs in two deployments
EpicInstalled, signed off, and live — and still not doing the job, because nobody owned what happened after the go-live.
The Crumple ZoneEssays on the gap between an automated recommendation and the person who has to carry it out.
04Governance belongs in the product.
Evaluation, monitoring, escalation, and correction work best built into the product’s daily workflows, not handed to a committee outside them.
Recurs in two deployments
EthotechnicsApproval, correction, and escalation published as open, versioned standards.
AndwiseCompliance review was a routed step in the flow, signed off by a named reviewer.
05A deployment is finished when corrections reach every record it touched.
An error corrected at the source remains active everywhere it already propagated.
Recurs in two deployments
RefractDownstream systems receive what changed and when.
Fast Harm, Slow RepairA protocol that measures how far a wrong output travels before the correction catches up with it.
06Evaluate workflows before models.
A model that performs well in evaluation can still fail at the integration point where busy clinicians have to use it.
One instance so far
TranscarentFour programs shipped on one shared decision architecture because routing was the binding constraint.
07You can get an approval later; you cannot undo a harm.
When the risk is clinical, launching early costs more later than it saves now.
One instance so far
EpicThe escalation route that had to exist before the software could honestly be called live.
08Hospital IT trusts the system around the model.
Hospital IT, security, and legal teams review the system around a model — access boundaries, audit trails — as closely as the interface.
One instance so far
DoximityHospital security reviews cleared the product on documented policies and procedures.
The operating model
AI deployment requires continuing authorization: when a system may influence decisions, what conditions limit that authority, how people can challenge it, and what evidence or behavior reopens review.
A model recommends a treatment change. At every state, the authorization is a dated record that can be superseded — never a permission granted once.
State transitions for the treatment-change recommendation. Rows are in order; a state is entered only when the row above it exits. The last row shows
three of the six review steps, one per column; the checklist has all six.
State
Trigger
Who acts
Evidence required
What propagates
State 0Not authorized
A use is proposed.
Named deployment and escalation owners
Defined clinical or operational decision
Named deployment and escalation owners
Specified affected population and exclusions
Nothing. The proposed use cannot influence care or workflow.
State 1ObservedThe recommendation is observed but cannot influence care.
State 0 exits when A bounded use case, accountable owner, and evaluation plan are approved.
—
Silent or retrospective evaluation
Error taxonomy and exception review
Baseline comparison against current practice
Nothing to care or workflow. Outputs are captured for evaluation.
State 2Constrained useAfter a silent evaluation, it may inform a narrow workflow under human review.
State 1 exits when Observed performance and failure modes justify a limited prospective deployment.
Explicit human review on each recommendation
Prospective workflow validation
Documented override and escalation paths
Monitored safety, equity, and operational indicators
Recommendations into one narrow workflow, within limited population scope and predefined stop conditions.
State 3Routine use
State 2 exits when The deployment performs acceptably inside its stated scope and the institution can pause, correct, or roll it back.
Independent review, with challenge rights preserved
Stable prospective performance
Independent evaluation sized to the risk of the use
Operational readiness for correction and rollback
Routine use within the defined scope, watched for change and reviewed on a schedule.
Any stateReopenedA new safety signal suspends the authorization immediately, from whichever state it had reached.
01 Detect Notice a change that could matter — in the evidence, the model’s behavior, policy, the data, the workflow, the people it is used on, or the outcomes being observed.
The clinical safety lead opens the review and makes the decision.
04 Review → Case for review Give the people allowed to decide the change (traced to its source), the original reasoning, how the system has actually performed, any objections, and what is still unknown.
06 Propagate → Everything that relied on it, corrected Update the approval record, and every workflow, instruction, screen, monitoring rule and person that relied on the old decision.
The loop opens on a change, not on a review cycle. A system that can only be reviewed on a schedule cannot be corrected in between.
Decision-readiness checklist
One page. It opens on the three tests a deployment has to pass — explainable, challengeable, correctable — and ends in either a decision you can rely on or a list of what still needs systems work.
When the evidence, a policy or a model changes, making sure everything that relied on the old answer is updated and corrected — not only the system that changed.
2011Georgia Tech RNA
Modeled thermodynamics and barrier kinetics deciding whether a molecular process completes or stalls.
View record →
2024Refract & NextConsensus
Built deterministic change-detection bots and claim-trajectory scoring against public records.
View record →
2025Fast Harm, Slow Repair
Designed a draft test for how long harm lasts and how far a correction reaches; it covers one of twelve planned cases.
View record →
I study what changes when a decision that used to end with a person ends in software instead: where errors travel, who absorbs the work they create, and whether the system can repair itself when the information under it changes.
Three cases, and which of the questions each one asks
Case
Mechanism
Who absorbs the cost
Which questions
Emergency department loudspeaker
Patients are called for triage by name over a loudspeaker. For a Deaf patient the mechanism deciding who waits is inaudible.
The Deaf patient; what it costs is measured in mortality.
Who waits?
What cannot be allowed to fail?
Epic: a misrouted clinical alert
A misrouted alert could bury a critical lab result.
Whichever clinician trusted the queue; nobody in the routing system answered for it.
Who absorbs error?
Who covers for the system?
Andwise: monetization
The monetization paths most likely to fund growth would have made employers or financial institutions the customer.
In 2012 I helped build a medication-recommendation system for type 2 diabetes, and we measured the thing that now worries me most about deployed models: not whether the recommendation was right, but what happened to the clinician’s own judgment once they had seen it.
Internists n=262%→92%change +30 pts
Familiar with the rules n=464%→86%change +22 pts
Endocrinologists n=468%→76%change +8 pts
Unfamiliar with the rules n=271%→71%change +0 pts
Agreement with the algorithm, before and after the clinician saw its recommendation. Blinded validation, twenty patient data sets per reviewer, six reviewers in total. Percentages are rounded; the subgroups nest rather than partition, so the reviewer counts do not sum to six. Rows are ordered by how far each group moved.
The specialists barely moved and the generalists moved most. The subgroup that did not move at all was the one that did not know how the algorithm worked — two reviewers, which is an observation about two people rather than a result. Taken together the rows point one way: the readers who understood the tool deferred to it most.
A recommendation moves the judgment beside it, and it moves different readers by different amounts. This course project’s data shows the direction but cannot size the effect. The rest of this framework is built to stop that: a model’s output becoming an institution’s policy without anyone deciding it should.
The rule the measurement above argues for. Software can update what the evidence says; only a
named person can change what the institution does, and that change is recorded. “No action” is one of the choices, because a system where
declining to act is not a recordable choice will drift into acting by default.
Jain, K., Patel, K., Rowland, J., Yong, C. DiaMonD (DIAbetes MONitoring and Dosing) System. Wallace H. Coulter Department of Biomedical Engineering, Georgia Tech / Emory University. Advised by Dr. Lawrence Phillips, MD.
Go deeper
Case studiesWhere this came from: five deployments through hospital security review, federal quality reporting, and clinical sign-off, with the decisions and contemporaneous records behind them.
PrinciplesWhat 14 years of clinical products left me unwilling to ship without.
Review LabOne composite deployment, six months in, with new information on the table. Make the call and see what it commits the organization to.