Decision-readiness checklist

What I work through to decide whether an automated decision is ready to be relied on — or what still needs systems work before it is. Nothing here is proprietary: it is the operating model stated as questions.

Take it, print it, put it in your review template. For the reasoning behind a particular line, or a second pair of eyes on a live deployment, email me.

1 · Three tests

If a deployment fails any of these, the answer is not a better model. Every later section is a way of making one of these three true.

Explainable The people relying on a deployment can inspect the basis for its current authorization: intended use, evidence, assumptions, known limits, and accountable owner.

Challengeable A clinician, operator, patient advocate, or reviewer can contest a result or deployment condition through a defined route that does not depend on informal access or personal discretion.

Correctable When evidence, policy, performance, or operating conditions change, the institution can review the deployment, make a new decision, and carry the correction to every workflow it affects.

2 · The authorization record

Eight things a reviewer should be able to read off one record. A deployment with an unanswerable line has not cleared the first rung of the ladder in the next section — it is at State 0, not authorized, regardless of how well the model performs.

The decision or workflow being supported

Authorized users, population, setting, and exclusions

Current model, prompt, policy, data, and integration versions

Evidence and assumptions that justify using it

Known limitations and required human checks

Named deployment, clinical, and escalation owners

Effective date, review date, and superseded authorization

Stop conditions and rollback route

3 · Evidence the current state requires

Authority does not advance because the last state went well; it advances when the exit condition is met.

Are a bounded use case, an accountable owner, and an evaluation plan approved?

No: State 0 Not authorized. Check its evidence below.
Yes: Go to question 2.

Do observed performance and failure modes justify a limited prospective deployment?

No: State 1 Observed. Check its evidence below.
Yes: Go to question 3.

Does the deployment perform acceptably inside its stated scope, and can the institution pause, correct, or roll it back?

No: State 2 Constrained use. Check its evidence below.
Yes: Go to question 4.

Has a change to the model, data, users or evidence reopened review, or has the approval expired?

No: State 3 Routine use. Check its evidence below; authorization stays conditional.
Yes: Review is open, or the prior approval has expired. Start again at question 1.

State 0 Not authorized

Defined clinical or operational decision
Named deployment and escalation owners
Specified affected population and exclusions

Exit A bounded use case, accountable owner, and evaluation plan are approved.

State 1 Observed

Silent or retrospective evaluation
Error taxonomy and exception review
Baseline comparison against current practice

Exit Observed performance and failure modes justify a limited prospective deployment.

State 2 Constrained use

Prospective workflow validation
Documented override and escalation paths
Monitored safety, equity, and operational indicators

Exit The deployment performs acceptably inside its stated scope and the institution can pause, correct, or roll it back.

State 3 Routine use

Stable prospective performance
Independent evaluation sized to the risk of the use
Operational readiness for correction and rollback

Exit Approval stays conditional: any change to the model, data, users or evidence reopens review, and every approval has an expiry date.

4 · When the facts change

The part most reviews never specify. Before authorizing, confirm each of these has a named person and a route, not just an assumption that someone will notice.

Detect — Change signal Notice a change that could matter — in the evidence, the model’s behavior, policy, the data, the workflow, the people it is used on, or the outcomes being observed.

Connect — Map of everything that depends on it Trace which of the approval’s assumptions, which people, which workflows and which later decisions the change could affect.

Triage — Review priority and interim controls Judge how much it matters, how urgent it is, whether it can be undone, how many people it reaches, and whether the system can keep running while the review happens.

Review — Case for review Give the people allowed to decide the change (traced to its source), the original reasoning, how the system has actually performed, any objections, and what is still unknown.

Decide — Recorded decision Make the decision and write it down: keep, caveat, narrow, expand, monitor, escalate, suspend, or retire.

Propagate — Everything that relied on it, corrected Update the approval record, and every workflow, instruction, screen, monitoring rule and person that relied on the old decision.