CAIOS: A new standard for clinical AI oversight

Healthcare practices are adopting AI at pace, but the governance frameworks available to them were not designed for clinical use. ISO 42001 addresses enterprise AI management. The NIST AI Risk Management Framework provides broad risk principles. Neither tells a radiology practice director how to document their AI tool register, what performance metrics to track, or how to maintain evidence of human oversight over AI-assisted diagnoses.

The Clinical AI Oversight Specification — CAIOS — was developed to fill that gap. It is a purpose-built evidentiary standard for healthcare practices that use AI tools in clinical decision-making. This article explains what CAIOS is, how it is structured, and how practices can start preparing.

What CAIOS is

CAIOS is a governance specification designed for the clinical context. It defines five domains of oversight that a healthcare practice should implement when using AI tools in clinical workflows. Unlike broad AI governance frameworks, CAIOS is prescriptive: it tells practices what to document, what to monitor, and what evidence to maintain.

The specification was developed by a clinician-researcher with direct experience in diagnostic imaging, published clinical AI research, and an understanding of the regulatory and medico-legal environment facing Australian healthcare practices. It was submitted for peer review to the Journal of Medical Imaging and Radiation Oncology (JMIRO) in March 2026.

CAIOS is not a certification scheme (though certification is a planned extension). It is a standard that practices can adopt today to structure their AI governance in a way that satisfies regulatory expectations, insurer requirements, and medico-legal defensibility.

The five domains

CAIOS is organised into five domains, each addressing a distinct aspect of clinical AI governance:

Domain 01: Tool registration

Every AI tool in clinical use must be formally registered in the practice's governance system. Registration captures the tool's identity (name, vendor, version), regulatory status (TGA classification, ARTG number), intended clinical purpose, and use boundaries.

Why it matters: You cannot govern what you have not identified. A tool register is the foundation of every other governance activity. Without it, risk assessments, performance monitoring, and incident management are untethered from the tools they should be tracking.

What CAIOS requires: A maintained register reviewed at least quarterly. Each entry must include intended use boundaries — not just what the tool does, but what it is not validated to do. Version changes must be logged.

Domain 02: Risk management

Each registered tool must have a documented risk assessment that identifies clinical, operational, and data risks, assesses their likelihood and severity, and documents the controls in place to mitigate them.

Why it matters: Risk assessment before deployment is the governance activity that most directly reduces patient harm. It forces the practice to think through failure modes — what happens when the AI misses a finding, generates a false positive, or goes offline during clinical hours.

What CAIOS requires: A risk assessment for each tool, completed before clinical deployment and reviewed at least annually or when the tool is updated. Assessments must name the individual responsible for each risk control.

Domain 03: Performance monitoring

Practices must track the real-world performance of their AI tools against clinical ground truth. This is not about relying on vendor-published validation data — it is about measuring how the tool performs in your clinical environment, on your patient population.

Why it matters: An AI tool validated on a European dataset may perform differently in an Australian practice with a different patient demographic. Performance drift after model updates is common and often goes undetected without structured monitoring.

What CAIOS requires: Defined performance metrics for each tool (concordance rate, false positive rate, detection sensitivity), a monitoring frequency (at minimum quarterly), and documented reviews of monitoring data with escalation thresholds.

Domain 04: Human oversight

The radiologist retains final clinical authority over all AI-assisted findings. CAIOS requires practices to document how human oversight is maintained in their workflows, including how AI outputs are reviewed, how disagreements between AI and clinician are handled, and how override authority is preserved.

Why it matters: This is the domain that regulators, insurers, and courts focus on most intensely. The question is never \"did you use AI?\" — it is \"did the clinician retain meaningful oversight?\" Workflow documentation that demonstrates systematic human review is the most defensible evidence a practice can produce.

What CAIOS requires: Documented clinical workflows showing how AI outputs are integrated into the reporting process. Policies on radiologist override authority. Evidence that practitioners are trained on the AI tools they use, including their limitations.

Domain 05: Governance documentation

The final domain ties the other four together. Practices must maintain a documented governance policy, assign governance responsibilities to named individuals, and maintain an audit trail of governance activities — meeting minutes, review dates, policy updates, incident reports.

Why it matters: Governance without documentation is invisible. If you cannot produce evidence that governance activities occurred, you cannot demonstrate compliance. The audit trail is what transforms good intentions into a defensible framework.

What CAIOS requires: A governance policy reviewed at least annually. A named governance lead. Documented evidence of governance activities including reviews, training, incident responses, and policy updates.

Why existing frameworks fall short

CAIOS was developed because existing AI governance frameworks do not address the clinical context with sufficient specificity:

ISO 42001 is an enterprise AI management system standard. It provides a framework for organisations to manage their AI systems at an organisational level — policy, lifecycle management, impact assessments. It is valuable for large health systems managing enterprise AI strategy, but it does not address clinical-specific requirements: radiologist override authority, concordance monitoring, clinical incident management, or patient consent for AI-assisted diagnosis.

NIST AI Risk Management Framework provides broad principles for AI risk management — govern, map, measure, manage. It is a useful conceptual model, but it is not prescriptive enough for a practice director who needs to know exactly what to document, how often to review it, and what evidence to maintain.

RANZCR Chapter 9 sets expectations for AI governance in radiology but is a standard of practice, not a detailed specification. It tells practices what they should do but leaves significant discretion on how. CAIOS provides the how — a structured specification that practices can implement to satisfy Chapter 9's expectations.

The frameworks are complementary, not competing. A practice can adopt CAIOS as its operational governance specification while recognising Chapter 9 as the regulatory standard it satisfies.

Peer review and publication

CAIOS was submitted to the Journal of Medical Imaging and Radiation Oncology (JMIRO) for peer review in March 2026. The decision to publish through a peer-reviewed journal, rather than releasing it as a white paper or industry standard, reflects the intent to establish CAIOS as a credible, academically validated specification.

Peer review subjects the specification to scrutiny from clinical and governance experts, which strengthens its authority as a standard that regulators, insurers, and courts can reference.

How practices can prepare

You do not need to wait for CAIOS to be published to begin implementing its principles. The five domains map directly to governance activities that are already expected by RANZCR Chapter 9 and medical indemnity insurers:

• Build a tool register (Domain 01)

• Conduct risk assessments for each AI tool (Domain 02)

• Establish performance monitoring (Domain 03)

• Document human oversight workflows (Domain 04)

• Create a governance policy and maintain an audit trail (Domain 05)

Practices that have these five elements in place will find CAIOS alignment straightforward when the specification is published. Those that start now will have a mature framework by the time the standard is formally available.

The goal of CAIOS is not to add bureaucracy to clinical practice. It is to provide the structure that makes AI governance sustainable, defensible, and — critically — proportionate to the risk. Clinical AI is here to stay. The question is whether it is governed well enough to protect patients, practitioners, and practices when it matters most.