AI safety and governance

Useful AI must also be accountable.

A practical centre for clinicians and healthcare leaders learning to evaluate AI without surrendering privacy, equity or professional judgment.

Ten operating principles

A baseline for responsible clinical AI.

Adapt these principles to the risk, intended use and jurisdiction of each system.

  1. 01

    Define the intended use

    State the patient population, user, decision, workflow and safe fallback before assessing a tool.

  2. 02

    Validate in context

    Do not assume that vendor or research performance will transfer to a different population or care setting.

  3. 03

    Protect patient information

    Use only approved systems and the minimum necessary data; never place identifiable patient information into an unapproved public AI service.

  4. 04

    Preserve human judgment

    Specify who reviews outputs, how uncertainty is handled and when users must override or escalate.

  5. 05

    Test for inequity

    Examine subgroup performance, access barriers, language, disability and the possibility of unequal downstream harm.

  6. 06

    Communicate transparently

    Explain what the system does, its limitations and how it affects decisions whenever that information is relevant to users or patients.

  7. 07

    Assign accountability

    Name owners for clinical safety, privacy, cybersecurity, procurement, incident response and change control.

  8. 08

    Monitor over time

    Track drift, incidents, overrides, outcome changes and software updates after deployment.

  9. 09

    Plan for failure

    Maintain downtime procedures and a safe non-AI workflow for unavailable, uncertain or degraded systems.

  10. 10

    Educate continuously

    Train teams in privacy, bias, hallucination, evidence appraisal, workflow effects and patient communication.

Immediate safeguards

Do not proceed when…

  • The intended use or responsible clinical owner is unclear.
  • Patient data would enter an unapproved environment.
  • Evidence does not represent the intended patient population.
  • Users cannot understand, challenge or safely ignore the output.
  • There is no monitoring, incident response or downtime plan.
Continue with primary guidance

Independent, non-affiliate sources

WHO ethics and governance guidance ↗FDA transparency principles ↗NIST AI Risk Management Framework ↗Browse the full resource library →