CEIAI Governance Database
Back to the database
Record STMT-W70794CAdvisory

Autonomy, artificial intelligence and robotics: Technical aspects of human control

Report International lawGlobaladopted

This report by the International Committee of the Red Cross (ICRC) examines the technical aspects of human control over autonomous systems, artificial intelligence, and robotics. Drawing on insights from the civilian sector, it analyzes the challenges of predictability, reliability, and human-machine interaction in autonomous weapon systems. The document highlights how AI and machine learning introduce inherent unpredictability and bias, reinforcing the urgent need for states to establish limits on autonomy in weapon systems.

Summary and content fields are drawn from the document text and are pending editorial review.

Who issued it

International Committee of the Red CrossIntergovernmental
Lead issuer
Legal forceAdvisory
Under whose authorityInternational law
Judged fromfull text

Who it governs

National governmentAI developers and providersResearchers and academicsInternational community

This report is for policymakers, diplomats, military experts, and researchers examining the regulation and governance of autonomous weapon systems and military AI.

Sectors
Defence and security · weapon systems and armed conflictResearch and science · robotics and artificial intelligence

What the document contains

What's covered

The document addresses the technical dimensions of human control, predictability, reliability, machine learning, computer vision, and civilian safety standards as they relate to autonomous weapon systems and international humanitarian law.

What's missing

It does not provide specific legal text for a binding international treaty, nor does it detail technical source code or proprietary military specifications.

Key takeaways
  • Autonomous weapon systems introduce inherent unpredictability regarding the specific target, timing, and location of attacks.
  • Human-on-the-loop control is not a panacea due to issues like automation bias, lack of situational awareness, and the moral buffer.
  • Machine learning and AI introduce a black box problem, lack of explainability, and algorithmic bias that exacerbate unpredictability.
  • Civilian safety standards for autonomous systems emphasize rigorous human supervision, fail-safes, and operational constraints.
  • States must urgently work to establish clear limits on autonomy in weapon systems to ensure compliance with international humanitarian law.
Principles it states
Human responsibility for decisions on the use of weapon systems and the use of force must be retained.The purpose of AI is to augment rather than replace human intelligence.
Outline
· Executive Summary
· 1. Introduction
· 2. Autonomy in Weapon Systems
· 3. Human Control
· 4. Predictability and Reliability
· 5. Algorithms, AI and Machine Learning
· 6. Computer Vision and Image Recognition
· 7. Standards in Civilian Autonomous Systems
· 8. Conclusions
IssuedAugust 2019
Length16,357 words
FormatNot recorded
Scopeglobal

How it scores

Pending review

Each score is backed by a quote verified against the document text. Editorial review has not yet confirmed them.

Accountability

3/5

Clear responsibility for AI outcomes

the moral buffer, where the human operator shifts moral responsibility and accountability to the machine as a perceived legitimate authority.

3.4

screening score · 2026-10-23

Fairness

3/5

Non-discrimination and equitable treatment

Bias in AI and machine learning algorithms is a core problem that can have many facets.

5.2.1

screening score · 2026-10-23

Human Oversight

5/5

Humans remain in control of consequential decisions

The International Committee of the Red Cross (ICRC) has emphasized the need to maintain human control over weapon systems and the use of force, to ensure compliance with international law and to satisfy ethical concerns.

EXECUTIVE SUMMARY

screening score · 2026-10-23

Enforcement

0/5

Consequences and compliance mechanisms with real force

N/A

screening score · 2026-10-23

Topics

autonomous_weapons artificial_intelligence human_control international_humanitarian_law machine_learning