Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations
This National Institute of Standards and Technology (NIST) report establishes a comprehensive taxonomy and terminology for adversarial machine learning (AML). It surveys the AML literature to categorize predictive and generative AI attacks, attacker goals and capabilities, and relevant mitigation strategies. The publication is designed to inform future standards and practice guides by providing a common language for assessing and managing AI system security.
Summary and content fields are drawn from the document text and are pending editorial review.
Who issued it
Who it governs
This document is intended for individuals and groups responsible for designing, developing, deploying, evaluating, and governing AI systems.
What the document contains
The document covers a structured taxonomy of attacks on predictive and generative AI systems, including attack classifications by learning stage, attacker goals, capabilities, and knowledge. It details evasion, poisoning, privacy attacks, supply chain vulnerabilities, and prompt injections, alongside corresponding mitigation strategies and open research challenges.
The document does not make recommendations on organizational risk tolerance, quantify attack likelihoods, or establish legal standards, regulations, or mandatory requirements.
- The report establishes a unified taxonomy and terminology for adversarial machine learning across predictive and generative AI systems.
- Attacks on AI encompass multiple lifecycle stages, ranging from training data poisoning to inference-time evasion, privacy leaks, and prompt injections.
- Generative AI models introduce unique vulnerabilities such as direct and indirect prompt injections, supply chain attacks, and retrieval-augmented generation risks.
- Mitigations and defenses require assessing attacker goals, capabilities, and knowledge across different data modalities.
- Open challenges include scale limits, theoretical bounds on adversarial robustness, and trade-offs between various trustworthy AI attributes.
· Executive Summary · 1. Introduction · 2. Predictive AI Taxonomy · 3. Generative AI Taxonomy · 4. Discussion and Remaining Challenges · Appendix: Glossary
How it scores
Each score is backed by a quote verified against the document text. Editorial review has not yet confirmed them.
Accountability
1/5Clear responsibility for AI outcomes
The intended primary audience for this document includes individuals and groups who are responsible for designing, developing, deploying, evaluating, and governing AI systems.
Audience
screening score · 2026-10-23
Fairness
1/5Non-discrimination and equitable treatment
Even though these factors might be exploited by an adversary, they are not within the scope of the literature on AML or this report.
Executive Summary
screening score · 2026-10-23
Human Dignity
0/5Respect for the inherent worth of persons
1. Introduction
screening score · 2026-10-23
Human Oversight
1/5Humans remain in control of consequential decisions
Similarly, in the medical feld where more and more ML models are being deployed to assist doctors, there is the potential for medical record leaks from ML models that can expose deeply personal information
1. Introduction
screening score · 2026-10-23
Privacy
3/5Protection of personal data and private life
This creates serious risks for user privacy online. The more often a piece of information appears in a dataset, the more likely a model is to leak it in response to random or specifcally designed queries or prompts.
1. Introduction
screening score · 2026-10-23
Transparency
2/5Openness about how AI systems work and are used
With the exception of BLOOM and LLaMA, most of the companies developing such models do not release detailed information about the data sets that have been used to build their language models
1. Introduction
screening score · 2026-10-23
Enforcement
0/5Consequences and compliance mechanisms with real force
Background
screening score · 2026-10-23