Guidance on privacy and developing and training generative AI models
This regulatory guidance from the Office of the Australian Information Commissioner explains how the Privacy Act 1988 and the Australian Privacy Principles apply to the development and training of generative artificial intelligence models. It outlines legal obligations regarding privacy by design, model accuracy, dataset collection through scraping or third-party sources, secondary uses of personal information, and transparency requirements. The document emphasizes that developers must take a cautious approach, minimize personal information, and obtain consent or establish valid exceptions when using personal data for AI training.
Summary and content fields are drawn from the document text and are pending editorial review.
Who issued it
Who it governs
This guidance is for developers and organizations subject to the Privacy Act who design, build, train, adapt, or fine-tune generative AI models or systems.
What the document contains
The document covers privacy obligations under the Privacy Act 1988 during the planning, design, and dataset compilation stages of developing and fine-tuning generative AI models. It addresses specific Australian Privacy Principles (APPs 1, 3, 5, 6, and 10) relating to privacy impact assessments, accuracy, data scraping, third-party datasets, secondary use of data, and transparency.
It does not address considerations during the testing, deployment, or operational use of generative AI models, nor does it cover non-privacy legal regimes.
- The Privacy Act applies to the collection, use, and disclosure of personal information used to train generative AI models.
- Publicly available data does not automatically mean it can be legally used to train or fine-tune generative AI models.
- Developers must take reasonable steps to ensure accuracy, commensurate with increased risks, using high-quality datasets and appropriate testing.
- Sensitive information generally requires explicit consent to be collected and scraped images or recordings may contain sensitive information.
- Where secondary use of data for AI training is not within reasonable expectations or related to the primary purpose, developers should seek consent or offer an opt-out.
· Top five takeaways · Quick reference guide · Privacy by design · Accuracy when training AI models · Privacy considerations when collecting and processing the training dataset · Collection obligations (APP 3) · Use and disclosure obligations (APP 6) · Notice and transparency obligations (APP 1 and APP 5) · Other considerations · Checklist
How it scores
Each score is backed by a quote verified against the document text. Editorial review has not yet confirmed them.
Accountability
4/5Clear responsibility for AI outcomes
Under APP 10, developers have an obligation to take reasonable steps to ensure the personal information collected, used and disclosed is accurate.
Accuracy
screening score · 2026-10-23
Fairness
2/5Non-discrimination and equitable treatment
As AI systems learn from source data which may contain inherent bias, this bias may be replicated in their outputs through inferences made based on gender, race or age and have discriminatory effects.
Privacy by design (APP 1)
screening score · 2026-10-23
Human Dignity
2/5Respect for the inherent worth of persons
Generative AI models have unique and powerful capabilities, and many use cases can pose significant privacy risks for individuals as well as broader ethical risks and harms.
Quick reference guide
screening score · 2026-10-23
Human Oversight
1/5Humans remain in control of consequential decisions
for example use in decisions that will have a legal or similarly significant effect on an individual’s rights
Accuracy when training AI models (APP 10)
screening score · 2026-10-23
Privacy
5/5Protection of personal data and private life
The Privacy Act applies to the collection, use and disclosure of personal information to train generative AI models, just as it applies to all uses of AI that involve personal information.
Quick reference guide
screening score · 2026-10-23
Transparency
4/5Openness about how AI systems work and are used
To ensure good privacy practice, developers should also ensure they update their privacy policies and notifications with clear and transparent information about their use of AI generally.
Transparency
screening score · 2026-10-23
Enforcement
3/5Consequences and compliance mechanisms with real force
Where a developer cannot clearly establish that a secondary use for an AI-related purpose was within reasonable expectations and related to a primary purpose, to avoid regulatory risk they should seek consent for that use and/or offer individuals a meaningful and informed ability to opt-out of such a use.
Top five takeaways
screening score · 2026-10-23