CEIAI Governance Database
Back to the database
Record STMT-W803966Voluntary

Responsible Personalisation: The Double-Edged Sword of Personalisation in Human-Robot Interaction

RESPONSIBLE PERSONALISATION: THE DOUBLE-EDGED SWORD OF PERSONALISATION IN HUMAN-ROBOT INTERACTION

White paper Institutional policyGlobalin force

This academic article presents a lifecycle-based and context-sensitive framework for responsible personalisation in human-robot interaction (HRI). It evaluates key ethical risks such as autonomy erosion, biased user modeling, manipulation, dehumanisation, and privacy violations across short- and long-term, open- and closed-domain contexts. Finally, it translates these insights into actionable design recommendations and open research questions for researchers and practitioners.

Summary and content fields are drawn from the document text and are pending editorial review.

Who issued it

Institut de Robòtica i Informàtica Industrial, CSIC-UPCAcademic
Lead issuer
Universität AugsburgAcademic · DE
Co-issuer
Free University of Bozen-BolzanoAcademic · IT
Co-issuer
San Francisco State UniversityAcademic · US
Co-issuer
Nantes UniversitéAcademic · FR
Co-issuer
Bielefeld UniversityAcademic · DE
Co-issuer
Singapore-MIT Alliance for Research and TechnologyAcademic · SG
Co-issuer
Massachusetts Institute of TechnologyAcademic · US
Co-issuer
Oakland UniversityAcademic · US
Co-issuer
Legal forceVoluntary
Under whose authorityInstitutional policy
Judged fromfull text

Who it governs

Researchers and academicsCompaniesAI developers and providersAI deployers and users

This paper is for robotics researchers, designers, developers, and practitioners seeking to understand and mitigate the ethical risks of personalised robotic systems.

Sectors
Research and science · human-robot interaction researchCross-sector

What the document contains

What's covered

The document addresses the definition and mechanics of personalization in human-robot interaction through an embodiment-aware lens, provides an integrative risk analysis covering autonomy erosion, bias, dehumanization, manipulation, and privacy, and outlines actionable design recommendations and open research questions.

What's missing

The text does not provide binding legal regulations or a mandatory code of conduct, relying instead on conceptual frameworks, ethical reflections, and voluntary design recommendations.

Key takeaways
  • Physical embodiment in social robots amplifies both the benefits and ethical risks of personalization compared to disembodied AI systems.
  • Personalization in HRI follows a six-phase lifecycle spanning design, data collection, modeling, interaction, evaluation, and end-of-life considerations.
  • Key ethical risks associated with personalized HRI include autonomy erosion, biased user modeling, dehumanization, manipulation, and privacy violations.
  • The manifestation of personalization risks varies based on interaction duration (short- vs. long-term) and domain specificity (open- vs. closed-domain).
  • Actionable recommendations include critically evaluating the necessity of personalization, mitigating overtrust, enabling user overrides, auditing models for bias, and defining responsibility.
Principles it states
Human-centred, accountable, and responsible personalised robotic systems.Preservation of user autonomy and meaningful opportunities for decision-making.Transparency and legibility of data collection and inference practices.
Outline
· 1 Introduction
· 2 Foundations for Responsible Personalisation in HRI
  - 2.1 Embodiment Driven Affordances of Robots
  - 2.2 Definitions
  - 2.3 The Lifecycle Mechanics of Personalisation
  - 2.4 Types of Social Human-Robot Interactions
· 3 Risks of Personalisation
  - 3.1 Erosion of User Autonomy
  - 3.2 Biases and Stereotypes in User Models
  - 3.3 Dehumanisation
  - 3.4 Manipulation
  - 3.5 Privacy violation
  - 3.6 Why these risks are amplified in HRI
· 4 Best Practices to Mitigate the Double-Edged Sword
  - 4.1 Recommendation 1: Critically evaluate to what extent robot personalisation is necessary or helpful for the given context
  - 4.2 Recommendation 2: Intentionally mitigate overtrust and overengagement with personalised robots
  - 4.3 Recommendation 3: Enable user oversight and overrides of robot behaviour
  - 4.4 Recommendation 4: Design and continuously audit personalisation models for bias and fairness
  - 4.5 Recommendation 5: Define responsibility for harms and successes of personalised systems
  - 4.6 Example of Usage of the Framework
· 5 Call to Actions
Issued7 July 2026
Length23,509 words · 36 pages
FormatNot recorded
Scopeglobal

How it scores

Pending review

Each score is backed by a quote verified against the document text. Editorial review has not yet confirmed them.

Human Dignity

3/5

Respect for the inherent worth of persons

Dehumanisation consists of treating or perceiving people as not (fully) human [63, 64, 65]. Dehumanisation has multiple antecedents and harmful consequences

3.3 Dehumanisation

screening score · 2026-10-23

Privacy

4/5

Protection of personal data and private life

This means that users should be able to access their own data collected by a robot and delete that data as desired

4.3 Recommendation 3

screening score · 2026-10-23

Impact Assessment

3/5

Required assessment of an AI system's risks/impacts before or during use

We present a lifecycle-based and context-sensitive framework for personalised HRI, grounded in an embodiment-aware perspective.

ABSTRACT

screening score · 2026-10-23

Topics

Physical AI Governancesupporting
robot_personalization ethical_risks human_robot_interaction embodiment privacy_violations