The rise of artificial intelligence (AI) and its widespread availability offers significant growth opportunities for businesses. However, it necessitates a robust governance framework to ensure compliance with regulatory requirements, especially under the European Union’s (EU) Artificial Intelligence Act (AI Act) (see our Guide to the AI Act) and the EU General Data Protection Regulation (GDPR). The reason GDPR compliance is so important is that (personal) data is a key pillar of AI. For AI to function effectively, it requires good quality and abundant data so that it can be trained to identify patterns and relationships. Additional personal data is often gathered during deployment and incorporated into AI to assist with individual decision-making.
In this series of five blog posts, we discuss GDPR compliance throughout the AI development life cycle and when using AI.
This is our fourth episode.
Data Protection by Design
GDPR compliance plays a key role throughout the AI development life cycle, starting from the very first stages. This reflects one of the key requirements and guiding principles of the GDPR called data protection by design (Article 25 GDPR). Businesses are required to implement appropriate technical and organizational measures, such as pseudonymization, at both the determination stage of processing methods and during the processing itself. These measures should aim to implement data protection principles, such as data minimization, and integrate necessary safeguards into the processing to ensure GDPR compliance and protect individuals’ data protection rights.
AI Development Life Cycle
The AI development life cycle encompasses four distinct phases: planning, design, development, and deployment. In this context, in accordance with the terminology of the EU AI Act, we will refer to both AI models and AI systems.
- AI models are a component of an AI system and are the engines that drive the functionality of AI systems. AI models require the addition of further components, such as a user interface, to become AI systems.
- AI systems present two characteristics: (1) they operate with varying levels of autonomy and (2) they infer from the input they receive how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments.
In this blog post, we focus on the fourth phase of the AI development life cycle: deployment. We already discussed the first and second phases (planning, design, and development) in our previous blog posts (see here, here, and here).
The Deployment Phase
The fourth phase of the AI development life cycle involves making AI accessible for real-world use, tracking performance, addressing drifts, and adjusting through monitoring and maintenance.
Implementing data protection by design establishes a strong foundation for GDPR compliance, but it is not sufficient on its own. GDPR compliance is an ongoing process that necessitates continuous monitoring and appropriate processes throughout the lifespan of the AI model or system to ensure that all the issues discussed in the previous episodes remain properly addressed at all times.
Monitoring and Processes
- Monitoring. Once an AI model or system is deployed, continuous monitoring is crucial to ensure it maintains strong performance over time as well as GDPR compliance. By analyzing key metrics and incorporating user feedback, the model’s predictions should be regularly evaluated. A drop in accuracy or performance indicates that updates or retraining may be necessary, effectively closing the loop in the AI life cycle. This continuous evaluation is vital for the model to remain adaptable and accurate in its real-time application.
- Processes. Ensuring appropriate processes is particularly important to comply with GDPR requirements concerning individuals’ rights and notification of security breaches.
Individuals’ Rights
It is essential to establish processes that address individuals’ requests for information, access to their personal data, portability, object, rectification, erasure, restriction, and object (see episode 3). These rights are applicable throughout the entire life cycle of an AI system, encompassing both the personal data utilized in training datasets and the data processed during the system’s operational phase.
Security
- Ensuring Continued Security. Another key measure to ensure ongoing GDPR compliance is ensuring the continued security of AI. Malicious actors may attempt prompt injection attacks, using harmful prompts to bypass safeguards, gain unauthorized access, extract personal data, or manipulate outputs. Additionally, attackers might design inputs specifically to capture the model’s responses, gradually building a dataset of input-output pairs to train a replica model, essentially copying the original’s functionality (see episode 3). To prevent such threats, it is important to understand how the model is being used or misused – even if that is different from what was originally expected – and align that usage with established assessment frameworks to ensure safe and responsible operation (see episode 3).
- Notifying Security Breaches. It is imperative to have processes in place to notify relevant authorities of security breaches without undue delay and, where feasible, no later than 72 hours after having become aware of the breach. No notification is required where the personal data breach is unlikely to result in a risk to the rights and freedoms of the individuals concerned. If that is not the case, the breach must also be communicated to the individuals concerned without undue delay.
The authors would like to thank Ekaterina Fakirova for her assistance in preparing this blog post.
