AI is no longer “the future.” It’s already embedded in recruitment tools, credit scoring, customer service, recommendation engines, and internal decision support systems. And when AI works with (or around) personal data, you’re almost immediately in GDPR territory.
Many organisations treat GDPR as a simple checklist. In reality, there are a few less obvious challenges—both legal and technical—that can introduce new risks, even when you think you’re doing everything right. Let’s start from the basics and then land in four key insights.
First: What Is GDPR, Really?
GDPR is the EU’s framework for how organisations are allowed to handle personal data. At its core, it tries to answer three simple questions:
- Are we allowed to use this data?
- Are we handling it in a fair and secure way?
- Can the individual understand and influence what’s happening?
What Counts as Personal Data?
Any information that can be linked to a person, directly or indirectly. For example:
- Names, email addresses, phone numbers
- Customer IDs, IP addresses, cookie identifiers
- Location data
- Things you can infer about someone (more on that soon)
What Does “Processing” Mean?
Almost everything: collecting, storing, analysing, sharing, training an AI model on the data, or using AI to make decisions.
1) The “Right to Explanation” Is More Nuanced Than Most Think
When discussing AI and GDPR, the “right to explanation” often comes up. Here’s the nuance: GDPR does not grant a simple, absolute right to have an algorithm explained line by line. However, if an AI system makes an automated decision that significantly affects you (for example in recruitment or credit scoring), you have the right to receive a meaningful explanation at a general level. You should also be able to request human review, challenge the decision, and express your point of view. This puts responsibility on companies to build AI systems that can be justified and governed—even when the model itself is difficult to fully interpret.
Many modern models operate as “black boxes”: so complex that it can be hard to explain exactly why a specific outcome occurred—sometimes even for the team that built it.
So what do you do if you can’t explain everything?
Good practice is less about dissecting the black box—and more about building accountability around it:
- Event logging: what data was used, which model version, what decision was made, when, and by whom
- Outcome validation: does the model perform equally well across different groups? Is there bias?
- Process controls: who is allowed to use the model, when is human review required, how are appeals handled?
The point: transparency is not a feature. It’s a combination of technology, process, and governance.
2) “Public Data” Is Not a Free Buffet
A common misconception is:
“If it’s publicly available online, we can use it.”
GDPR doesn’t primarily care where you found the data—it cares about:
- what type of data it is, and
- what you intend to do with it
A classic example is companies collecting images from open sources for facial recognition. The issue is that facial data used for identification often qualifies as biometric data, which is considered highly sensitive under GDPR and requires a clear legal basis and strict handling.
A practical rule of thumb:
“Public” does not mean “free to use.” It just means “visible.”
3) AI Can Memorise—and Accidentally Leak—Personal Data
This is where things get very real. Even if you never intend to expose training data, the model itself can sometimes “remember” parts of it. This can lead to privacy risks through so-called privacy attacks, such as:
Membership inference: attempting to determine whether a specific person’s data was part of the training set
Model inversion: attempting to reconstruct original data by querying the model in specific ways
The key implication is strategic: the AI model itself can become an information asset that needs protection—almost like a database. This changes how you should think about security, access control, logging, and incident management.
4) Even How You Scroll Can Become High-Risk Data
When we talk about profiling, many think of “likes” and “interests.” But AI can also infer insights from more subtle behaviour:
- how long someone stays on a post
- how quickly they scroll
- what content they pause on, without clicking
These signals can be used to infer things about an individual. And here’s a GDPR nuance many miss:
Even inferred data can be considered personal data—if it can be linked to an identifiable individual.
This matters because incorrect assumptions or aggressive profiling can lead to:
- discriminatory outcomes
- decisions that are hard to understand or challenge
- “invisible” risks where individuals don’t even know what is being assumed about them
Summary: Trustworthy AI Is About More Than Compliance
GDPR is not a one-time checkbox. It’s a way of working. And AI makes some classic GDPR questions sharper:
- What is truly “minimum necessary data” when models tend to want everything?
- How do we ensure control when outcomes aren’t always easy to explain?
- How do we handle the fact that the model itself can become a privacy risk?
If you’re using AI in ways that involve personal data, aim for this:
- Be clear about why you use the data (purpose must be legitimate and understandable)
- Don’t use more data than necessary (AI loves “everything,” GDPR prefers “just enough”)
- Stay in control of the risks (security, logging, testing, human oversight)
Most importantly: using AI in a way people can trust is not just about compliance. It’s a competitive advantage. And maybe the most important question we need to ask—both as companies and as a society—is this:
As AI becomes more convenient and invisible, will we demand more transparency, or simply get used to not knowing what happens to our data?
