GDPR & Web Scraping: Personal Data, Fines & a Checklist

GDPR does not ban scraping — it governs personal data. Here is exactly when it applies, why legitimate interest is your only realistic basis, what the recent EU fines punished, and a checklist engineers can actually run.

GDPR does not ban web scraping. It regulates the processing of personal data, and scraping only collides with it when the data you collect can identify a person. Get that distinction right and most compliance questions answer themselves; get it wrong and you inherit the exposure that recent EU enforcement has made very concrete. This is an engineer's guide, not a lecture: when GDPR actually applies to a scrape, why legitimate interest is realistically your only lawful basis, the data-minimization and notification rules that regulators keep punishing people for ignoring, and a checklist you can run before a job starts. It is informative guidance and not legal advice — for a specific project, involve a data protection lawyer.

The only question that matters first: is it personal data?

GDPR defines personal data as any information relating to an identified or identifiable person, and it applies to EU residents' data regardless of where your servers sit. The practical line for scrapers is clean. Scraping non-personal data generally sits outside GDPR entirely: product prices and specifications, stock and market figures, news and blog content, real estate listings, public government statistics. The moment your dataset contains names, email addresses, phone numbers, photos, profiles — or the less obvious identifiers like IP addresses, cookie IDs and device fingerprints — you are processing personal data and the full weight of the regulation attaches. Health, biometric and similar 'special category' data raises the bar further, generally requiring explicit consent. If your target is prices and catalogue data, you are mostly in the clear; the broader legal picture beyond GDPR is in is web scraping legal in 2026.

Why legitimate interest is your only realistic basis

GDPR's Article 6 lists six lawful bases, but only one fits large-scale scraping. Consent is impractical — you cannot obtain informed agreement from millions of people whose data you have not yet collected. Contractual necessity fails because you have no relationship with the data subjects. That leaves legitimate interest, and the EDPB's ChatGPT Taskforce and France's CNIL both confirm it as the standard basis — with conditions. It demands a documented three-part test: the interest is genuinely legitimate, the processing is necessary and not excessive, and it does not override the rights and reasonable expectations of the people whose data you take. Write that Legitimate Interest Assessment down before you scrape; 'if you cannot prove compliance, you are not compliant' is the operating principle regulators apply.

Checklist showing which scraped data sits outside GDPR, like prices and news, versus data that triggers GDPR, like names, emails, IP addresses and profiles — The trigger is personal data of EU residents and nothing else. Prices, stock and news are usually clear; anything that identifies a person pulls the whole regulation in.

What the recent fines actually punished

The enforcement of the last two years tells you exactly where the risk concentrates. The Dutch DPA fined Clearview AI EUR 30.5 million in 2024 for building a facial-recognition database from scraped public images without informing people or offering access and deletion — the Italian regulator had already levied EUR 20 million for the same model. France's CNIL fined KASPR EUR 240,000 in December 2024 for scraping LinkedIn contact details, including from users who had restricted their visibility, across a database of roughly 160 million contacts. Poland's DPA imposed EUR 220,000 on a data broker that scraped public business registries covering millions of people and then argued that notifying them was a 'disproportionate effort' — the regulator rejected that outright. The ceiling on all of this is the GDPR maximum of EUR 20 million or 4% of global annual revenue, whichever is higher.

Read across those cases and the pattern is consistent: the violations were failing to inform people, ignoring their rights, collecting restricted or excessive personal data, and continuing after being told to stop. None of them turned on 'scraping is illegal' — they turned on how personal data was handled.

The engineer's checklist

Translate the principles into steps you run, not a policy you file. Before and during any job that touches personal data:

Minimize at the point of collection. Only keep fields you actually need for your documented purpose. If you need comment text, do not store the commenter's name and pseudonym alongside it.
Limit to freely accessible data. CNIL guidance says stay on public pages that require no login — do not scrape behind authentication or past restricted-visibility settings.
Respect technical objections. robots.txt and CAPTCHA are signals a site does not want automated collection; ignoring them weakens your legal position. Details in robots.txt in practice.
Set retention limits. Delete personal data once it has served its purpose; indefinite storage is a storage-limitation breach.
Plan for Article 14 and DSARs. Be able to inform data subjects and to handle access and deletion requests — the failure that sank the Clearview and Poland cases.
Run a DPIA for large-scale or higher-risk collection, and keep the documentation.

Two of these are literally code. Minimization is a filter that drops PII you did not set out to collect:

KEEP = {"product", "price", "currency", "rating", "review_text"}
PII  = {"reviewer_name", "email", "user_id", "ip"}

def minimize(record):
    # keep only declared fields; never persist PII you did not need
    return {k: v for k, v in record.items() if k in KEEP and k not in PII}

And respecting the site's stated wishes is a check you make before the first request, using the standard library:

import urllib.robotparser as rp

def allowed(url, ua="MyResearchBot"):
    r = rp.RobotFileParser()
    r.set_url(url.rstrip("/").split("/", 3)[0] + "//" +
              url.split("/")[2] + "/robots.txt")
    r.read()
    return r.can_fetch(ua, url)

Stat panel of recent EU GDPR fines for web scraping personal data: Clearview AI, KASPR LinkedIn scraping, a Poland registry case, and the 4 percent revenue cap — The fines punished handling, not scraping itself: no notification, ignored rights, excessive or restricted data, and carrying on after being told to stop.

Where proxies fit — and don't

Proxies are infrastructure for reaching public data reliably and from the right geography; they are not a compliance shortcut. Routing a scrape through a scraper API or residential IPs does not change whether you have a lawful basis, and it does not launder personal data you should not be collecting. Where they genuinely help is keeping you on the compliant path: collecting only public, non-personal data at scale — prices, catalogues, market signals — which is exactly the low-risk work most teams actually need. If your project does touch people's data, such as B2B contact research, treat compliance as the design constraint from the start, as we frame in LinkedIn public data and compliance.

Frequently asked questions

Is web scraping legal under GDPR?

Scraping is not banned by GDPR, but processing personal data of EU residents requires a lawful basis and compliance with the regulation's principles. If you scrape only non-personal data — prices, stock levels, news — GDPR generally does not apply. If your data identifies people, you need a documented basis (usually legitimate interest), data minimization, retention limits and a way to honour data subject rights.

What personal data can I scrape without breaching GDPR?

The safest position is to avoid personal data altogether: product, price, stock, market, news and public statistical data typically fall outside GDPR. When you must process personal data, limit it to freely accessible public information, collect only what your documented purpose requires, and be ready to inform people and handle deletion requests. Avoid special-category data like health information unless you have explicit consent.

Does GDPR apply if my company is outside the EU?

Yes. GDPR applies based on whose data you process, not where you are located. If you scrape personal data of individuals in the EU or EEA, the regulation applies regardless of your company's country. Scraping data that concerns only non-EU individuals falls under those jurisdictions' laws instead, but the EU cases show regulators will pursue foreign operators handling EU residents' data.

What is the maximum GDPR fine for scraping?

The GDPR ceiling is EUR 20 million or 4% of global annual revenue, whichever is higher. Real scraping penalties have ranged from six figures — EUR 220,000 for a registry scrape, EUR 240,000 for LinkedIn contact scraping — to EUR 30.5 million against Clearview AI. The common thread is mishandling personal data: no notification, ignored rights, and excessive collection.

The compliant path is narrower than 'scrape anything public' and much wider than 'scraping is illegal'. Stay on non-personal data where you can, document legitimate interest and minimize where you cannot, respect the signals sites give you, and keep the receipts. Do that and GDPR is a design constraint, not a threat.

Collect public, non-personal data with the Scraper API