Pharmacy Price Monitoring: Building GoodRx-Style Data the Right Way
US drug prices swing wildly by pharmacy, ZIP and discount card — and public tools already expose them. Here's how to collect that data at scale, where the official datasets are, and how to stay on the compliant side of a sensitive field.
US prescription prices are famously chaotic: the same drug can cost wildly different amounts depending on the pharmacy, the ZIP code, and which discount card you flash at the counter. That variance is exactly why price-transparency tools exist and why the underlying data is valuable for research, comparison sites and market analysis. It's also a sensitive field, so the right way to collect it is compliance-first — public, non-personal price data only. This guide covers what makes the data worth monitoring, where the official sources are, how to capture geo spreads at scale, and where the line sits. It's practical information, not legal or medical advice.
Why the data is worth collecting
The spreads are enormous. A RAND analysis of 2022 data found US prices across all drugs were 278% of prices in 33 comparison countries — and for brand-name originator drugs, 422%. Generics buck the trend: US unbranded generics, which make up about 90% of US prescription volume, were actually cheaper at 67% of other countries' prices. Even within the US, discount tools compare prices across more than 70,000 pharmacies and advertise savings of up to 80%. That combination — huge cross-country gaps, and huge within-country variance — is what makes systematic price monitoring useful, whether you're building a comparison service, tracking a market, or researching affordability.
Start with official datasets
Before scraping anything, check whether the data is already published. A lot of it is. Medicaid publishes the National Average Drug Acquisition Cost (NADAC) as an open, weekly-updated dataset — pharmacy acquisition costs by drug, no scraping required. Several states run their own transparency tools; Florida's, for example, lets you compare retail prices for the 400 most-dispensed drugs by county. Government and academic sources (RAND, ASPE) publish the cross-country comparisons. Pulling structured data from an official API is faster, cleaner and lower-risk than scraping a consumer site — always exhaust it first.
import requests
# NADAC is an open Medicaid dataset -- query it directly, no scraping
r = requests.get(
"https://data.medicaid.gov/api/1/datastore/query/<dataset-id>/0",
params={"limit": 500, "offset": 0},
timeout=30,
)
rows = r.json().get("results", [])
for row in rows[:5]:
print(row["ndc_description"], row["nadac_per_unit"], row["effective_date"])
Capture the geo spread
The interesting signal in consumer price tools is local: cash and discount prices for a drug change by ZIP, because pharmacy competition and negotiated rates differ by area. To capture that spread you query a public price page for the same drug across a set of ZIPs, and — since these tools personalise by the visitor's location — route each request through a residential exit in the matching region so you see local prices, not one default. The output is a per-ZIP price matrix per drug: which pharmacies are cheapest where, and how wide the spread runs.
import requests, time, random
PROXY = "http://USER:PASS@gate.quantumproxies.io:8000"
proxies = {"http": PROXY, "https": PROXY}
def cash_prices(drug, zip_code):
r = requests.get(f"https://example-rx.com/{drug}",
params={"zip": zip_code}, proxies=proxies, timeout=20)
time.sleep(random.uniform(2, 4)) # polite pacing
return parse_prices(r.text) # [{pharmacy, price}, ...]
zips = ["10001", "33101", "90001", "60601"] # NY, Miami, LA, Chicago
for z in zips:
rows = cash_prices("atorvastatin-20mg", z)
best = min(rows, key=lambda x: x["price"])
print(z, best["pharmacy"], best["price"])Geo accuracy is the whole point, so a residential pool with city and country targeting across 200+ locations is what makes per-ZIP monitoring real. This is the same location-first approach behind grocery and delivery price intel.
There's a second axis worth capturing: the payer spread. The same drug has a cash price, a discount-card price and, separately, whatever insurance negotiates — and the cheapest of those is often not the one a patient assumes. Because discount tools surface cash and coupon prices side by side, a monitor can record both per pharmacy and flag where the discount price undercuts the list price by the widest margin. Snapshot on a schedule and you also see prices move over time, which is where the analysis gets interesting: seasonal shifts, new generics entering the market, and pharmacies repricing in response to competitors down the street. Store the drug, dosage, form, pharmacy, ZIP, price type and a timestamp on every row so the dataset stays clean and queryable.
Get geo-targeted residential proxies
Handle the rendering and the walls
Consumer price sites tend to render results with JavaScript and defend against bots, because their pricing is their product. A raw request often returns an empty shell, and heavy polling from one IP gets challenged. Rotating residential IPs plus polite pacing keeps a monitor running; when a site renders client-side or throws persistent challenges, a Scraper API that renders the page and returns structured data is the cleaner path. Keep the request rate gentle regardless — these are health-adjacent services, and being a good citizen of the site is part of doing this responsibly.
The compliance line
This is where discipline matters more than in ordinary price scraping. Collect advertised cash and discount prices, pharmacy names and locations, drug names and dosages, and official public datasets — all non-personal, publicly displayed information. Do not touch patient records, prescriptions, insurance or claims data, anything behind a login, or any health information tied to an individual — that's personal data under laws like the GDPR and can be sensitive health data on top. Keep your dataset to prices and pharmacies, aggregate rather than track individuals, and you stay well inside the line. Our guide on GDPR and scraping personal data covers the reasoning in full.
Frequently asked questions
Can you scrape prescription drug prices?
Advertised cash and discount prices shown publicly on comparison and pharmacy sites can be collected, and much of the reference data is published as official open datasets. The important limits are compliance-driven: stay on public, non-personal price data, avoid anything behind a login, and never collect patient or health information about individuals.
Where can I get official drug price data?
Medicaid publishes NADAC, an open weekly dataset of pharmacy acquisition costs, queryable through its data API. Several states run price-transparency tools — Florida's covers the 400 most-dispensed drugs by county. RAND and ASPE publish cross-country comparisons. Start with these official sources before scraping any consumer site; they're cleaner and lower-risk.
Why do drug prices vary by location?
Cash and discount prices reflect local pharmacy competition and negotiated rates, which differ by area, so the same drug costs different amounts by ZIP. Consumer tools also personalise results to the visitor's location. To capture the real spread, query multiple ZIPs and route each request through a residential exit in the matching region using geo-targeted proxies.
Do I need proxies for pharmacy price monitoring?
For per-ZIP geo accuracy, yes — you need exits in the regions you're pricing, which means geo-targeted residential proxies. They also spread requests so scheduled monitoring from one address doesn't get blocked. Official datasets like NADAC need no proxies at all, so pull those directly and reserve proxies for the consumer-site geo data.
Pharmacy price data is valuable precisely because it's so variable — 278% cross-country gaps, wide per-ZIP spreads, 70,000-plus pharmacies pricing differently. Build the pipeline the right way: exhaust official datasets first, capture geo spreads with location-targeted residential proxies, render guarded sites with a Scraper API, and draw a hard line at public prices with no personal data. Do that and you get a clean, defensible price-monitoring dataset in a field where compliance is the whole game.