# How to Scrape Reddit Data in 2026: JSON Endpoints and the 1,000 Cap

Since Reddit's API went paid, the goldmine got harder to reach — but the public .json endpoint still works. Here's how to page it, beat the 1,000-item ceiling, and collect at scale without getting rate-limited.

Reddit is one of the richest opinion datasets on the web — real people arguing about products, tools, brands and news in searchable communities. Since Reddit moved its official API to paid pricing in 2023 (widely reported at $0.24 per 1,000 calls, which shut down third-party apps like Apollo), collecting that data at scale got trickier. But there's a route the pricing change didn't close: the public `.json` endpoint that backs every Reddit page. This guide covers how to page it, how to beat the 1,000-item ceiling, and how to collect without tripping the rate limit.

## Append .json to any Reddit URL

Almost every Reddit URL returns structured JSON if you add `.json`. No OAuth, no client secret — just an HTTP GET. A subreddit listing, a post with its comment tree, a user's history: all of it. Start here:

```python
import requests

headers = {"User-Agent": "research-script/1.0 by u/yourname"}
proxies = {"http": "http://USER:PASS@residentialpremium.quantumproxies.io:8080",
           "https": "http://USER:PASS@residentialpremium.quantumproxies.io:8080"}

url = "https://www.reddit.com/r/webscraping/new.json?limit=100"
r = requests.get(url, headers=headers, proxies=proxies, timeout=20)
data = r.json()

for child in data["data"]["children"]:
    post = child["data"]
    print(post["title"], post["ups"], post["num_comments"])
```

Set a descriptive User-Agent — Reddit is stricter with generic ones. The JSON gives you title, score (`ups`), upvote ratio, comment count, author and timestamps directly, so there's nothing to parse out of HTML.

## Pagination with the after token

Listings page with a cursor, not page numbers. Each response includes an `after` token (the fullname of the last item, like `t3_abc123`); pass it back to get the next batch. Loop until `after` comes back null:

```python
def fetch_listing(subreddit, sort="new", pages=10):
    after, out = None, []
    for _ in range(pages):
        url = f"https://www.reddit.com/r/{subreddit}/{sort}.json?limit=100"
        if after:
            url += f"&after={after}"
        data = requests.get(url, headers=headers, proxies=proxies, timeout=20).json()
        out += [c["data"] for c in data["data"]["children"]]
        after = data["data"].get("after")
        if not after:
            break
    return out
```

![Four sorts of one subreddit, new, top, hot and controversial, each drawn as its own window of about 1,000 items; the card lists the other ways past the cap: time filters on top from hour to all, keyword search for older posts, incremental polling of new, and no cap on comments inside one post.](https://res.cloudinary.com/dmqliab0t/image/upload/f_auto,q_auto,w_1200/v1785335171/blog/generated/qp-b9-035-diag1.png)
*The 1,000-item cap is per sort — combining sorts and time filters is how you get past it.*

## The 1,000-item ceiling (and how to break it)

Here's the limit that surprises people: any Reddit listing stops at roughly 1,000 items. Page a subreddit's `new` feed and after ~1,000 posts the `after` token runs dry, no matter how many more posts exist. This is a platform-wide behaviour, not a scraper bug. It does **not** apply to comments inside a single post — you can pull thousands of those. To collect more than 1,000 posts from a community, multiply your views of it:

- **Combine sorts.** Scrape `new`, then `top`, `hot` and `controversial`. Each is a different 1,000-item window into the same subreddit, and the overlap is partial — together they surface far more unique posts.
- **Add time filters.** On `top`, iterate `?t=hour`, `day`, `week`, `month`, `year`, `all` to reveal different slices.
- **Use search.** Keyword searches surface older posts that fell off the main lists entirely.
- **Run incrementally.** Poll `new` on a schedule and dedup by post id, so you capture everything before it ages past the 1,000-item window.

```python
# combine sorts to break past the 1,000-item cap
seen, posts = set(), []
for sort in ("new", "top", "hot", "controversial"):
    for post in fetch_listing("webscraping", sort=sort, pages=10):
        if post["id"] not in seen:
            seen.add(post["id"])
            posts.append(post)
print(f"{len(posts)} unique posts across four sorts")
```

## Rate limits and staying unblocked

One IP polling the JSON endpoints hard gets throttled quickly — Reddit returns 429s and eventually blocks. Two things keep a collection job healthy: pace it (a short delay between requests beats a burst that trips the limiter) and spread it across IPs so no single address carries the whole load. Routing through [residential proxies](https://quantumproxies.io/residential-proxies) lets a research crawl look like many ordinary readers rather than one aggressive client, and a [Scraper API](https://quantumproxies.io/scraper-api) handles that rotation and pacing for you. If you're hitting 429s constantly, our guide on [rate limits and backoff](https://quantumproxies.io/blog/fix-429-too-many-requests-scraping) covers the pacing maths.

## What to do with it: listening at scale

The reason to collect Reddit data is to listen. Track mentions of your brand or a competitor across communities, run sentiment on the comment trees, watch a niche subreddit for recurring problems worth building around, or spot a trend before it hits the mainstream. For anything past keyword matching, feeding the raw text through an [LLM extraction step](https://quantumproxies.io/blog/ai-web-scraping-llm-extraction) turns messy threads into structured signals — themes, complaints, feature requests. Reddit's public data is fair game to collect, but it is people's posts; if you plan to process usernames or personal detail, our overview of [scraping legality in 2026](https://quantumproxies.io/blog/is-web-scraping-legal-2026) is worth a read (informative, not legal advice).

![The Reddit collection loop in four steps: page the .json listing with limit=100 and the after token, pace requests across residential exits, dedup by fullname, and feed the posts into brand listening; the card shows the JSON fields used at each step: after, the post fullname, and title, ups and comment count.](https://res.cloudinary.com/dmqliab0t/image/upload/f_auto,q_auto,w_1200/v1785335171/blog/generated/qp-b9-035-diag2.png)
*Page the JSON, pace it across clean IPs, dedup by fullname, and the data feeds brand listening or trend research.*

[Collect Reddit data on clean residential IPs](https://quantumproxies.io/residential-proxies)

## Frequently asked questions

### Can you still scrape Reddit after the API changes?

Yes. The official API is now paid, but the public `.json` endpoint behind every Reddit page still returns structured data over a plain HTTP GET with no OAuth. It's rate-limited, so pace requests and spread them across IPs, but for research-scale collection it remains the most practical route. `old.reddit.com` and RSS feeds are additional lightweight sources.

### Why does Reddit stop at 1,000 posts?

Reddit caps any single listing — a subreddit sort, a user's history, a search — at about 1,000 items platform-wide; the pagination cursor simply stops. It's not your scraper. Get past it by combining sorts (new, top, hot, controversial), applying time filters, using search, and running incremental jobs that capture posts before they age out of the window.

### Do I need proxies to scrape Reddit?

For a handful of requests, no. For sustained collection you do: a single IP hammering the JSON endpoints gets 429-throttled and then blocked. Residential proxies spread the load so the traffic reads as many ordinary readers, and pacing between requests keeps you under the limiter. Together they let a research crawl run continuously without tripping defences.

### Is scraping Reddit legal?

Collecting publicly visible pages is broadly defensible, but Reddit's terms restrict automated access and the data includes user-generated content and usernames. Keep to public data, respect rate limits, and be careful with anything that identifies individuals. This is general information, not legal advice — check Reddit's current terms and your jurisdiction before a large project.

The API going paid didn't lock the door — it just moved the handle. The `.json` endpoint, cursor pagination, and a strategy for the 1,000-item cap get you the data; residential IPs and polite pacing keep the collection running. From there it's a listening problem, and Reddit is one of the best signals you can point it at. If you also want the API-pricing story on the other platforms, our post on [X/Twitter data without the API](https://quantumproxies.io/blog/twitter-x-data-without-api) maps the same terrain.

[Scrape Reddit at scale with a Scraper API](https://quantumproxies.io/scraper-api)