How to scrape YouTube transcripts at scale (2026)
By the YouTubeTranscriptAPI engineering team · Published · Updated
Your transcript scraper ran fine on your laptop. Then you deployed it, and YouTube started saying no.
To scrape YouTube transcripts at scale in 2026, you need three things a laptop script doesn't: residential IP addresses, a way to handle YouTube's proof-of-origin tokens, and a pipeline that treats blocks, empty responses and missing captions as separate outcomes. The proxy bandwidth itself is cheap. Keeping the scraper working is the real cost, and it's why a hosted API usually wins below about 200,000 transcripts a month.
I read five of the pages that rank for this. Four of them sell a scraper or earn from one, and their numbers are their own. None of them measured what a scraper actually downloads, so I did.
Below you'll find those measurements, the proxy math at 1,000 to 100,000 videos a month, and a script that fetches thousands of transcripts without any of it.
What does it take to scrape YouTube transcripts at scale?
Scraping YouTube transcripts at scale takes more than a loop over video IDs. You need IP addresses YouTube trusts, usually rotating residential proxies, because most cloud IPs are blocked. You need proof-of-origin tokens, or caption requests come back empty. And you need retries, rate limits and a queue that survives a crash, since at 10,000 videos even a 1% failure rate is 100 videos to chase.
Five things change when you go from 50 videos to 50,000:
1. Where the requests come from. A home connection gets through. A cloud server mostly doesn't. 2. How many come at once. YouTube answers too many requests with HTTP 429. 3. What "success" means. A 200 with an empty body is a failure that looks like a success. 4. Who fixes it. When YouTube changes its player, someone has to update the scraper that week. 5. What it costs. Every attempt, blocked or not, uses proxy bandwidth you pay for.
The rest of this guide takes these one at a time.
Why does a YouTube transcript scraper work on your laptop but fail on a server?
A scraper fails on a server because YouTube judges the IP address before anything else. Home connections get through; addresses from AWS, Google Cloud, Azure and other data centers are mostly blocked. A popular open-source Python library for YouTube transcripts says so on its own PyPI page, and its maintainer recommends rotating residential proxies, because static proxies get banned after extended use.
You'll see this as a RequestBlocked or IpBlocked error, an HTTP 403, or YouTube's "Sign in to confirm you're not a bot" page.
Cookies don't fix it. A developer building a cloud video service tested seven approaches in March 2026 and wrote it up on DEV. Valid cookies worked from a laptop and failed from the cloud server with the bot check. Only a residential proxy got through.
That conclusion matches the library's PyPI page: YouTube decides on IP reputation before it looks at cookies.
So the first cost of scraping at scale is buying IP addresses that look like someone's home.
What do YouTube blocks look like at scale?
At scale, YouTube blocks show up as five different responses, and only some of them are worth retrying. A 429 means slow down. A 403 or the bot check means your IP is burned. An empty 200 means a missing proof-of-origin token. A video with no captions isn't a block at all. A scraper that treats them all as "failed" wastes proxy money retrying things that will never work.
| What you see | Usually means | Retry? | Fix |
|---|---|---|---|
| HTTP 429 Too Many Requests | Too many requests from one IP | Yes, after waiting | Slow down, honor Retry-After |
| "Sign in to confirm you're not a bot" or 403 | IP flagged as a data center or abuser | Not from that IP | Different IP, usually residential |
| HTTP 200 with an empty body | Caption request had no PO token | No | Token tooling, or an API |
| No caption tracks on the page | The video has no captions | No | Skip, or use speech-to-text |
| 5xx or a timeout | Temporary problem | Yes, with backoff | Retry a few times, then log it |
The empty 200 is the nasty one. I saw it myself on November 19, 2026.
I fetched a watch page for a popular music video from a home connection, took the caption URL the page lists, and requested it. YouTube answered HTTP 200 with 0 bytes. No error, no text.
That matches the yt-dlp PO token guide, which lists subtitle requests from the web client among those that need a proof-of-origin token. The token comes from YouTube's player code running in a browser.
If your pipeline counts a 200 as a success, you'll ship empty transcripts and never know. Check the body, every time.
Do you need residential proxies to scrape YouTube transcripts?
Yes, if you scrape YouTube transcripts from a cloud server at any real volume. Data center proxies are blocked the same way your cloud IP is, so they rarely help. Rotating residential proxies route each request through a home connection and cost $1.00 to $8.00 per GB, according to a September 2026 check of nine providers' pricing pages. Your bill depends on how many bytes each attempt pulls.
That price range comes from WebScraping.AI's comparison, which checked each provider's live pricing page on September 28, 2026. The cheapest pay-as-you-go option was $1.00 per GB. The priciest list rate was $8.00.
One line from that comparison is worth remembering: you pay for the bytes in a 403 exactly as you pay for the bytes in a 200. Blocks cost money.
Running the scraper from a home machine avoids proxies entirely. It can work for a small, steady job on one connection. It isn't a production system, and that one IP can get blocked too if you push it.
How much bandwidth does scraping one YouTube transcript use?
Scraping one YouTube transcript over plain HTTP uses about 0.15 MB if your client accepts compression, and about 1.4 MB if it doesn't, based on three watch pages we measured on November 19, 2026. The caption file adds a little more. A headless browser is far heavier: Bright Data's docs put a browser-loaded page at 20 MB or more. That choice moves your proxy bill by a factor of 100.
I fetched three watch pages once each from a home connection and recorded the sizes:
| Video | Compressed on the wire | Uncompressed |
|---|---|---|
| A popular music video | 158 KB | 1.49 MB |
| The first video ever uploaded | 138 KB | 1.37 MB |
| A music video with over 5 billion views | 147 KB | 1.43 MB |
The page is mostly JavaScript and JSON. Turning on gzip cuts it by about 90%, and it's an easy setting to miss.
The browser number comes from Bright Data's guide to reducing scraping costs. It says an HTTP client downloads about 500 KB of HTML for a typical page, while a browser loads 20 MB or more of resources for the same page.
Why would you use a browser at all? Because the PO token comes from YouTube's player code, and running that code is the reliable way to get one. That's how a browser ends up in the loop, even when all you want is a text file.
How much does it cost to scrape 10,000 YouTube transcripts?
Scraping 10,000 YouTube transcripts a month costs $2.40 to $19.20 in proxy bandwidth with a lean, compressed HTTP scraper, or $240 to $1,920 if every attempt loads a headless browser, assuming 20% of attempts get retried. Add the time to keep it working: 4 hours a month at the $64.44 US median hourly pay for software developers is $257.76. A hosted API costs $20 to $25.50 for the same 10,000.
These numbers come from the estimate command in our script. It multiplies attempts by megabytes by proxy price, then adds upkeep:
| Scraping profile | GB a month | At $1/GB | At $3.50/GB | At $8/GB |
|---|---|---|---|---|
| HTTP, compressed (0.2 MB a try) | 2.4 | $2.40 | $8.40 | $19.20 |
| HTTP, uncompressed (1.5 MB) | 18 | $18 | $63 | $144 |
| Headless browser (20 MB) | 240 | $240 | $840 | $1,920 |
The hourly rate is the BLS 2025 median for software developers: $134,040 a year, or $64.44 an hour. That's salary only, before benefits.
The 4 hours a month is my assumption, not a measurement. It's the time to notice a break, read the yt-dlp issue tracker, patch the token code and re-run the failed queue. Put your own number in with --hours-per-month.
To be fair to scrapers, the bandwidth on a lean HTTP setup is cheap. If the token handling is already solved and nobody's time counts, $8.40 a month for 10,000 transcripts beats our $20.
The catch is that token handling doesn't stay solved. On August 17, 2026, yt-dlp users started getting 403 errors without changing anything, and the fix shipped in a release two days later, according to VideoProc's troubleshooting guide.
Is it cheaper to scrape YouTube transcripts or use an API?
A hosted transcript API is cheaper for most teams below about 200,000 transcripts a month, once you count upkeep. With 4 hours of monthly maintenance, a lean compressed scraper at $3.50 per GB only beats our $2 per 1,000 above roughly 222,000 transcripts a month. If maintenance really takes 1 hour, the crossover drops to about 55,000. A browser-based scraper never catches up.
Across volumes, with the lean scraper at $3.50 per GB and 4 hours of upkeep, it looks like this:
| Transcripts a month | Lean scraper, all in | YouTubeTranscriptAPI Annual | YouTubeTranscriptAPI Monthly |
|---|---|---|---|
| 1,000 | $258.60 | $2.50 | $3.00 |
| 10,000 | $266.16 | $20.00 | $25.50 |
| 100,000 | $341.76 | $200.00 | $250.50 |
Those plan prices are from our pricing page today. Videos without captions return a 404 and aren't charged, and neither are errors or 429s.
The math is simple. Upkeep is flat, so it dominates at low volume. Bandwidth grows with volume, but slowly, because each transcript is small.
So who should scrape? Teams pulling hundreds of thousands of transcripts a month, with an engineer who already owns the token code and the proxy account. Everyone else is paying salary to save a few dollars of API credits.
If you're still deciding between building and buying, our guide to choosing an API or a tool walks through the other factors.
How do you build a transcript pipeline that survives failures?
A transcript pipeline survives failures when every video ends in a named state, like done, no captions or failed, stored somewhere that outlives the process. Add a shared rate limit, retries that honor Retry-After, a check that a 200 actually contains text, and a credit or budget guard. Then a crash or a block costs you minutes, not a rerun of 10,000 videos.
This applies whether you scrape or use an API. Our script does it with an API key:
- A SQLite file holds the queue. Each video is pending, done, no_captions or failed. - 24 workers share one rate limit of 280 requests a minute, under our 300-a-minute limit. - 429s, 408s and 5xx errors are retried with the wait the server asks for, up to five tries. - A 200 with an empty transcript is marked failed, not done. - A 402, out of credits, stops every worker and leaves the rest pending for tomorrow.
Every status code the API can return is listed in our errors reference. Missing captions come back as a 404, which the script stores as no_captions so you can count them.
That count matters. If 15% of a channel has no captions, you want to know before you promise a client "every video."
One more habit that pays at scale: never fetch the same video twice. Captions on an old video rarely change, so keep what you have and only fetch new IDs.
A scraper that re-reads a 5,000-video channel every night pays for 5,000 proxy requests to learn about the 3 new uploads. Keep a list of what's done, which is exactly what the queue file is for.
On our side, a transcript someone already fetched comes back from our store in about a second, marked X-Cache-Status: HIT. Our caching docs explain when a stored copy is used.
Is it legal to scrape YouTube transcripts?
YouTube's terms of service forbid accessing the service with robots or scrapers, except for public search engines and anyone with YouTube's written permission. Breaking a site's terms isn't automatically illegal, and the law depends on your country and use. At scale the practical risk is that YouTube can block you, change its endpoints or ask you to stop.
I covered the terms in more detail, with the exact wording and date, in our guide to the free YouTube transcript API question.
The short version for this article: the bigger your scraping operation, the more it depends on YouTube tolerating it. That's a business risk worth pricing in, next to the proxies and the hours.
This isn't legal advice. If you're building a commercial product on scraped data, talk to someone who can look at your specific case.
How do you fetch 10,000 YouTube transcripts without scraping?
To fetch 10,000 YouTube transcripts without running a scraper, put the video IDs or URLs in a text file and run our script with an API key. It fetches them in parallel at up to 280 a minute, saves each transcript as JSON, skips videos you already have, and stops cleanly if credits run out. Expect 10,000 videos to take between 40 minutes and about 3 hours.
Download transcripts_at_scale.py. It needs Python 3.9 or newer and nothing else.
export YT_API_KEY=your_key # 100 free credits, no card
python transcripts_at_scale.py run ids.txt --out transcripts --workers 24Why such a wide range? A video nobody has fetched before takes 10 to 30 seconds, and a stored one about a second. With 24 workers that's roughly 50 to 280 transcripts a minute, depending on the mix.
The output folder gets one <video_id>.json per transcript and a queue.db file. Stop it any time with Ctrl+C and run the same command again: finished videos are skipped.
Need a channel's video IDs first? Our guide to YouTube channel transcripts in bulk lists a channel's newest 100 videos for 20 credits and shows how to feed them in.
And before you decide anything, price your own setup:
python transcripts_at_scale.py estimate --per-month 50000 --hours-per-month 2It prints the scraping cost for each profile at three proxy prices next to both of our plans. Change the inputs until they match your reality.
Frequently asked questions
How do I scrape YouTube transcripts without getting blocked?
Run the scraper from IP addresses YouTube trusts, usually rotating residential proxies, since most cloud server IPs are blocked. Keep request rates low and honor Retry-After on 429 responses. Handle proof-of-origin tokens, or caption requests return an empty 200. Even then, expect breaks when YouTube updates its player. A hosted transcript API handles all of this for you and charges per transcript.
Why does my YouTube transcript scraper work locally but not on AWS?
YouTube blocks most IP addresses that belong to cloud providers like AWS, Google Cloud and Azure. Your laptop uses a home IP, which YouTube trusts, so the same code works there. On the server you'll see RequestBlocked, a 403 or a "Sign in to confirm you're not a bot" page. Cookies don't help; residential proxies or a hosted API do.
How much do residential proxies cost for scraping YouTube?
Residential proxies cost $1.00 to $8.00 per GB at September 2026 list prices. A compressed HTTP request for a YouTube watch page is about 0.15 MB, so 10,000 transcripts with 20% retries uses about 2.4 GB, or $2.40 to $19.20. A headless browser uses roughly 100 times more. Blocked requests use bandwidth too.
Why does my YouTube transcript scraper return empty transcripts?
Most likely YouTube wants a proof-of-origin token on the caption request. Without one, the caption URL answers HTTP 200 with an empty body, which we reproduced on November 19, 2026. The yt-dlp PO token guide lists subtitle requests from the web client among those that need the token. Check that every 200 actually contains text.
How many YouTube transcripts can I scrape per hour?
YouTube doesn't publish a limit for caption requests, and blocks depend on your IP's reputation as much as on speed. From a single IP, expect 429 errors well before you reach thousands an hour. With YouTubeTranscriptAPI the limit is 300 requests a minute per key, which is 18,000 an hour, and rate-limited requests aren't charged.
Is it cheaper to build a YouTube transcript scraper or use an API?
For most teams an API is cheaper. Proxy bandwidth for a lean scraper is small, about $8.40 for 10,000 transcripts at $3.50 per GB, but keeping it working takes engineering time. At 4 hours a month, a scraper only beats a $2 per 1,000 API above about 222,000 transcripts a month. A browser-based scraper costs more at every volume.
Price your scraper before you scale it
Run estimate with your real volume, your proxy price and an honest number of hours. If the API column is lower, stop maintaining the scraper and move the IDs to run.
If the scraper still wins, keep it, and add the empty-200 check today. That one bug is the most common way a scraper fails without telling anyone.