Reference designs and worked examples drawn from production systems built on the YouTube Data API. Covers large-scale crawling, Xapian-backed full-text search and classification, BM25 ranking, NLP pipelines, and brand-safety / contextual-targeting applications for advertising on social video platforms.
The pages cover two different ways of working with YouTube. The search-engine design uses the official YouTube Data API v3: an API key for public metadata, or OAuth for user-scoped data, and a quota measured in units per day where search requests are much more expensive than list requests by ID. A production crawler therefore discovers video IDs cheaply, batches detail lookups, and stores raw responses so the index can be rebuilt without spending quota again. The text is indexed with Xapian, ranked with BM25 and enriched with NLP features for brand-safety classification.
The two cleanup pages solve a smaller problem: removing every item from the Watch Later or Liked playlists. These are personal playlists with limited API support, so the scripts drive the YouTube web interface from the browser developer console, clicking menu items with delays, retries and lazy-load handling.
| Task | Approach | Notes |
|---|---|---|
| Collect metadata for many public videos or channels | Data API v3 | Plan around the daily quota; cache responses |
| Build search or classification over video text | API + local index | See the search-engine design |
| Bulk-clear Watch Later | Browser script | Runs as you, in your session |
| Bulk-unlike videos | Browser script | Slow by design to avoid rate limits |
Browser scripts depend on YouTube's page structure and can break when the interface changes; read them before pasting them into a console, since console code runs with your account's full permissions.