How a Researcher Scraped 4.5B TikTok Videos by Reverse-Engineering the App's Private API
4.5B Posts Scraped from TikTok
A technical deep dive into scraping TikTok's mobile API, which is faster and richer than the web endpoints. The author details the four prerequisites for success: a valid device credential, request signature, regional host, and TLS fingerprint. They also share a free dataset of 4.5 billion videos on Hugging Face, collected using this method.
A device that registered and then immediately started querying the API is, from ByteDance's side, an install that never launched.
- 1vuio0pswjnm7
1787990085 | X and Meta's past data scraping lawsuit losses and how it would relate to nitter | https://www.reuters.com/legal/musks-x-corp-loses-lawsuit-aga... | https://news.ycombinator.com/item?id=49487864
Perhaps Google can succeed where others like Meta and X have failed, or perhaps not
https://storage.courtlistener.com/recap/gov.uscourts.cand.46...
- Retr0id
Can any brave soul wade through the LLM prose to provide a human-readable summary?
- moinism
> Everything described here is a private Go repository. One-time payment, permanent access, complete source.
> $699 one time · lifetime access
Not open-source apparently.
And I cant find the reddit post but I think I read that videos/assets are not actually pre-downloaded, they have to be requested through Tiktok API using the provided code. So if Tiktok patches, the code will need updates too.
- nomilk
> Three things to notice, because each one bites later:
Very LLMish language!
- smallerize
There's no way this dataset is going to survive on HF, right? It will be hit with so many DMCA takedowns.