High-Concurrency Crawler Optimization Notes – From 1 to 10000 QPS | Hacker News Reader