Request Throttling
Deliberately slowing down how often a scraper sends requests to stay under a site's limits, helping avoid bans, server overload, and detection.
Full Definition
Request throttling is the practice of intentionally limiting the rate at which a scraper sends HTTP requests to a target server. Instead of sending requests as fast as possible, throttling introduces delays — either fixed (e.g. 1 second between requests) or randomised (e.g. 0.5–3 seconds uniformly random) to mimic human browsing patterns. Randomised delays are more effective at evading detection than fixed intervals, which create a rhythmic pattern that bot-detection systems can identify. Good throttling respects both per-IP limits and per-account limits, especially when multiple scraping workers share the same account session.
