Page History

When Traffic Server is used as a delivery server for a video live streaming event, it's possible that there are a huge number of concurrent requests for the same object. Depending on the type of the object being requested, the cache lookup for those objects can result in either a stale copy of the object (e.g manifest files) or a complete cache miss (e.g segment files). ATS currently supports different types of connection collapse (e.g. read-while-write functionality - https://docs.trafficserver.apache.org/en/latest/admin/http-proxy-caching.en.html#read-while-writer*, stale-while-revalidate writer functionality, stale_while_revalidate plugin etc), but, in order for the read-while-write to writer to kick-in, ATS requires the complete response headers for the object be received and validated. Until this happens, any number of incoming requests for the same object that result in a cache miss or a cache stale would be forwarded to the origin. For a scenario such as a live event, this leaves a sufficiently significant window, where there could be 100's of requests being forwarded to the origin for the same object. This scenario is what is commonly known as a Thundering Herd. It has been observed during production that this results in significant increase in latency for the objects waiting in read-while-write state.

Note that, there are also a couple of related settings proxy.config.http.cache.open_read_retry_time and proxy.config.http.cache.max_open_read_retries (https://docs.trafficserver.apache.org/en/latest/admin/http-proxy-caching.en.html#open-read-retry-timeout) that open-read-retry-timeout and max-open-read-retries that can alleviate the thundering herd to some extent, by re-trying to acquire the read lock for the object. With these settings configured, ATS would retry to get the read lock for the configured amount of attempts each with the configured duration and if the read lock is still not available (due to the write lock being held by the first request that was forwarded to the origin - for e.g. due to the fact that the response headers have not been received yet), then all the waiting requests would simply be forwarded to the origin (by disabling cache for each of them).

...