Squid has a feature they coined [Collapsed Forwarding|http://wiki.squid-cache.org/Features/CollapsedForwarding]. In a nutshell, it allows the intermediary to piggy back multiple clients on a single origin server connections. There are a few ways we could accomplish this, but they all would share the same property: this is only safe to do for cacheable objects.

h2. Problem space and issues

The problem to solve is a form of stampeding heard: When a cached object goes stale, instead of potentially letting 100's or 1000's of requests slip through to the origin, we want to minimize this as much as possible. In a best case scenario,  a redux to 1 origin connection could be achieved. One issue here is to know if a request is cacheable (and therefore shareable). On a complete cache miss (cold cache), this can be difficult at best. On a cache

ATS already has three (and a half) features that are in place to alleviate this problem, and we will discuss those three here as well. The discussion should also include options of improving upon these existing features, such that we would not need more complexity.

h3. Fuzzy logic

ATS has three configuration options related to pre-fetching objects *before* they go stale in cache:

  {RECT_CONFIG, "proxy.config.http.cache.fuzz.time", RECD_INT, "240", RECU_DYNAMIC, RR_NULL, RECC_NULL, NULL, RECA_NULL}
  ,
  {RECT_CONFIG, "proxy.config.http.cache.fuzz.min_time", RECD_INT, "0", RECU_DYNAMIC, RR_NULL, RECC_NULL, NULL, RECA_NULL}
  ,
  {RECT_CONFIG, "proxy.config.http.cache.fuzz.probability", RECD_FLOAT, "0.005", RECU_DYNAMIC, RR_NULL, RECC_NULL, NULL, RECA_NULL}

{code}
CONFIG proxy.config.http.cache.fuzz.time INT 240

CONFIG proxy.config.http.cache.fuzz.min_time INT 0

CONFIG proxy.config.http.cache.fuzz.probability FLOAT 0.005

{code}
The intent here is that there's a small chance (*fuzz.probability* == 0.5%) that a request for an object will be revalidated, starting 240 seconds before it goes stale. For objects getting few requests per second, this would likely not trigger, but then this feature is not necessary anyways since odds are only 1 or a small number of connections would hit origin upon objects going stale. The defaults are a good compromise, for objects getting roughly 4 requests / second or more, it's guaranteed to trigger a revalidate event within the 240s. These configs are also overridable per remap rule or via a plugin, so can be adjusted per request if necessary.

Finally, the *fuzz.min_time* is there to be able to handle requests with a TTL less than *fuzz.time*; In this case, the revalidation happens in a function between *fuzz.min_time* and *fuzz.time*, and it gets closer to expiring, becoming more likely. By default this setting is not enabled, but should be enabled anytime you have objects with small TTLs. Note that this option predates overridable configurations, so you can achieve something similar with a plugin or *remap.config* conf_remap.so configs.

These configurations are similar to Squid's{color:#000000} {color}{*}refresh_stale_hit* configuration option.

h3. Read While Write

This feature is pretty similar to how Squid actually implements Collapsed Forwarding. When ATS goes to fetch something from cache, and upon receiving the response from origin, any number of clients can be allowed to start serving out of the partially filled cache object. The difference (Ed: I think?) is that Squid allows this as soon as we go to origin, whereas ATS can not do it until we get the response header. The reason for this is that we make no distinction between cache refresh, and cold cache, so we have no way to know if a response is going to be cacheable, and therefore allow read-while-write functionality.

The configurations necessary to enable this in ATS are:

{code}
CONFIG proxy.config.cache.enable_read_while_writer INT 1

CONFIG proxy.config.http.background_fill_active_timeout INT 0

CONFIG proxy.config.http.background_fill_completed_threshold FLOAT 0.000000

CONFIG proxy.config.cache.max_doc_size INT 0

{code}
All for configurations are required, for the following reasons:
# Obviously the feature should be enabled to begin with. It's off by default (as of 4.0.x, we might want to modify this for 5.x)
# The background fill option must be enabled, and allowed to kick in for every request. This is necessary, in case the owning consumer ("client session") goes away, someone needs to take over the session.
# The max_doc_size must be unlimited, since we potentially don't know the size of the object, we can't allow it to disconnect due to going over this limit.

Once all this enabled, you have something that is very close, but not quite the same, as Squid's Collapsed Forwarding.

Open Retry Timeout


h3.