DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
Squid
...
has
...
a
...
feature
...
they
...
coined
...
...
...
.
...
In
...
a
...
nutshell,
...
it
...
allows
...
the
...
intermediary
...
to
...
piggy
...
back
...
multiple
...
clients
...
on
...
a
...
single
...
origin
...
server
...
connections.
...
There
...
are
...
a
...
few
...
ways
...
we
...
could
...
accomplish
...
this,
...
but
...
they
...
all
...
would
...
share
...
the
...
same
...
property:
...
this
...
is
...
only
...
safe
...
to
...
do
...
for
...
cacheable
...
objects.
...
Problem
...
space
...
and
...
issues
...
The
...
problem
...
to
...
solve
...
is
...
a
...
form
...
of
...
stampeding
...
heard:
...
When
...
a
...
cached
...
object
...
goes
...
stale,
...
instead
...
of
...
potentially
...
letting
...
100's
...
or
...
1000's
...
of
...
requests
...
slip
...
through
...
to
...
the
...
origin,
...
we
...
want
...
to
...
minimize
...
this
...
as
...
much
...
as
...
possible.
...
In
...
a
...
best
...
case
...
scenario, a redux to 1 origin connection could be achieved. One issue here is to know if a request is cacheable (and therefore shareable). On a complete cache miss (cold cache), this can be difficult at best. On a cache
ATS already has three (and a half) features that are in place to alleviate this problem, and we will discuss those three here as well. The discussion should also include options of improving upon these existing features, such that we would not need more complexity.
Fuzzy logic
ATS has three configuration options related to pre-fetching objects before they go stale in cache:
| Code Block |
|---|
a redux to 1 origin connection could be achieved. One issue here is to know if a request is cacheable (and therefore shareable). On a complete cache miss (cold cache), this can be difficult at best. On a cache ATS already has three (and a half) features that are in place to alleviate this problem, and we will discuss those three here as well. The discussion should also include options of improving upon these existing features, such that we would not need more complexity. h3. Fuzzy logic ATS has three configuration options related to pre-fetching objects *before* they go stale in cache: {RECT_CONFIG, "proxy.config.http.cache.fuzz.time", RECD_INT, "240", RECU_DYNAMIC, RR_NULL, RECC_NULL, NULL, RECA_NULL} , {RECT_CONFIG, "proxy.config.http.cache.fuzz.min_time", RECD_INT, "0", RECU_DYNAMIC, RR_NULL, RECC_NULL, NULL, RECA_NULL} , {RECT_CONFIG, "proxy.config.http.cache.fuzz.probability", RECD_FLOAT, "0.005", RECU_DYNAMIC, RR_NULL, RECC_NULL, NULL, RECA_NULL} {code} CONFIG proxy.config.http.cache.fuzz.time INT 240 CONFIG proxy.config.http.cache.fuzz.min_time INT 0 CONFIG proxy.config.http.cache.fuzz.probability FLOAT 0.005 {code} |
The
...
intent
...
here
...
is
...
that
...
there's
...
a
...
small
...
chance
...
(
...
fuzz.probability
...
==
...
0.5%)
...
that
...
a
...
request
...
for
...
an
...
object
...
will
...
be
...
revalidated,
...
starting
...
240
...
seconds
...
before
...
it
...
goes
...
stale.
...
For
...
objects
...
getting
...
few
...
requests
...
per
...
second,
...
this
...
would
...
likely
...
not
...
trigger,
...
but
...
then
...
this
...
feature
...
is
...
not
...
necessary
...
anyways
...
since
...
odds
...
are
...
only
...
1
...
or
...
a
...
small
...
number
...
of
...
connections
...
would
...
hit
...
origin
...
upon
...
objects
...
going
...
stale.
...
The
...
defaults
...
are
...
a
...
good
...
compromise,
...
for
...
objects
...
getting
...
roughly
...
4
...
requests
...
/
...
second
...
or
...
more,
...
it's
...
guaranteed
...
to
...
trigger
...
a
...
revalidate
...
event
...
within
...
the
...
240s.
...
These
...
configs
...
are
...
also
...
overridable
...
per
...
remap
...
rule
...
or
...
via
...
a
...
plugin,
...
so
...
can
...
be
...
adjusted
...
per
...
request
...
if
...
necessary.
...
Finally,
...
the
...
fuzz.min_time
...
is
...
there
...
to
...
be
...
able
...
to
...
handle
...
requests
...
with
...
a
...
TTL
...
less
...
than
...
fuzz.time
...
;
...
In
...
this
...
case,
...
the
...
revalidation
...
happens
...
in
...
a
...
function
...
between
...
fuzz.min_time
...
and
...
fuzz.time
...
,
...
and
...
it
...
gets
...
closer
...
to
...
expiring,
...
becoming
...
more
...
likely.
...
By
...
default
...
this
...
setting
...
is
...
not
...
enabled,
...
but
...
should
...
be
...
enabled
...
anytime
...
you
...
have
...
objects
...
with
...
small
...
TTLs.
...
Note
...
that
...
this
...
option
...
predates
...
overridable
...
configurations,
...
so
...
you
...
can
...
achieve
...
something
...
similar
...
with
...
a
...
plugin
...
or
...
remap.config
...
conf_remap.so
...
configs.
...
These
...
configurations
...
are
...
similar
...
to
...
Squid's
...
refresh_stale_hit
...
configuration
...
option.
...
Read
...
While
...
Write
...
This
...
feature
...
is
...
pretty
...
similar
...
to
...
how
...
Squid
...
actually
...
implements
...
Collapsed
...
Forwarding.
...
When
...
ATS
...
goes
...
to
...
fetch
...
something
...
from
...
cache,
...
and
...
upon
...
receiving
...
the
...
response
...
from
...
origin,
...
any
...
number
...
of
...
clients
...
can
...
be
...
allowed
...
to
...
start
...
serving
...
out
...
of
...
the
...
partially
...
filled
...
cache
...
object.
...
The
...
difference
...
(Ed:
...
I
...
think?)
...
is
...
that
...
Squid
...
allows
...
this
...
as
...
soon
...
as
...
we
...
go
...
to
...
origin,
...
whereas
...
ATS
...
can
...
not
...
do
...
it
...
until
...
we
...
get
...
the
...
response
...
header.
...
The
...
reason
...
for
...
this
...
is
...
that
...
we
...
make
...
no
...
distinction
...
between
...
cache
...
refresh,
...
and
...
cold
...
cache,
...
so
...
we
...
have
...
no
...
way
...
to
...
know
...
if
...
a
...
response
...
is
...
going
...
to
...
be
...
cacheable,
...
and
...
therefore
...
allow
...
read-while-write
...
functionality.
...
The
...
configurations
...
necessary
...
to
...
enable
...
this
...
in
...
ATS
...
are:
| Code Block |
|---|
} CONFIG proxy.config.cache.enable_read_while_writer INT 1 CONFIG proxy.config.http.background_fill_active_timeout INT 0 CONFIG proxy.config.http.background_fill_completed_threshold FLOAT 0.000000 CONFIG proxy.config.cache.max_doc_size INT 0 {code} All for configurations are |
All four configurations are required,
...
for
...
the
...
following
...
reasons:
...
- Obviously
...
- the
...
- feature
...
- should
...
- be
...
- enabled
...
- to
...
- begin
...
- with.
...
- It's
...
- off
...
- by
...
- default
...
- (as
...
- of
...
- 4.0.x,
...
- we
...
- might
...
- want
...
- to
...
- modify
...
- this
...
- for
...
- 5.x)
...
- The
...
- background
...
- fill
...
- option
...
- must
...
- be
...
- enabled,
...
- and
...
- allowed
...
- to
...
- kick
...
- in
...
- for
...
- every
...
- request.
...
- This
...
- is
...
- necessary,
...
- in
...
- case
...
- the
...
- owning
...
- consumer
...
- ("client
...
- session")
...
- goes
...
- away,
...
- someone
...
- needs
...
- to
...
- take
...
- over
...
- the
...
- session.
...
- The
...
- max_doc_size
...
- must
...
- be
...
- unlimited,
...
- since
...
- we
...
- potentially
...
- don't
...
- know
...
- the
...
- size
...
- of
...
- the
...
- object,
...
- we
...
- can't
...
- allow
...
- it
...
- to
...
- disconnect
...
- due
...
- to
...
- going
...
- over
...
- this
...
- limit.
...
Once
...
all
...
this
...
enabled,
...
you
...
have
...
something
...
that
...
is
...
very
...
close,
...
but
...
not
...
quite
...
the
...
same,
...
as
...
Squid's
...
Collapsed
...
Forwarding.
...
Open
...
Retry
...
Timeout
The third configuration option, which is really only suitable when you know everything is cacheable, are related how to deal with locking out cache access until the origin response has come back. Combined with read-while-write this would completely mimic the behavior of Squid's Collapsed Forwarding. The downside is that non-cacheable content would also be serialized, which can severely affect latency. Since ATS now supports setting these settings per-request or remap rule, you can configure this to be suitable for your setup much more easily.
The configurations are (with defaults):
| Code Block |
|---|
CONFIG proxy.config.http.cache.max_open_read_retries INT -1
CONFIG proxy.config.http.cache.open_read_retry_time INT 10
|
The default means that the feature is disabled, and every connection is allowed to go to origin instantly. When enabled, you will try max_open_read_retries times, each with a open_read_retry_time timeout. Lets say you set these to 5 and 10 respectively, connections will wait up to 50ms for a response to come back from origin from a previous request, until this request is allowed through.
I don't know if Squid has similar features, it seems to have a 30,000ms timeout, which is not tunable. It also seems incredibly high.
Stale While Revalidate plugin
There is a plugin that implements Stale While Revalidate, which is documented in RFC 5861. With this plugin, you are allowed to serve objects stale in cache, while one request goes to origin to fetch the new version. Combined with read-while-write, and the fuzzy logic feature, this is a good alternative to the Open-Retry feature. The downside is that the current implementation is not complete, and needs changes to the core / cache to solve the problem completely.