Page History

This plugin exploits RFC 6429, Metalink/HTTP: Mirrors and Hashes, and RFC 3230, Instance Digests in HTTP, to detect duplicate downloads from different mirrors and redirect clients to mirrors that are already cached.h2

Problem

Many download sites or content distribution networks distribute the same files from different mirrors and users do not know which mirrors are already cached. These sites often present users with a simple download button that redirects them to a mirror, but it does not predictably redirect them to the same mirror, or to a mirror that is already cached, so users cannot predict whether a download will take seconds or hours, which is frustrating.h2

Behavior

Given a response with a "Location: ..." header and a "Digest: SHA-256=..." header, such as from MirrorBrain, if the URL in the "Location: ..." header is not already cached but the cache already contains content that matches the digest in the "Digest: SHA-256=..." header, then this plugin will rewrite the "Location: ..." header with the cached URL. This should redirect clients to mirrors that are already cached.h2

Install

The code is up on GitHub. You can either download a zip archive or download the code with Git:

  $ git clone https://github.com/jablko/dedup/zipball/master

...

Follow the instructions in the Programmers Guide to compile the plugin:

  $ tsxs -I proxy/api -C metalink.cc -o metalink.so

Finally add "metalink.so" to the Traffic Server plugin.config file to load the plugin