Page History

...

It implements TS_HTTP_SEND_RESPONSE_HDR_HOOK to check and potentially rewrite the " Location : ..." and " Digest : SHA-256=..." headers after responses are cached. It doesn't do it before they're cached because the contents of the cache can change after responses are cached. It uses TSCacheRead() to check if the URL in the " Location : ..." header is already cached. In future, the plugin should also check if the URL is fresh or not.

It implements TS_HTTP_READ_RESPONSE_HDR_HOOK and a null transformtransformation to compute the SHA-256 digest for content as it's added to the cache. It uses SHA256_Init(), SHA256_Update(), and SHA256_Final() from OpenSSL to compute the digest, then it uses TSCacheWrite() to associate the digest with the request URL. This adds a new cache object where the key is the digest and the object is the request URL.

...

The "Digest: SHA-256=..." header is also more efficient than "Link: <...>; rel=duplicate" headers because it involves a constant number of cache lookups. RFC 6249 requires a "Digest: SHA-256=..." header or "Link: <...>; rel=duplicate" headers MUST be ignored:

If Instance Digests are not provided by the Metalink servers, the Link header fields pertaining to this specification MUST be ignored.

Metalinks contain whole file hashes as described in Section 6, and MUST include SHA-256, as specified in [FIPS-180-3].

Alex Rousskov pointed out a project for Squid to implement Duplicate Transfer Detection:

...