These are the highlights of new features or additions to existing features in v5.0.x. For the complete list of all changes see this JIRA Release Notes.

New records.config layout

We have cleaned up the default records.config file, reducing it's size to roughly 1/3. Now only the very commonly used settings are included. In addition, we've also modified the default values for a number of settings, for more details on that, see the Upgrade to 5.0.x page.

New configurations

  • proxy.config.http.cache.range.write: The old behavior (before v5.0.0) was to always lock the object in cache on a Range, even it is unlikely that the object fetched can be cached. The new behavior is to not attempt such a lock, unless this new configuration is enabled. The use case for this is if your origin ignores the Range: header, and responds with the full (complete) object anyways.

Changed default configurations

Removed configurations

  • proxy.config.http.cache.when_to_add_no_cache_to_msie_requests
  • proxy.config.log.xuid_logging_enabled

Jira: TS-2274, TS-2319

Experimental SPDY Support

There is now SPDY 3.1 support for ATS, which requires the spdylay library to build. This is a compile time option and will need to be enabled when the server is built from source code. 

Jira: TS-2431

TLS / SSL and HTTPS improvements

ALPN support

We now support ALPN in addition to NPN, for TLS based protocol negotiation. This is a prerequisite for HTTP/2 support going forward.

Jira: TS-2239

Passphrase protected TLS certificates

We now provide a mechanism for providing pass phrases for unlocking the private key of a TLS certificate. From the docs:

ssl_key_pass_dialog=[builtin|exec:/path/to/program]

  Method used to provide a pass phrase for encrypted private keys.
  Two options are supported: builtin and exec
  builtin - Requests passphrase via stdin/stdout. Useful for debugging.
  exec: - Executes a program and uses the stdout output for the pass
  phrase.

Example:

  ssl_cert_name=foo.pem ssl_key_pass_dialog="exec:/usr/bin/mypass foo"

Jira: TS-612

TLS session metrics

A whole slew of new metrics are made available for TLS sessions, such as how often keys are reused, which ciphers are used etc.

Jira: TS-2169

Docs: traffic_line

LuaJit and Lua

LuaJit is now included in the ATS source. A new LuaJit based plugin is also available, albeit still experimental and it will change, the APIs are not frozen. Going forward, the intent is to include more and more Lua features in the core, including for configurations.

Jira: TS-2711

Logging

New action: WIPE_FIELD_VALUE

For custom log formats, a new action has been added, named WIPE_FIELD_VALUE. With this action, you can scrub field values before they get logged, which can be useful for hiding sensitive information such as username or passwords.

Jira: TS-2636

New log tag: %<pitag> and %<piid>

With new APIs (see below) it is now possible for a plugin to tag an internal request with a plugin tag and ID. This can thus be used in the logs, for further correlation of requests and events.

Jira: TS-2838

New and improved plugins 

A number of new plugins and feature additions to existing plugins are available in this version.

Promoted to stable (and renamed): tcpinfo (was: tcp_info)

This plugin produces information pertaining to every TCP session. This is now considered a stable plugin, and has gone through a few iterations of improvements.

Jira: TS-2713, TS-2714

Docs: tcpinfo

Renamed: stale_while_revalidate (was rfc5861)

This plugin now has a much better name.

Jira: TS-2737

Improved: regex_remap

Several new options are now available, using the @ formats:

  • @caseless: Make the regular expressions case insensitive.
  • @lowercase_subtitutions: Make all regular expression substitutions be lower cased.
  • @proxy.config....=<value>: Modify an overridable records.config configuration.

Jira: TS-2646

Docs:  regex_remap

Improved: header_rewrite

A fairly significant number of improvements were made to this plugin.

  • Make the destination rewrites generically available (not just for remap rules)
  • The hooks management are improved such that you can now trigger rules from a remap.config rule, but a different hook. This is useful for example when you want to send a per-remap specific header to the origin server.
  • Conditions on internal requests
  • Conditions on the Client IP

Jira: TS-2631, TS-2834, TS-2522

Docs: header_rewrite

stats_over_http

The metrics from this plugin will now include all metrics, not just the process metrics.

Jira:  TS-2620

Docs: stats_over_http

New: background_fetch

This is a new, experimental, plugin to perform background fetches of full objects when clients sends Range: requests only. Since we (currently) do not cache partial objects in the cache, this is a way to allow such objects to make it into cache. Together with the read-while-writer feature, this can improve cache hit ratio and performance dramatically. 

Jira: TS-2554

Docs: background_fetch

New: collapsed_connection

This plugin allows for finer granular control over origin server connections, collapsing client requests into one server session. This is similar to read-while-writer.

Jira: TS-2604

New: s3_auth

The Amazon S3 services has an optional authentication component. This plugin allows ATS to make origin requests to S3 using the AWS authentication scheme. Currently this only implements the v2 specification of the APIs.

Jira: TS-2611

Docs:  s3_auth

New: url_sig

This plugin validates cryptographically signed URLs.

Jira: TS-2732

New: regex_revalidate

This plugin allow you to use regular expressions to force revalidation of content.

Jira: TS-2804

traffic_shell removed

The command line tool traffic_shell has been removed.  There was little interest in maintaining this command line tool. For those who still need this functionality, an intermediary perl script is available in the code, traffic_shell.pl.

Jira: TS-1665

traffic_line improvements

The traffic_line command line has a new option, '-m', which allows you to look for statistics and configurations matching a certain regular expression. This is a very convenient and quick way to explore the metrics and configurations system, and is a great introspection tool. In addition, better feedback is done when modifying configurations via command line.

Jira: TS-2637, TS-2628

Docs: traffic_line

Modified and deprecated Plugin APIs (incompatible changes from 4.2.x)

Change TSRecordType enum values to powers of two

This API is now basically a bit map, allowing for OR'ing bits together. In most cases, recompiling your plugins should suffice, but they are not ABI compatible with versions prior to 5.0.x.

Jira: TS-2088

Eliminate TSHttpTxnSetHttpRetBody()

This reduces memory requirements for the HTTP state by 25%. Instead, use the less confusing TSHttpTxnErrorBodySet() API.

Jira: TS-2657

Docs: TSHttpTxnErrorBodySet()

Deprecation and changed memory ownership for TSRedirectUrlSet() and TSRedirectUrlGet()

Before, the  TSRedirectUrlSet() would make an implicit strdup() of the incoming string, assuming ownership. With these changes, the plugin retains ownership of the string, and is responsible for freeing it (if necessary). This can save significant memory allocations in some cases. Instead of these APIs, use TSHttpTxnRedirectUrlSet() and TSHttpTxnRedirectUrlGet().

Jira: TS-2693

New Plugin APIs

The following sections discuss changes and additions to public plugin APIs.

Name lookups of internal state values

These are primarily useful together with debugging, and allows to lookup a textual representation of internal state values. The following APIs were added:

tsapi const char* TSHttpServerStateNameLookup(TSServerState state);
tsapi const char* TSHttpHookNameLookup(TSHttpHookID hook);
tsapi const char* TSHttpEventNameLookup(TSEvent event);

Jira: TS-2598

Docs: TSDebug()

HTTP Connect with Plugin tag and ID

This new API goes together with the new logging tags, %<pitag> and %<piid> (see above under logging). It is a super set of the existing TSHttpConnect() API, with the addition of a plugin tag and a ID number.

tsapi TSVConn TSHttpConnectWithPluginId(struct sockaddr const* addr, char const* tag, int64_t id);

 

Jira: TS-2838

Docs: TSHttpConnectWithPluginId()

API for detecting if a Session is internal or not

This is a corollary to the existing TSHttpIsInternalRequest() API.

 TSReturnCode TSHttpIsInternalSession(TSHttpSsn ssnp);

Jira: TS-2855

Docs: TSHttpIsInternalSession()

Determine if a transaction would be cacheable

This new API can look at a particular request / response header, and apply the internal logic (based on current configurations) to decide if the request would be cacheable. This is usable for plugins that wishes to fetch things proactively into the cache.

 int TSHttpTxnIsCacheable(TSHttpTxn txnp, TSMBuffer request, TSMBuffer response);

Jira:  TS-1622

Get the SSL object of a session

The following new API allows a plugin to get access to the underlying  TLS (OpenSSL) object for a TLS session.

 void* TSHttpSsnSSLConnectionGet(TSHttpSsn ssnp);

Jira: TS-2210

Set/Get() a redirect URL for ATS to follow

These two APIs replaces the old, deprecated APIs for setting the redirect URL.

 void TSHttpTxnRedirectUrlSet(TSHttpTxn txnp, const char* url, const int url_len);
 const char* TSHttpTxnRedirectUrlGet(TSHttpTxn txnp, int* url_len_ptr);

Jira: TS-2707

Get current redirection retry count

This new API allows a plugin to request how many times a request has gone through redirects, and followed those. This is useful in plugins where you want to retry request to a different server, follow redirects etc. With this API, you can both short circuit (abort) the redirects, or apply different redirects based on the count. This API is typically used combined with the previous redirection APIs.

 int TSHttpTxnRedirectRetries(TSHttpTxn txnp); 

Jira: TS-2692

Create a TSVConn from a file descriptor

TSVConn TSVConnFdCreate(int fd);

This new API creates a TSVConn from a socket. This is helpful for integrating Traffic Server with external resources.

Jira: TS-2810

Docs: TSVConnFdCreate()

5.0.1 Security Update

Due to a security issue in 5.0.0, a possibly incompatible change was made to 5.0.1.

To monitor the state of the proxy, a nanny process (traffic_cop) periodically probes the proxy with a specific URL (http://127.0.0.1:8083/synthetic.txt). The security change was to do additional checks on the incoming heartbeat probe. This can cause Traffic Server to fail by preventing the health check probe from succeeding.

Symptoms

  • The traffic_server and/or traffic_manager process is restarted very frequently (more than once a minute)
  • There are error messages in the error.log file of the form

20140806.13h33m54s CONNECT: could not connect to 127.0.0.1 for 'http://127.0.0.1:8083/synthetic.txt' (setting last failure time)
20140806.13h33m54s RESPONSE: sent 192.168.56.196 status 502 (Server Hangup) for 'http://127.0.0.1:8083/synthetic.txt'

  • There are syslog messages of the form

traffic_cop[13730]: (http test) received non-200 status(502)
traffic_cop[13730]: server heartbeat failed [1]

Causes

Anything that modifies the heartbeat connection and request from the expected form. Known causes

  • iptables NAT masquerade.
    If connections on the loopback are affected this will break the heartbeat because the connection source address will be a host address, not the loopback address. To fix this, change your iptables rules to not perform NAT masquerade on the heartbeat connection. Two possible approaches are
    • Explicitly disable MASQUERADE on the loopback interface and addresses (make these the first rules in the NAT POSTROUTING list
      iptables --insert POSTROUTING --table nat -o lo -j RETURN
      iptables --append POSTROUTING --table nat -s 127.0.0.1 -j RETURN
      iptables --append POSTROUTING --table nat -d 127.0.0.1 -j RETURN
    • Apply MASQUERADE explicitly to the non-loopback interfaces. If you have eth0 and eth1 then use
      iptables --append --table nat -o eth0 -j MASQUERADE
      iptables --append --table nat -o eth1 -j MASQUERADE
  • Generic remap rules, such as "map / http://some.other.place.com"
    To fix this, either
    • add an additional rule to prevent the generic rule from triggering on heartbeat connections, such as map /synthetic.txt http://localhost:8083/synthetic.txt
    • apply the fix from Unable to render Jira issues macro, execution error. . TS-2934 (remove this when wiki is fixed and preceding link works)
  • No labels