Page History

...

mark_out and tos_out are overridable but mark_in and tos_in are not. In addition to configuring them globally in records.config you can override them per transaction with the conf_remap plugin, the set-config header_rewrite operator, or TSHttpTxnConfigIntSet() *however* they are only consulted when the socket is created which happens before SEND_REQUEST_HDR_HOOK so you must set them in READ_REQUEST_HDR_HOOK or at some other earlier time. See Connection::open(). It might be possible to update an existing socket when you change the configuration variables by implementing a callback.

API Functions

ClientPacketMarkSet() and ClientPacketTosSet() mark traffic destined for the client and ServerPacketMarkSet() and ServerPacketTosSet() mark traffic destined for the origin. Sometimes you can mark traffic sent *from* the origin with the Netfilter CONNMARK target. ClientPacketMarkSet() and ServerPacketMarkSet() set the packet mark and ClientPacketTosSet() and ServerPacketTosSet() set the ToS/DiffServ Field. API functions for the connection mark and PCP Field haven't been implemented but you can call TSHttpSsnClientFdGet() or TSHttpTxnClientFdGet() to get the client socket and call setsockopt() to set the option yourself.

...

Nothing related to the PCP Field has been added to Traffic Server but you can call TSHttpSsnClientFdGet() or TSHttpTxnClientFdGet() to get the client socket and call setsockopt() to set the option yourself. There's a feature request (TS-3037) to implement the PCP Field in Traffic Server.

...

Use the tsxs utility to compile it:

Code Block

language	bash


  $ tsxs -o priority.so priority.cc

Add lines like the following to remap.config to configure it:

Code Block

language	bash
title	remap.config


  map <a class="external-link" href="http://example.com" rel="nofollow">http://example.com</a> <a class="external-link" href="http://example.com" rel="nofollow">http://example.com</a> @plugin=priority.so @pparam=4
  regex_map http://.*\.example.com http://$0 @plugin=priority.so @pparam=4

...

You can copy marks from the ToS/DiffServ or PCP fields to the connection mark with the Netfilter CONNMARK target, to mark traffic sent *from* the origin, even if Traffic Server is on a separate device. In the following example, if you set the ToS/DiffServ Field (0x0c, 0x1c, etc.) on traffic sent to the origin, it will add both the origin request and response to the same iproute2 class (2:1, 2:3, etc.):

Code Block

language	bash


  iptables -t mangle -A POSTROUTING -m tos --tos 0x0c -j CONNMARK --set-mark 1
  iptables -t mangle -A POSTROUTING -m connmark --mark 1 -j CLASSIFY --set-class 2:1
  iptables -t mangle -A POSTROUTING -m tos --tos 0x1c -j CONNMARK --set-mark 2
  iptables -t mangle -A POSTROUTING -m connmark --mark 2 -j CLASSIFY --set-class 2:3

...

Use the tsxs utility to compile the plugin:

Code Block

language	bash


  $ tsxs -o tos.so tos.cc

If the content was already cached (cache hit) then the plugin sets the ToS/DiffServ Field on the client response to 0x0c. Edit the source code to change the value or implement a configuration variable to set it. You could adapt it to mark other cache lookup statuses (see TSHttpTxnCacheLookupStatusGet()) or instead set the packet mark, connection mark, or PCP Field.

...

This is handy for communicating more detail to iproute2 etc. For example here's how to divide upstream bandwidth equally among all clients with iproute2 and SFQ:

Code Block

language	bash
title	records.config


  # The source of origin requests and destination of origin responses is
  # the address of the client
  CONFIG proxy.config.http.server_ports STRING 8080:tr-out

Code Block

language	bash


  # Remember if traffic originated from our internet connection
  iptables -t mangle -A PREROUTING -i eth0.2 -j MARK --set-mark 1/1

  ifconfig ifb0 up

  # A qdisc is required before we can add a filter
  insmod sch_prio
  tc qdisc add dev br-lan root handle 1 prio

  # Shape only traffic originating from our internet connection
  # (packet mark 1/1)
  insmod cls_u32
  insmod act_mirred
  tc filter add dev br-lan parent 1: protocol ip pref 1 u32 match mark 1 1 flowid 1:1 action mirred egress redirect dev ifb0

  # Don&apos;t shape traffic (reorder/delay/drop) while there&apos;s available
  # capacity.  Unfortunately available capacity must be manually
  # configured and fine-tuned.  The following assumes isolated
  # up/downstream capacity (full-duplex).
  insmod sch_tbf
  tc qdisc add dev eth0.2 root handle 1 tbf rate .5mbit burst 5k latency 70ms
  tc qdisc add dev ifb0 root handle 1 tbf rate 2.5mbit burst 5k latency 70ms

  # Schedule an equal amount of traffic for each client
  insmod sch_sfq
  tc qdisc add dev eth0.2 parent 1: handle 2 sfq
  tc qdisc add dev ifb0 parent 1: handle 2 sfq

  # Divide downstream traffic into clients by destination IP address.
  # Divide upstream traffic into clients by *Netfilter connection
  # tracking* source IP address (after NAT all upstream traffic shares the
  # same source IP address).
  insmod cls_flow
  tc filter add dev eth0.2 parent 2: pref 1 handle 1 flow hash keys nfct-src divisor 1024
  tc filter add dev ifb0 parent 2: protocol ip pref 1 handle 1 flow hash keys dst divisor 1024

...

If you need to distinguish one transaction from another you're required to use configuration variables for origin traffic but header_rewrite operators or API functions for client traffic. I think this asymmetry unnecessarily burdens the administrator with implementation details. It would be more consistent and more user friendly to just implement configuration variables for each socket option and make them work in all scenarios (origin and client traffic, records.config, the conf_remap plugin, the set-config header_rewrite operator, and TSHttpTxnConfigIntSet()). It might be possible to update an existing socket when you change the configuration variables by implementing a callback.

...

Be careful of ICMP redirects, they can sometimes cause clients to route non-web traffic to the proxy.

Code Block

language	bash
title	records.config


  # The source of client responses and destination of client requests is
  # the address of the origin.  The source of origin requests and
  # destination of origin responses is the address of the client.
  CONFIG proxy.config.http.server_ports STRING 8080:tr-full

Code Block

language	bash
title	remap.config


  # Give high priority to Wikipedia, low priority to YouTube
  map <a class="external-link" href="http://wikipedia.org" rel="nofollow">http://wikipedia.org</a> <a class="external-link" href="http://wikipedia.org" rel="nofollow">http://wikipedia.org</a> @plugin=conf_remap.so @pparam=proxy.config.net.sock_packet_tos_out=0x0c
  regex_map http://.*\.wikipedia\.org http://$0 @plugin=conf_remap.so @pparam=proxy.config.net.sock_packet_tos_out=0x0c
  map <a class="external-link" href="http://youtube.org" rel="nofollow">http://youtube.org</a> <a class="external-link" href="http://youtube.org" rel="nofollow">http://youtube.org</a> @plugin=conf_remap.so @pparam=proxy.config.net.sock_packet_tos_out=0x1c
  regex_map http://.*\.youtube\.org http://$0 @plugin=conf_remap.so @pparam=proxy.config.net.sock_packet_tos_out=0x1c

Code Block

language	bash


  # Remember if traffic originated from our internet connection
  iptables -t mangle -A PREROUTING -i eth0.2 -j MARK --set-mark 1/1

  # Route web traffic to the proxy server except traffic already
  # originating from it.  Matching web traffic by port number isn&apos;t
  # perfect but it&apos;s good enough.  This is the MAC address of the proxy
  # server.  Because it&apos;s configured to make origin connections
  # transparent this is the only way to match traffic already originating
  # from it:
  # <a class="external-link" href="http://thread.gmane.org/gmane.comp.security.firewalls.netfilter.general/45405" rel="nofollow">http://thread.gmane.org/gmane.comp.security.firewalls.netfilter.general/45405</a>
  iptables -t mangle -A PREROUTING -m mac --mac-source 00:22:15:d2:1e:61 -j RETURN
  iptables -t mangle -A PREROUTING -p tcp --dport 80 -j MARK --set-mark 2/2
  iptables -t mangle -A PREROUTING -i eth0.2 -p tcp --sport 80 -j MARK --set-mark 2/2

  # Web traffic is medium priority by default but the proxy server further
  # breaks down some high/low priority traffic.  It communicates this by
  # setting the ToS/DiffServ Field (it uses the pool of codepoints reserved
  # for experimental or local use, 0x0c/0x0c).  Mark the connection to
  # remember the priority and apply the same classification to response
  # traffic (on which the ToS/DiffServ Field is not set).
  iptables -t mangle -A POSTROUTING -m tos --tos 0x0c -j CONNMARK --set-mark 1
  iptables -t mangle -A POSTROUTING -m connmark --mark 1 -j CLASSIFY --set-class 2:1
  iptables -t mangle -A POSTROUTING -m tos --tos 0x1c -j CONNMARK --set-mark 2
  iptables -t mangle -A POSTROUTING -m connmark --mark 2 -j CLASSIFY --set-class 2:3

  # Route web traffic to the proxy server
  ip route add table 1 via 192.168.1.2
  ip rule add fwmark 2/2 table 1

  ifconfig ifb0 up

  # A qdisc is required before we can add a filter
  insmod sch_prio
  tc qdisc add dev br-lan root handle 1 prio

  # Shape only traffic originating from our internet connection
  # (packet mark 1/1)
  insmod cls_u32
  insmod act_mirred
  tc filter add dev br-lan parent 1: protocol ip pref 1 u32 match mark 1 1 flowid 1:1 action mirred egress redirect dev ifb0

  # Don&apos;t shape traffic (reorder/delay/drop) while there&apos;s available
  # capacity.  Unfortunately available capacity must be manually
  # configured and fine-tuned.  The following assumes isolated
  # up/downstream capacity (full-duplex).
  insmod sch_tbf
  tc qdisc add dev eth0.2 root handle 1 tbf rate .5mbit burst 5k latency 70ms
  tc qdisc add dev ifb0 root handle 1 tbf rate 2.5mbit burst 5k latency 70ms

  # Schedule traffic according to three priorities
  tc qdisc add dev eth0.2 parent 1: handle 2 prio
  tc qdisc add dev ifb0 parent 1: handle 2 prio

  # For each priority schedule an equal amount of traffic for each client
  insmod sch_sfq
  tc qdisc add dev eth0.2 parent 2:1 handle 3 sfq
  tc qdisc add dev ifb0 parent 2:1 handle 3 sfq
  tc qdisc add dev eth0.2 parent 2:2 handle 4 sfq
  tc qdisc add dev ifb0 parent 2:2 handle 4 sfq
  tc qdisc add dev eth0.2 parent 2:3 handle 5 sfq
  tc qdisc add dev ifb0 parent 2:3 handle 5 sfq

  # Divide downstream traffic into clients by destination IP address.
  # Divide upstream traffic into clients by *Netfilter connection
  # tracking* source IP address (after NAT all upstream traffic shares the
  # same source IP address).
  insmod cls_flow
  tc filter add dev eth0.2 parent 3: pref 1 handle 1 flow hash keys nfct-src divisor 1024
  tc filter add dev ifb0 parent 3: protocol ip pref 1 handle 1 flow hash keys dst divisor 1024
  tc filter add dev eth0.2 parent 4: pref 1 handle 1 flow hash keys nfct-src divisor 1024
  tc filter add dev ifb0 parent 4: protocol ip pref 1 handle 1 flow hash keys dst divisor 1024
  tc filter add dev eth0.2 parent 5: pref 1 handle 1 flow hash keys nfct-src divisor 1024
  tc filter add dev ifb0 parent 5: protocol ip pref 1 handle 1 flow hash keys dst divisor 1024

...

Space shortcuts

Page tree

Versions Compared

Old Version 3

New Version 4

Key

API Functions