Versions Compared

Old Version 14

changes.mady.by.user theseusyang

Saved on

compared with

New Version 15

changes.mady.by.user theseusyang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

注意: OS TCP buffer size 限制必须足够大来调节  Geode Socket Buffer  的设置. 如果不是, Geode 的值将设置到 OS 限制 – 而不是请求的值.

  • TCP/IP Keep Alive

Geode supports 支持 TCP KeepAlive to prevent socket connections from being timed out来防止 Socket 连接超时.

The gemfire.enableTcpKeepAlive system property prevents connections that appear idle from being timed out (for example, by a firewall.) When configured to true, Geode enables the SO_KEEPALIVE option for individual sockets. This operating system-level setting allows the socket to send verification checks (ACK requests) to remote systems in order to determine whether or not to keep the socket connection alive.

Note: The time intervals for sending the first ACK KeepAlive request, the subsequent ACK requests and the number of requests to send before closing the socket is configured on the operating system level. See

系统属性放置空闲的连接发生超时 (例如, 通过一个防火墙.) 当配置设置为 true 时, 对于每个 Sockets 来说, Geode 启用了 SO_KEEPALIVE 选项. OS级别的设置允许 Socket 发送验证检查 (ACK requests) 到远程系统为了确定是否保持socket 连接是活跃的.

注意: 对于发送第一个 ACK KeepAlive 请求的时间间隔, 后续的 ACK 请求和请求数量 在关闭 Socket 之前进行发送.

默认情况下, 此系统属性被设置为 By default, this system property is set to true.

  • TCP/IP Peer-to-Peer Handshake Timeouts

This property governs the amount of time a peer will wait to complete the TCP/IP handshake process. You can change the connection handshake timeouts for TCP/IP connections with the system property p2p.handshakeTimeoutMs.

  • 握手超时

此属性管理时间量,  一个 Peer 将等待完成 TCP/IP 握手处理. 你能够更改连接握手超时时间, 带有系统属性p2p.handshakeTimeoutMs 的 TCP/IP 连接.

默认设置是 The default setting is 59,000 milliseconds (59 seconds).

This sets the handshake timeout to 75,000 milliseconds for a Java application对于一个 Java 应用来说, 此设置握手超时到  75,000 毫秒:

-Dp2p.handshakeTimeoutMs=7500075000

此属性可以通过命令行来设置The properties are passed to the cache server on the gfsh command line:

 gfsh>start server --name=server1 --J=-Dp2p.handshakeTimeoutMs=75000
Linux TCP/IP通信设置

The following table summarizes the recommended 如下的表总结了推荐的 TCP/IP settings for Linux. These settings are in the 设置. 这些设置位于 /etc/sysctl.conf file文件

Setting设置Recommended Value推荐值Rationale基本原理
net.core.netdev_max_backlog30000Set maximum number of packets, queued on the INPUT side, when the interface receives packets faster than kernel can process them. Recommended setting is for 10GbE links. For 1GbE links use 8000.
net.core.wmem_max67108864Set max to 16MB (16777216) for 1GbE links and 64MB (67108864) for 10GbE links.
net.core.rmem_max67108864Set max to 16MB (16777216) for 1GbE links and 64MB (67108864) for 10GbE links.
net.ipv4.tcp_congestion_controlhtcpThere seem to be bugs in both bic and cubic (the default) for a number of versions of the Linux kernel up to version 2.6.33. The kernel version for Redhat 5.x is 2.6.18-x and 2.6.32-x for Redhat 6.x
net.ipv4.tcp_congestion_window10This is the default for Linux operating systems based on Linux kernel 2.6.39 or later.
net.ipv4.tcp_fin_timeout10This setting determines the time that must elapse before TCP/IP can release a closed connection and reuse its resources. During this TIME_WAIT state, reopening the connection to the client costs less than establishing a new connection. By reducing the value of this entry, TCP/IP can release closed connections faster, making more resources available for new connections. The default value is 60. The recommened setting lowers its to 10. You can lower this even further, but too low, and you can run into socket close errors in networks with lots of jitter.
net.ipv4.tcp_keepalive_interval30This determines the wait time between isAlive interval probes. Default value is 75. Recommended value reduces this in keeping with the reduction of the overall keepalive time.
net.ipv4.tcp_keepalive_probes5How many keepalive probes to send out before the socket is timed out. Default value is 9. Recommended value reduces this to 5 so that retry attempts will take 2.5 minutes.
net.ipv4.tcp_keepalive_time600Set the TCP Socket timeout value to 10 minutes instead of 2 hour default. With an idle socket, the system will wait tcp_keepalive_time seconds, and after that try tcp_keepalive_probes times to send a TCP KEEPALIVE in intervals of tcp_keepalive_intvl seconds. If the retry attempts fail, the socket times out.
net.ipv4.tcp_low_latency1Configure TCP for low latency, favoring low latency over throughput
net.ipv4.tcp_max_orphans16384Limit number of orphans, each orphan can eat up to 16M (max wmem) of unswappable memory
net.ipv4.tcp_max_tw_buckets1440000Maximal number of timewait sockets held by system simultaneously. If this number is exceeded time-wait socket is immediately destroyed and warning is printed. This limit exists to help prevent simple DoS attacks.
net.ipv4.tcp_no_metrics_save1Disable caching TCP metrics on connection close
net.ipv4.tcp_orphan_retries0Limit number of orphans, each orphan can eat up to 16M (max wmem) of unswappable memory
net.ipv4.tcp_rfc13371Enable a fix for RFC1337 - time-wait assassination hazards in TCP
net.ipv4.tcp_rmem10240 131072 33554432Setting is min/default/max. Recommed increasing the Linux autotuning TCP buffer limit to 32MB
net.ipv4.tcp_wmem10240 131072 33554432Setting is min/default/max. Recommed increasing the Linux autotuning TCP buffer limit to 32MB
net.ipv4.tcp_sack1Enable select acknowledgments启用 选择确认
net.ipv4.tcp_slow_start_after_idle0By default, TCP starts with a single small segment, gradually increasing it by one each time. This results in unnecessary slowness that impacts the start of every request.
net.ipv4.tcp_syncookies0Many default Linux installations use SYN cookies to protect the system against malicious attacks that flood TCP SYN packets. The use of SYN cookies dramatically reduces network bandwidth, and can be triggered by a running Geode cluster. If your Geode cluster is otherwise protected against such attacks, disable SYN cookies to ensure that Geode network throughput is not affected. 
NOTE: if SYN floods are an issue and SYN cookies can’t be disabled, try the following: 
net.ipv4.tcp_max_syn_backlog="16384"
net.ipv4.tcp_synack_retries="1" 
net.ipv4.tcp_max_orphans="400000"
net.ipv4.tcp_timestamps1Enable timestamps as defined in RFC1323:
net.ipv4.tcp_tw_recycle1This enables fast recycling of TIME_WAIT sockets. The default value is 0 (disabled). Should be used with caution with load balancers.
net.ipv4.tcp_tw_reuse1This allows reusing sockets in TIME_WAIT state for new connections when it is safe from protocol viewpoint. Default value is 0 (disabled). It is generally a safer alternative to tcp_tw_recycle. The tcp_tw_reuse setting is particularly useful in environments where numerous short connections are open and left in TIME_WAIT state, such as web servers and loadbalancers.
net.ipv4.tcp_window_scaling1Turn on window scaling which can be an option to enlarge the transfer window:

In addition, increasing the size of transmit queue can also help TCP throughput. Add the following command to 另外, 增加传输队列的大小也能够帮助提升 TCP 吞吐量. 添加如下的命令到 /etc/rc.local to accomplish this来完成.

/sbin/ifconfig eth0 txqueuelen 10000

...