Network tuning can reduce Transmission Control Protocol (TCP) stack delay by changing connection settings and can improve throughput by changing TCP buffers.<\/p>\n
A Windows system needs the least tuning while a Solaris system needs the most tuning. The following information pertains to each system specified, and might improve WebSphere\u00ae eXtreme Scale performance. You should tune according to your network and application load.<\/p>\n
HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Services\\<\/pre>\nTcpip\\Parameters<\/pre>\nMaxFreeTcbs = dword:00011940<\/pre>\nMaxHashTableSize = dword:00010000<\/pre>\nMaxUserPort = dword:0000fffe<\/pre>\nTcpTimedWaitDelay = dword:0000001e<\/pre>\nSolaris<\/h2>\n
ndd -set \/dev\/tcp tcp_time_wait_interval 60000<\/pre>\nfndd -set \/dev\/tcp tcp_keepalive_interval 15000<\/pre>\nndd -set \/dev\/tcp tcp_fin_wait_2_flush_interval 67500<\/pre>\nndd -set \/dev\/tcp tcp_conn_req_max_q 16384<\/pre>\nndd -set \/dev\/tcp tcp_conn_req_max_q0 16384<\/pre>\nndd -set \/dev\/tcp tcp_xmit_hiwat 400000<\/pre>\nndd -set \/dev\/tcp tcp_recv_hiwat 400000<\/pre>\nndd -set \/dev\/tcp tcp_cwnd_max 2097152<\/pre>\nndd -set \/dev\/tcp tcp_ip_abort_interval 20000<\/pre>\nndd -set \/dev\/tcp tcp_rexmit_interval_initial 4000<\/pre>\nndd -set \/dev\/tcp tcp_rexmit_interval_max 10000<\/pre>\nndd -set \/dev\/tcp tcp_rexmit_interval_min 3000<\/pre>\nndd -set \/dev\/tcp tcp_max_buf 4194304<\/pre>\nAIX\u00ae<\/h2>\n
\/usr\/sbin\/no -o tcp_sendspace=65536<\/pre>\n\/usr\/sbin\/no -o tcp_recvspace=65536<\/pre>\n\/usr\/sbin\/no -o udp_sendspace=65536<\/pre>\n\/usr\/sbin\/no -o udp_recvspace=65536<\/pre>\n\/usr\/sbin\/no -o somaxconn=10000<\/pre>\n\/usr\/sbin\/no -o tcp_nodelayack=1<\/pre>\n\/usr\/sbin\/no \u2013o tcp_keepinit=40<\/pre>\n\/usr\/sbin\/no \u2013o tcp_keepintvl=10<\/pre>\nLINUX<\/h2>\n
sysctl -w net.ipv4.tcp_timestamps=0<\/pre>\nsysctl -w net.ipv4.tcp_tw_reuse=1<\/pre>\nsysctl -w net.ipv4.tcp_tw_recycle=1<\/pre>\nsysctl -w net.ipv4.tcp_fin_timeout=30<\/pre>\nsysctl -w net.ipv4.tcp_keepalive_time=1800<\/pre>\nsysctl -w net.ipv4.tcp_rmem=\"4096 87380 8388608\"<\/pre>\nsysctl -w net.ipv4.tcp_wmem=\"4096 87380 8388608\"<\/pre>\nsysctl -w net.ipv4.tcp_max_syn_backlog=4096<\/pre>\nHP-UX<\/h2>\n
ndd -set \/dev\/tcp tcp_ip_abort_cinterval 20000<\/pre>\n<\/h1>\n
<\/h1>\n
Linux Tweaking<\/h1>\n
Raising netowrk limits for broadband under Linux
\n<\/em>2003.04.01 16:03 by Philip<\/a>
\nKeywords: TCP Window<\/a>, WAN<\/a>, TCP\/IP<\/a>, tweak<\/a>, Linux<\/a><\/p>\n<\/p>\n
The TCP\/IP parameters for tweaking a Linux-based machine for fast internet connections are located in \/proc\/sys\/net\/… (assuming 2.1+ kernel). This location is volatile, and changes are reset at reboot. There are a couple of methods for reapplying the changes at boot time, ilustrated below.<\/p>\n
Locating the TCP\/IP related parameters<\/strong><\/p>\n
All TCP\/IP tunning parameters are located under \/proc\/sys\/net\/… For example, here is a list of the most important tunning parameters, along with short description of their meaning:<\/p>\n
\/proc\/sys\/net\/core\/rmem_max – Maximum TCP Receive Window
\n\/proc\/sys\/net\/core\/wmem_max – Maximum TCP Send Window
\n\/proc\/sys\/net\/ipv4\/tcp_rmem – memory reserved for TCP receive buffers
\n\/proc\/sys\/net\/ipv4\/tcp_wmem – memory reserved for TCP send buffers
\n\/proc\/sys\/net\/ipv4\/tcp_timestamps – timestamps (RFC 1323) add 12 bytes to the TCP header…
\n\/proc\/sys\/net\/ipv4\/tcp_sack – tcp selective acknowledgements.
\n\/proc\/sys\/net\/ipv4\/tcp_window_scaling – support for large TCP Windows (RFC 1323). Needs to be set to 1 if the Max TCP Window is over 65535.<\/p>\nKeep in mind everything under \/proc is volatile, so any changes you make are lost after reboot.<\/p>\n
There are some additional internal memory buffers for the TCP Window, allocated for each connection:
\n\/proc\/sys\/net\/ipv4\/tcp_rmem – memory reserved for TCP rcv buffers (reserved memory per connection default)
\n\/proc\/sys\/net\/ipv4\/tcp_wmem – memory reserved for TCP snd buffers (reserved memory per connection default)<\/p>\nThe tcp_rmem and tcp_wmem contain arrays of three parameter values: the 3 numbers represent minimum, default and maximum memory values. Those 3 values are used to bound autotunning and balance memory usage<\/a> while under global memory stress.<\/p>\n
Applying TCP\/IP Parameters at System Boot<\/strong><\/p>\n
You can edit \/etc\/rc.local, or \/etc\/boot.local depending on your distribution so the parameters get automatically reapplied at boot time. The TCP\/IP parameters should be self-explanatory: we’re basically setting the TCP Window to 256960, disabling timestamps (to avoid 12 byte header overhead), enabling tcp window scaling, and selective acknowledgements:<\/p>\n
echo 256960 > \/proc\/sys\/net\/core\/rmem_default\r\necho 256960 > \/proc\/sys\/net\/core\/rmem_max\r\necho 256960 > \/proc\/sys\/net\/core\/wmem_default\r\necho 256960 > \/proc\/sys\/net\/core\/wmem_max\r\n\r\necho 0 > \/proc\/sys\/net\/ipv4\/tcp_timestamps \r\necho 1 > \/proc\/sys\/net\/ipv4\/tcp_sack \r\necho 1 > \/proc\/sys\/net\/ipv4\/tcp_window_scaling<\/pre>\nChange the values above as desired, depending on your internet connection and maximum bandwidth\/latency. There are other parameters you can change from the default if you’re confident in what you’re doing – just find the correct syntax of the values in \/proc\/sys\/net\/… and add a line<\/a> in the above code analogous to the others. To revert to the default parameters, you can just comment or delete the above code from \/etc\/rc.local and restart.<\/p>\n
Another method to reapply the values upon boot is to include the following in your \/etc\/sysctl.conf (adjust RWIN values as needed):<\/p>\n
net.core.rmem_default = 256960\r\nnet.core.rmem_max = 256960 \r\nnet.core.wmem_default = 256960\r\nnet.core.wmem_max = 256960 \r\n\r\nnet.ipv4.tcp_timestamps = 0 \r\nnet.ipv4.tcp_sack = 1 \r\nnet.ipv4.tcp_window_scaling = 1<\/pre>\nNotes:<\/em>
\nExecute <\/em>sysctl -p<\/strong> to make these new settings take effect.<\/em>
\nTo manually set the MTU value under Linux, use the command: <\/em>ifconfig eth0 mtu 1500<\/strong> (where 1500 is the desired MTU size)<\/em><\/em><\/p>\nChanging Current Values without rebooting<\/strong><\/p>\n
The current TCP\/IP parameters can be edited without the need for reboot in the following locations:<\/p>\n
\/proc\/sys\/net\/core\/<\/strong>
\n<\/strong>rmem_default = Default Receive Window
\nrmem_max = Maximum Receive Window
\nwmem_default = Default Send Window
\nwmem_max = Maximum Send Window<\/p>\n\/proc\/sys\/net\/ipv4\/<\/strong>
\n<\/strong>You’ll find timestamps, window scalling, selective acknowledgements, etc.<\/p>\nKeep in mind the values in \/proc will be reset upon reboot. You still need to add the code in \/etc\/rc.local or \/etc\/boot.local in order to have the changes applied at boot time as described above.<\/em><\/p>\n
Other TCP Parameters to consider<\/strong><\/p>\n
TCP_FIN_TIMEOUT<\/strong>
\n<\/strong>This setting determines the time that must elapse before TCP\/IP can release a closed connection and reuse its resources. During this TIME_WAIT state, reopening the connection to the client costs less than establishing a new connection. By reducing the value of this entry, TCP\/IP can release closed connections faster, making more resources available for new connections. Addjust this in the presense of many connections sitting in the TIME_WAIT state:<\/p>\n# echo 30 > \/proc\/sys\/net\/ipv4\/tcp_fin_timeout<\/strong>
\n<\/strong>(default: 60 seconds, recommended 15-30 seconds)<\/p>\nNotes:
\nYou can use any of the earlier described methods to reapply these settings at boot time.
\nHere is a quick way to view the number of connections and their states:<\/p>\nnetstat -tan | grep ‘:80 ‘ | awk ‘{print $6}’ | sort | uniq -c<\/strong><\/p>\n
TCP_KEEPALIVE_INTERVAL<\/strong>
\nThis determines the wait time between isAlive interval probes. To set:<\/p>\necho 30 > \/proc\/sys\/net\/ipv4\/tcp_keepalive_intvl<\/strong>
\n(default: 75 seconds, recommended: 15-30 seconds)<\/p>\nTCP_KEEPALIVE_PROBES<\/strong>
\nThis determines the number of probes before timing out<\/a>. To set:<\/p>\necho 5 > \/proc\/sys\/net\/ipv4\/tcp_keepalive_probes<\/strong>
\n(default: 9, recommended 5)<\/p>\nTCP_TW_RECYCLE<\/strong>
\n<\/strong>It enables fast recycling of TIME_WAIT sockets. The default value<\/a> is 0 (disabled). The sysctl documentation incorrectly states the default as enabled. It can be changed to 1 (enabled) in many cases. Known to cause some issues with hoststated (load balancing and fail over) if enabled, should be used with caution.<\/p>\necho 1 > \/proc\/sys\/net\/ipv4\/tcp_tw_recycle<\/strong>
\n(boolean, default: 0)<\/p>\nTCP_TW_REUSE<\/strong>
\nThis allows reusing sockets in TIME_WAIT state for new connections when it is safe from protocol viewpoint. Default value is 0 (disabled). It is generally a safer alternative to tcp_tw_recycle<\/p>\necho 1 > \/proc\/sys\/net\/ipv4\/tcp_tw_reuse<\/strong>
\n<\/strong>(boolean, default: 0)<\/p>\nNote: The tcp_tw_reuse setting is particularly useful in environments where numerous short connections are open and left in TIME_WAIT state, such as web servers. Reusing the sockets can be very effective in reducing server load.<\/p>\n
Kernel Recompile Option<\/strong><\/p>\n
There is another method one can use to set TCP\/IP parameters, involving kernel recompile… If you’re brave enough. Look for the parameters in the following files:
\n\/LINUX-SOURCE-DIR\/include\/linux\/skbuff.h<\/strong>
\nLook for<\/a> SK_WMEM_MAX & SK_RMEM_MAX
\n\/LINUX-SOURCE-DIR\/include\/net\/tcp.h<\/strong>
\nLook for MAX_WINDOW & MIN_WINDOW<\/p>\n","protected":false},"excerpt":{"rendered":"Operating systems and network tuning <\/p>\n
Network tuning can reduce Transmission Control Protocol (TCP) stack delay by changing connection settings and can improve throughput by changing TCP buffers.<\/p>\n
Operating systems <\/p>\n
A Windows system needs the least tuning while a Solaris system needs the most tuning. The following information pertains to each system specified, and might improve […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/746"}],"collection":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=746"}],"version-history":[{"count":1,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/746\/revisions"}],"predecessor-version":[{"id":747,"href":"https:\/\/mohan.sg\/index.php?rest_route=\/wp\/v2\/posts\/746\/revisions\/747"}],"wp:attachment":[{"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=746"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=746"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mohan.sg\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=746"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}