TCP Fast Open

Metrics

/proc/net/netstat provides a bunch of TFO-related metrics:

• TCPFastOpenActive: number of successful outbound TFO connections
• TCPFastOpenActiveFail: number of SYN-ACK packets received that did not acknowledge data sent in the SYN packet and caused a retransmissions without SYN data. Note that the original SYN packet contained a cookie + data, this is not the number of connections to servers that didn’t support TFO
• TCPFastOpenPassive: number of successful inbound TFO connections
• TCPFastOpenPassiveFail: number of inbound SYN packets with TFO cookie that was invalid
• TCPFastOpenCookieReqd: number of inbound SYN packets requesting TFO with TFO set but no cookie
• TCPFastOpenListenOverflow: number of inbound SYN packets that will have TFO disabled because the socket has exceeded the max queue length

Other interesting metrics are:

• TCPSynRetrans: number of SYN and SYN/ACK retransmits to break down retransmissions into SYN, fast-retransmits, timeout retransmits, etc.
• TCPOrigDataSent: number of outgoing packets with original data (excluding retransmission but including data-in-SYN). This counter is different from TcpOutSegs because TcpOutSegs also tracks pure ACKs. TCPOrigDataSent is more useful to track the TCP retransmission rate.

Server key generation

RAND=$(openssl rand -hex 16)
NEWKEY=${RAND:0:8}-${RAND:8:8}-${RAND:16:8}-${RAND:24:8}
echo "net.ipv4.tcp_fastopen_key=$NEWKEY" > /etc/sysctl.d/50-tcp_fastopen_key.conf
chmod 600 /etc/sysctl.d/50-tcp_fastopen_key.conf; chown root /etc/sysctl.d/50-tcp_fastopen_key.conf
sysctl -p /etc/sysctl.d/50-tcp_fastopen_key.conf
unset RAND NEWKEY

References

• Shaving your RTT with TCP Fast Open