Ubuntu Linux iptables firewall rules to prevent TCP package leakage. WWW.Smythies.com

Ubuntu Linux iptables firewall rules to prevent package leakage. WWW.Smythies.com

The problem: Unroutable TCP type packets, from reserved for local use only networks (RFC1918), being sent out to internet. For example, my Ubuntu Linux server 64-bit version 10.10 sending packets from 192.168.111.100 instead of performing NAT (Network Address Translation) such that they appear to be from 209.121.28.192.

The fix: Many reference iptables firewall rules examples list only the following two lines for the FORWARD section:

$IPTABLES -A FORWARD -i $EXTIF -o $INTIF -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT

To prevent leakage, or rogue, packets from escaping to internet another rule is required in the FORWARD chain (you might need similar in your INPUT chain, if your system is not just a server/router but also used as a desktop):

$IPTABLES -A FORWARD -i $INTIF -p tcp -m state --state INVALID -j DROP
$IPTABLES -A FORWARD -i $EXTIF -o $INTIF -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT

See also addendum notes further below. A device on your LAN (Local Area Network) that sends bad packets with another protocol might also cause packet leakage, so maybe delete the protocol specification. I say "maybe", because I do not have a good way to test this:

$IPTABLES -A FORWARD -i $INTIF -m state --state INVALID -j DROP
$IPTABLES -A FORWARD -i $EXTIF -o $INTIF -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT

The concern: Without the workaround, shouldn't the packets still go through NAT (Network Address Translation)? Is this a bug?

see also:
Netfilter.org - SNAT is failing to maquerade some TCP RST packets - bugzilla
Netfilter.org - NATed TCP-connections fail arbitrarily - bugzilla

The short answer (from bugzilla SNAT postings above) is that the NAT engine simply can not know what to do with an INVALID packet, so it is passed without address translation. So such packets need to be identified and dropped. This requirement is incredibly NOT obvious in most documentation.

Related point form notes:

For TCP connections, Linux tends to use a "half-duplex" close sequence where either side of the session can initiate connection termination via a single 2 way FIN-ACK handshake (which puts the connection into the CLOSE_WAIT state), instead of a full 4 way FIN-ACK handshake.
The default CLOSE_WAIT timeout is now 60 seconds. At one time it was 3 days. The rate of occurrence of this issue can be greatly reduced (by a factor of between 10 and 100) by increasing the CLOSE_WAIT timeout (I now use 3600 seconds, for example)("sudo sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait=3600" or "echo 3600 >/proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_timeout_close_wait"). On my system the length of the conntrack table was not significantly effected by this change.
Busy WWW servers will request TCP connection closure just a few seconds (sometimes as short as 5 seconds) after sending the last packet of data, thus inititating the CLOSE_WAIT state in the Ubuntu Server.
Many references found on the WWW contain misleading information on these issues.
By far (99.9%) of the leakage packets are a result of confusion about a terminated, or terminating, TCP session. There is rarely, if ever, an issue with opening a TCP session.

For reference, my current iptables rules script as a text file. (which started from the generally available "rc.firewall-iptables-stronger" version 0.88s) There are many comments in the history header and within the script body.

Example of TCP Packet leakage: The following two wireshark screen captures show the external and internal network during a packet leakage event. In this example it is the local client which first initiates termination of the TCP connection, which is not the usual senario.
Note that this example is actually not a typical representation of the issue. I did the screen shots and such before I had a good grasp of the issue. However, I am leaving this example in because I believe it is exactly like Netfilter.org - NATed TCP-connections fail arbitrarily - bugzilla

Observe, starting at number 6779, several packets being sent out to internet without Network Address Translation (NAT). Also notice packets 6825 and 6826 from ports 52181 and 52182 that do go through NAT again.

Example packet leakage to external network.

For reference, the corresponding internal network packets are shown in the following wireshark screen capture.

Example packet leakage. Corresponding internal network packets.

A more typical example of TCP Packet leakage: The following wireshark screen shot shows the external network packets during a leakage event. After that is a tcpdump segment (wireshark screen shot not available) for the corresponding internal network packets. This example is the most basic, the TCP connection is properly terminated as of the FIN ACK and ACK exchange of packets 117723 and 117724 and the conntrack table forgets about the connection. Subsequently, the internal client at 192.168.111.100 tries to close the non-existant TCP connection via FIN ACK packets and those go out to internet without NAT.

Example packet leakage to external network.

The tcpdump output for the same packets on the internal network:

2011-01-25 15:25:49.121815 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 113881:115341, ack 1059, win 63, length 1460
2011-01-25 15:25:49.126246 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 115341:116801, ack 1059, win 63, length 1460
2011-01-25 15:25:49.128475 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [.], ack 116801, win 16425, length 0
2011-01-25 15:25:49.141022 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 116801:118261, ack 1059, win 63, length 1460
2011-01-25 15:25:49.145446 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 118261:119721, ack 1059, win 63, length 1460
2011-01-25 15:25:49.146962 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [.], ack 119721, win 16425, length 0
2011-01-25 15:25:49.149664 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 119721:121181, ack 1059, win 63, length 1460
2011-01-25 15:25:49.154351 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 121181:122641, ack 1059, win 63, length 1460
2011-01-25 15:25:49.156031 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [.], ack 122641, win 16425, length 0
2011-01-25 15:25:49.158761 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 122641:124101, ack 1059, win 63, length 1460
2011-01-25 15:25:49.162955 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 124101:125561, ack 1059, win 63, length 1460
2011-01-25 15:25:49.164434 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [.], ack 125561, win 16425, length 0
2011-01-25 15:25:49.167389 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 125561:127021, ack 1059, win 63, length 1460
2011-01-25 15:25:49.172046 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 127021:128481, ack 1059, win 63, length 1460
2011-01-25 15:25:49.173528 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [.], ack 128481, win 16425, length 0
2011-01-25 15:25:49.176275 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 128481:129941, ack 1059, win 63, length 1460
2011-01-25 15:25:49.180678 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 129941:131401, ack 1059, win 63, length 1460
2011-01-25 15:25:49.185109 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [.], seq 131401:132861, ack 1059, win 63, length 1460
2011-01-25 15:25:49.186230 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [.], ack 131401, win 16425, length 0
2011-01-25 15:25:49.246483 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [P.], seq 132861:134067, ack 1059, win 63, length 1206
2011-01-25 15:25:49.248004 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [.], ack 134067, win 16425, length 0
2011-01-25 15:26:03.854483 IP 91.189.94.12.80 > 192.168.111.100.57975: Flags [F.], seq 134067, ack 1059, win 63, length 0
2011-01-25 15:26:03.855683 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [.], ack 134068, win 16425, length 0
2011-01-25 15:28:27.514040 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [F.], seq 1059, ack 134068, win 16425, length 0
2011-01-25 15:28:28.328373 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [F.], seq 1059, ack 134068, win 16425, length 0
2011-01-25 15:28:29.932232 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [F.], seq 1059, ack 134068, win 16425, length 0
2011-01-25 15:28:33.129838 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [F.], seq 1059, ack 134068, win 16425, length 0
2011-01-25 15:28:39.524965 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [F.], seq 1059, ack 134068, win 16425, length 0
2011-01-25 15:28:52.324922 IP 192.168.111.100.57975 > 91.189.94.12.80: Flags [R.], seq 1060, ack 134068, win 0, length 0

Example of TCP session termination with packet leakage fixed (with TCP RST REJECT): The following wireshark screen captures show cases where packet leakage would have occured if not for the additional rule in the FORWARD chain. However, in these examples the packets were not DROPped, they were REJECTed with the TCP RST flag set. The rules:

$IPTABLES -A FORWARD -i $INTIF -p tcp -m state --state INVALID -j REJECT --reject-with tcp-reset
$IPTABLES -A FORWARD -i $EXTIF -o $INTIF -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT

For reference, the complete iptables rules script for this configuration as a text file (See above for my actual current iptables rule script).

Case 1: An apple Macbook (at 192.168.111.106):
The Ubuntu Linux server will set the TCP connection into the CLOSE_WAIT state as of packets 3778 and 3779. The Macbook tries to send a FIN, ACK 1 minute and 32 seconds later, after the CLOSE_WAIT time of 1 minute has expired. The iptables rule triggers and packet 15823 is inserted and sent back to the macbook.

Example.

The corresponding external packets showing no leakage packets:

Case 2: A windows computer (at 192.168.111.100):
The Ubuntu Linux server will set the TCP connection into the CLOSE_WAIT state as of packets 1766 and 1767. The windows computer tries to send a RST 5 minutes and 18 seconds later, after the CLOSE_WAIT time of 1 minute has expired. The iptables rule triggers and the packet is REJECTED. However, no packet is inserted and sent back to the windows computer. The system does seem to permit replying to a RST packet with another RST packet. (I did not take an external screen shot for this case, but there was no packet leakage).

Example.

Note: For the inserted REJECT packets to be delivered (Macbook case), the default OUPUT chain rule needs to set to ACCEPT rather than DROP.

Example of TCP session termination with packet leakage fixed (with DROP): The following wireshark screen captures show cases where packet leakage would have occured if not for the additional rule in the FORWARD chain. In these examples the packets were set to DROP and the default OUTPUT policy was DROP. The rules:

$IPTABLES -A FORWARD -i $INTIF -p tcp -m state --state INVALID -j DROP
$IPTABLES -A FORWARD -i $EXTIF -o $INTIF -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A FORWARD -i $INTIF -o $EXTIF -j ACCEPT

Case 1: An apple Macbook (at 192.168.111.106):
Again, the Ubuntu Linux server will set the TCP connection into the CLOSE_WAIT state as of packets 2753 and 2754. The Macbook tries to send a FIN, ACK 1 minute and 8 seconds later, after the CLOSE_WAIT time of 1 minute has expired. The iptables rule triggers and DROPs the packet. Since the Macbook doesn't get an ACK (or a RST) it tries 10 more times over the next 5 minutes.
Note that in this case the WWW host is asking for the TCP connection termination and that it does so only 5 seconds after the last data packet.

Example.

The corresponding external packets showing no leakage packets:

Case 2: A windows computer: See case 2 above, as the results are the same.

ip_conntrack_tcp_loose: Some rather obscure references suggest setting ip_conntrack_tcp_loose=0. All I can say is I tried it and it did not solve the packet leakage issue. On my Ubuntu Server 64 bit version 10.10 system this is set to 1 by default. If by a mistake during installation, or by design, I do not know. I have not determined how to make it 0 and to persist through a system re-boot. It could be set by the iptables script, and mine has a commented out rule to that effect.

Addendum notes: After over 2.5 years without a single leakage, or un-NATted, packet escaping to internet from my network, there was an incident where one did. The cause was a poorly formed ICMP packet from a smart phone (something that should never occur, but did). The proposed solution is to delete the protocol specifier on the INVALID check in the forward chain (see way above).

The internal bad packet with a bogus header length:

Example.

The external leakage, Un-NATted, bad packet with a bogus header length:

References:
Netfilter.org - SNAT is failing to maquerade some TCP RST packets - bugzilla
Netfilter.org - NATed TCP-connections fail arbitrarily - bugzilla
default value of nf_conntrack_tcp_timeout_close_wait discussion
The Linux Documentation Project - Linux IP Masquerade HOWTO
The Linux Documentation Project - stronger firewall example.
CLOSE_WAIT description
Iptables Tutorial 1.2.2
Linuxtopia Firewall Iptables
netfilter.org NAT-HOWTO
Packet Filtering Notes - Prabhaker Mateti
Ubuntu forums - an example ruleset.
wikipedia TCP
tcp_fin_timeout description
tcp_close_wait_interval description
FIN_WAIT_2 description
Easy Firewall Generator
Bodhi Zazen's (Ubuntu Guru) IPTables Primer
Ockers notes on Un-NATted packets

Ubuntu Linux iptables rules to prevent TCP package leakage. WWW.Smythies.com emaildoesnotwork@smythies.com 2011.02.13 Updated 2013.11.01