Did you observe the following problems on router ?
1. Router with two or more interfaces forwarding the packets has problem of 'reply packet is seen on the interface but not forwarded to another interface', given iptables rules are correct.
2. Reply packet is seen on iptables mangle PREROUTING chain after that it is disappeared.
Packet is not sent to iptables FORWARD chain after routing.
3. packet source address is not directly routable via the interface it is not reachable, packet received on the interface but sent out or forwarded.
4. A non default route public interface traffic received that packet but not forwarded, given iptables rules are correct.
If you observe the above problems then it is the problem of rp_filter setting on the interface.
What is linux rp_filter (reverse path forwarding)
/proc/sys/net/ipv4/conf/<interface>/rp_filter
rp_filter - 0 disable or
0. no source validation
rp_filter - 1 strict
1. When a machine with interface rp_filter enabled receives a packet on the interface, if this received packet source is not reachable (routable) via this interface (the interface on which the packet received) then the packet is dropped.
If it is routable through the interface which it came, then the machine will accept the packet
If it is not routable through the interface, which it came, then the machine will drop that packet.
rp_filter - 2 loose
if the received packet source address is routable with any of the routes on the any of the interface on the router then it accepts the packet.
Example:
Router has two public interfaces eth1, eth2.
eth1 is default route interface
eth2 is non default route interface but it has policy based routes.
rp_filter on eth2 is set to 1, its ip is 10.147.52.100 .
A packet with (src, dst) 10.233.70.100 ---- 10.147.52.100 is received on eth2. Now eth2 will check whether 10.233.70.100 is reachable with its routes. If the packet is not reachable then it will drop the packet. That means its reverse path is not reachable.
conf/{all,interface}/rp_filter
rp_filter takes the max of {all,interface} value and then takes the action. if all->1 and interface value is 2 then interface rp_filter will take effect.
/proc/sys/net/ipv4/conf/eth5/log_martians
This will log the dropped packets of the rp_filter in dmesg.
#dmesg
[61162.827562] ll header: 00000000: 1e 00 18 00 00 9b 38 90 a5 27 4b d4 08 00 ......8..'K...
[61163.842472] IPv4: martian source 10.1.3.145 from 8.8.8.8, on dev eth2
Destination mac: 1e 00 18 00 00 9b
Source mac: 38 90 a5 27 4b d4
Type: 08 is ip header
When rp_filter takes the action:
1. rp_filter checks the packet after the SNAT source nat get applied on the packet post routing
2. On return traffic after prerouting, conntrack changes the destination ip address of the NAT packet. Before forwarding the packet rp_filter checks whether source ip reachable from this interface then takes the action.
Let me show the problem in the setup.
Setup image:
Video explanation:
