Why CentOS ignore packets when the route for outbound traffic differs from the route of incoming traffic?

To resolve this issue, set the net.ipv4.conf.all.rp_filter kernel tunable parameter value to 2.

sysctl -w net.ipv4.conf.all.rp_filter=2

To make this change persistent across reboots, add the tunable to the /etc/sysctl.conf file.

Strict filtering means that when a packet comes into the system, the kernel looks up the source IP in its routing table to see if the interface the packet came in on is the same interface it would use to send a packet to that IP.

If the interfaces are the same, the packet has passed the strict filtering test and is processed normally. If the interfaces differ, the packet is discarded and not processed further. In CentOS 7 and later, the IPReversePathFilter counter goes up.

The main effect of strict filtering is that the system will only talk to a remote IP address through a specific interface. Set up static routes to control which interface responds to a particular IP or network remotely.

The filtering method is controlled globally by the sysctl net.ipv4.conf.all.rp_filter

Setting the sysctl net.ipv4.conf.all.rp_filter to 2 (loose) will override the settings for each interface and turn off the strict check. Setting net.ipv4.conf.all.rp_filter to 0 (disabled) doesn't change the settings for each interface, so it's not a good idea.

Note that if the strict check is not done, the system may answer a packet through a different interface than the one it came in on. Whether this leads to the expected connectivity depends on many things outside of the system, like how the physical network is set up and how the firewall is set up.

Check the value of the rp_filter sysctls.

$ sysctl -a 2>/dev/null | grep "\.rp_filter"
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.enp0s31f6.rp_filter = 0
net.ipv4.conf.lo.rp_filter = 0
net.ipv4.conf.tun0.rp_filter = 0
net.ipv4.conf.virbr0.rp_filter = 0
net.ipv4.conf.virbr0-nic.rp_filter = 0
net.ipv4.conf.virbr1.rp_filter = 0
net.ipv4.conf.virbr1-nic.rp_filter = 0
net.ipv4.conf.wlp58s0.rp_filter = 0

Check the IPReversePathFilter SNMP counter for CentOS 7 and higher. If strict filtering means that packets are being ignored, this counter will go up each time.

$ nstat -rsz | grep IPReversePathFilter
TcpExtIPReversePathFilter       52537                  0.0

$ netstat -s | grep IPReversePathFilter
IPReversePathFilter: 52537

Do a route lookup for a remote IP whose packets seem to be ignored by the system to find out which interface the system will use to connect to that remote IP. If this is not the same interface that the packets from the remote IP come in on, strict enforcement will throw them away.

$ ip route get <remote IP>