[RFC PATCH net] ipv4: ip_gre: Fix drops of small packets in ipgre_xmit

Anton Danilov posted 1 patch 2 months, 1 week ago
There is a newer version of this series
net/ipv4/ip_gre.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
[RFC PATCH net] ipv4: ip_gre: Fix drops of small packets in ipgre_xmit
Posted by Anton Danilov 2 months, 1 week ago
Regression Description:

Depending on the GRE tunnel device options, small packets are being
dropped. This occurs because the pskb_network_may_pull function fails due
to insufficient space in the network header. For example, if only the key
option is specified for the tunnel device, packets of sizes up to 27
(including the IPv4 header itself) will be dropped. This affects both
locally originated and forwarded packets.

How to reproduce (for local originated packets):

  ip link add dev gre1 type gre ikey 1.9.8.4 okey 1.9.8.4 \
          local <your-ip> remote <any-ip>

  ip link set mtu 1400 dev gre1
  ip link set up dev gre1
  ip address add 192.168.13.1/24 dev gre1
  ping -s 1374 -c 10 192.168.13.2
  tcpdump -vni gre1
  tcpdump -vni <your-ext-iface> 'ip proto 47'
  ip -s -s -d link show dev gre1

Solution:

Use the pskb_may_pull function instead the pskb_network_may_pull.

Fixes: 80d875cfc9d3 ("ipv4: ip_gre: Avoid skb_pull() failure in ipgre_xmit()")

Signed-off-by: Anton Danilov <littlesmilingcloud@gmail.com>
---
 net/ipv4/ip_gre.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/ipv4/ip_gre.c b/net/ipv4/ip_gre.c
index 5f6fd382af38..115272ba2726 100644
--- a/net/ipv4/ip_gre.c
+++ b/net/ipv4/ip_gre.c
@@ -664,7 +664,7 @@ static netdev_tx_t ipgre_xmit(struct sk_buff *skb,
 
 		tnl_params = (const struct iphdr *)skb->data;
 
-		if (!pskb_network_may_pull(skb, pull_len))
+		if (!pskb_may_pull(skb, pull_len))
 			goto free_skb;
 
 		/* ip_tunnel_xmit() needs skb->data pointing to gre header. */
-- 
2.39.2
Re: [RFC PATCH net] ipv4: ip_gre: Fix drops of small packets in ipgre_xmit
Posted by Eric Dumazet 2 months, 1 week ago
On Sat, Sep 21, 2024 at 11:55 PM Anton Danilov
<littlesmilingcloud@gmail.com> wrote:
>
> Regression Description:
>
> Depending on the GRE tunnel device options, small packets are being
> dropped. This occurs because the pskb_network_may_pull function fails due
> to insufficient space in the network header. For example, if only the key
> option is specified for the tunnel device, packets of sizes up to 27
> (including the IPv4 header itself) will be dropped. This affects both
> locally originated and forwarded packets.
>
> How to reproduce (for local originated packets):
>
>   ip link add dev gre1 type gre ikey 1.9.8.4 okey 1.9.8.4 \
>           local <your-ip> remote <any-ip>
>
>   ip link set mtu 1400 dev gre1
>   ip link set up dev gre1
>   ip address add 192.168.13.1/24 dev gre1
>   ping -s 1374 -c 10 192.168.13.2

This size does not match the changelog ? (packets of sizes up to 27...)

>   tcpdump -vni gre1
>   tcpdump -vni <your-ext-iface> 'ip proto 47'
>   ip -s -s -d link show dev gre1

Please provide a real selftest, because in this particular example,
the path taken by the packets should not reach the
pskb_network_may_pull(skb, pull_len)),
because dev->header_ops should be NULL ?
Re: [RFC PATCH net] ipv4: ip_gre: Fix drops of small packets in ipgre_xmit
Posted by Anton Danilov 2 months, 1 week ago
On Sun, Sep 22, 2024 at 12:20:03PM +0200, Eric Dumazet wrote:

Hi Eric,

> Please provide a real selftest, because in this particular example,
> the path taken by the packets should not reach the
> pskb_network_may_pull(skb, pull_len)),
> because dev->header_ops should be NULL ?

I sincerely apologize for providing an inaccurate example of the commands 
needed to reproduce the issue. I understand that this may have caused
confusion, and I'm truly sorry for any inconvenience.

Here are the correct commands and their results.


  ip l add name mgre0 type gre local 192.168.71.177 remote 0.0.0.0 ikey 1.9.8.4 okey 1.9.8.4
  ip l s mtu 1400 dev mgre0
  ip a add 192.168.13.1/24 dev mgre0
  ip l s up dev mgre0
  ip n add 192.168.13.2 lladdr 192.168.69.50 dev mgre0
  
  
  ip -s -s -d l ls dev mgre0
    19: mgre0@NONE: <NOARP,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
        link/gre 192.168.71.177 brd 0.0.0.0 promiscuity 0  allmulti 0 minmtu 0 maxmtu 0 
        gre remote any local 192.168.71.177 ttl inherit ikey 1.9.8.4 okey 1.9.8.4 \
          addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 \
          tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 

        RX:  bytes packets errors dropped  missed   mcast           
                 0       0      0       0       0       0 
        RX errors:  length    crc   frame    fifo overrun
                         0      0       0       0       0 
        TX:  bytes packets errors dropped carrier collsns           
                 0       0      0       0       0       0 
        TX errors: aborted   fifo  window heartbt transns
                         0      0       0       0       0 


  ping -n -c 10 -s 1374 192.168.13.2
    PING 192.168.13.2 (192.168.13.2) 1374(1402) bytes of data.
    
    --- 192.168.13.2 ping statistics ---
    10 packets transmitted, 0 received, 100% packet loss, time 9237ms


  ip -s -s -d l ls dev mgre0
    19: mgre0@NONE: <NOARP,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
        link/gre 192.168.71.177 brd 0.0.0.0 promiscuity 1  allmulti 0 minmtu 0 maxmtu 0 
        gre remote any local 192.168.71.177 ttl inherit ikey 1.9.8.4 okey 1.9.8.4 \
          addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 \
          tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536

        RX:  bytes packets errors dropped  missed   mcast           
                 0       0      0       0       0       0 
        RX errors:  length    crc   frame    fifo overrun
                         0      0       0       0       0 
        TX:  bytes packets errors dropped carrier collsns           
             13960      10      0      10       0       0 
        TX errors: aborted   fifo  window heartbt transns
                     0      0       0       0       0 


  tcpdump -vni mgre0
    tcpdump: listening on mgre0, link-type LINUX_SLL (Linux cooked v1), snapshot length 262144 bytes
    21:51:19.481523 IP (tos 0x0, ttl 64, id 52595, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 1, length 1376
    21:51:19.481547 IP (tos 0x0, ttl 64, id 52595, offset 1376, flags [none], proto ICMP (1), length 26)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    21:51:20.526751 IP (tos 0x0, ttl 64, id 53374, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 2, length 1376
    21:51:20.526773 IP (tos 0x0, ttl 64, id 53374, offset 1376, flags [none], proto ICMP (1), length 26)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    21:51:21.550751 IP (tos 0x0, ttl 64, id 54124, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 3, length 1376
    21:51:21.550775 IP (tos 0x0, ttl 64, id 54124, offset 1376, flags [none], proto ICMP (1), length 26)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    21:51:22.574748 IP (tos 0x0, ttl 64, id 55109, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 4, length 1376
    21:51:22.574766 IP (tos 0x0, ttl 64, id 55109, offset 1376, flags [none], proto ICMP (1), length 26)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    21:51:23.598748 IP (tos 0x0, ttl 64, id 56011, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 5, length 1376
    21:51:23.598771 IP (tos 0x0, ttl 64, id 56011, offset 1376, flags [none], proto ICMP (1), length 26)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    21:51:24.622758 IP (tos 0x0, ttl 64, id 57009, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 6, length 1376
    21:51:24.622783 IP (tos 0x0, ttl 64, id 57009, offset 1376, flags [none], proto ICMP (1), length 26)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    21:51:25.646748 IP (tos 0x0, ttl 64, id 57277, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 7, length 1376
    21:51:25.646775 IP (tos 0x0, ttl 64, id 57277, offset 1376, flags [none], proto ICMP (1), length 26)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    21:51:26.670750 IP (tos 0x0, ttl 64, id 57869, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 8, length 1376
    21:51:26.670773 IP (tos 0x0, ttl 64, id 57869, offset 1376, flags [none], proto ICMP (1), length 26)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    21:51:27.694751 IP (tos 0x0, ttl 64, id 58317, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 9, length 1376
    21:51:27.694774 IP (tos 0x0, ttl 64, id 58317, offset 1376, flags [none], proto ICMP (1), length 26)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    21:51:28.718751 IP (tos 0x0, ttl 64, id 58558, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 10, length 1376
    21:51:28.718775 IP (tos 0x0, ttl 64, id 58558, offset 1376, flags [none], proto ICMP (1), length 26)
        192.168.13.1 > 192.168.13.2: ip-proto-1


  tcpdump -vni enp11s0.100 'ip proto 47'
    tcpdump: listening on enp11s0.100, link-type EN10MB (Ethernet), snapshot length 262144 bytes
    21:51:19.481696 IP (tos 0x0, ttl 64, id 32563, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 52595, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 1, length 1376
    21:51:20.526767 IP (tos 0x0, ttl 64, id 33363, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 53374, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 2, length 1376
    21:51:21.550768 IP (tos 0x0, ttl 64, id 34260, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 54124, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 3, length 1376
    21:51:22.574761 IP (tos 0x0, ttl 64, id 34922, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 55109, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 4, length 1376
    21:51:23.598764 IP (tos 0x0, ttl 64, id 35042, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 56011, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 5, length 1376
    21:51:24.622775 IP (tos 0x0, ttl 64, id 36024, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 57009, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 6, length 1376
    21:51:25.646766 IP (tos 0x0, ttl 64, id 36133, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 57277, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 7, length 1376
    21:51:26.670766 IP (tos 0x0, ttl 64, id 36417, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 57869, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 8, length 1376
    21:51:27.694767 IP (tos 0x0, ttl 64, id 37006, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 58317, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 9, length 1376
    21:51:28.718767 IP (tos 0x0, ttl 64, id 37825, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 58558, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 10, length 1376


  ping -n -c 10 -s 1376 192.168.13.2
    PING 192.168.13.2 (192.168.13.2) 1376(1404) bytes of data.
    
    --- 192.168.13.2 ping statistics ---
    10 packets transmitted, 0 received, 100% packet loss, time 9198ms


  ip -s -s -d l ls dev mgre0
    19: mgre0@NONE: <NOARP,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
        link/gre 192.168.71.177 brd 0.0.0.0 promiscuity 0  allmulti 0 minmtu 0 maxmtu 0 
        gre remote any local 192.168.71.177 ttl inherit ikey 1.9.8.4 okey 1.9.8.4 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 
        RX:  bytes packets errors dropped  missed   mcast           
                 0       0      0       0       0       0 
        RX errors:  length    crc   frame    fifo overrun
                         0      0       0       0       0 
        TX:  bytes packets errors dropped carrier collsns           
             28200      30      0      10       0       0 
        TX errors: aborted   fifo  window heartbt transns
                         0      0       0       0       0 


  tcpdump -vni mgre0
    tcpdump: listening on mgre0, link-type LINUX_SLL (Linux cooked v1), snapshot length 262144 bytes
    22:01:34.176810 IP (tos 0x0, ttl 64, id 40388, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 1, length 1376
    22:01:34.176830 IP (tos 0x0, ttl 64, id 40388, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:35.183742 IP (tos 0x0, ttl 64, id 40516, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 2, length 1376
    22:01:35.183765 IP (tos 0x0, ttl 64, id 40516, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:36.207750 IP (tos 0x0, ttl 64, id 40684, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 3, length 1376
    22:01:36.207774 IP (tos 0x0, ttl 64, id 40684, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:37.230738 IP (tos 0x0, ttl 64, id 41578, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 4, length 1376
    22:01:37.230756 IP (tos 0x0, ttl 64, id 41578, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:38.254761 IP (tos 0x0, ttl 64, id 42099, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 5, length 1376
    22:01:38.254789 IP (tos 0x0, ttl 64, id 42099, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:39.278748 IP (tos 0x0, ttl 64, id 42506, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 6, length 1376
    22:01:39.278771 IP (tos 0x0, ttl 64, id 42506, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:40.302738 IP (tos 0x0, ttl 64, id 42527, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 7, length 1376
    22:01:40.302754 IP (tos 0x0, ttl 64, id 42527, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:41.326733 IP (tos 0x0, ttl 64, id 42989, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 8, length 1376
    22:01:41.326749 IP (tos 0x0, ttl 64, id 42989, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:42.350750 IP (tos 0x0, ttl 64, id 43576, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 9, length 1376
    22:01:42.350773 IP (tos 0x0, ttl 64, id 43576, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:43.374743 IP (tos 0x0, ttl 64, id 44118, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 10, length 1376
    22:01:43.374762 IP (tos 0x0, ttl 64, id 44118, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1

    
  tcpdump -vni enp11s0.100 'ip proto 47'
    tcpdump: listening on enp11s0.100, link-type EN10MB (Ethernet), snapshot length 262144 bytes
    22:01:34.176825 IP (tos 0x0, ttl 64, id 5066, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 40388, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 1, length 1376
    22:01:34.176832 IP (tos 0x0, ttl 64, id 5067, offset 0, flags [DF], proto GRE (47), length 56)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
        IP (tos 0x0, ttl 64, id 40388, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:35.183758 IP (tos 0x0, ttl 64, id 5567, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 40516, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 2, length 1376
    22:01:35.183768 IP (tos 0x0, ttl 64, id 5568, offset 0, flags [DF], proto GRE (47), length 56)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
        IP (tos 0x0, ttl 64, id 40516, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:36.207767 IP (tos 0x0, ttl 64, id 5741, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 40684, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 3, length 1376
    22:01:36.207778 IP (tos 0x0, ttl 64, id 5742, offset 0, flags [DF], proto GRE (47), length 56)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
        IP (tos 0x0, ttl 64, id 40684, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:37.230751 IP (tos 0x0, ttl 64, id 5785, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 41578, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 4, length 1376
    22:01:37.230758 IP (tos 0x0, ttl 64, id 5786, offset 0, flags [DF], proto GRE (47), length 56)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
        IP (tos 0x0, ttl 64, id 41578, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:38.254780 IP (tos 0x0, ttl 64, id 5937, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 42099, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 5, length 1376
    22:01:38.254795 IP (tos 0x0, ttl 64, id 5938, offset 0, flags [DF], proto GRE (47), length 56)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
        IP (tos 0x0, ttl 64, id 42099, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:39.278764 IP (tos 0x0, ttl 64, id 6876, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 42506, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 6, length 1376
    22:01:39.278775 IP (tos 0x0, ttl 64, id 6877, offset 0, flags [DF], proto GRE (47), length 56)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
        IP (tos 0x0, ttl 64, id 42506, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:40.302749 IP (tos 0x0, ttl 64, id 7410, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 42527, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 7, length 1376
    22:01:40.302757 IP (tos 0x0, ttl 64, id 7411, offset 0, flags [DF], proto GRE (47), length 56)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
        IP (tos 0x0, ttl 64, id 42527, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:41.326744 IP (tos 0x0, ttl 64, id 7913, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 42989, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 8, length 1376
    22:01:41.326753 IP (tos 0x0, ttl 64, id 7914, offset 0, flags [DF], proto GRE (47), length 56)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
        IP (tos 0x0, ttl 64, id 42989, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:42.350766 IP (tos 0x0, ttl 64, id 8422, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 43576, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 9, length 1376
    22:01:42.350776 IP (tos 0x0, ttl 64, id 8423, offset 0, flags [DF], proto GRE (47), length 56)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
        IP (tos 0x0, ttl 64, id 43576, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
    22:01:43.374756 IP (tos 0x0, ttl 64, id 9410, offset 0, flags [DF], proto GRE (47), length 1424)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
        IP (tos 0x0, ttl 64, id 44118, offset 0, flags [+], proto ICMP (1), length 1396)
        192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 10, length 1376
    22:01:43.374766 IP (tos 0x0, ttl 64, id 9411, offset 0, flags [DF], proto GRE (47), length 56)
        192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
        IP (tos 0x0, ttl 64, id 44118, offset 1376, flags [none], proto ICMP (1), length 28)
        192.168.13.1 > 192.168.13.2: ip-proto-1
Re: [RFC PATCH net] ipv4: ip_gre: Fix drops of small packets in ipgre_xmit
Posted by Eric Dumazet 2 months ago
On Sun, Sep 22, 2024 at 10:23 PM Anton Danilov
<littlesmilingcloud@gmail.com> wrote:
>
> On Sun, Sep 22, 2024 at 12:20:03PM +0200, Eric Dumazet wrote:
>
> Hi Eric,
>
> > Please provide a real selftest, because in this particular example,
> > the path taken by the packets should not reach the
> > pskb_network_may_pull(skb, pull_len)),
> > because dev->header_ops should be NULL ?
>
> I sincerely apologize for providing an inaccurate example of the commands
> needed to reproduce the issue. I understand that this may have caused
> confusion, and I'm truly sorry for any inconvenience.
>
> Here are the correct commands and their results.
>
>
>   ip l add name mgre0 type gre local 192.168.71.177 remote 0.0.0.0 ikey 1.9.8.4 okey 1.9.8.4
>   ip l s mtu 1400 dev mgre0
>   ip a add 192.168.13.1/24 dev mgre0
>   ip l s up dev mgre0
>   ip n add 192.168.13.2 lladdr 192.168.69.50 dev mgre0

This looks much better. I was hoping that we could capture this in a
new test (added in tools/testing/selftests/net)

Please also move the buggy "tnl_params = (const struct iphdr
*)skb->data;" line as in :

diff --git a/net/ipv4/ip_gre.c b/net/ipv4/ip_gre.c
index 5f6fd382af38a32d9e22633cdb2e9fd01f1795e4..f1f31ebfc7934467fd10776c3cb221f9cff9f9dd
100644
--- a/net/ipv4/ip_gre.c
+++ b/net/ipv4/ip_gre.c
@@ -662,11 +662,11 @@ static netdev_tx_t ipgre_xmit(struct sk_buff *skb,
                if (skb_cow_head(skb, 0))
                        goto free_skb;

-               tnl_params = (const struct iphdr *)skb->data;
-
-               if (!pskb_network_may_pull(skb, pull_len))
+               if (!pskb_may_pull(skb, pull_len))
                        goto free_skb;

+               tnl_params = (const struct iphdr *)skb->data;
+
                /* ip_tunnel_xmit() needs skb->data pointing to gre header. */
                skb_pull(skb, pull_len);
                skb_reset_mac_header(skb);


>
>
>   ip -s -s -d l ls dev mgre0
>     19: mgre0@NONE: <NOARP,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
>         link/gre 192.168.71.177 brd 0.0.0.0 promiscuity 0  allmulti 0 minmtu 0 maxmtu 0
>         gre remote any local 192.168.71.177 ttl inherit ikey 1.9.8.4 okey 1.9.8.4 \
>           addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 \
>           tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536
>
>         RX:  bytes packets errors dropped  missed   mcast
>                  0       0      0       0       0       0
>         RX errors:  length    crc   frame    fifo overrun
>                          0      0       0       0       0
>         TX:  bytes packets errors dropped carrier collsns
>                  0       0      0       0       0       0
>         TX errors: aborted   fifo  window heartbt transns
>                          0      0       0       0       0
>
>
>   ping -n -c 10 -s 1374 192.168.13.2
>     PING 192.168.13.2 (192.168.13.2) 1374(1402) bytes of data.
>
>     --- 192.168.13.2 ping statistics ---
>     10 packets transmitted, 0 received, 100% packet loss, time 9237ms
>
>
>   ip -s -s -d l ls dev mgre0
>     19: mgre0@NONE: <NOARP,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
>         link/gre 192.168.71.177 brd 0.0.0.0 promiscuity 1  allmulti 0 minmtu 0 maxmtu 0
>         gre remote any local 192.168.71.177 ttl inherit ikey 1.9.8.4 okey 1.9.8.4 \
>           addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 \
>           tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536
>
>         RX:  bytes packets errors dropped  missed   mcast
>                  0       0      0       0       0       0
>         RX errors:  length    crc   frame    fifo overrun
>                          0      0       0       0       0
>         TX:  bytes packets errors dropped carrier collsns
>              13960      10      0      10       0       0
>         TX errors: aborted   fifo  window heartbt transns
>                      0      0       0       0       0
>
>
>   tcpdump -vni mgre0
>     tcpdump: listening on mgre0, link-type LINUX_SLL (Linux cooked v1), snapshot length 262144 bytes
>     21:51:19.481523 IP (tos 0x0, ttl 64, id 52595, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 1, length 1376
>     21:51:19.481547 IP (tos 0x0, ttl 64, id 52595, offset 1376, flags [none], proto ICMP (1), length 26)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     21:51:20.526751 IP (tos 0x0, ttl 64, id 53374, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 2, length 1376
>     21:51:20.526773 IP (tos 0x0, ttl 64, id 53374, offset 1376, flags [none], proto ICMP (1), length 26)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     21:51:21.550751 IP (tos 0x0, ttl 64, id 54124, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 3, length 1376
>     21:51:21.550775 IP (tos 0x0, ttl 64, id 54124, offset 1376, flags [none], proto ICMP (1), length 26)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     21:51:22.574748 IP (tos 0x0, ttl 64, id 55109, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 4, length 1376
>     21:51:22.574766 IP (tos 0x0, ttl 64, id 55109, offset 1376, flags [none], proto ICMP (1), length 26)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     21:51:23.598748 IP (tos 0x0, ttl 64, id 56011, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 5, length 1376
>     21:51:23.598771 IP (tos 0x0, ttl 64, id 56011, offset 1376, flags [none], proto ICMP (1), length 26)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     21:51:24.622758 IP (tos 0x0, ttl 64, id 57009, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 6, length 1376
>     21:51:24.622783 IP (tos 0x0, ttl 64, id 57009, offset 1376, flags [none], proto ICMP (1), length 26)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     21:51:25.646748 IP (tos 0x0, ttl 64, id 57277, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 7, length 1376
>     21:51:25.646775 IP (tos 0x0, ttl 64, id 57277, offset 1376, flags [none], proto ICMP (1), length 26)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     21:51:26.670750 IP (tos 0x0, ttl 64, id 57869, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 8, length 1376
>     21:51:26.670773 IP (tos 0x0, ttl 64, id 57869, offset 1376, flags [none], proto ICMP (1), length 26)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     21:51:27.694751 IP (tos 0x0, ttl 64, id 58317, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 9, length 1376
>     21:51:27.694774 IP (tos 0x0, ttl 64, id 58317, offset 1376, flags [none], proto ICMP (1), length 26)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     21:51:28.718751 IP (tos 0x0, ttl 64, id 58558, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 10, length 1376
>     21:51:28.718775 IP (tos 0x0, ttl 64, id 58558, offset 1376, flags [none], proto ICMP (1), length 26)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>
>
>   tcpdump -vni enp11s0.100 'ip proto 47'
>     tcpdump: listening on enp11s0.100, link-type EN10MB (Ethernet), snapshot length 262144 bytes
>     21:51:19.481696 IP (tos 0x0, ttl 64, id 32563, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 52595, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 1, length 1376
>     21:51:20.526767 IP (tos 0x0, ttl 64, id 33363, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 53374, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 2, length 1376
>     21:51:21.550768 IP (tos 0x0, ttl 64, id 34260, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 54124, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 3, length 1376
>     21:51:22.574761 IP (tos 0x0, ttl 64, id 34922, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 55109, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 4, length 1376
>     21:51:23.598764 IP (tos 0x0, ttl 64, id 35042, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 56011, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 5, length 1376
>     21:51:24.622775 IP (tos 0x0, ttl 64, id 36024, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 57009, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 6, length 1376
>     21:51:25.646766 IP (tos 0x0, ttl 64, id 36133, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 57277, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 7, length 1376
>     21:51:26.670766 IP (tos 0x0, ttl 64, id 36417, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 57869, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 8, length 1376
>     21:51:27.694767 IP (tos 0x0, ttl 64, id 37006, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 58317, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 9, length 1376
>     21:51:28.718767 IP (tos 0x0, ttl 64, id 37825, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 58558, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 63847, seq 10, length 1376
>
>
>   ping -n -c 10 -s 1376 192.168.13.2
>     PING 192.168.13.2 (192.168.13.2) 1376(1404) bytes of data.
>
>     --- 192.168.13.2 ping statistics ---
>     10 packets transmitted, 0 received, 100% packet loss, time 9198ms
>
>
>   ip -s -s -d l ls dev mgre0
>     19: mgre0@NONE: <NOARP,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
>         link/gre 192.168.71.177 brd 0.0.0.0 promiscuity 0  allmulti 0 minmtu 0 maxmtu 0
>         gre remote any local 192.168.71.177 ttl inherit ikey 1.9.8.4 okey 1.9.8.4 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536
>         RX:  bytes packets errors dropped  missed   mcast
>                  0       0      0       0       0       0
>         RX errors:  length    crc   frame    fifo overrun
>                          0      0       0       0       0
>         TX:  bytes packets errors dropped carrier collsns
>              28200      30      0      10       0       0
>         TX errors: aborted   fifo  window heartbt transns
>                          0      0       0       0       0
>
>
>   tcpdump -vni mgre0
>     tcpdump: listening on mgre0, link-type LINUX_SLL (Linux cooked v1), snapshot length 262144 bytes
>     22:01:34.176810 IP (tos 0x0, ttl 64, id 40388, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 1, length 1376
>     22:01:34.176830 IP (tos 0x0, ttl 64, id 40388, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:35.183742 IP (tos 0x0, ttl 64, id 40516, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 2, length 1376
>     22:01:35.183765 IP (tos 0x0, ttl 64, id 40516, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:36.207750 IP (tos 0x0, ttl 64, id 40684, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 3, length 1376
>     22:01:36.207774 IP (tos 0x0, ttl 64, id 40684, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:37.230738 IP (tos 0x0, ttl 64, id 41578, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 4, length 1376
>     22:01:37.230756 IP (tos 0x0, ttl 64, id 41578, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:38.254761 IP (tos 0x0, ttl 64, id 42099, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 5, length 1376
>     22:01:38.254789 IP (tos 0x0, ttl 64, id 42099, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:39.278748 IP (tos 0x0, ttl 64, id 42506, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 6, length 1376
>     22:01:39.278771 IP (tos 0x0, ttl 64, id 42506, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:40.302738 IP (tos 0x0, ttl 64, id 42527, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 7, length 1376
>     22:01:40.302754 IP (tos 0x0, ttl 64, id 42527, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:41.326733 IP (tos 0x0, ttl 64, id 42989, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 8, length 1376
>     22:01:41.326749 IP (tos 0x0, ttl 64, id 42989, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:42.350750 IP (tos 0x0, ttl 64, id 43576, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 9, length 1376
>     22:01:42.350773 IP (tos 0x0, ttl 64, id 43576, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:43.374743 IP (tos 0x0, ttl 64, id 44118, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 10, length 1376
>     22:01:43.374762 IP (tos 0x0, ttl 64, id 44118, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>
>
>   tcpdump -vni enp11s0.100 'ip proto 47'
>     tcpdump: listening on enp11s0.100, link-type EN10MB (Ethernet), snapshot length 262144 bytes
>     22:01:34.176825 IP (tos 0x0, ttl 64, id 5066, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 40388, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 1, length 1376
>     22:01:34.176832 IP (tos 0x0, ttl 64, id 5067, offset 0, flags [DF], proto GRE (47), length 56)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
>         IP (tos 0x0, ttl 64, id 40388, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:35.183758 IP (tos 0x0, ttl 64, id 5567, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 40516, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 2, length 1376
>     22:01:35.183768 IP (tos 0x0, ttl 64, id 5568, offset 0, flags [DF], proto GRE (47), length 56)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
>         IP (tos 0x0, ttl 64, id 40516, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:36.207767 IP (tos 0x0, ttl 64, id 5741, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 40684, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 3, length 1376
>     22:01:36.207778 IP (tos 0x0, ttl 64, id 5742, offset 0, flags [DF], proto GRE (47), length 56)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
>         IP (tos 0x0, ttl 64, id 40684, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:37.230751 IP (tos 0x0, ttl 64, id 5785, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 41578, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 4, length 1376
>     22:01:37.230758 IP (tos 0x0, ttl 64, id 5786, offset 0, flags [DF], proto GRE (47), length 56)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
>         IP (tos 0x0, ttl 64, id 41578, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:38.254780 IP (tos 0x0, ttl 64, id 5937, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 42099, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 5, length 1376
>     22:01:38.254795 IP (tos 0x0, ttl 64, id 5938, offset 0, flags [DF], proto GRE (47), length 56)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
>         IP (tos 0x0, ttl 64, id 42099, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:39.278764 IP (tos 0x0, ttl 64, id 6876, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 42506, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 6, length 1376
>     22:01:39.278775 IP (tos 0x0, ttl 64, id 6877, offset 0, flags [DF], proto GRE (47), length 56)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
>         IP (tos 0x0, ttl 64, id 42506, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:40.302749 IP (tos 0x0, ttl 64, id 7410, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 42527, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 7, length 1376
>     22:01:40.302757 IP (tos 0x0, ttl 64, id 7411, offset 0, flags [DF], proto GRE (47), length 56)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
>         IP (tos 0x0, ttl 64, id 42527, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:41.326744 IP (tos 0x0, ttl 64, id 7913, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 42989, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 8, length 1376
>     22:01:41.326753 IP (tos 0x0, ttl 64, id 7914, offset 0, flags [DF], proto GRE (47), length 56)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
>         IP (tos 0x0, ttl 64, id 42989, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:42.350766 IP (tos 0x0, ttl 64, id 8422, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 43576, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 9, length 1376
>     22:01:42.350776 IP (tos 0x0, ttl 64, id 8423, offset 0, flags [DF], proto GRE (47), length 56)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
>         IP (tos 0x0, ttl 64, id 43576, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>     22:01:43.374756 IP (tos 0x0, ttl 64, id 9410, offset 0, flags [DF], proto GRE (47), length 1424)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 1404
>         IP (tos 0x0, ttl 64, id 44118, offset 0, flags [+], proto ICMP (1), length 1396)
>         192.168.13.1 > 192.168.13.2: ICMP echo request, id 54476, seq 10, length 1376
>     22:01:43.374766 IP (tos 0x0, ttl 64, id 9411, offset 0, flags [DF], proto GRE (47), length 56)
>         192.168.71.177 > 192.168.69.50: GREv0, Flags [key present], key=0x1090804, length 36
>         IP (tos 0x0, ttl 64, id 44118, offset 1376, flags [none], proto ICMP (1), length 28)
>         192.168.13.1 > 192.168.13.2: ip-proto-1
>