| 1 | Subject: [PATCH] SUNRPC: fix shutdown of NFS TCP client socket | ||
|---|---|---|---|
| 2 | |||
| 3 | NFS server Duplicate Request Cache (DRC) algorithms rely on NFS clients | 1 | NFS server Duplicate Request Cache (DRC) algorithms rely on NFS clients |
| 4 | reconnecting using the same local TCP port. Unique NFS operations are | 2 | reconnecting using the same local TCP port. Unique NFS operations are |
| 5 | identified by the per-TCP connection set of XIDs. This prevents file | 3 | identified by the per-TCP connection set of XIDs. This prevents file |
| 6 | corruption when non-idempotent NFS operations are retried. | 4 | corruption when non-idempotent NFS operations are retried. |
| 7 | 5 | ||
| ... | ... | ||
| 14 | the same local port but fails with EADDRNOTAVAIL (99). This forces the | 12 | the same local port but fails with EADDRNOTAVAIL (99). This forces the |
| 15 | socket to use a different local TCP port to reconnect to the remote NFS | 13 | socket to use a different local TCP port to reconnect to the remote NFS |
| 16 | server. | 14 | server. |
| 17 | 15 | ||
| 18 | State Transition and Events: | 16 | State Transition and Events: |
| 19 | TCP_CLOSING(8) | 17 | TCP_CLOSE_WAIT(8) |
| 20 | TCP_LAST_ACK(9) | 18 | TCP_LAST_ACK(9) |
| 21 | connect(fail EADDRNOTAVAIL(99)) | 19 | connect(fail EADDRNOTAVAIL(99)) |
| 22 | TCP_CLOSE(7) | 20 | TCP_CLOSE(7) |
| 23 | bind on new port | 21 | bind on new port |
| 24 | connect success | 22 | connect success |
| 25 | 23 | ||
| 26 | dmesg excerpts showing reconnect switching from local port of 688 to 1014: | 24 | dmesg excerpts showing reconnect switching from TCP local port of 926 to |
| 27 | [590701.200229] NFS: mkdir(0:61/560857152), testQ | 25 | 763 after commit 7c81e6a9d75b: |
| 28 | [590701.200231] NFS call mkdir testQ | 26 | [13354.947854] NFS call mkdir testW |
| 29 | [590701.200259] RPC: xs_tcp_send_request(224) = 0 | ||
| 30 | [590751.883111] RPC: xs_tcp_state_change client 0000000051f4e000... | ||
| 31 | [590751.883146] RPC: state 8 conn 1 dead 0 zapped 1 sk_shutdown 1 | ||
| 32 | [590751.883160] RPC: xs_data_ready... | ||
| 33 | [590751.883232] RPC: xs_tcp_state_change client 0000000051f4e000... | ||
| 34 | [590751.883235] RPC: state 9 conn 0 dead 0 zapped 1 sk_shutdown 3 | ||
| 35 | [590751.883238] RPC: xs_tcp_state_change client 0000000051f4e000... | ||
| 36 | [590751.883239] RPC: state 9 conn 0 dead 0 zapped 1 sk_shutdown 3 | ||
| 37 | [590751.883283] RPC: xs_connect scheduled xprt 0000000051f4e000 | ||
| 38 | [590751.883314] RPC: xs_bind 0.0.0.0:688: ok (0) | ||
| 39 | [590751.883321] RPC: worker connecting xprt 0000000051f4e000 via tcp | ||
| 40 | to 10.101.1.30 (port 2049) | ||
| 41 | [590751.883330] RPC: 0000000051f4e000 connect status 99 connected 0 | ||
| 42 | sock state 7 | ||
| 43 | [590751.883342] RPC: xs_tcp_state_change client 0000000051f4e000... | ||
| 44 | [590751.883343] RPC: state 7 conn 0 dead 0 zapped 1 sk_shutdown 3 | ||
| 45 | [590751.883356] RPC: xs_connect scheduled xprt 0000000051f4e000 | ||
| 46 | [590751.883383] RPC: xs_bind 0.0.0.0:1014: ok (0) | ||
| 47 | [590751.883388] RPC: worker connecting xprt 0000000051f4e000 via tcp | ||
| 48 | to 10.101.1.30 (port 2049) | ||
| 49 | [590751.883420] RPC: 0000000051f4e000 connect status 115 connected 0 | ||
| 50 | sock state 2 | ||
| 51 | ... | 27 | ... |
| 52 | 28 | [13405.654781] RPC: xs_tcp_state_change client 00000000037d0f03... | |
| 29 | [13405.654813] RPC: state 8 conn 1 dead 0 zapped 1 sk_shutdown 1 | ||
| 30 | [13405.654826] RPC: xs_data_ready... | ||
| 31 | [13405.654892] RPC: xs_tcp_state_change client 00000000037d0f03... | ||
| 32 | [13405.654895] RPC: state 9 conn 0 dead 0 zapped 1 sk_shutdown 3 | ||
| 33 | [13405.654899] RPC: xs_tcp_state_change client 00000000037d0f03... | ||
| 34 | [13405.654900] RPC: state 9 conn 0 dead 0 zapped 1 sk_shutdown 3 | ||
| 35 | [13405.654950] RPC: xs_connect scheduled xprt 00000000037d0f03 | ||
| 36 | [13405.654975] RPC: xs_bind 0.0.0.0:926: ok (0) | ||
| 37 | [13405.654980] RPC: worker connecting xprt 00000000037d0f03 via tcp | ||
| 38 | to 10.101.6.228 (port 2049) | ||
| 39 | [13405.654991] RPC: 00000000037d0f03 connect status 99 connected 0 | ||
| 40 | sock state 7 | ||
| 41 | [13405.655001] RPC: xs_tcp_state_change client 00000000037d0f03... | ||
| 42 | [13405.655002] RPC: state 7 conn 0 dead 0 zapped 1 sk_shutdown 3 | ||
| 43 | [13405.655024] RPC: xs_connect scheduled xprt 00000000037d0f03 | ||
| 44 | [13405.655038] RPC: xs_bind 0.0.0.0:763: ok (0) | ||
| 45 | [13405.655041] RPC: worker connecting xprt 00000000037d0f03 via tcp | ||
| 46 | to 10.101.6.228 (port 2049) | ||
| 47 | [13405.655065] RPC: 00000000037d0f03 connect status 115 connected 0 | ||
| 48 | sock state 2 | ||
| 53 | 49 | ||
| 54 | State Transition and Events with patch applied: | 50 | State Transition and Events with patch applied: |
| 55 | TCP_CLOSING(8) | 51 | TCP_CLOSE_WAIT(8) |
| 56 | TCP_LAST_ACK(9) | 52 | TCP_LAST_ACK(9) |
| 57 | TCP_CLOSE(7) | 53 | TCP_CLOSE(7) |
| 58 | connect(reuse of port succeeds) | 54 | connect(reuse of port succeeds) |
| 59 | 55 | ||
| 60 | dmesg excerpts showing reconnect on same port (936): | 56 | dmesg excerpts showing reconnect on same TCP local port of 936 with patch |
| 57 | applied: | ||
| 61 | [ 257.139935] NFS: mkdir(0:59/560857152), testQ | 58 | [ 257.139935] NFS: mkdir(0:59/560857152), testQ |
| 62 | [ 257.139937] NFS call mkdir testQ | 59 | [ 257.139937] NFS call mkdir testQ |
| 63 | ... | 60 | ... |
| 64 | [ 307.822702] RPC: state 8 conn 1 dead 0 zapped 1 sk_shutdown 1 | 61 | [ 307.822702] RPC: state 8 conn 1 dead 0 zapped 1 sk_shutdown 1 |
| 65 | [ 307.822714] RPC: xs_data_ready... | 62 | [ 307.822714] RPC: xs_data_ready... |
| ... | ... | ||
| 84 | [ 314.916292] RPC: worker connecting xprt 00000000ce702f14 via tcp | 81 | [ 314.916292] RPC: worker connecting xprt 00000000ce702f14 via tcp |
| 85 | to 10.101.1.30 (port 2049) | 82 | to 10.101.1.30 (port 2049) |
| 86 | [ 314.916342] RPC: 00000000ce702f14 connect status 115 connected 0 | 83 | [ 314.916342] RPC: 00000000ce702f14 connect status 115 connected 0 |
| 87 | sock state 2 | 84 | sock state 2 |
| 88 | 85 | ||
| 89 | Fixes: 7c81e6a9d75b (SUNRPC: Tweak TCP socket shutdown in the RPC client) | 86 | Fixes: 7c81e6a9d75b ("SUNRPC: Tweak TCP socket shutdown in the RPC client") |
| 90 | Signed-off-by: Siddharth Rajendra Kawar <sikawar@microsoft.com> | 87 | Signed-off-by: Siddharth Rajendra Kawar <sikawar@microsoft.com> |
| 91 | --- | 88 | --- |
| 89 | Changes in v2: | ||
| 90 | - Fixed definition of TCP state 8 in commit message [Trond Myklebust] | ||
| 91 | - Removed redundant closing of socket [Trond Myklebust] | ||
| 92 | - Changed handling of TCP_LAST_ACK | ||
| 92 | diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c | 93 | diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c |
| 93 | index XXXXXXX..XXXXXXX 100644 | 94 | index XXXXXXX..XXXXXXX 100644 |
| 94 | --- a/net/sunrpc/xprtsock.c | 95 | --- a/net/sunrpc/xprtsock.c |
| 95 | +++ b/net/sunrpc/xprtsock.c | 96 | +++ b/net/sunrpc/xprtsock.c |
| 96 | @@ -XXX,XX +XXX,XX @@ static void xs_tcp_shutdown(struct rpc_xprt *xprt) | 97 | @@ -XXX,XX +XXX,XX @@ static void xs_tcp_shutdown(struct rpc_xprt *xprt) |
| 98 | switch (skst) { | ||
| 99 | case TCP_FIN_WAIT1: | ||
| 100 | case TCP_FIN_WAIT2: | ||
| 101 | + case TCP_LAST_ACK: | ||
| 97 | break; | 102 | break; |
| 98 | case TCP_ESTABLISHED: | 103 | case TCP_ESTABLISHED: |
| 99 | case TCP_CLOSE_WAIT: | 104 | case TCP_CLOSE_WAIT: |
| 100 | + case TCP_LAST_ACK: | ||
| 101 | kernel_sock_shutdown(sock, SHUT_RDWR); | ||
| 102 | trace_rpc_socket_shutdown(xprt, sock); | ||
| 103 | break; | diff view generated by jsdifflib |