[PATCH RFC 0/8] Reduce lock contention in the NFS client

Chuck Lever posted 8 patches 3 weeks, 4 days ago
There is a newer version of this series
fs/nfs/inode.c                             | 18 ++++++++-
include/linux/sunrpc/xprt.h                |  8 +++-
kernel/workqueue.c                         |  3 ++
net/sunrpc/sched.c                         | 42 ++++++++++++++------
net/sunrpc/svcsock.c                       |  6 +--
net/sunrpc/xprt.c                          | 63 +++++++++++++++---------------
net/sunrpc/xprtrdma/rpc_rdma.c             | 14 +++----
net/sunrpc/xprtrdma/svc_rdma_backchannel.c |  8 ++--
net/sunrpc/xprtsock.c                      | 18 ++++-----
9 files changed, 111 insertions(+), 69 deletions(-)
[PATCH RFC 0/8] Reduce lock contention in the NFS client
Posted by Chuck Lever 3 weeks, 4 days ago
Under a 4KB NFSv3 workload on 100GbE RDMA, roughly 150 RPC worker
threads drive the client, and lock contention dominates its CPU
profile: up to 53% of non-idle cycles are spent in
native_queued_spin_lock_slowpath.

An earlier RFC put this fix in the workqueue core, falling back
automatically to a finer scope when the configured scope
degenerates to a single pod. Tejun rejected a finer default for
unbound workqueues and suggested two alternatives: a sharded
scope between CACHE and SMT, and a scope change on the NFS
workqueues themselves [1]. The first became the
WQ_AFFN_CACHE_SHARD default, whose 8-core shards still leave a
12-core single-LLC system with only two pools. This series does
the second.

My test harness is transport-limited, so the freed cycles appear as
reduced submission latency variance and lower tail completion
latency rather than higher IOPS.

[1] https://lore.kernel.org/all/aYUVVuIidMpuYy3j@slm.duckdns.org/

---
Chuck Lever (8):
      SUNRPC: Use atomic_t for XID allocation
      SUNRPC: Execute initial async RPC states in caller's context
      SUNRPC: Split recv_lock out of xprt->queue_lock
      Set WQ_SYSFS on key NFS-related workqueues
      workqueue: Export the functions needed for WQ attribute modification
      SUNRPC: Reduce rpciod workqueue contention
      NFS: Reduce nfsiod workqueue contention
      SUNRPC: Reduce xprtiod workqueue contention

 fs/nfs/inode.c                             | 18 ++++++++-
 include/linux/sunrpc/xprt.h                |  8 +++-
 kernel/workqueue.c                         |  3 ++
 net/sunrpc/sched.c                         | 42 ++++++++++++++------
 net/sunrpc/svcsock.c                       |  6 +--
 net/sunrpc/xprt.c                          | 63 +++++++++++++++---------------
 net/sunrpc/xprtrdma/rpc_rdma.c             | 14 +++----
 net/sunrpc/xprtrdma/svc_rdma_backchannel.c |  8 ++--
 net/sunrpc/xprtsock.c                      | 18 ++++-----
 9 files changed, 111 insertions(+), 69 deletions(-)
---
base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
change-id: 20260831-performance-e465e621c1c0

Best regards,
--  
Chuck Lever <cel@kernel.org>