From nobody Fri Jul 24 22:54:52 2026 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C2672206A7 for ; Wed, 22 Jul 2026 18:21:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784744522; cv=none; b=CB302exiNGY4P4soEDMCoK9bKPQmkJ3URsHQcrmVw7CvHhSQlyQMkpSCKeMBh8VUAPR2EzgeguEDO6WZrzshcsUKtBO0JPreA4Abb2DZFkNysBVZJKSfZaHoaSXC/G0MyyhZvGoEtrYLCBkZ8/aPP/+t8Pvq/cha9U6gBynO6Tk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784744522; c=relaxed/simple; bh=P8FWy1tUNZy254/t3Q2SdlHTPBy6/tOA7OG8L5olLgM=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=uUdlfAkGNd26UcJORW71yfSuVjO6fUE7XK50K+qvN6SveBMHP0WVgmHxGLyu60m+Cy8MpRubF2uhOXDLlSRLViPdjFUWobiQhPom0f+NOSGo4YXMt2US/AEFDUle0XScWPbZ8w6hP7DAukLqOlRWflOR6RLgQ0ki2nmb+KnpulA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=truenas.com; spf=pass smtp.mailfrom=truenas.com; dkim=pass (2048-bit key) header.d=truenas.com header.i=@truenas.com header.b=WgISf6CO; arc=none smtp.client-ip=209.85.216.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=truenas.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=truenas.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=truenas.com header.i=@truenas.com header.b="WgISf6CO" Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-381216921aaso12104821a91.1 for ; Wed, 22 Jul 2026 11:21:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=truenas.com; s=google; t=1784744519; x=1785349319; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=6370lSCMfomCtQziBDgTY+lShjBYnGlG2dgy/Dr1HuI=; b=WgISf6COxU57iNyDQNgpFvhLZWdj/Lvt48ZT/YATYf+HScj7kKHJI+LTPxW1KuIlG0 snR1Mc5AT7LCOUNJNCvK178q4zFtTFAd15SaLBYD+idAWTDG7PSdquSUkwMVeslZ3/Pp p/5nAkm9F72v1vf4TrfM3iZ5Ry//VuTa1zo4nFJii/ewI3ejOxq7ct15BsqxtVYbkLKz JKeW7dkpC/G0HT7eIVNcaNE6OwswryLjU2JC4Jkc6DEimR+GFunGvN0APM/VZVIru2cQ Hj/HauwVDs+sMipgs0FQah/mVENyT0dx63Z4mCQPIWRIkNODsQ58aSwSyQsPF4tPO2/2 FV8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784744519; x=1785349319; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=6370lSCMfomCtQziBDgTY+lShjBYnGlG2dgy/Dr1HuI=; b=awiROCFNuqL8XLizZBXtTcmnUu5VG/mQiuIzSH84pZiBVJwrO2HmWMvbIz9odtJcnq 4wm1t8KIgyRV/w4pk0+0ctLgZNabznkWgkQ0C66GX9uB6tSFRuL6BpYNQIFyPmJyR0wW 4wyrNi+0mXbyI+2BLIExsMauvrDBCVdEDJC841v0/7uYhaduy/WDyjT2/Ib6Sl7x5eUk FlF3M8zQM6MKj131KoWzY+2CG5vu0bOKmWG+4JWQ6WIZq6iIjCcXG1agGsyl0cm9AMsX 97sVu0goZ9DZTeELPm+K/BsJTpmQ7lWhCo5BTwZ5Zb+n5bb4nJPVSo1PXKshJ9xjEfcL cV+Q== X-Forwarded-Encrypted: i=1; AHgh+RqFK4fIl/I3UmOQ23eS1hdej9Gt1hhulFgv46vRoV4zSMHbeZ8AyCpPSi4oTCnGUfCEPfcXTxtoiLUFcZI=@vger.kernel.org X-Gm-Message-State: AOJu0Yxt///06VPqdj75OPPj4U+kdEVRnsaDPvFD5EvDyuUkC8eHRQ6e Mkk57dzN1sfZM3r7Q6tUOhvXpNYAuBep2o0V9AwPDWCNQjwSKbFoNT7q0kkjWyETCA== X-Gm-Gg: AR+sD13OOwLo8rD5LUUZ6LxJDDePbQpJ/w0gPVDFfSHuL/4S0mY0L+obil/96oc/klN +VcSMt66k6rJxvKgFpFUGMGbTi05Le0/EU8DifBoiCCzuDVmLgbEK+peBbLnvcP98dLJ+tscfRA RcUFWF3CaxC8NaLcJ1+Zui1rYU8umfOra+T9AbJWFaZ03ofnItBFBGqySEynaUBQAA4i/RCwNya BI5lqPh5cgsUKlZ/VMzelKmVIteSBBXVHTH05WCYhzgZMx0wwVkPSn8DW3aRks5QTDDT/Yki+0r xdfC5nylGZ6w6E4l8xtwO37veUEvSvwcSrmVlVNzvnutObyERZD5doAGn1q5txe6gMLtwzakI5P xlwOQNw4XoWpQlfACDzonX2tinBBugXw6377NAYxpsGE4YmxL4BJRD8AdvWKz9cqXM0+SPw5BZn EyyTNj X-Received: by 2002:a17:90b:1c8e:b0:37f:ed7e:7e42 with SMTP id 98e67ed59e1d1-38e4b410b8bmr23878928a91.14.1784744519428; Wed, 22 Jul 2026 11:21:59 -0700 (PDT) Received: from hamza-PC ([2400:adc1:158:c700:28c8:66ff:fe97:3f21]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-13d130f55c6sm14443009c88.15.2026.07.22.11.21.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jul 2026 11:21:58 -0700 (PDT) From: Ameer Hamza To: cel@kernel.org, jlayton@kernel.org, neil@brown.name, okorniev@redhat.com, Dai.Ngo@oracle.com, tom@talpey.com Cc: trondmy@kernel.org, anna@kernel.org, linux-nfs@vger.kernel.org, linux-kernel@vger.kernel.org, frederic@kernel.org, alexander.motin@truenas.com, caleb.stjohn@truenas.com, Ameer Hamza Subject: [PATCH] SUNRPC: Restore NUMA_NO_NODE for svc thread allocations in global mode Date: Wed, 22 Jul 2026 23:20:11 +0500 Message-ID: <20260722182012.2063936-1-ameer.hamza@truenas.com> X-Mailer: git-send-email 2.53.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Commit d57e43b72bf2 ("SUNRPC: Update svcxdr_init_decode() to call xdr_set_scratch_folio()") changed svc_pool_map_get_node() to return numa_mem_id() instead of NUMA_NO_NODE, because __folio_alloc_node() cannot accept NUMA_NO_NODE. That return value is not equivalent: it is evaluated in the context of the task creating the nfsd threads, once per thread created, and it is passed to kthread_create_on_node() and to the per-thread allocations in svc_prepare_thread(). Since commit d1a89197589c ("kthread: Default affine kthread to its preferred NUMA node"), the node argument of kthread_create_on_node() no longer only places the task structure and stack: a kthread created with a real node id normally affines itself to that node's CPUs when it is first woken to run its thread function. All nfsd threads are typically started together, by one task writing to /proc/fs/nfsd/threads, so under the default pool_mode=3Dglobal each nfsd thread is now affined to the local-memory node of the CPU its creating iteration happened to run on - typically the same node for every thread. The CPUs of the other nodes are then unable to run nfsd at all, and the threads' allocations - svc_rqst structures, page pointer arrays, newly allocated task stacks, and the per-RPC pages allocated at run time - all prefer that one node. Restore the NUMA_NO_NODE behaviour that global mode has had since commit 11fd165c68b7 ("sunrpc: use better NUMA affinities"), and handle NUMA_NO_NODE at the one call site that cannot take it by resolving it to numa_mem_id() there, exactly as alloc_pages_node() did for the scratch page before the conversion. The mapped percpu and pernode branches are unchanged. Unpooled services such as lockd and the NFS client callback service also take this fallback when no percpu or pernode map is active, restoring their thread placement in that case. A bisect of a 2x NFS READ throughput regression between v6.17 and v6.18 converged on d57e43b72bf2. On the affected 4-node server every nfsd thread comes up with its CPU affinity restricted to the CPUs of a single node; with this change the threads are runnable on all CPUs again and the observed regression is resolved. Fixes: d57e43b72bf2 ("SUNRPC: Update svcxdr_init_decode() to call xdr_set_s= cratch_folio()") Signed-off-by: Ameer Hamza Reviewed-by: Jeff Layton --- The pool-mode rework in nfsd-testing already removes global mode, so this fix is based on nfsd-fixes and aimed there and at stable (v6.18+), where the regression ships. Tested on a v6.18-based 4-node server: without this patch all nfsd threads come up affined to one node; with it they are runnable on all CPUs. net/sunrpc/svc.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index ae9ec4bf34f7..16b1ca1a7ae3 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -352,7 +352,7 @@ static int svc_pool_map_get_node(unsigned int pidx) if (m->mode =3D=3D SVC_POOL_PERNODE) return m->pool_to[pidx]; } - return numa_mem_id(); + return NUMA_NO_NODE; } /* * Set the given thread's cpus_allowed mask so that it @@ -711,7 +711,9 @@ svc_prepare_thread(struct svc_serv *serv, struct svc_po= ol *pool, int node) rqstp->rq_server =3D serv; rqstp->rq_pool =3D pool; =20 - rqstp->rq_scratch_folio =3D __folio_alloc_node(GFP_KERNEL, 0, node); + rqstp->rq_scratch_folio =3D __folio_alloc_node(GFP_KERNEL, 0, + node =3D=3D NUMA_NO_NODE ? + numa_mem_id() : node); if (!rqstp->rq_scratch_folio) goto out_enomem; =20 base-commit: 5c4851e4cdfc3204128f88991be64a6e3929d3d0 --=20 2.53.0