From nobody Mon May 25 00:55:39 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 690763C0606; Wed, 20 May 2026 09:23:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268993; cv=none; b=Lo3pjOOUWouZxPZsDTW2Bcqeh06JWJs925+NcDOfHwrnujJ3qStB5TWuyp3zP86KMj9ulaIb5MielukW36aShDvs44uKN79W5VDhBMXJi70ydOYOWCrhMbgEO/LbncevOxh+TlJ3/dbDab+xHu+GNTMBylJuRDZvNKSDxAdXdbc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268993; c=relaxed/simple; bh=nbr5dP1YOyyVLUxt7iV3D1/g+/nZYc1UjbTdnhs+sd4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aSFpCSv584xtVsTQtsU2ETWw09EnU9SZBivc6v4khnUKiw3XaTDYXQA415C02SVQ6abSZub2cMeAiWw19tc+jpcZZngXPKTqxGUJ3CL+09ZFDDGG3EPHj9j7pC0h8FA0Yl7fZzmHss4fyQFwhkDk8ch/GHRsxpoeJYnPp9GH7NU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=lrmZy2Q3; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="lrmZy2Q3" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64K09WNM935604; Wed, 20 May 2026 09:22:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=516jMG73Ak/9zh4f1 47ayfk2V9oKc/KnWcfL8ZufVrE=; b=lrmZy2Q3IS6azqc76ITQrTmLacxj0t/iO kssmjktESVxcQ+R1fWgVQHuMKagXgdQ76M6ut2DRGZWOLg6BtuB9PZbU6RwAC16s Og+Xii8hZ5E474gXnrxIzQnPoVZNLf7BMtNl5Tvmkf5pMBkfmR28bzJQKRuFi2b/ XECTJW62Scg8FNgcduyWngW0SSRosOqVNdg0rLE0s3Ym2Vf6cl6gALMMOae71wDx rALgUtW3idFMq3BZdaEZuIeJ6vjvEYAG5cw6yXHZM+GGXQFsBz9ie2C/uve9ze5J aSLpk9FVaGz2AJVUz2Fu6Ey9FqhtaHfioph3tzH3I+zFA5tltqe4A== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h88gbjg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:48 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64K994jx032267; Wed, 20 May 2026 09:22:48 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4e72wq6vwt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:47 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64K9MilN49480052 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 May 2026 09:22:44 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8062620040; Wed, 20 May 2026 09:22:44 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 525C22004B; Wed, 20 May 2026 09:22:44 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 20 May 2026 09:22:44 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v3 1/9] s390/alternatives: Add new ALT_TYPE_PERCPU type Date: Wed, 20 May 2026 11:22:35 +0200 Message-ID: <20260520092243.264847-2-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260520092243.264847-1-hca@linux.ibm.com> References: <20260520092243.264847-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: rdFkMtFyexIomHqUgmwR83tsTiIl8ssw X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIwMDA4NyBTYWx0ZWRfXxTzdQgw2jX5V AnH+WzvDNpkTBtRmmcmEIMz04JJ+YPdtHJW5rlnb7NtuTOjP/5MZOyjgfut10i3Qzba7pfHwkJX dKyIlvszJ3UPmuBTeLdPS2+PtiN8OBsz0TZLxsZmSfEqA7ksbRgHcV7r2fyrJEonnf3fmx5WxuM 2zG/k49+TAzmhiq5mSyBKTrVJh2YibIeNVT7DqibP/CcH/ASDWmAWmTy+sBwtA6C2pueZU2yFOz 1cCGp4f5CN/2OlJbgBXfysCTv+No/2uTstTOnOEzQf7HMWEmle1NnztUBZmelmGNmtoNAPRxMQs fbuYd66AkfZrRr5Nne8AJsrBcQAFwo5vZVCShEf4r85bILa2hjjE/0ZbnYMkWeatHs3LR4wbXTG 6DkR6opaU6DOHgRvTdnDEandzDG2ND07NB59PvScunhnAehY9zlN6eOcPxQsyIS8taJoEszOCFo EwBitfQ+sGMX1dMve9g== X-Proofpoint-GUID: rdFkMtFyexIomHqUgmwR83tsTiIl8ssw X-Authority-Analysis: v=2.4 cv=apyCzyZV c=1 sm=1 tr=0 ts=6a0d7d68 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=V8glGbnc2Ofi9Qvn3v5h:22 a=VnNF1IyMAAAA:8 a=XZhbzCa44lJJqUdH0vgA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-20_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501 impostorscore=0 bulkscore=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605200087 Content-Type: text/plain; charset="utf-8" The upcoming percpu section code uses two mviy instructions to guard the beginning and end of a percpu code section. The first mviy instruction writes the register number, which contains the percpu address to lowcore. This indicates both the beginning of a percpu code section and which register contains the percpu address. During compile time the mviy instruction is generated in a way that its base register contains the percpu register, and the immediate field is zero. This needs to be patched so that the base register is zero, and the immediate field contains the register number. For example 101424: eb 00 23 c0 00 52 mviy 960(%r2),0 needs to be patched to 101424: eb 20 03 c0 00 52 mviy 960(%r0),2 Provide a new ALT_TYPE_PERCPU alternative type which handles this specific instruction patching. In addition it also handles the relocated lowcore case, where the displacement of the mviy instruction has a different value. Signed-off-by: Heiko Carstens --- arch/s390/boot/alternative.c | 7 +++++++ arch/s390/include/asm/alternative.h | 5 +++++ arch/s390/kernel/alternative.c | 25 +++++++++++++++++++++++-- 3 files changed, 35 insertions(+), 2 deletions(-) diff --git a/arch/s390/boot/alternative.c b/arch/s390/boot/alternative.c index 19ea7934b918..ad078a2b1192 100644 --- a/arch/s390/boot/alternative.c +++ b/arch/s390/boot/alternative.c @@ -22,6 +22,9 @@ static void alt_debug_all(int type) case ALT_TYPE_SPEC: alt_debug.spec =3D 1; break; + case ALT_TYPE_PERCPU: + alt_debug.percpu =3D 1; + break; } } =20 @@ -115,6 +118,7 @@ void alt_debug_setup(char *str) alt_debug_all(ALT_TYPE_FACILITY); alt_debug_all(ALT_TYPE_FEATURE); alt_debug_all(ALT_TYPE_SPEC); + alt_debug_all(ALT_TYPE_PERCPU); return; } while (*str) { @@ -130,6 +134,9 @@ void alt_debug_setup(char *str) case ALT_TYPE_SPEC: alt_debug_all(ALT_TYPE_SPEC); break; + case ALT_TYPE_PERCPU: + alt_debug_all(ALT_TYPE_PERCPU); + break; } if (*str !=3D ';') break; diff --git a/arch/s390/include/asm/alternative.h b/arch/s390/include/asm/al= ternative.h index 1c56480def9e..9ca2e49338a2 100644 --- a/arch/s390/include/asm/alternative.h +++ b/arch/s390/include/asm/alternative.h @@ -34,6 +34,7 @@ #define ALT_TYPE_FACILITY 0 #define ALT_TYPE_FEATURE 1 #define ALT_TYPE_SPEC 2 +#define ALT_TYPE_PERCPU 3 =20 #define ALT_DATA_SHIFT 0 #define ALT_TYPE_SHIFT 20 @@ -51,6 +52,10 @@ ALT_TYPE_SPEC << ALT_TYPE_SHIFT | \ (facility) << ALT_DATA_SHIFT) =20 +#define ALT_PERCPU(num) (ALT_CTX_EARLY << ALT_CTX_SHIFT | \ + ALT_TYPE_PERCPU << ALT_TYPE_SHIFT | \ + (num) << ALT_DATA_SHIFT) + #ifndef __ASSEMBLER__ =20 #include diff --git a/arch/s390/kernel/alternative.c b/arch/s390/kernel/alternative.c index 02d04ae621ba..a79a11879c2f 100644 --- a/arch/s390/kernel/alternative.c +++ b/arch/s390/kernel/alternative.c @@ -28,6 +28,7 @@ struct alt_debug { unsigned long facilities[MAX_FACILITY_BIT / BITS_PER_LONG]; unsigned long mfeatures[MAX_MFEATURE_BIT / BITS_PER_LONG]; int spec; + int percpu; }; =20 static struct alt_debug __bootdata_preserved(alt_debug); @@ -48,8 +49,18 @@ static void alternative_dump(u8 *old, u8 *new, unsigned = int len, unsigned int ty a_debug("[%d/%3d] %016lx: %s -> %s\n", type, data, kptr, oinsn, ninsn); } =20 +struct insn_siy { + u64 opc1 : 8; + u64 i2 : 8; + u64 b1 : 4; + u64 dl1 : 12; + u64 dh1 : 8; + u64 opc2 : 8; +} __packed; + void __apply_alternatives(struct alt_instr *start, struct alt_instr *end, = unsigned int ctx) { + struct insn_siy insn_siy; struct alt_debug *d; struct alt_instr *a; bool debug, replace; @@ -63,6 +74,8 @@ void __apply_alternatives(struct alt_instr *start, struct= alt_instr *end, unsign for (a =3D start; a < end; a++) { if (!(a->ctx & ctx)) continue; + old =3D (u8 *)&a->instr_offset + a->instr_offset; + new =3D (u8 *)&a->repl_offset + a->repl_offset; switch (a->type) { case ALT_TYPE_FACILITY: replace =3D test_facility(a->data); @@ -76,14 +89,22 @@ void __apply_alternatives(struct alt_instr *start, stru= ct alt_instr *end, unsign replace =3D nobp_enabled(); debug =3D d->spec; break; + case ALT_TYPE_PERCPU: + replace =3D true; + insn_siy =3D *(struct insn_siy *)old; + if (test_machine_feature(MFEATURE_LOWCORE)) + insn_siy =3D *(struct insn_siy *)new; + insn_siy.i2 =3D insn_siy.b1; + insn_siy.b1 =3D 0; + new =3D (u8 *)&insn_siy; + debug =3D d->percpu; + break; default: replace =3D false; debug =3D false; } if (!replace) continue; - old =3D (u8 *)&a->instr_offset + a->instr_offset; - new =3D (u8 *)&a->repl_offset + a->repl_offset; if (debug) alternative_dump(old, new, a->instrlen, a->type, a->data); s390_kernel_write(old, new, a->instrlen); --=20 2.51.0 From nobody Mon May 25 00:55:39 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A10873C0617; Wed, 20 May 2026 09:23:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268996; cv=none; b=LEeDthiVK8efgjJz8MQWcVbMnWieN1OpFf7Z3bmGlwVWrgIAvuaI8y+H2T8pJ3HHu2tvVnVyBKEgU1Vm70Lh4W7gZinh2VOy0VpDEm/G9/2DkSAT7JfNvWY0++Qjw36VdzEXO8LTSfVozWGoLNXZbyNcshhm0V1TaCZeVf7v/yk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268996; c=relaxed/simple; bh=nw2gXo4BmpZ6cFZINnEtgkzD02rHCkUXLgcbHVrmcR0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=agUxXjA3zIj7xs+9Djl7rAIbP3lfmyKhq+DQQsG5t80z9tcH7TeLlIfF4RWUuKuIAY4ohhI85TX3a3l7/Gxbd/i4xKLbHrbQpswOj5rmUIky9pmGvNM4ei+ctKtk0RsouxoABrWJ9GzAcpdA8mmW2awXhsFdSgaSyIW4ki1G+G0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=mdT9l1nn; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="mdT9l1nn" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64K5E6w61950620; Wed, 20 May 2026 09:22:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=U9xE2zFCeHpTpmk4m McEXIEPakktS28quEpSGwuesuI=; b=mdT9l1nn93Lk4cvaCwSL3GIQkq9Jwhz2+ 1uqnmxBh6NnI0Aquc+fGHb6tWg6dlapyryJmXI/g/FyOXzzcx2upGtYYped7weR7 o4tAJz1iEQLY38xlYzBrs92Bf+y3RJ55tLs/XBwblcSvrFPEeNAz92W4cUXSWpw5 qQ7ic8kutodaZALFO90ZLT8Xs6mODQQ1hzzORug/cbo7QKx1CqFJxyN5s4gc6pBI Un5dzruNpItu05ppQ7gHyARUdOHQdWP2VfvikuWCJgcx9f6l9H2Fm/hMDjwiHi0G H1VRBv2ky5oS1SVOMtMPKduHb/HZ6EqZZbUUp54IJezI9nQWueS/A== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h88gbjh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:49 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64K99Cto022933; Wed, 20 May 2026 09:22:48 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4e754gef61-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:48 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64K9Mink49480060 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 May 2026 09:22:45 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C440F20040; Wed, 20 May 2026 09:22:44 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8651120043; Wed, 20 May 2026 09:22:44 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 20 May 2026 09:22:44 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v3 2/9] s390/percpu: Infrastructure for more efficient this_cpu operations Date: Wed, 20 May 2026 11:22:36 +0200 Message-ID: <20260520092243.264847-3-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260520092243.264847-1-hca@linux.ibm.com> References: <20260520092243.264847-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: wCC1aJX2SNKA2cv1uZ5_2c23375bLO0S X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIwMDA4NyBTYWx0ZWRfX1RJc0GDOrDds ZEFtNnJSojTk2z+cs/wvQWhhTtvW2pJRmQ9Xu3+/0rNEHQOYw3ss/R4qZ7vlDpG/VAjwTW4l13V KQZxbwcekjgDMOWwqetNJSZOlHi8DMgUa4lgaAm0Iqla3zVBTDL6sxhvzbNqqfNhdy1V5cuPG/h kCO9az+W1Xpsaf+jEmMOkbaKrk3bG9+o/Hz5G0S6NfhQ6+duEmCihQX8lxVPIGNtz0tZfw04ewh dIuGhwbmNvvFVX643VoTGyHKjUvnVeNQg9+ZLWuNYPAEwIqApjROR8sssZa6w8dO8Ic2PCsXdv7 cvJVoUOvpwzBdOt//9AuK6QJCQC0FzTR0gxfADXSJya03aOVBivI+FfOM0tw013UlX9AEemlqiA FJTxGSFZk8GifcsQZeu6shFevdjIXkQEYG1sizdDTy9XAs+Ckpo2T3bKcTe95VcWlJGHo/v6JrE 4SRbDGCodag1DaVMOTw== X-Proofpoint-GUID: wCC1aJX2SNKA2cv1uZ5_2c23375bLO0S X-Authority-Analysis: v=2.4 cv=apyCzyZV c=1 sm=1 tr=0 ts=6a0d7d69 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=V8glGbnc2Ofi9Qvn3v5h:22 a=VnNF1IyMAAAA:8 a=f0XghBoHWfAVVSya4KAA:9 a=RwcZUMTHd8_6jkUI:21 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-20_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501 impostorscore=0 bulkscore=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605200087 Content-Type: text/plain; charset="utf-8" With the intended removal of PREEMPT_NONE this_cpu operations based on atomic instructions, guarded with preempt_disable()/preempt_enable() pairs become more expensive: the preempt_disable() / preempt_enable() pairs are not optimized away anymore during compile time. In particular the conditional call to preempt_schedule_notrace() after preempt_enable() adds additional code and register pressure. E.g. this simple C code sequence DEFINE_PER_CPU(long, foo); long bar(long a) { return this_cpu_add_return(foo, a); } generates this code: 11a976: eb af f0 68 00 24 stmg %r10,%r15,104(%r15) 11a97c: b9 04 00 ef lgr %r14,%r15 11a980: b9 04 00 b2 lgr %r11,%r2 11a984: e3 f0 ff c8 ff 71 lay %r15,-56(%r15) 11a98a: e3 e0 f0 98 00 24 stg %r14,152(%r15) 11a990: eb 01 03 a8 00 6a asi 936,1 <- __preem= pt_count_add(1) 11a996: c0 10 00 d2 ac b5 larl %r1,1b70300 <- address= of percpu var 11a9a0: e3 10 23 b8 00 08 ag %r1,952 <- add per= cpu offset 11a9a6: eb ab 10 00 00 e8 laag %r10,%r11,0(%r1) <- atomic = op 11a9ac: eb ff 03 a8 00 6e alsi 936,-1 <- __preem= pt_count_dec_and_test() 11a9b2: a7 54 00 05 jnhe 11a9bc 11a9b6: c0 e5 00 76 d1 bd brasl %r14,ff4d30 11a9bc: b9 e8 b0 2a agrk %r2,%r10,%r11 11a9c0: eb af f0 a0 00 04 lmg %r10,%r15,160(%r15) 11a9c6 07 fe br %r14 Even though the above example is more or less the worst case, since the branch to preempt_schedule_notrace() requires a stackframe, which otherwise wouldn't be necessary, there is also the conditional jnhe branch instruction. Get rid of the conditional branch with the following code sequence: 11a8e6: c0 30 00 d0 c5 0d larl %r3,1b33300 11a8ec: b9 04 00 43 lgr %r4,%r3 11a8f0: eb 00 43 c0 00 52 mviy 960,4 11a8f6: e3 40 03 b8 00 08 ag %r4,952 11a8fc: eb 52 40 00 00 e8 laag %r5,%r2,0(%r4) 11a902: eb 00 03 c0 00 52 mviy 960,0 11a908: b9 08 00 25 agr %r2,%r5 11a90c 07 fe br %r14 The general idea is that this_cpu operations based on atomic instructions are guarded with mviy instructions: - The first mviy instruction writes the register number, which contains the percpu address variable to lowcore. This also indicates that a percpu code section is executed. - The first instruction following the mviy instruction must be the ag instruction which adds the percpu offset to the percpu address register. - Afterwards the atomic percpu operation follows. - Then a second mviy instruction writes a zero to lowcore, which indicates the end of the percpu code section. - In case of an interrupt/exception/nmi the register number which was written to lowcore is copied to the exception frame (pt_regs), and a zero is written to lowcore. - On return to the previous context it is checked if a percpu code section was executed (saved register number not zero), and if the process was migrated to a different cpu. If the percpu offset was already added to the percpu address register (instruction address does _not_ point to the ag instruction) the content of the percpu address register is adjusted so it points to percpu variable of the new cpu. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/entry-percpu.h | 76 ++++++++++++++++++++++++++++ arch/s390/include/asm/lowcore.h | 3 +- arch/s390/include/asm/percpu.h | 54 ++++++++++++++++++++ arch/s390/include/asm/ptrace.h | 2 + arch/s390/kernel/irq.c | 26 +++++++--- arch/s390/kernel/nmi.c | 6 +++ arch/s390/kernel/traps.c | 6 +++ 7 files changed, 165 insertions(+), 8 deletions(-) create mode 100644 arch/s390/include/asm/entry-percpu.h diff --git a/arch/s390/include/asm/entry-percpu.h b/arch/s390/include/asm/e= ntry-percpu.h new file mode 100644 index 000000000000..e25108e773ab --- /dev/null +++ b/arch/s390/include/asm/entry-percpu.h @@ -0,0 +1,76 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef ARCH_S390_ENTRY_PERCPU_H +#define ARCH_S390_ENTRY_PERCPU_H + +#include +#include +#include +#include +#include + +static __always_inline void percpu_entry(struct pt_regs *regs) +{ + struct lowcore *lc =3D get_lowcore(); + + if (user_mode(regs)) + return; + regs->cpu =3D lc->cpu_nr; + regs->percpu_register =3D lc->percpu_register; + lc->percpu_register =3D 0; +} + +static __always_inline bool percpu_code_check(struct pt_regs *regs) +{ + unsigned int insn, disp; + struct kprobe *p; + + if (user_mode(regs) || !regs->percpu_register) + return false; + /* + * Within a percpu code section - check if the percpu base register + * needs to be updated. This is the case if the PSW does not point to + * the ADD instruction within the section. + * - AG %rx,percpu_offset_in_lowcore(%r0,%r0) + * which adds the percpu offset to the percpu base register. + */ + lockdep_assert_preemption_disabled(); +again: + insn =3D READ_ONCE(*(u16 *)psw_bits(regs->psw).ia); + if (unlikely(insn =3D=3D BREAKPOINT_INSTRUCTION)) { + p =3D get_kprobe((void *)psw_bits(regs->psw).ia); + /* + * If the kprobe is concurrently removed on a different CPU + * it might not be found anymore. However text must have + * been restored - try again. + */ + if (!p) + goto again; + insn =3D p->opcode; + } + if ((insn & 0xff0f) !=3D 0xe300) + return false; + disp =3D offsetof(struct lowcore, percpu_offset); + if (machine_has_relocated_lowcore()) + disp +=3D LOWCORE_ALT_ADDRESS; + insn =3D (disp & 0xff000) >> 4 | (disp & 0x00fff) << 16 | 0x8; + if (*(u32 *)(psw_bits(regs->psw).ia + 2) !=3D insn) + return false; + return true; +} + +static __always_inline void percpu_fixup(struct pt_regs *regs) +{ + struct lowcore *lc =3D get_lowcore(); + unsigned char reg; + + reg =3D regs->percpu_register; + lc->percpu_register =3D reg; + /* Check if process has been migrated to a different CPU. */ + if (regs->cpu =3D=3D lc->cpu_nr) + return; + /* Fixup percpu base register */ + regs->gprs[reg] -=3D __per_cpu_offset[regs->cpu]; + regs->gprs[reg] +=3D lc->percpu_offset; +} + +#endif diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcor= e.h index 50ffe75adeb4..cd1ddfdb5d35 100644 --- a/arch/s390/include/asm/lowcore.h +++ b/arch/s390/include/asm/lowcore.h @@ -165,7 +165,8 @@ struct lowcore { __u32 spinlock_index; /* 0x03b0 */ __u8 pad_0x03b4[0x03b8-0x03b4]; /* 0x03b4 */ __u64 percpu_offset; /* 0x03b8 */ - __u8 pad_0x03c0[0x0400-0x03c0]; /* 0x03c0 */ + __u8 percpu_register; /* 0x03c0 */ + __u8 pad_0x03c1[0x0400-0x03c1]; /* 0x03c1 */ =20 __u32 return_lpswe; /* 0x0400 */ __u32 return_mcck_lpswe; /* 0x0404 */ diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index b18a96f3a334..1af622a8aa67 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -60,6 +60,60 @@ #define this_cpu_or_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) #define this_cpu_or_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) =20 +/* + * Macros to be used for percpu code section based on atomic instructions. + * + * Avoid the need to use preempt_disable() / preempt_disable() pairs and t= he + * conditional preempt_schedule_notrace() function calls which come with + * this. The idea is that this_cpu operations based on atomic instructions= are + * guarded with mviy instructions: + * + * - The first mviy instruction writes the register number, which contains= the + * percpu address variable to lowcore. This also indicates that a percpu + * code section is executed. + * + * - The first mviy instruction following the mviy instruction must be the= ag + * instruction which adds the percpu offset to the percpu address regist= er. + * + * - Afterwards the atomic percpu operation follows. + * + * - Then a second mviy instruction writes a zero to lowcore, which indica= tes + * the end of the percpu code section. + * + * - In case of an interrupt/exception/nmi the register number which was + * written to lowcore is copied to the exception frame (pt_regs), and a = zero + * is written to lowcore. + * + * - On return to the previous context it is checked if a percpu code sect= ion + * was executed (saved register number not zero), and if the process was + * migrated to a different cpu. If the percpu offset was already added to + * the percpu address register (instruction address does _not_ point to = the + * ag instruction) the content of the percpu address register is adjuste= d so + * it points to percpu variable of the new cpu. + * + * Inline assemblies making use of this typically have a code sequence lik= e: + * + * MVIY_PERCPU(...) <- start of percpu code section + * AG_ALT(...) <- add percpu offset; must be the second instruction + * atomic_op <- atomic op + * MVIY_ALT(...) <- end of percpu code section + */ + +#define MVIY_PERCPU(disp, dispalt, base) \ + ALTERNATIVE(" mviy " disp "(" base " ),0\n", \ + " mviy " dispalt "(" base " ),0\n", \ + ALT_PERCPU(0)) + +#define MVIY_ALT(disp, dispalt, base) \ + ALTERNATIVE(" mviy " disp "(" base " ),0\n", \ + " mviy " dispalt "(" base " ),0\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) + +#define AG_ALT(disp, dispalt, reg) \ + ALTERNATIVE(" ag " reg ", " disp "(%%r0)\n", \ + " ag " reg ", " dispalt "(%%r0)\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) + #ifndef MARCH_HAS_Z196_FEATURES =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) diff --git a/arch/s390/include/asm/ptrace.h b/arch/s390/include/asm/ptrace.h index aaceb1d9110a..495e310c3d6d 100644 --- a/arch/s390/include/asm/ptrace.h +++ b/arch/s390/include/asm/ptrace.h @@ -134,6 +134,8 @@ struct pt_regs { }; unsigned long flags; unsigned long last_break; + unsigned int cpu; + unsigned char percpu_register; }; =20 /* diff --git a/arch/s390/kernel/irq.c b/arch/s390/kernel/irq.c index d10a17e6531d..92fdc2ae96f8 100644 --- a/arch/s390/kernel/irq.c +++ b/arch/s390/kernel/irq.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "entry.h" =20 DEFINE_PER_CPU_SHARED_ALIGNED(struct irq_stat, irq_stat); @@ -142,10 +143,13 @@ static int irq_pending(struct pt_regs *regs) =20 void noinstr do_io_irq(struct pt_regs *regs) { - irqentry_state_t state =3D irqentry_enter(regs); - struct pt_regs *old_regs =3D set_irq_regs(regs); - bool from_idle; + bool from_idle, percpu_needs_fixup; + struct pt_regs *old_regs; + irqentry_state_t state; =20 + percpu_entry(regs); + state =3D irqentry_enter(regs); + old_regs =3D set_irq_regs(regs); from_idle =3D test_and_clear_cpu_flag(CIF_ENABLED_WAIT); if (from_idle) update_timer_idle(); @@ -170,21 +174,26 @@ void noinstr do_io_irq(struct pt_regs *regs) do_irq_async(regs, IO_INTERRUPT); } while (machine_is_lpar() && irq_pending(regs)); =20 + percpu_needs_fixup =3D percpu_code_check(regs); irq_exit_rcu(); - set_irq_regs(old_regs); irqentry_exit(regs, state); =20 if (from_idle) regs->psw.mask &=3D ~(PSW_MASK_EXT | PSW_MASK_IO | PSW_MASK_WAIT); + if (percpu_needs_fixup) + percpu_fixup(regs); } =20 void noinstr do_ext_irq(struct pt_regs *regs) { - irqentry_state_t state =3D irqentry_enter(regs); - struct pt_regs *old_regs =3D set_irq_regs(regs); - bool from_idle; + bool from_idle, percpu_needs_fixup; + struct pt_regs *old_regs; + irqentry_state_t state; =20 + percpu_entry(regs); + state =3D irqentry_enter(regs); + old_regs =3D set_irq_regs(regs); from_idle =3D test_and_clear_cpu_flag(CIF_ENABLED_WAIT); if (from_idle) update_timer_idle(); @@ -206,12 +215,15 @@ void noinstr do_ext_irq(struct pt_regs *regs) =20 do_irq_async(regs, EXT_INTERRUPT); =20 + percpu_needs_fixup =3D percpu_code_check(regs); irq_exit_rcu(); set_irq_regs(old_regs); irqentry_exit(regs, state); =20 if (from_idle) regs->psw.mask &=3D ~(PSW_MASK_EXT | PSW_MASK_IO | PSW_MASK_WAIT); + if (percpu_needs_fixup) + percpu_fixup(regs); } =20 static void show_msi_interrupt(struct seq_file *p, int irq) diff --git a/arch/s390/kernel/nmi.c b/arch/s390/kernel/nmi.c index 94fbfad49f62..d43cc18fe9be 100644 --- a/arch/s390/kernel/nmi.c +++ b/arch/s390/kernel/nmi.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -363,6 +364,7 @@ NOKPROBE_SYMBOL(s390_backup_mcck_info); */ void notrace s390_do_machine_check(struct pt_regs *regs) { + bool percpu_needs_fixup; static int ipd_count; static DEFINE_SPINLOCK(ipd_lock); static unsigned long long last_ipd; @@ -374,6 +376,7 @@ void notrace s390_do_machine_check(struct pt_regs *regs) unsigned long mcck_dam_code; int mcck_pending =3D 0; =20 + percpu_entry(regs); irq_state =3D irqentry_nmi_enter(regs); =20 if (user_mode(regs)) @@ -495,7 +498,10 @@ void notrace s390_do_machine_check(struct pt_regs *reg= s) if (mcck_pending) schedule_mcck_handler(); =20 + percpu_needs_fixup =3D percpu_code_check(regs); irqentry_nmi_exit(regs, irq_state); + if (percpu_needs_fixup) + percpu_fixup(regs); } NOKPROBE_SYMBOL(s390_do_machine_check); =20 diff --git a/arch/s390/kernel/traps.c b/arch/s390/kernel/traps.c index 1b5c6fc431cc..fb16e9bee80b 100644 --- a/arch/s390/kernel/traps.c +++ b/arch/s390/kernel/traps.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -329,6 +330,7 @@ static void (*pgm_check_table[128])(struct pt_regs *reg= s); void noinstr __do_pgm_check(struct pt_regs *regs) { struct lowcore *lc =3D get_lowcore(); + bool percpu_needs_fixup; irqentry_state_t state; unsigned int trapnr; union teid teid; @@ -349,6 +351,7 @@ void noinstr __do_pgm_check(struct pt_regs *regs) current->thread.gmap_int_code =3D regs->int_code & 0xffff; return; } + percpu_entry(regs); state =3D irqentry_enter(regs); if (user_mode(regs)) { update_timer_sys(); @@ -385,7 +388,10 @@ void noinstr __do_pgm_check(struct pt_regs *regs) pgm_check_table[trapnr](regs); out: local_irq_disable(); + percpu_needs_fixup =3D percpu_code_check(regs); irqentry_exit(regs, state); + if (percpu_needs_fixup) + percpu_fixup(regs); } =20 /* --=20 2.51.0 From nobody Mon May 25 00:55:39 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68DCB3BFE4C; Wed, 20 May 2026 09:23:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268994; cv=none; b=BV4StmmnA9p4duR6vHbilgNfHcwVWhRrdY4F84GlMVpU+WmHpb0UYK2MQMYqaeoXvPIW5J8JmAaVZgeyKswSiPgFZrHCOWNM1/N+DUm0ZfvH5+nrz2xqcUzGIE0g4TE8CBtSEmEgUjwMCGF9aKr4w2R/9cXVE18ikF40UPXF67w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268994; c=relaxed/simple; bh=LK/9+eVDGDlX+wVK9pqJWT+ZFJOaNa7xi51Ag4LlZYg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ge99nuiosD//7Tke7p0rCDERF+wSUqXPmsmIwo/FikkDSh+DK6c2CiSKzPF6WkD1vVTvtDulAfw0aWzb6ExaU3g6NcYXiPKqoUPFgu6egRIubxcfCUHdUMnOwh1ufeFgI6RswCK11X7+uH2C6rbPhGFCr9H4ooQU4gmTJ4Fp9Uo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=sUazi2aU; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="sUazi2aU" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64K2YJ5Z733638; Wed, 20 May 2026 09:22:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=vxdFKml+j9Y8jaQUN SlogzILQtLAndqgJwMrzL8AbWI=; b=sUazi2aUERxEGkPFh8f0sTcKCxd6Boyw0 eD+1pYmeXDFHDceo/GE9JyX4r7NqQlyw3bKsGbPwOUEJmYbBYqUxYEmsOvRi4tvp XPFe31giUKkDaNPHaHW+9NBw4roK3zyDqa56W8d4qrtAGOzxUQvZVVLb8QpXJ7bs TSjyTSK+4Q4esVUDM012gvloOFP+LdWE3L70wzDVP5T5pHVwmRGh15wEMtXPb6lt BCp/675IjmqgBIdnVdyXph3CR2Ftibsa6FzBb+QlprHh92VcO4uUhW6cJYFsokl5 ryZ9P5m5VXEw84HbiiKtKsI62KEtIr3i5E7OitO5jgbioOX3Kx6Og== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6hb8gb2e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:49 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64K99E8i021778; Wed, 20 May 2026 09:22:48 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4e74dhpkup-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:48 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64K9MjM356885606 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 May 2026 09:22:45 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 02A5720043; Wed, 20 May 2026 09:22:45 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CA6F32004B; Wed, 20 May 2026 09:22:44 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 20 May 2026 09:22:44 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v3 3/9] s390/percpu: Add missing do { } while (0) constructs Date: Wed, 20 May 2026 11:22:37 +0200 Message-ID: <20260520092243.264847-4-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260520092243.264847-1-hca@linux.ibm.com> References: <20260520092243.264847-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIwMDA4NyBTYWx0ZWRfX4eH/AtowW13D bRxE9/nZ4crh4tapHrYvSqBzHNhPv5AG1IYVB7DBpuKuEDj24Ao6iGWdoBvAgCK6ZObl6q21muv ljlgAel/IeEKP2YXkqobvJ8Z7XSTBEOhsXMc/Vf/7SPaYfR1IJ9llhRQ6f888icHzueSc1ImP4+ BsbJw38DSBOvulEt7gvIxZupYefmWjYYLHByuQNxma0OJD4H9cuxhgR2YpO+h0qEuOf7f+wrRKs mdBOXlFxUjrJSYbGbBANlNpQFHzc8NKNlVJ2VFMBYCDYdST9t0tIplQAL3iDV6v+x2AQb+YjNGQ Bf7t3RgORfQH2fFaEKsOUmxIx9kJhBpjJkqGdD5oh1H8hPpVoksWBBLbx2bORCcI/2wHLzj2ZfZ tttrc72Q9fyLI2ICyzBdB4l2BMshC0oZeBSjvC8bK0zSYq0Spq69t9d5TC407gZAvj7HzR4uGv5 otZ/+Hk9MSKXx7tGSmA== X-Proofpoint-GUID: VDvhlEaP8o7DTnx5Gc5nR-mYkrpCqZkx X-Proofpoint-ORIG-GUID: VDvhlEaP8o7DTnx5Gc5nR-mYkrpCqZkx X-Authority-Analysis: v=2.4 cv=aYBRWxot c=1 sm=1 tr=0 ts=6a0d7d69 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=c92rfblmAAAA:8 a=VnNF1IyMAAAA:8 a=VwQbUJbxAAAA:8 a=V-yoBbGPs5LSgb0Mvs4A:9 a=GvGzcOZaWPEFPQC_NcjD:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-20_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0 bulkscore=0 suspectscore=0 lowpriorityscore=0 clxscore=1015 spamscore=0 phishscore=0 priorityscore=1501 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605200087 Content-Type: text/plain; charset="utf-8" Add missing do { } while (0) constructs in order to avoid potential build failures. Reported-by: Sashiko Closes: https://sashiko.dev/#/patchset/20260319120503.4046659-1-hca%40linux= .ibm.com Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 1af622a8aa67..c8fc8b320a86 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -128,7 +128,7 @@ #else /* MARCH_HAS_Z196_FEATURES */ =20 #define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ -{ \ +do { \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ @@ -149,7 +149,7 @@ : "cc"); \ } \ preempt_enable_notrace(); \ -} +} while (0) =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_add(pcp, val, "laa", "asi",= int) #define this_cpu_add_8(pcp, val) arch_this_cpu_add(pcp, val, "laag", "agsi= ", long) @@ -174,7 +174,7 @@ #define this_cpu_add_return_8(pcp, val) arch_this_cpu_add_return(pcp, val,= "laag") =20 #define arch_this_cpu_to_op(pcp, val, op) \ -{ \ +do { \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ @@ -186,7 +186,7 @@ : [val__] "d" (val__) \ : "cc"); \ preempt_enable_notrace(); \ -} +} while (0) =20 #define this_cpu_and_4(pcp, val) arch_this_cpu_to_op(pcp, val, "lan") #define this_cpu_and_8(pcp, val) arch_this_cpu_to_op(pcp, val, "lang") --=20 2.51.0 From nobody Mon May 25 00:55:39 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3446834DCC8; Wed, 20 May 2026 09:22:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268981; cv=none; b=kFwRdvqG0+qoMDH9FHZZ9yxFf5s/bEY6VUw7hd4xYCTe98PiI5KT7qBgR5gjIyFAJ8o8nryRuG2TZ1+HnRz+eVNHMnXS6abPUKEm+U0sbMlKET0BP9IoxhTw9GPKTJxjSfRrAhtQClhmhDLcRsCfsY2QVF4DBlw6KMqBP/4dPEk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268981; c=relaxed/simple; bh=5gaQNpuYvawh0wrEaPibXQTlaY2dPhZHPydLMZIsIfU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qOtNMUc1tiY1YCfSyy77lrAecUM3mZjVgvzA4aKBQ87PQdVGie0M8LxP+mxhqdvBiIO2XTz0ABtg3zE6v4qYkGRbDoVZzLoaSsvsI4xMH4c5/jNfpzQjfw7DcJdUab40UOv8uLe1q3MWm4yEkUWuF5zoCoZjTrZ42S57zOpjw0A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=E7kwQH2U; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="E7kwQH2U" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64K6VPBP1839474; Wed, 20 May 2026 09:22:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=7ua+SCOOYqK79bzMR hDsvBmjg4cNCt1vhxkEacZsI3I=; b=E7kwQH2UWnKLuiI8hSNkbQGh3aY9V4t0K UoyFM3/uyULoEI+rI+R61C4zg+fGWks0C0OgYg6Q3YlywQ49eBkTkm9GzoayCmVB /1qvpGzLAo6Wncf4T+R/ojjGsbSbotLGK7CPpeeUqP9vLFWYodE1LxtxKiFpYeEu 0YFM2eMPk1higSpI4X2ukuZb4hVMsdONg8FBjTbftNSkIqApf3xTfgqU2l5JodaL H1XPpFPkwGeA9TIFtiVIWzig/TKiFN8fNlSgijuGdYhFaWSanGKPqrcZo90dWhP5 Hcs7rqNpWqqT5R1rDwHVcXcWX+/BW+fsnKKy22MKEg4azpDV5/+/w== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h8ms95n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:49 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64K99ANB022924; Wed, 20 May 2026 09:22:48 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4e754gef63-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:48 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64K9Mje256885608 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 May 2026 09:22:45 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 36A3120040; Wed, 20 May 2026 09:22:45 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 08FEE2004D; Wed, 20 May 2026 09:22:45 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 20 May 2026 09:22:44 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v3 4/9] s390/percpu: Use new percpu code section for arch_this_cpu_add() Date: Wed, 20 May 2026 11:22:38 +0200 Message-ID: <20260520092243.264847-5-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260520092243.264847-1-hca@linux.ibm.com> References: <20260520092243.264847-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 5fGnuyjwJINd0jsd7YW2t-teeW7EsY69 X-Authority-Analysis: v=2.4 cv=GYMnWwXL c=1 sm=1 tr=0 ts=6a0d7d69 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=VnNF1IyMAAAA:8 a=SujZmW7ZfIunUtho7CIA:9 X-Proofpoint-ORIG-GUID: 5fGnuyjwJINd0jsd7YW2t-teeW7EsY69 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIwMDA4NyBTYWx0ZWRfX8IKYmmfaRxaI 4q7QhmL0ZwZyotL6r7TYQymRCaecWJQ5n2UYecbBmA3SyUi/LvwEDcttr4xO4PBUicKg6I/RVN7 UJBwm6ovmmP/FXBZg1nuPrr5g4XZG4yelhlByqRwTZNHX7GDmIIwGwuyCzDI6wONIExxo/nMVuM LCvLJAHH0VYJcQ0BSD1OXLMdqbAsjfPvoUE8g/DznnHWdrk+dkn24r/i4LwsaYS0FYNF63k/VBQ l7qlKT0O5aGnvrJoDdw59xb3me1KXZO5vMbdG7O99Niyb3dCAPaVbXr9MTQ2FRi7XitjPENJDUZ dQIhq1qL1IWZasPbZTnohUW73Z/+VBs+Ip4zDMBSeMSPXpkOa+KmAcznfuWsykTgJaij8Y57CBX t34xPLUwsxNAWwq+FjKVLj0LKl89Psh4Qq+UGL1XJ9RiHavQJG6o2GKTsonMYHXvay/BHO7p4B6 b1Ali8I0P9otsFpTHKQ== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-20_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 clxscore=1015 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605200087 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_add() to make use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~76kb (defconfig). Also more than 5300 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. With: DEFINE_PER_CPU(long, foo); void bar(long a) { this_cpu_add(foo, a); } Old arch_this_cpu_add() looks like this: 00000000000000c0 : c0: c0 04 00 00 00 00 jgnop c0 c6: eb 01 03 a8 00 6a asi 936,1 cc: c4 18 00 00 00 00 lgrl %r1,cc ce: R_390_GOTENT foo+0x2 d2: e3 10 03 b8 00 08 ag %r1,952 d8: eb 22 10 00 00 e8 laag %r2,%r2,0(%r1) de: eb ff 03 a8 00 6e alsi 936,-1 e4: a7 a4 00 05 jhe ee e8: c0 f4 00 00 00 00 jg e8 ea: R_390_PC32DBL __s390_indirect_jump_r14+0x2 ee: c0 f4 00 00 00 00 jg ee f0: R_390_PLT32DBL preempt_schedule_notrace+0x2 New arch_this_cpu_add() looks like this: 00000000000000c0 : c0: c0 04 00 00 00 00 jgnop c0 c6: c4 38 00 00 00 00 lgrl %r3,c6 c8: R_390_GOTENT foo+0x2 cc: b9 04 00 43 lgr %r4,%r3 d0: eb 00 43 c0 00 52 mviy 960(%r0),4 d6: e3 40 03 b8 00 08 ag %r4,952 dc: eb 52 40 00 00 e8 laag %r5,%r2,0(%r4) e2: eb 00 03 c0 00 52 mviy 960,0 e8: c0 f4 00 00 00 00 jg e8 ea: R_390_PC32DBL __s390_indirect_jump_r14+0x2 Note that the conditional function call is removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 65 ++++++++++++++++++++++------------ 1 file changed, 43 insertions(+), 22 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index c8fc8b320a86..459603c7305a 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -127,28 +127,49 @@ =20 #else /* MARCH_HAS_Z196_FEATURES */ =20 -#define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ -do { \ - typedef typeof(pcp) pcp_op_T__; \ - pcp_op_T__ val__ =3D (val); \ - pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - if (__builtin_constant_p(val__) && \ - ((szcast)val__ > -129) && ((szcast)val__ < 128)) { \ - asm volatile( \ - op2 " %[ptr__],%[val__]" \ - : [ptr__] "+Q" (*ptr__) \ - : [val__] "i" ((szcast)val__) \ - : "cc"); \ - } else { \ - asm volatile( \ - op1 " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ - : "cc"); \ - } \ - preempt_enable_notrace(); \ +#define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ +do { \ + unsigned long lc_pcpr, lc_pcpo; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ val__ =3D (val); \ + pcp_op_T__ old__, *ptr__; \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + if (__builtin_constant_p(val__) && \ + ((szcast)val__ > -129) && ((szcast)val__ < 128)) { \ + asm volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op2 " 0(%[ptr__]),%[val__]\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "i" ((szcast)val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + } else { \ + asm volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op1 " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + } \ } while (0) =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_add(pcp, val, "laa", "asi",= int) --=20 2.51.0 From nobody Mon May 25 00:55:39 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00EE33C1F4F; Wed, 20 May 2026 09:23:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268996; cv=none; b=EnpSmvjrDEUn85yjsNgUZoy5lcY1aBTccoG3tKCVlX8gDrXwTMt2e1SnSL4tsd75PXXkZad1U9jp+rV6UkAUKu/o5ge13xGl0AJPqh8E7M2SvbE4M6ovMUqGzJ1ehNJSIJEF2i1T4YZLHacsF/bk+1GksiiYgFGWhc1MyNb0uQo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268996; c=relaxed/simple; bh=kzT4GgF9TOm1SzSZAp0GUl4LgLUdpgjBPWkgjzhwqmg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MNVUkqnvExZMlnK4p/IukE8k0iVRE7r3rfUb6g6I/0R5XCSwl1O3fhQmtSA/Vw0Xd22mEKg6/+u4T3xTFnzB1/9yVPtI0Y71ddVTx9V2Ok6QHEY2YwhNtwVyS2wdDkWGja6VOfxiiJEsyA53SvMVhR/jh5J/HY9GFNjEZYesxko= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=KG4zIwr3; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="KG4zIwr3" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64JJpaxc069628; Wed, 20 May 2026 09:22:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=aNuC9aZ0O1QMyLAoh 8AbEsJP6D3xOXXADPlZtT+pV30=; b=KG4zIwr3Nmoa9CJ7d2iJ0SVa2sXfy3ZZH RkJuwaX8WAASBK9mZ9bSOPfJEYg9kMmecaCpXgWjUssuV4NnC7kilMzzMcv8yQ4q 2gHZmbHGz9nFzrlodDBQDW9k6iRDPoQOOTNGhmuoGUyxKvB9SN//DElicH6wjCBT an/EF8BP69v03a8XmPa8k3DyrAmiIOuhYKdLt6E10HCRdkKr1CkHiTI8Rbec+xe9 I68yacz6x+Fsxwafkbnpk8sozGrjbDRPQtzVYZ+AUe9Cjk4u4UEca6VbDCqbpjrD zZXMO07kSCOT/Vq82wuGZhwCfEYuTlMmf3uezuzzCeCo0p4lmQ1+g== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h8ms95p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:49 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64K997Hk002414; Wed, 20 May 2026 09:22:49 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4e75ky6dbm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:48 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64K9MjBs55116218 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 May 2026 09:22:45 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6920020040; Wed, 20 May 2026 09:22:45 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3C95520043; Wed, 20 May 2026 09:22:45 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 20 May 2026 09:22:45 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v3 5/9] s390/percpu: Use new percpu code section for arch_this_cpu_add_return() Date: Wed, 20 May 2026 11:22:39 +0200 Message-ID: <20260520092243.264847-6-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260520092243.264847-1-hca@linux.ibm.com> References: <20260520092243.264847-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: vZiFTyv7LifXqyfj1ckzHsmaWIl8VL7O X-Authority-Analysis: v=2.4 cv=GYMnWwXL c=1 sm=1 tr=0 ts=6a0d7d69 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=VnNF1IyMAAAA:8 a=gp5uoB9lSsFOMp1OYIcA:9 X-Proofpoint-ORIG-GUID: vZiFTyv7LifXqyfj1ckzHsmaWIl8VL7O X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIwMDA4NyBTYWx0ZWRfX/mFR7XGklOmF ZZVjaYLtbZOmbfJLKCM3kMnroFR7XnFz5S+oimnrluRxD2ikL48S46WPus1bgmJQtsAWQFShvB7 w1mynhPqz3whbLdnOl9OAT3A0oK2AqvCZcSljyy/hKVIksf0TN1ubB45s9IXXUY2bQ5de5LIbxR fg29iyUEDSHz7xe08V0b7ExlOLz2S+EcJj1byUzQ+mraowV/hPIaBYYZXnL1p+JWTWQSROzx2Xh 0esFqfHTaDokEP9MESWvGOD7+gYTZeNpjQfiwDN83vEJ4bDuDTFt6nJHaqdsV26vLVJtDLTQEuG TVEy6MtmRjvIsaiDf7pVbO893lp5FGf2KQ9Gqph443QHq2qc6lc/e0ax0q46LHCjuThtCq2kRv3 OJBVPLTym7ft+AG3Xx2IxS9EFAbaA8rdSk73eizY+GjynkseCRthzP5TUd7eFK61P8sigzT9UiD Nc9L/bzPS7vaM1vCtyg== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-20_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 clxscore=1015 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605200087 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_add_return() to make use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~4k (defconfig). Also 66 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 459603c7305a..b3da863954c5 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -177,17 +177,29 @@ do { \ =20 #define arch_this_cpu_add_return(pcp, val, op) \ ({ \ + unsigned long lc_pcpr, lc_pcpo; \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - asm volatile( \ - op " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ : "cc"); \ - preempt_enable_notrace(); \ old__ + val__; \ }) =20 --=20 2.51.0 From nobody Mon May 25 00:55:39 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93EED3BF67E; Wed, 20 May 2026 09:22:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268980; cv=none; b=hDVHr61ppaXhKcXT7X/GVgZzLwCmZPilQ9aPvAVakDdcD0O1adAtKRdTRp1Z/UIHUReSuA5Ki2G0WHR5nTST3KcK2Gd4WqFwHe0q9pcV++9WesBX94xJY186TDnFgLFP1clQMRUBUdEigw3X3kU3shOT20sB/ILMTO4zsmI2jt8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268980; c=relaxed/simple; bh=RoxvrAdCNQnqfNn/2BxPuqYQKFNyXqblRZvfi1+sszc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JUXMvRIaOP9PRGtlsBqD5iB7Q9sfaHEIg7sikQ3ImB4Tu3sSbdM6fyNILf3Kfl0plo77mT5ZWjxLl6erDhRV4adzC7uDS9k6zCXG8igfuDWh671OPVH/Noh8lkeaeMTg16XK8tGIanmGX8ylJ9YFtvmW8oAS7aQUO0cqD/Tg2ek= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=lDjHOdOj; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="lDjHOdOj" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64JKBLdI3252227; Wed, 20 May 2026 09:22:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=eRoHlgFcQDL41Y6Qv +2GL67detecv8uwRK4C+x5v6LU=; b=lDjHOdOjdigPUYkPvOeQXTcUAUnwFTfls Vr3jSmH6SCTz70TP1Ab3dvdlcW0EZAW6K0KVRtjxLC4sbjWCse+eRE3ZJBTaEEVt XU01ZC7xpyIb5Qs7ihuY9foO2Kh0L65hJesk17bpq7uJ8tnLu07jLPkSiswWvjhl LX0b3tpL8b1EUcEDnXQ3Aa2lcZ8WRw6NLpAcpR/fEfU/ac0fJ7JqY3Hsk3Vbp04X MvGpS96RDrBDC2SikuttC+ogyp6BGEwz2uxTJDVT/ryHgJUqZYtkqybDkiwruYy3 5Z5nnLn5I8A2P7bNuTH4E9GHt+n7jFyYMhYTkWSHpPq/5EjdNYMew== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6haw8cay-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:49 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64K994k0032267; Wed, 20 May 2026 09:22:49 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4e72wq6vwv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:48 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64K9MjdS55116220 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 May 2026 09:22:45 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9C33520043; Wed, 20 May 2026 09:22:45 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6F5D62004B; Wed, 20 May 2026 09:22:45 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 20 May 2026 09:22:45 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v3 6/9] s390/percpu: Use new percpu code section for arch_this_cpu_[and|or]() Date: Wed, 20 May 2026 11:22:40 +0200 Message-ID: <20260520092243.264847-7-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260520092243.264847-1-hca@linux.ibm.com> References: <20260520092243.264847-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIwMDA4NyBTYWx0ZWRfX7pwQOesG7DSE XKu+Xy2xN4h7La33DO6ZukiV36H9KxfuXexTkp25VXa4SQx5rVtQMCxHyXM9SVtY0sLh+xKSvMm Zwsy3L+ZmONZABaMd7z5uf6fhblYI+UYKzkxaz/TlytCtyM5VNOMOMZvz8uRrzO38CrUsUxmaZ0 liH6MXnzDdksk8zSUCgC/rMY/fJjIDhecMG/CDE4UqXzrJV7g5xhZndS342I6MS4zcLWFX+tx3B 1fQJtqUwa0n07wHLqfOagCj+PBP7a774Vf7GxXQBd6Eu2lVZhh6WKeDf8vvJo+1tW/B1JLSW8Vk r9aa9Kb35z5tspFDoGwUO5CETfo/8FrP5Szd/7C+vfActyWk3Ro0VNMq62tWwDWv/wOkknEM3ZE yyfGsVRNp6u5OoaZXmdKn1Ge5lp9o9u73vHmy+V+oZwtmSlyfa+pHFvCqfhHnt030lVi5tqtw9H 8EnC726M3fNpUgP95Mw== X-Authority-Analysis: v=2.4 cv=Np/htcdJ c=1 sm=1 tr=0 ts=6a0d7d69 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=4dLYTkTKlH5g-JvQF6AA:9 X-Proofpoint-ORIG-GUID: 99llkpKqlZUeiKNSlUI_LsE7omIaqKMU X-Proofpoint-GUID: 99llkpKqlZUeiKNSlUI_LsE7omIaqKMU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-20_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 spamscore=0 clxscore=1015 priorityscore=1501 impostorscore=0 lowpriorityscore=0 suspectscore=0 adultscore=0 phishscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605200087 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_[and|or]() to make use of the new percpu code section infrastructure. There is no user of this_cpu_and() and only one user of this_cpu_or() within the kernel. Therefore this conversion has hardly any effect, and also removes only preempt_schedule_notrace() function call. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index b3da863954c5..fa3dc3de5f8b 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -208,17 +208,29 @@ do { \ =20 #define arch_this_cpu_to_op(pcp, val, op) \ do { \ + unsigned long lc_pcpr, lc_pcpo; \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - asm volatile( \ - op " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ : "cc"); \ - preempt_enable_notrace(); \ } while (0) =20 #define this_cpu_and_4(pcp, val) arch_this_cpu_to_op(pcp, val, "lan") --=20 2.51.0 From nobody Mon May 25 00:55:39 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93C8B3BED66; Wed, 20 May 2026 09:22:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268980; cv=none; b=MzpIZhMNblrE8fAsQJSwSHNVBo5zTNJQMc0cn/fSccyx3j+FPTG35mCF8g6SHXhObx7wQ4fe/dtTRzmFqbp6BtrisDeiPl6el6Eq154KTDbqkyGbqKvP2L+8rvbu/zQmpX4YCKQ0GzZAv5bilUkSPTCkIu02NBUtKgUUH3Wy7Fc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268980; c=relaxed/simple; bh=PPMUfyzHj1wJrKxOQ1wNO8oKi1gXYEX3a83zcx6MgDo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WgrFgBVw6+MCtiAToukpR25eJ++1m+ur3SxEEhni3ct2k0kgjch2RgZvapZgiGMgml9q190b6IGePsERK/FTWVdhAnWgBYCNiEm+nMLyDECjaGdoI5xPvO2Yztq0WG3UsigdUoZ/hL16Q3r3FY2qejZd2emwTI7AfOma7ueG2a0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=hV3NNzzP; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="hV3NNzzP" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64JLADRm3907161; Wed, 20 May 2026 09:22:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=WS1QX18NMJTzeEbjT v10Xg8Ic9yy6JorqopS7BB+yU8=; b=hV3NNzzP8WYwBlo+J58dnedF9HDlPmTLm Itmrki/ps1DtqJT40HgZu2RQykn6kvKlry4k6hrBUG2i6DSE4CcgSgEHLTVOlWPs aK9D3turdGOyN9pxv7mE+/eYepyMUACzQKi/b9CRIRbzGd8dWWxDmdXl2rAdIjHx flHo60kU7/6FwUiRrAYuoPitww9MClt1Q9xsFprhbT/OmoBDS230bNtBnm5Nq4ej ngU6NHWo0j+el4OqQdvRLyZMqbyuyuoCPVyVJwnJApdFrcE0O7J9iPjw+iN9M1R3 TG63tKoE1XuHPfCqWO+7XT8nwlZhDS483IiCbTHf4IkChahodsJ8w== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6hb8gb2f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:49 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64K99ANC022924; Wed, 20 May 2026 09:22:49 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4e754gef64-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:49 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64K9Mj6l55116222 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 May 2026 09:22:45 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CEFF620043; Wed, 20 May 2026 09:22:45 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A1E3120040; Wed, 20 May 2026 09:22:45 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 20 May 2026 09:22:45 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v3 7/9] s390/percpu: Provide arch_this_cpu_read() implementation Date: Wed, 20 May 2026 11:22:41 +0200 Message-ID: <20260520092243.264847-8-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260520092243.264847-1-hca@linux.ibm.com> References: <20260520092243.264847-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIwMDA4NyBTYWx0ZWRfX5be0+wc1KcNS YbOBcwf51DGir+jTRR7srv4fXYVtNsefDoPGLy5l4rJBErNn2V6fY5s7xEOyQdFjg3tTjLQ54fL f5M4sfyAaYyY7LSFMLaFhFlr667qzveY3cQ1jZYklgeNyHmd6CxvvJXQrsgAYyavX+4gN8qV2sH 0lCHRBvuWB0ZgT1bTDczlnc5xa1yqEM338G1jhnj3415XxdT1zbAQmEZn5bnxdslFMsss/lZ58w a0OJ7jxxgS+xAzoj2HsHxwWFA0yRoBvM3tgu5jtCHPrmTpj9LxHUorF5e5bPbmKZxRUVvPc/NWN VA2WJLTi523sgH6ODMCzG/ETkoD4DC0ycRvLKjKJeRgWmFZRi+6qf7t4ncLJ5vINkuHVH/RI+/k qwvphU+NWcMmQ/mpBe9kofdIYpkHjs3gFqO81OAaVt1+TA+YeIX9GI/TO659EU9g32uTYVU/1f6 H7qPRzeX4nsn6uj3nOQ== X-Proofpoint-GUID: PsXHdtYS-ltAz4RKGlL5uocqu2WfP9X5 X-Proofpoint-ORIG-GUID: PsXHdtYS-ltAz4RKGlL5uocqu2WfP9X5 X-Authority-Analysis: v=2.4 cv=aYBRWxot c=1 sm=1 tr=0 ts=6a0d7d6a cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=VnNF1IyMAAAA:8 a=4bOSrtcVyj7iNoH5sBwA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-20_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0 bulkscore=0 suspectscore=0 lowpriorityscore=0 clxscore=1015 spamscore=0 phishscore=0 priorityscore=1501 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605200087 Content-Type: text/plain; charset="utf-8" Provide an s390 specific implementation of arch_this_cpu_read() instead of the generic variant. The generic variant uses preempt_disable() / preempt_enable() pair and READ_ONCE(). Get rid of the preempt_disable() / preempt_enable() pairs by providing an own variant which makes use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~1k (defconfig). Also 87 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index fa3dc3de5f8b..abd774fdd73a 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -240,6 +240,37 @@ do { \ =20 #endif /* MARCH_HAS_Z196_FEATURES */ =20 +#define arch_this_cpu_read(pcp, op) \ +({ \ + unsigned long lc_pcpr, lc_pcpo, res__; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ *ptr__; \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[res__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [res__] "=3D&d" (res__), [ptr__] "+&a" (ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (*ptr__), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + (pcp_op_T__)res__; \ +}) + +#define this_cpu_read_1(pcp) arch_this_cpu_read(pcp, "ic") +#define this_cpu_read_2(pcp) arch_this_cpu_read(pcp, "lh") +#define this_cpu_read_4(pcp) arch_this_cpu_read(pcp, "l") +#define this_cpu_read_8(pcp) arch_this_cpu_read(pcp, "lg") + #define arch_this_cpu_cmpxchg(pcp, oval, nval) \ ({ \ typedef typeof(pcp) pcp_op_T__; \ --=20 2.51.0 From nobody Mon May 25 00:55:39 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A52A3BF67A; Wed, 20 May 2026 09:23:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268982; cv=none; b=YNXel2JmKdXhoLbVNSwW6QGTHjoUA4Jt4nBmxkSvOYqDt/5ByrIUhIgTq+Td9++OkvqjhJqx9H7OWZozYPtYoiin9lITAUOJACeQ8iJSuE/+yyGRvtU8u0lqzKXbqn9agWbwcfXF3ytes4bG0lTnZa+Uo8m/CkZLHvB4TyU4xew= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268982; c=relaxed/simple; bh=NK+6vbofr75aKt/BHup9RBCtvaVedcUrgp4zmxiMUaI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nK7vEh+DXJP0DR1VzCTOEgUhvhuB+tuxJrmXC2bHaews2PHTjmiE6BdUPLOgW2VhuV0r963HTRddRLwXJ6H1KBok8ksnizprsXRsGW06+qBavVcDvYJkH6oWn8OCKEJEHHYI+LKfvCm//B2FO9vAAEFMJSCRs/K7y7O8cYQBQVo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=MtbBDYRm; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="MtbBDYRm" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64K6mYG41522114; Wed, 20 May 2026 09:22:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=8fPvVyXaWPChPzMtg UrBINFyjiKpfEVImi8OFhjKjNg=; b=MtbBDYRmJKFpCFJlkWOH8C2r2aaeAXtXF uiCraUF5/G3JrBOPdpNHzk7XTpMCHiEmolD2ueTmSi6aTM2QSHjyiZ6vjiTkuRZ6 lfshnljLf1RoFKOHtBx/WbjIXxbQffzICxyIBbR91s/BB8m3zv82OFAwnDJMCKhe hBPetOqE5LcZ6EiF8gvk9adqeQ6j1DqhU8tOslmE25HFclblK5zzQvT83zIQHGDF jtaRmh6xpsMhw5fVdRufec5/ATzTuIq7nmFPWsa2K2Kr4ivDnHVT3hR8hq1N2rcO 1UD15TaPWKJUf84+6ND3VJDIG43TiZZAJz3k7MLiX9BLRZQgCIe8Q== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h9y1c1m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:51 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64K99DUl021766; Wed, 20 May 2026 09:22:49 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4e74dhpkur-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:49 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64K9Mkcx15597984 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 May 2026 09:22:46 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0D61A20040; Wed, 20 May 2026 09:22:46 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D52B12004B; Wed, 20 May 2026 09:22:45 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 20 May 2026 09:22:45 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v3 8/9] s390/percpu: Provide arch_this_cpu_write() implementation Date: Wed, 20 May 2026 11:22:42 +0200 Message-ID: <20260520092243.264847-9-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260520092243.264847-1-hca@linux.ibm.com> References: <20260520092243.264847-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIwMDA4NyBTYWx0ZWRfX2HIBWq0WOVWh 1UVbyETMYcDuT7vvKP+uPvpbiNnW08ifeKJ+sOXDbNFm5dWn7M85ByD0cVZVvkzuJe9EOL8VnN+ yEsxyVhYNg8WxRl0gkcuiVHgBf4TZyHGA4wan3nYpBt9qyztjFgWrdqz5ZVZphvInzIFN9rN/qn izrfpyqy+3dOy5Kr6rea/s3qN+9ExMfSNQNOBoejQJ9cR1vMGKtYEsqy+mKK/LlQa5l70FPDxL4 lNGP9+ilJBfjHAH6vNtKe2rEY34v4wpj7pk+h7ABNE2UAaTjuNHNhFZu4/PTBkFnCwjaMvGyBLY Tqyk5h6t7zirnsCkFWniZ2rKKh6iRXyiBbvNVZ4Ybyi3tXRXhtH1aa3APDQZvDPjEN4K+Xj5neh vyYTZpR9RApTZaxIHsFABjUOzAaB8dbnXZ6//D9U3PrnQriYUJC7lnl0z540egFJf+Wl25cYem4 IGrIW7LZ7lrbcs0+hZw== X-Authority-Analysis: v=2.4 cv=BNuDalQG c=1 sm=1 tr=0 ts=6a0d7d6b cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=VnNF1IyMAAAA:8 a=ObmbXjF690NQJU6EaucA:9 X-Proofpoint-ORIG-GUID: xnxRidgd9SIKD7Ohgg76JL6KU12IulxN X-Proofpoint-GUID: xnxRidgd9SIKD7Ohgg76JL6KU12IulxN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-20_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 priorityscore=1501 malwarescore=0 impostorscore=0 suspectscore=0 lowpriorityscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605200087 Content-Type: text/plain; charset="utf-8" Provide an s390 specific implementation of arch_this_cpu_write() instead of the generic variant. The generic variant uses a quite expensive raw_local_irq_save() / raw_local_irq_restore() pair. Get rid of this by providing an own variant which makes use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~1k (defconfig). Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index abd774fdd73a..44867e7ed0df 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -271,6 +271,36 @@ do { \ #define this_cpu_read_4(pcp) arch_this_cpu_read(pcp, "l") #define this_cpu_read_8(pcp) arch_this_cpu_read(pcp, "lg") =20 +#define arch_this_cpu_write(pcp, val, op) \ +do { \ + unsigned long lc_pcpr, lc_pcpo; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ *ptr__, val__ =3D (val); \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [ptr__] "+&a" (ptr__), "=3Dm" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ +} while (0) + +#define this_cpu_write_1(pcp, val) arch_this_cpu_write(pcp, val, "stc") +#define this_cpu_write_2(pcp, val) arch_this_cpu_write(pcp, val, "sth") +#define this_cpu_write_4(pcp, val) arch_this_cpu_write(pcp, val, "st") +#define this_cpu_write_8(pcp, val) arch_this_cpu_write(pcp, val, "stg") + #define arch_this_cpu_cmpxchg(pcp, oval, nval) \ ({ \ typedef typeof(pcp) pcp_op_T__; \ --=20 2.51.0 From nobody Mon May 25 00:55:39 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6CE333BED5C; Wed, 20 May 2026 09:22:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268981; cv=none; b=Uv0zg9w+LsccGfybyIzuEt+NvfzNiLf7hMs3TftUoZ52EaBFcChIauY4QqcP3XrQaa7Vx26zMFl7UWgp+M4KzrjfHnhOCwvcxgg4iGJY91wzfci+iXGQO1mU5bqpIVSVe0aOuNs32GcacmMVgPdav9ISbYdV0o5vKYISAeW5n/w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779268981; c=relaxed/simple; bh=BzayYQmn6krl5L89iOH64pLU+6QjEDmdFo4YUxmhDuQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=i1OJL2A/hxV4DOokkxNkzB5QySR1QTUpAg0x1OmVOwLsHVA1z+nnW5nJJoWaE/SogYwLzmmPIy0VeIYAVoofOYXF/W91/03tE1ZKDZLmHIw0zzM92FZxWTAEpnJL7q8EhqHrlhVrZROJcKDl2mVTQlEzs7da01L6rRaF3xwooyg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=dPFinNop; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="dPFinNop" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64K6k2Hd513145; Wed, 20 May 2026 09:22:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=FvXcslh19P7qgkCvd upIof8+0RzjLylKgqHtlxcqnfI=; b=dPFinNopQRuj+hlQRiUaQexMyBT9uL66Q KmB+FMo23m9aRsMnfiawZ5ujAstUffYduCCBS4JgAmd++vR0ggNa3igKOwlhmmJw V04J6VJlTZDRQz9mr+zkFIMGMNUBTEtIC0i/RI7Lss6VQUYmEqgMYuSIuWydRULt S+H0sotIPaHzTGlVX8Jdic7pE3cDQ2tQxyN6aEs7vsoNfYie57uFIJTKkR2zrYCz pQ0eY2aVD3e3YjVZREkhv/I7QgI8DItnTKCDXu2R1gYAK6kuvHcOifwhzyx7BTKe 9d4tVDx+gVtn+PIrKCvmJSQCm6EraNu3ZBtd1kuS8KiK6So6GojEQ== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h751a35-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:50 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64K99As2022921; Wed, 20 May 2026 09:22:49 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4e754gef67-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 May 2026 09:22:49 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64K9MkO430016176 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 20 May 2026 09:22:46 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4063820040; Wed, 20 May 2026 09:22:46 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1394F20043; Wed, 20 May 2026 09:22:46 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 20 May 2026 09:22:46 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v3 9/9] s390/percpu: Remove one and two byte this_cpu operation implementation Date: Wed, 20 May 2026 11:22:43 +0200 Message-ID: <20260520092243.264847-10-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260520092243.264847-1-hca@linux.ibm.com> References: <20260520092243.264847-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=ffCdDUQF c=1 sm=1 tr=0 ts=6a0d7d6a cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=U7nrCbtTmkRpXpFmAIza:22 a=VnNF1IyMAAAA:8 a=e14ZIUwkndthOOMI8LgA:9 X-Proofpoint-ORIG-GUID: TTftSxFEt0VrS1p77gwSjAo5GlvmhOX8 X-Proofpoint-GUID: TTftSxFEt0VrS1p77gwSjAo5GlvmhOX8 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIwMDA4NyBTYWx0ZWRfX7lN1wnvAceLM tnCyD0VkTZxULFlz7Bi7VHoW6LHrgdoFu1kzlteEF/4zQEoElV2XJIpb6DmkbXXOrQarP/lBuJJ l0rmJqpPA6Q+04wsdPteYrWWFhg3rlC5S/7Hpu9UYFfQBwd/RLeiAlKjCnMF+dQWVfm4WhYuhM9 v50xt291JU+GfyPbG/z3wqyxnAsdr3UnupzfSAvCeUWP7Cye+QQpG1Pc3jaeFMhnqc407byhDU6 6uBVz5+cptKnuq8WRwVvqB2+2AwkklgM/XwezIePgQvOlQyfRe6kkzz2UDpW9DiY0ETLrcNk4oJ VTAXmR25LByW8yXQuLKqt32COckAPCcmme9Y+Ds+H/n2abKfuZAUKtqKQR+KnkAzbCIIbcAizyz GZcQ5mHTh43wSPUI4fOyYwOMXvWgshDR403m1VoT/aBejUhyQW+SudysxBX3TG109M4yQzY9n4B WsDqBxq3q+ioYaRZckQ== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-20_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 spamscore=0 phishscore=0 suspectscore=0 adultscore=0 clxscore=1015 impostorscore=0 lowpriorityscore=0 bulkscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605200087 Content-Type: text/plain; charset="utf-8" There are no one and two byte this_cpu operations within the kernel (defconfig). However even if there would be, the s390 implementation, which uses a cmpxchg loop, generates a very large code sequence due to the lack of native one and two byte cmpxchg instructions. Remove the s390 implementation and use the generic implementation. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 13 ------------- 1 file changed, 13 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 44867e7ed0df..72237cb962c2 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -51,15 +51,6 @@ new__; \ }) =20 -#define this_cpu_add_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) -#define this_cpu_add_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) -#define this_cpu_add_return_1(pcp, val) arch_this_cpu_to_op_simple(pcp, va= l, +) -#define this_cpu_add_return_2(pcp, val) arch_this_cpu_to_op_simple(pcp, va= l, +) -#define this_cpu_and_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, &) -#define this_cpu_and_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, &) -#define this_cpu_or_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) -#define this_cpu_or_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) - /* * Macros to be used for percpu code section based on atomic instructions. * @@ -313,8 +304,6 @@ do { \ ret__; \ }) =20 -#define this_cpu_cmpxchg_1(pcp, oval, nval) arch_this_cpu_cmpxchg(pcp, ova= l, nval) -#define this_cpu_cmpxchg_2(pcp, oval, nval) arch_this_cpu_cmpxchg(pcp, ova= l, nval) #define this_cpu_cmpxchg_4(pcp, oval, nval) arch_this_cpu_cmpxchg(pcp, ova= l, nval) #define this_cpu_cmpxchg_8(pcp, oval, nval) arch_this_cpu_cmpxchg(pcp, ova= l, nval) =20 @@ -345,8 +334,6 @@ do { \ ret__; \ }) =20 -#define this_cpu_xchg_1(pcp, nval) arch_this_cpu_xchg(pcp, nval) -#define this_cpu_xchg_2(pcp, nval) arch_this_cpu_xchg(pcp, nval) #define this_cpu_xchg_4(pcp, nval) arch_this_cpu_xchg(pcp, nval) #define this_cpu_xchg_8(pcp, nval) arch_this_cpu_xchg(pcp, nval) =20 --=20 2.51.0