From nobody Mon Apr 6 10:31:00 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 73B3F3CCFB8; Thu, 19 Mar 2026 12:05:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921916; cv=none; b=s6UsKFNmJLbnWP1KGyGryquzM7fTXHERVGcqYxGO3pgEGHZeGQ7cR2PKXxuwyTzoQbzS62LqKmDO1XK9WJMcLO/NE0wmX66nKYI3Uww2dDV1XlNC198WA6wUtiCROGCKzrtuX/XCVItQPYxltxGNkf/dXXi/42oC/idXuu98XWk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921916; c=relaxed/simple; bh=6GJTxaWwR7SsXFuSvJ9EEx8IeTvptmfJsOIwIwetdhI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cGbVjD5V2XLCMOc8r1zZXq73Z045tUYK3vlkHNBEISjdlcy7sanx5E/QfmRu9mobgZgjp+TznabK5swfIrZS5y4A4p5IeEg11Qu+QlMwJYfjleTmc+EbDqC0GobOFvNsEsNdSAqtSaTUHzvD0aKd/eaF/+ufHYoYVhfysght7/M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=ttlSSEWZ; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="ttlSSEWZ" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62IN5GPX437268; Thu, 19 Mar 2026 12:05:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=xs2bj+fROmAG6N3/N 6RjvdGdbnsG02BdiCjb16hzr3g=; b=ttlSSEWZUohDkZqKICaethVFIDV2iBU2K liINTm516w0XW3/vmicLli3YkOalC8InylCRMrfaBBrs1+LAEm3XwwRTZkBVGnwk eOdiFead3tvMe1vy8nGAYLML6+p6NOygWoTwZgZ1SX4iNGCzqKCca+phxF5zCOuK QzoBKBNN+JzKCaJgmE4X2aBwboB52SLPmpM5bvUHjh898lElc389aNj+Sk/eWJJn hsubd3i5g9P+So226XUEv8Djo9prUxBPH1iCDGFzh7tot+uQyIwjLn+C3XBLbK2l TgwPoGxlg3ikIzaruocBIuru4bOsLMPe61Utlwv765ngoCD6+1F8w== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvx3d60n5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:08 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62J8hlF5004559; Thu, 19 Mar 2026 12:05:07 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwj0sjf83-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:07 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62JC53Uk51053020 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 19 Mar 2026 12:05:03 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A9A3120040; Thu, 19 Mar 2026 12:05:03 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 865242004B; Thu, 19 Mar 2026 12:05:03 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 19 Mar 2026 12:05:03 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v2 1/9] s390/percpu: Provide arch_raw_cpu_ptr() Date: Thu, 19 Mar 2026 13:04:55 +0100 Message-ID: <20260319120503.4046659-2-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260319120503.4046659-1-hca@linux.ibm.com> References: <20260319120503.4046659-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=arO/yCZV c=1 sm=1 tr=0 ts=69bbe674 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=V8glGbnc2Ofi9Qvn3v5h:22 a=VnNF1IyMAAAA:8 a=FJXwAgvRweZ6vCOBEk0A:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE5MDA5NCBTYWx0ZWRfX1uxA2EuyAFxg Nb6uMUnf7PMOAkEzDKHBcRxNbH9RMmnIsB6GBaPUQ/7ijuMTX3aqiHE6/gTc0gjpzTP+ixeKKoh FO9O3wxBVe404SoH++nIgbzS93JMzz5zbfH8dhIE2czoOAdrvbQMiL4zhooZVRlAvWyksk6BJpW MXenMInyUb0P9w49raygF5R9GaQ5n07Wgawn6eNZi1I9u6BJz9q/wtuREanhAXKkCN0ydtSBAR3 tIWBqdeee6lEik5d2QrUarG/9r4Vd5wmJECE8/j0+bgMzmYD/SWx3xb+hRZbebRiDiQ6pd5JHJE Ezy4z2CNlboyEqLWXkmNyZ+mI141x9rjuXdfC5ijJaxvhWMPkkIrXNVZElYVVerhcfFpQYqTMbb /7aytZcT7e/g7XB6tGBRmXWVszvwGcrr3gcxB/TNUUDpkYGveJVgetUms5IYtE7qS+hSwZcmMWj HcIOtgp1VaE+eDPD76g== X-Proofpoint-GUID: B5hlhpo5eTrEX_mBSH3Gt2iDk79EGFPc X-Proofpoint-ORIG-GUID: B5hlhpo5eTrEX_mBSH3Gt2iDk79EGFPc X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-19_01,2026-03-19_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 spamscore=0 lowpriorityscore=0 impostorscore=0 adultscore=0 bulkscore=0 suspectscore=0 malwarescore=0 clxscore=1015 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603190094 Content-Type: text/plain; charset="utf-8" Provide an s390 specific arch_raw_cpu_ptr() implementation which avoids the detour over get_lowcore() to get the lowcore pointer. The inline assembly is implemented with an alternative so that relocated lowcore (percpu offset is at a different address) is handled correctly. This turns code like this 102f78: a7 39 00 00 lghi %r3,0 102f7c: e3 20 33 b8 00 08 ag %r2,952(%r3) which adds the percpu offset to register r2 into a single instruction 102f7c: e3 20 33 b8 00 08 ag %r2,952(%r0) and also avoids the need of a base register, thus reducing register pressure. With defconfig bloat-o-meter -t provides this result: add/remove: 12/26 grow/shrink: 183/3391 up/down: 14880/-41950 (-27070) Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 5899f57f17d1..b18a96f3a334 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -12,6 +12,24 @@ */ #define __my_cpu_offset get_lowcore()->percpu_offset =20 +#define arch_raw_cpu_ptr(_ptr) \ +({ \ + unsigned long lc_percpu, tcp_ptr__; \ + \ + tcp_ptr__ =3D (__force unsigned long)(_ptr); \ + lc_percpu =3D offsetof(struct lowcore, percpu_offset); \ + asm_inline volatile( \ + ALTERNATIVE("ag %[__ptr__],%[offzero](%%r0)\n", \ + "ag %[__ptr__],%[offalt](%%r0)\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) \ + : [__ptr__] "+d" (tcp_ptr__) \ + : [offzero] "i" (lc_percpu), \ + [offalt] "i" (lc_percpu + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + (TYPEOF_UNQUAL(*(_ptr)) __force __kernel *)tcp_ptr__; \ +}) + /* * We use a compare-and-swap loop since that uses less cpu cycles than * disabling and enabling interrupts like the generic variant would do. --=20 2.51.0 From nobody Mon Apr 6 10:31:00 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 490883CFF7B; Thu, 19 Mar 2026 12:05:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921920; cv=none; b=Z2cvVGJ3+KgL71G/kpKc3QL80HCdLk/KHtpDesxgYc92uUjIMWolrRJ7rItkzcwCGX7mIYLeus8scx+yQGZuDS8NqWJbbWzBXDDx3qQa9KWVQxlaz3b8RsMcihtcuyhaPL/h4iKYlEV3P+ijAIn6wyoPNgKAihJRKrpTUTrIrk4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921920; c=relaxed/simple; bh=a6Pk+lB1YjWy8VvHrIJcmLXrklJ4QI1HGjq0wZRfwfo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PSWhWEIwLqU/jTO3E2KPc4DbTduW0wlfh0dE4QsBsdQO3ye17Htg1Bxpbe1HNnEEQmEz9NNX0q9NXIk5zev3yo/ElbEH+lLjW4hTKYWhm+d5lf4zkc5SGh4XJgT1ZkolCBf+bkCzZeZfZjGblTbFZJWpzwiGPdVzzb7GYa0Tpqg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=s92Kwcz8; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="s92Kwcz8" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62ILEObb853168; Thu, 19 Mar 2026 12:05:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=yCw+fBUHhrNy61rDK pKlydSK0YDq4zvqsM5L/Owc8yk=; b=s92Kwcz8EPAD2MrGVexMKaBHQ1/Bx/1NW BscenORwX4Y9lim/6dYufO8Lv7cCNjC1OftpsEmfHkdyIkATTHyqPnC8zDHIJs3W 1+0K5CjUVQA7RQXvLQ2R7YKCLNtFZbyx7wBEfbhg4d/gKEuXolKM4IEnvS9sRU8/ 5lMzfUEEjhbiutEW2xuCA+DRND/165QQBDGLeciTruk5m2AJt2Yx2YuolAh0xLd3 mdfjOavfrjR8Lq3gf6+KBxst5v0IJ0x0mB1y48eU7KRzBKsvyUTLnIFZp74tVF2i b8P7+5hjjjCdKyDE6SPlsdY8LN86uuUx0lFC0S40POezMwINEzF0g== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvyaunyyt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:08 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62JALdRW028471; Thu, 19 Mar 2026 12:05:07 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwmq1j4dg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:07 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62JC53ut51053022 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 19 Mar 2026 12:05:04 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D1CA420040; Thu, 19 Mar 2026 12:05:03 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AEE0220043; Thu, 19 Mar 2026 12:05:03 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 19 Mar 2026 12:05:03 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v2 2/9] s390/alternatives: Add new ALT_TYPE_PERCPU type Date: Thu, 19 Mar 2026 13:04:56 +0100 Message-ID: <20260319120503.4046659-3-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260319120503.4046659-1-hca@linux.ibm.com> References: <20260319120503.4046659-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE5MDA5NCBTYWx0ZWRfX31vthNMHZTD0 yP+GZM1LvDLmufoW/eB21c5tEwIzj3lXfLji/ZOFM731JQvLzmLf8FqDjth3LqWnJEjbAeVptSp DWznBk3OXzavOEfADS6IKxHze74mCbq103SfeEpLAoZ03SpRYF4xGCDHKTYrbggejq3NYMDaBj5 S4OXc+eewd+fdR8tujZBYU+Vw1knpVFQsM+BpSnR5JM1q01TtkJL923mDFDIulG6+uLgEIf1W46 zM7RlXWNVk9QPcHH/zUVuu4gDX2gExaeExt2zeu57BKlSI5cx43RhMbzziWVEsJt3BWNsRx/gvh J+S7vYgefPZXW5KdsWCcNXwFoqkAkkXm2OLqKzVMjNdYykDv9FjS4OQnYkQ/EKKz92pes7cw7QJ 49pATObAzV1+cIp5mKKchfoxQN4mnqMU/4eIV2cBgVLIe/f7/SPZJ4pwaa+vHoECY7Q1NN8u7as AQ8N04RFOwuqSs/2Wvw== X-Authority-Analysis: v=2.4 cv=GIQF0+NK c=1 sm=1 tr=0 ts=69bbe674 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=XZhbzCa44lJJqUdH0vgA:9 X-Proofpoint-ORIG-GUID: TN7SP_IvDvwukJJHyMYXi7TBUfdcxszx X-Proofpoint-GUID: TN7SP_IvDvwukJJHyMYXi7TBUfdcxszx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-19_01,2026-03-19_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 bulkscore=0 lowpriorityscore=0 malwarescore=0 phishscore=0 impostorscore=0 suspectscore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603190094 Content-Type: text/plain; charset="utf-8" The upcoming percpu section code uses two mviy instructions to guard the beginning and end of a percpu code section. The first mviy instruction writes the register number, which contains the percpu address to lowcore. This indicates both the beginning of a percpu code section and which register contains the percpu address. During compile time the mvyi instruction is generated in a way that its base register contains the percpu register, and the immediate field is zero. This needs to be patched so that the base register is zero, and the immediate field contains the register number. For example 101424: eb 00 23 c0 00 52 mviy 960(%r2),0 needs to be patched to 101424: eb 20 03 c0 00 52 mviy 960(%r0),2 Provide a new ALT_TYPE_PERCPU alternative type which handles this specific instruction patching. In addition it also handles the relocated lowcore case, where the displacement of the mviy instruction has a different value. Signed-off-by: Heiko Carstens --- arch/s390/boot/alternative.c | 7 +++++++ arch/s390/include/asm/alternative.h | 5 +++++ arch/s390/kernel/alternative.c | 25 +++++++++++++++++++++++-- 3 files changed, 35 insertions(+), 2 deletions(-) diff --git a/arch/s390/boot/alternative.c b/arch/s390/boot/alternative.c index 19ea7934b918..ad078a2b1192 100644 --- a/arch/s390/boot/alternative.c +++ b/arch/s390/boot/alternative.c @@ -22,6 +22,9 @@ static void alt_debug_all(int type) case ALT_TYPE_SPEC: alt_debug.spec =3D 1; break; + case ALT_TYPE_PERCPU: + alt_debug.percpu =3D 1; + break; } } =20 @@ -115,6 +118,7 @@ void alt_debug_setup(char *str) alt_debug_all(ALT_TYPE_FACILITY); alt_debug_all(ALT_TYPE_FEATURE); alt_debug_all(ALT_TYPE_SPEC); + alt_debug_all(ALT_TYPE_PERCPU); return; } while (*str) { @@ -130,6 +134,9 @@ void alt_debug_setup(char *str) case ALT_TYPE_SPEC: alt_debug_all(ALT_TYPE_SPEC); break; + case ALT_TYPE_PERCPU: + alt_debug_all(ALT_TYPE_PERCPU); + break; } if (*str !=3D ';') break; diff --git a/arch/s390/include/asm/alternative.h b/arch/s390/include/asm/al= ternative.h index 1c56480def9e..9ca2e49338a2 100644 --- a/arch/s390/include/asm/alternative.h +++ b/arch/s390/include/asm/alternative.h @@ -34,6 +34,7 @@ #define ALT_TYPE_FACILITY 0 #define ALT_TYPE_FEATURE 1 #define ALT_TYPE_SPEC 2 +#define ALT_TYPE_PERCPU 3 =20 #define ALT_DATA_SHIFT 0 #define ALT_TYPE_SHIFT 20 @@ -51,6 +52,10 @@ ALT_TYPE_SPEC << ALT_TYPE_SHIFT | \ (facility) << ALT_DATA_SHIFT) =20 +#define ALT_PERCPU(num) (ALT_CTX_EARLY << ALT_CTX_SHIFT | \ + ALT_TYPE_PERCPU << ALT_TYPE_SHIFT | \ + (num) << ALT_DATA_SHIFT) + #ifndef __ASSEMBLER__ =20 #include diff --git a/arch/s390/kernel/alternative.c b/arch/s390/kernel/alternative.c index 02d04ae621ba..a79a11879c2f 100644 --- a/arch/s390/kernel/alternative.c +++ b/arch/s390/kernel/alternative.c @@ -28,6 +28,7 @@ struct alt_debug { unsigned long facilities[MAX_FACILITY_BIT / BITS_PER_LONG]; unsigned long mfeatures[MAX_MFEATURE_BIT / BITS_PER_LONG]; int spec; + int percpu; }; =20 static struct alt_debug __bootdata_preserved(alt_debug); @@ -48,8 +49,18 @@ static void alternative_dump(u8 *old, u8 *new, unsigned = int len, unsigned int ty a_debug("[%d/%3d] %016lx: %s -> %s\n", type, data, kptr, oinsn, ninsn); } =20 +struct insn_siy { + u64 opc1 : 8; + u64 i2 : 8; + u64 b1 : 4; + u64 dl1 : 12; + u64 dh1 : 8; + u64 opc2 : 8; +} __packed; + void __apply_alternatives(struct alt_instr *start, struct alt_instr *end, = unsigned int ctx) { + struct insn_siy insn_siy; struct alt_debug *d; struct alt_instr *a; bool debug, replace; @@ -63,6 +74,8 @@ void __apply_alternatives(struct alt_instr *start, struct= alt_instr *end, unsign for (a =3D start; a < end; a++) { if (!(a->ctx & ctx)) continue; + old =3D (u8 *)&a->instr_offset + a->instr_offset; + new =3D (u8 *)&a->repl_offset + a->repl_offset; switch (a->type) { case ALT_TYPE_FACILITY: replace =3D test_facility(a->data); @@ -76,14 +89,22 @@ void __apply_alternatives(struct alt_instr *start, stru= ct alt_instr *end, unsign replace =3D nobp_enabled(); debug =3D d->spec; break; + case ALT_TYPE_PERCPU: + replace =3D true; + insn_siy =3D *(struct insn_siy *)old; + if (test_machine_feature(MFEATURE_LOWCORE)) + insn_siy =3D *(struct insn_siy *)new; + insn_siy.i2 =3D insn_siy.b1; + insn_siy.b1 =3D 0; + new =3D (u8 *)&insn_siy; + debug =3D d->percpu; + break; default: replace =3D false; debug =3D false; } if (!replace) continue; - old =3D (u8 *)&a->instr_offset + a->instr_offset; - new =3D (u8 *)&a->repl_offset + a->repl_offset; if (debug) alternative_dump(old, new, a->instrlen, a->type, a->data); s390_kernel_write(old, new, a->instrlen); --=20 2.51.0 From nobody Mon Apr 6 10:31:00 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 368C03D1712; Thu, 19 Mar 2026 12:05:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921922; cv=none; b=fNOjU8hMEjsj9rGTglk6XVrkyVXEDBRAVhlRQFvVELgCA8dAzV0TMypN6+wJVR/abKp8VzafAlKNZE2XkmpBVDp0wljQNr6qzxzaeHIlM9GaonSH+PLGzmTJhZu77IPfa3toWwQhOI2FIyaKi5J6iq4gT2sra6VeEYwsFCwaXoM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921922; c=relaxed/simple; bh=U7N05ffFz21ixiq6isG1B9g9dCGcDx772ny1yBdiKcA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qcY2h0trCb2lU99T6HO0ZBfPjLdO53+GjquCoRe8riS0+ERFJxXkeNPGpzPZ48TAeVJqP/E/yJfXHo7jLhdZqoq1QpOLPD8AytjytiXirvQ5RSTefFrPFHjn4cQ39AxQ8unFGlAGxk9RXwhy1k0toYTVK6hmY3BZkNd45r6aEHo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=V8/P1PsE; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="V8/P1PsE" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62INvRuc578436; Thu, 19 Mar 2026 12:05:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=Qw/x8Y9kQvtRGQAZJ d2tqJpz5Vgi1suY+RmIIni1pmU=; b=V8/P1PsEv/XhBZJyYJBHPZLexmk3Bo1Op ring9PWJSBYuZL3zMgjIBJXUnDF2nyGFvr4PYI5yDqlLXkwFN41Kz0UFSECB+woD M5sTKlKKKUdpcAh420YQEtnuDQjRhFjFhEBfieYZW9QtW5sgGdec0XgCgu43H+se QaTb8Hv/Ru75ywuAI5XJbRDIePZ3dLd1gabvUFfaJC3VSiraI5OTvlmrrhRopndi cMyckPX1ebWS4m/o4odwobt6/vAGd490j5o8VhnsYUO1TjBOkmxBD60lXz6nZy6w jemUJNbty7Fe1RdxWJoWTVHNBupNBMLsSYCRnUM6Gmplh96OuEzug== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvw3j652j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:09 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62J8aQmM004585; Thu, 19 Mar 2026 12:05:07 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwj0sjf84-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:07 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62JC542x46596450 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 19 Mar 2026 12:05:04 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0F2D420040; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D77812004B; Thu, 19 Mar 2026 12:05:03 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 19 Mar 2026 12:05:03 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v2 3/9] s390/percpu: Infrastructure for more efficient this_cpu operations Date: Thu, 19 Mar 2026 13:04:57 +0100 Message-ID: <20260319120503.4046659-4-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260319120503.4046659-1-hca@linux.ibm.com> References: <20260319120503.4046659-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: _rEm_YqTCSmwmCkdhYHRkmGbsuVCUGj2 X-Proofpoint-ORIG-GUID: _rEm_YqTCSmwmCkdhYHRkmGbsuVCUGj2 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE5MDA5NCBTYWx0ZWRfX3kbakpEQmyZc wXtjbLw6XOpc1+qJOfTbXoAAXKZhOhVJ7lCRESBgcZ2O24PiH+uSx9UImjiHR8J1XqTnNsEHpSm iWYOvRs7KAo1Xtdi+YieZ+mHx/1uavQqRsFuiYkqpxHOZ51Lf+9flKaA2jAe/Vgd6b8m3pltOQZ DyYmSuX+cURv7IF83+6NxSuKj0Iw2mILPgZdD+TzkXCeVAVPxEa7vRfiW1aM6QBMnvy7aKBwuTy /hqzGVEw3bJ/hB61hYIWaptTW7rndP4ykezOKv1OfEt8NXXE9TGURmEEZkyzWqnzlS2f+xg6mK8 rRPBFjteVn466IVPhXsswPpgP9T3IO01nanP58JH28n27vk4Npfw8k0pLWPUbjCv4IqhvmzkJS1 JjtbiKQWAgn/o7rtzTsfJm9TrjXV9ZMYcQCsDsgvLRrZBEmFAI+rTOik63DDjLwMvHZsyS6+bQD rHIqOJk4NOZiA/Z34NA== X-Authority-Analysis: v=2.4 cv=Hf8ZjyE8 c=1 sm=1 tr=0 ts=69bbe675 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=VnNF1IyMAAAA:8 a=WWXEMnX_Ls2LmYDc0GUA:9 a=RwcZUMTHd8_6jkUI:21 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-19_01,2026-03-19_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 lowpriorityscore=0 impostorscore=0 bulkscore=0 suspectscore=0 priorityscore=1501 spamscore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603190094 Content-Type: text/plain; charset="utf-8" With the intended removal of PREEMPT_NONE this_cpu operations based on atomic instructions, guarded with preempt_disable()/preempt_enable() pairs become more expensive: the preempt_disable() / preempt_enable() pairs are not optimized away anymore during compile time. In particular the conditional call to preempt_schedule_notrace() after preempt_enable() adds additional code and register pressure. E.g. this simple C code sequence DEFINE_PER_CPU(long, foo); long bar(long a) { return this_cpu_add_return(foo, a); } generates this code: 11a976: eb af f0 68 00 24 stmg %r10,%r15,104(%r15) 11a97c: b9 04 00 ef lgr %r14,%r15 11a980: b9 04 00 b2 lgr %r11,%r2 11a984: e3 f0 ff c8 ff 71 lay %r15,-56(%r15) 11a98a: e3 e0 f0 98 00 24 stg %r14,152(%r15) 11a990: eb 01 03 a8 00 6a asi 936,1 <- __preem= pt_count_add(1) 11a996: c0 10 00 d2 ac b5 larl %r1,1b70300 <- address= of percpu var 11a9a0: e3 10 23 b8 00 08 ag %r1,952 <- add per= cpu offset 11a9a6: eb ab 10 00 00 e8 laag %r10,%r11,0(%r1) <- atomic = op 11a9ac: eb ff 03 a8 00 6e alsi 936,-1 <- __preem= pt_count_dec_and_test() 11a9b2: a7 54 00 05 jnhe 11a9bc 11a9b6: c0 e5 00 76 d1 bd brasl %r14,ff4d30 11a9bc: b9 e8 b0 2a agrk %r2,%r10,%r11 11a9c0: eb af f0 a0 00 04 lmg %r10,%r15,160(%r15) 11a9c6 07 fe br %r14 Even though the above example is more or less the worst case, since the branch to preempt_schedule_notrace() requires a stackframe, which otherwise wouldn't be necessary, there is also the conditional jnhe branch instruction. Get rid of the conditional branch with the following code sequence: 11a8e6: c0 30 00 d0 c5 0d larl %r3,1b33300 11a8ec: b9 04 00 43 lgr %r4,%r3 11a8f0: eb 00 43 c0 00 52 mviy 960,4 11a8f6: e3 40 03 b8 00 08 ag %r4,952 11a8fc: eb 52 40 00 00 e8 laag %r5,%r2,0(%r4) 11a902: eb 00 03 c0 00 52 mviy 960,0 11a908: b9 08 00 25 agr %r2,%r5 11a90c 07 fe br %r14 The general idea is that this_cpu operations based on atomic instructions are guarded with mvyi instructions: - The first mvyi instruction writes the register number, which contains the percpu address variable to lowcore. This also indicates that a percpu code section is executed. - The first instruction following the mvyi instruction must be the ag instruction which adds the percpu offset to the percpu address register. - Afterwards the atomic percpu operation follows. - Then a second mvyi instruction writes a zero to lowcore, which indicates the end of the percpu code section. - In case of an interrupt/exception/nmi the register number which was written to lowcore is copied to the exception frame (pt_regs), and a zero is written to lowcore. - On return to the previous context it is checked if a percpu code section was executed (saved register number not zero), and if the process was migrated to a different cpu. If the percpu offset was already added to the percpu address register (instruction address does _not_ point to the ag instruction) the content of the percpu address register is adjusted so it points to percpu variable of the new cpu. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/entry-percpu.h | 57 ++++++++++++++++++++++++++++ arch/s390/include/asm/lowcore.h | 3 +- arch/s390/include/asm/percpu.h | 54 ++++++++++++++++++++++++++ arch/s390/include/asm/ptrace.h | 2 + arch/s390/kernel/irq.c | 17 +++++++-- arch/s390/kernel/nmi.c | 3 ++ arch/s390/kernel/traps.c | 3 ++ 7 files changed, 134 insertions(+), 5 deletions(-) create mode 100644 arch/s390/include/asm/entry-percpu.h diff --git a/arch/s390/include/asm/entry-percpu.h b/arch/s390/include/asm/e= ntry-percpu.h new file mode 100644 index 000000000000..7ccfefdd8162 --- /dev/null +++ b/arch/s390/include/asm/entry-percpu.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef ARCH_S390_ENTRY_PERCPU_H +#define ARCH_S390_ENTRY_PERCPU_H + +#include +#include +#include +#include + +static __always_inline void percpu_entry(struct pt_regs *regs) +{ + struct lowcore *lc =3D get_lowcore(); + + if (user_mode(regs)) + return; + regs->cpu =3D lc->cpu_nr; + regs->percpu_register =3D lc->percpu_register; + lc->percpu_register =3D 0; +} + +static __always_inline void percpu_exit(struct pt_regs *regs) +{ + struct lowcore *lc =3D get_lowcore(); + unsigned int insn, disp; + unsigned char reg; + + if (user_mode(regs)) + return; + reg =3D regs->percpu_register; + if (!reg) + return; + lc->percpu_register =3D reg; + if (regs->cpu =3D=3D lc->cpu_nr) + return; + /* + * Within a percpu code section and process has been migrated to + * a different CPU. Check if the percpu base register needs to be + * updated. This is the case if the PSW does not point to the ADD + * instruction within the section + * - AG %rx,percpu_offset_in_lowcore(%r0,%r0) + * which adds the percpu offset to the percpu base register. + */ + if ((*(u16 *)psw_bits(regs->psw).ia & 0xff0f) !=3D 0xe300) + goto fixup; + disp =3D offsetof(struct lowcore, percpu_offset); + if (machine_has_relocated_lowcore()) + disp +=3D LOWCORE_ALT_ADDRESS; + insn =3D (disp & 0xff000) >> 4 | (disp & 0x00fff) << 16 | 0x8; + if (*(u32 *)(psw_bits(regs->psw).ia + 2) !=3D insn) + return; +fixup: + /* Fixup percpu base register */ + regs->gprs[reg] -=3D __per_cpu_offset[regs->cpu]; + regs->gprs[reg] +=3D lc->percpu_offset; +} + +#endif diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcor= e.h index 50ffe75adeb4..cd1ddfdb5d35 100644 --- a/arch/s390/include/asm/lowcore.h +++ b/arch/s390/include/asm/lowcore.h @@ -165,7 +165,8 @@ struct lowcore { __u32 spinlock_index; /* 0x03b0 */ __u8 pad_0x03b4[0x03b8-0x03b4]; /* 0x03b4 */ __u64 percpu_offset; /* 0x03b8 */ - __u8 pad_0x03c0[0x0400-0x03c0]; /* 0x03c0 */ + __u8 percpu_register; /* 0x03c0 */ + __u8 pad_0x03c1[0x0400-0x03c1]; /* 0x03c1 */ =20 __u32 return_lpswe; /* 0x0400 */ __u32 return_mcck_lpswe; /* 0x0404 */ diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index b18a96f3a334..05eb91428b42 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -60,6 +60,60 @@ #define this_cpu_or_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) #define this_cpu_or_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) =20 +/* + * Macros to be used for percpu code section based on atomic instructions. + * + * Avoid the need to use preempt_disable() / preempt_disable() pairs and t= he + * conditional preempt_schedule_notrace() function calls which come with + * this. The idea is that this_cpu operations based on atomic instructions= are + * guarded with mvyi instructions: + * + * - The first mvyi instruction writes the register number, which contains= the + * percpu address variable to lowcore. This also indicates that a percpu + * code section is executed. + * + * - The first mvyi instruction following the mvyi instruction must be the= ag + * instruction which adds the percpu offset to the percpu address regist= er. + * + * - Afterwards the atomic percpu operation follows. + * + * - Then a second mvyi instruction writes a zero to lowcore, which indica= tes + * the end of the percpu code section. + * + * - In case of an interrupt/exception/nmi the register number which was + * written to lowcore is copied to the exception frame (pt_regs), and a = zero + * is written to lowcore. + * + * - On return to the previous context it is checked if a percpu code sect= ion + * was executed (saved register number not zero), and if the process was + * migrated to a different cpu. If the percpu offset was already added to + * the percpu address register (instruction address does _not_ point to = the + * ag instruction) the content of the percpu address register is adjuste= d so + * it points to percpu variable of the new cpu. + * + * Inline assemblies making use of this typically have a code sequence lik= e: + * + * MVIY_PERCPU(...) <- start of percpu code section + * AG_ALT(...) <- add percpu offset; must be the second instruction + * atomic_op <- atomic op + * MVIY_ALT(...) <- end of percpu code section + */ + +#define MVIY_PERCPU(disp, dispalt, base) \ + ALTERNATIVE(" mviy " disp "(" base " ),0\n", \ + " mviy " dispalt "(" base " ),0\n", \ + ALT_PERCPU(0)) + +#define MVIY_ALT(disp, dispalt, base) \ + ALTERNATIVE(" mviy " disp "(" base " ),0\n", \ + " mviy " dispalt "(" base " ),0\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) + +#define AG_ALT(disp, dispalt, reg) \ + ALTERNATIVE(" ag " reg ", " disp "(%%r0)\n", \ + " ag " reg ", " dispalt "(%%r0)\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) + #ifndef MARCH_HAS_Z196_FEATURES =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) diff --git a/arch/s390/include/asm/ptrace.h b/arch/s390/include/asm/ptrace.h index aaceb1d9110a..495e310c3d6d 100644 --- a/arch/s390/include/asm/ptrace.h +++ b/arch/s390/include/asm/ptrace.h @@ -134,6 +134,8 @@ struct pt_regs { }; unsigned long flags; unsigned long last_break; + unsigned int cpu; + unsigned char percpu_register; }; =20 /* diff --git a/arch/s390/kernel/irq.c b/arch/s390/kernel/irq.c index d10a17e6531d..43c92b8faade 100644 --- a/arch/s390/kernel/irq.c +++ b/arch/s390/kernel/irq.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "entry.h" =20 DEFINE_PER_CPU_SHARED_ALIGNED(struct irq_stat, irq_stat); @@ -142,10 +143,13 @@ static int irq_pending(struct pt_regs *regs) =20 void noinstr do_io_irq(struct pt_regs *regs) { - irqentry_state_t state =3D irqentry_enter(regs); - struct pt_regs *old_regs =3D set_irq_regs(regs); + struct pt_regs *old_regs; + irqentry_state_t state; bool from_idle; =20 + percpu_entry(regs); + state =3D irqentry_enter(regs); + old_regs =3D set_irq_regs(regs); from_idle =3D test_and_clear_cpu_flag(CIF_ENABLED_WAIT); if (from_idle) update_timer_idle(); @@ -177,14 +181,18 @@ void noinstr do_io_irq(struct pt_regs *regs) =20 if (from_idle) regs->psw.mask &=3D ~(PSW_MASK_EXT | PSW_MASK_IO | PSW_MASK_WAIT); + percpu_exit(regs); } =20 void noinstr do_ext_irq(struct pt_regs *regs) { - irqentry_state_t state =3D irqentry_enter(regs); - struct pt_regs *old_regs =3D set_irq_regs(regs); + struct pt_regs *old_regs; + irqentry_state_t state; bool from_idle; =20 + percpu_entry(regs); + state =3D irqentry_enter(regs); + old_regs =3D set_irq_regs(regs); from_idle =3D test_and_clear_cpu_flag(CIF_ENABLED_WAIT); if (from_idle) update_timer_idle(); @@ -212,6 +220,7 @@ void noinstr do_ext_irq(struct pt_regs *regs) =20 if (from_idle) regs->psw.mask &=3D ~(PSW_MASK_EXT | PSW_MASK_IO | PSW_MASK_WAIT); + percpu_exit(regs); } =20 static void show_msi_interrupt(struct seq_file *p, int irq) diff --git a/arch/s390/kernel/nmi.c b/arch/s390/kernel/nmi.c index a55abbf65333..20fd319a3a8e 100644 --- a/arch/s390/kernel/nmi.c +++ b/arch/s390/kernel/nmi.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -374,6 +375,7 @@ void notrace s390_do_machine_check(struct pt_regs *regs) unsigned long mcck_dam_code; int mcck_pending =3D 0; =20 + percpu_entry(regs); irq_state =3D irqentry_nmi_enter(regs); =20 if (user_mode(regs)) @@ -496,6 +498,7 @@ void notrace s390_do_machine_check(struct pt_regs *regs) schedule_mcck_handler(); =20 irqentry_nmi_exit(regs, irq_state); + percpu_exit(regs); } NOKPROBE_SYMBOL(s390_do_machine_check); =20 diff --git a/arch/s390/kernel/traps.c b/arch/s390/kernel/traps.c index 1b5c6fc431cc..095937c2ca72 100644 --- a/arch/s390/kernel/traps.c +++ b/arch/s390/kernel/traps.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -349,6 +350,7 @@ void noinstr __do_pgm_check(struct pt_regs *regs) current->thread.gmap_int_code =3D regs->int_code & 0xffff; return; } + percpu_entry(regs); state =3D irqentry_enter(regs); if (user_mode(regs)) { update_timer_sys(); @@ -386,6 +388,7 @@ void noinstr __do_pgm_check(struct pt_regs *regs) out: local_irq_disable(); irqentry_exit(regs, state); + percpu_exit(regs); } =20 /* --=20 2.51.0 From nobody Mon Apr 6 10:31:00 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFF823CCFCC; Thu, 19 Mar 2026 12:05:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921918; cv=none; b=LCQ67KkNLYFSb9o9MHfGnkJt63W4puEDEtNSOUQlrGvYM0A+UNRzTpMexmoPFHvoqlsEgUgJKpPCodr8FRuiSD3mOp7YGipVU2kFhlHcpohZMZdHZD3c4luV0vVJH5AYrJVJjI5VqeOvmPiPgAhQKN88oe1c91xUtXllPC+vdUk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921918; c=relaxed/simple; bh=4/SQBSS8Itbjkdjiu7TQUdqMdqIBwjIyMhd0dKJy8ZE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ilbPQT0qoHbPuclWAqhW2y09QOyHa1M+2UHBWRFjq9jk//xF6voU9VehDPjClUSRbQb9A5AuMYtftvqx7i6p/wg0Pcdr8sCxQ8H3hNQ8ZdalLm7+2W2SBTTPBrWdTmcQBnzkiAHZZiNFO4Okgj+eS9YcQ3Phdfol3tqlNk0dmd0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Ab04MI7a; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Ab04MI7a" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62INMsDd753346; Thu, 19 Mar 2026 12:05:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=/8Rr5nt+h+JSZbITk HB/FWQiemw5Hn5dCzjOLpsxcrQ=; b=Ab04MI7ann2UGjruto1rdBqIypytq0C5B oq9vKotOLWhejB1YyUIm51YKAQx24OawChmoJz/RxtKTAWRE+EEY4iA60Y9Gc4F6 jTsuWyozeF7OGTpwJfAaHpG3mQJSJOWlnO1xP3Q/U9OPwh90R/ePu/3Gz9hjDxuW 0W+N44V9ech59mUQ/391EIK6CNsqJkVB9IV/odzCbJ1p0qEvyirG2hpg5fhRYk+b 8zkdrVWZzYYKwqSuhwWD54YM8zpGeLRNcfMq2ZBbsRvM43OtFZxrb7y7KTcPenb6 pvM1Xx6QCboqRlOttGlXncWcmoesoY3rnwFe6+hickDMNQLVwRoqQ== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvw3j652k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:08 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62J9Tctt015644; Thu, 19 Mar 2026 12:05:08 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4cwk0njb3s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:08 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62JC54Zp46596452 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 19 Mar 2026 12:05:04 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 370062004B; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1459B20043; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v2 4/9] s390/percpu: Use new percpu code section for arch_this_cpu_add() Date: Thu, 19 Mar 2026 13:04:58 +0100 Message-ID: <20260319120503.4046659-5-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260319120503.4046659-1-hca@linux.ibm.com> References: <20260319120503.4046659-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: dFfITtoFiG-2wNVHgkPN9CpOv8rBStbx X-Proofpoint-ORIG-GUID: dFfITtoFiG-2wNVHgkPN9CpOv8rBStbx X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE5MDA5NCBTYWx0ZWRfXxixSp/4wNlGf vINae4NNjm5LOj9bC126OMHLK5Fb7GO/FUpFWB2vVtr8DTEpHVpZljxJu2P74iJDpHA0MPp7kYr vw1G8zlBXrHNsB9GCCb21oUehrpE1RgF9w2pgo5/n0I3CCwkrbK9VuoCQvpzfFYe31RADbOuPiP 8+oYCzA41GTQ6cl9z+/Gmo+om/7+GNlCVKZdESzoFqdm+Tb9gOFgs+AbN1MAMXf5F/2E4NCDWvp JFfEapCJ6aRK/kdAn9QKs1Edq/9TfhEJgmspV75lMTR3gGhxZEY/9dXwmuIpDK9Mydf4zU3TXlE 5Y6MQn0ZDScCdTiDHQiT+GMKNBLZffkmax/JvSjgxh4B6HaiuqhCliBbE08Bf0tSpPQt+1PWDy+ LEIUykHshemfo0agFtqHbNnVANfyJbJFweBjbmt09yEz4IqICvstDT8gKN38m0QGGNtl36Asgpg 8wMpoJObcWSKRr1JBmw== X-Authority-Analysis: v=2.4 cv=Hf8ZjyE8 c=1 sm=1 tr=0 ts=69bbe674 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=VnNF1IyMAAAA:8 a=SujZmW7ZfIunUtho7CIA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-19_01,2026-03-19_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 lowpriorityscore=0 impostorscore=0 bulkscore=0 suspectscore=0 priorityscore=1501 spamscore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603190094 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_add() to make use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~76kb (defconfig). Also more than 5300 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. With: DEFINE_PER_CPU(long, foo); void bar(long a) { this_cpu_add(foo, a); } Old arch_this_cpu_add() looks like this: 00000000000000c0 : c0: c0 04 00 00 00 00 jgnop c0 c6: eb 01 03 a8 00 6a asi 936,1 cc: c4 18 00 00 00 00 lgrl %r1,cc ce: R_390_GOTENT foo+0x2 d2: e3 10 03 b8 00 08 ag %r1,952 d8: eb 22 10 00 00 e8 laag %r2,%r2,0(%r1) de: eb ff 03 a8 00 6e alsi 936,-1 e4: a7 a4 00 05 jhe ee e8: c0 f4 00 00 00 00 jg e8 ea: R_390_PC32DBL __s390_indirect_jump_r14+0x2 ee: c0 f4 00 00 00 00 jg ee f0: R_390_PLT32DBL preempt_schedule_notrace+0x2 New arch_this_cpu_add() looks like this: 00000000000000c0 : c0: c0 04 00 00 00 00 jgnop c0 c6: c4 38 00 00 00 00 lgrl %r3,c6 c8: R_390_GOTENT foo+0x2 cc: b9 04 00 43 lgr %r4,%r3 d0: eb 00 43 c0 00 52 mviy 960(%r4),0 d6: e3 40 03 b8 00 08 ag %r4,952 dc: eb 52 40 00 00 e8 laag %r5,%r2,0(%r4) e2: eb 00 03 c0 00 52 mviy 960,0 e8: c0 f4 00 00 00 00 jg e8 ea: R_390_PC32DBL __s390_indirect_jump_r14+0x2 Note that the conditional function call is removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 65 ++++++++++++++++++++++------------ 1 file changed, 43 insertions(+), 22 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 05eb91428b42..3c5364475e3e 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -127,28 +127,49 @@ =20 #else /* MARCH_HAS_Z196_FEATURES */ =20 -#define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ -{ \ - typedef typeof(pcp) pcp_op_T__; \ - pcp_op_T__ val__ =3D (val); \ - pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - if (__builtin_constant_p(val__) && \ - ((szcast)val__ > -129) && ((szcast)val__ < 128)) { \ - asm volatile( \ - op2 " %[ptr__],%[val__]" \ - : [ptr__] "+Q" (*ptr__) \ - : [val__] "i" ((szcast)val__) \ - : "cc"); \ - } else { \ - asm volatile( \ - op1 " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ - : "cc"); \ - } \ - preempt_enable_notrace(); \ +#define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ +{ \ + unsigned long lc_pcpr, lc_pcpo; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ val__ =3D (val); \ + pcp_op_T__ old__, *ptr__; \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + if (__builtin_constant_p(val__) && \ + ((szcast)val__ > -129) && ((szcast)val__ < 128)) { \ + asm volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op2 " 0(%[ptr__]),%[val__]\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "i" ((szcast)val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + } else { \ + asm volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op1 " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + } \ } =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_add(pcp, val, "laa", "asi",= int) --=20 2.51.0 From nobody Mon Apr 6 10:31:00 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C414F38F95E; Thu, 19 Mar 2026 12:05:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921919; cv=none; b=pL5Uiu2jKts4L4+1Gwuwhy0qlQVya7DCbq0hiHxwLU/yQPUUuVCL1aNLL8j2Ebm8+cEfna5QaNDSf8XkKnzWHSD0+kqVnMVQuo3/a1NHP6IDEH2rdy7f1I6o8s0LMCIxNYpz1474Za4XuuCJRA1qVYqszEFNzptpO/AAVrIRf6I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921919; c=relaxed/simple; bh=vXtUTowqziaicsjtIv7lvPZpUwK6soubRPCCqW1azRo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tbdu942eSkt0S9XoMBIgJaQxqFZ7UlZA0MLOdf/dfGk6iqS0xUSrlzHv0bG5fFINUQ4FuBJ4HOa8MvgrqR0MCFqYWLl1dTK0705uEz2kryUWqp+Y6fXY18YxAYNthCgWMbCJoAdH/wzrHgofkQWIjsvp3AfKW+Tslsi49R3DJgg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=ifYuhh76; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="ifYuhh76" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62J250wT921100; Thu, 19 Mar 2026 12:05:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=HJIc3+2rGvK7CufqI qAyPhxCgf2VMtEdXJhBQpESGcw=; b=ifYuhh76/4VOUIpdeTyIAzLqEdLw9K4sg xnKha60fH0NpvKJrJcsFO0ZDodjeeY/0cctPyNjuabscuWj5siLtnGjtdbCnRS3h Y8xYZRqGQthg8e3auZxFT2B2bulzVEjUqFJHYR0rLEC81h09mj/vqxuEO1jo7Kld oCJjk5BswMr79yf4vODdJoQEoVRJLupoV3gtn8UU9cWEGJF4UgG6F5c4gEldLOPb U1xhdG3OkWB4tMGBLAHjzwjIkF3EdElY9bETViwsapyvgW554H20c941egIC9KG0 /dl9K0VvYkuFvjX72N1b/qLN31MCOWkRUZtpOQpWEwhIQX6zIUsrA== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvyaunyyx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:08 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62J874S1005412; Thu, 19 Mar 2026 12:05:08 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwj0sjf86-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:08 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62JC54rg46596454 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 19 Mar 2026 12:05:04 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5EC3F20040; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3C7062004D; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v2 5/9] s390/percpu: Use new percpu code section for arch_this_cpu_add_return() Date: Thu, 19 Mar 2026 13:04:59 +0100 Message-ID: <20260319120503.4046659-6-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260319120503.4046659-1-hca@linux.ibm.com> References: <20260319120503.4046659-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE5MDA5NCBTYWx0ZWRfXycnCVulmdYbU S/19ffV5nQbqyM7fOaGFWL05l0HWdFZH20KgzEuSrRTgYuHbf5nwe9x4a5zPDb/5YV1HzMQlS2P 0aQ2R5zq/A7ORCETOLQhLjRqFZ4ijcalMsbO0GAJ0uotAFmwXvhI60wkRtcomK0g1vZdXD1xDHR jUIANTzkyIe/cfrUErNfqkSjT187sSqhuI3sgFmltOO/sSc5KZ+eaE6Paz4oMk3va/Qz49JNtUc 48uIPn3vxRwiemwkl5NASLDkUjFtfw8vLAF7Bdsc3v5WrujyceNjjnBg2+M7QnziU1YJCFqOflQ 5zQnYO7Y0sAk1q0b/nb2AR4UaS3ZmQC+UiEdRJ4rkgtZ8I7tDGIfHpLhFvmwu5tlPQr4A4SIBEB bwwfYdmU0I5/CBJv1CGpDnxYICfZXdmAbvKqMXD/WlrWC7wsZRgAA1omiacgCttFqiJpzlqQg5f x4/r6L6yxcs+ZCMfM1A== X-Authority-Analysis: v=2.4 cv=GIQF0+NK c=1 sm=1 tr=0 ts=69bbe674 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=AksqfXeGXF_AYWYsu5gA:9 X-Proofpoint-ORIG-GUID: gbszMLxAqzqX1ucRdpv1FLs2QiVlWZig X-Proofpoint-GUID: gbszMLxAqzqX1ucRdpv1FLs2QiVlWZig X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-19_01,2026-03-19_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 bulkscore=0 lowpriorityscore=0 malwarescore=0 phishscore=0 impostorscore=0 suspectscore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603190094 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_add_return() to make use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~4k (defconfig). Also 66 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 3c5364475e3e..2479b4748510 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -177,17 +177,29 @@ =20 #define arch_this_cpu_add_return(pcp, val, op) \ ({ \ + unsigned long lc_pcpr, lc_pcpo; \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - asm volatile( \ - op " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ : "cc"); \ - preempt_enable_notrace(); \ old__ + val__; \ }) =20 --=20 2.51.0 From nobody Mon Apr 6 10:31:00 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DAE013D1701; Thu, 19 Mar 2026 12:05:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921921; cv=none; b=BaOrh5vOoBSb6KhtoxHzPRFEy02Y7D46j1GXZuKmb4WrRSaw4KJw2h9BpbuUrKYvqvUgYeeqRaMGRSGMlfnBBVQEjvqnvseQRI/xAKghWDVQzGqFq6JLUC91h3qG/9n4a8LESf7tkVo6Mzz3PS866Xb5ro2CAP63aGu+wbLGHRc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921921; c=relaxed/simple; bh=yWVgz3Vqo70DLgRkPa0nVx949kF1zsRqNahtMIVo8nU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GOu79InOU1Dlj089Zq2x+8LMgdcEnWlAHB5jvgwxBkEBhtQamK0zBpzT392HnVtbGzgg+kGMPMatgcdh+GwZPW7WpBCK7mxbKDQoRQ+ENwgG0mxI+n9GTWIBqP8nwUkJpnsuehiIkMtsukj2KlS1ZAfhXV2mUur466BpMxHxPLA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=cMSPP7oW; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="cMSPP7oW" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62J0ZsbT1188668; Thu, 19 Mar 2026 12:05:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=KUCZUWot0q64d1Jv1 ay9Jsjjy4iXkIDQg9rwMkYwFbM=; b=cMSPP7oWl4Gur05J8mwLGC/On0hideRL6 UqlLzHI46iZ1rKYJ4WBa92Y43GjQahUsvHj9iFhuxeKnpWIyO/s2ZrGrXYIAbJ3G ma+TYkRC8MOWmmKDyhhtZwiVEOQ5IGQsi91vhDasI8BMpGLW7eyR+/WnsEulQIDZ fBTDUJ4UYyfCBEjGpT8myphIojrUESwhOYbRCGre/tN1vlK8tp2iCfJIptNMWDb8 6a7xRcjJQNOU3tcTZDrAdUv+wtMJpTOku3kwJVTubDjqnWrwgrWRiG8tUl+CredN /vIWZfdP7MLog5WGxDZd/w8UU1YLTSHj9vu1gqnYkMYemgXJ8jllg== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cx7vfs109-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:09 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62J8qZTX015746; Thu, 19 Mar 2026 12:05:08 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4cwk0njb3t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:08 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62JC54TV46596456 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 19 Mar 2026 12:05:04 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 865A12004B; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6412320043; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v2 6/9] s390/percpu: Use new percpu code section for arch_this_cpu_[and|or]() Date: Thu, 19 Mar 2026 13:05:00 +0100 Message-ID: <20260319120503.4046659-7-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260319120503.4046659-1-hca@linux.ibm.com> References: <20260319120503.4046659-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: rPWNwWVa92mKgaYZ6ZmJ2NiD6sR3zUYL X-Authority-Analysis: v=2.4 cv=KajfcAYD c=1 sm=1 tr=0 ts=69bbe675 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=U7nrCbtTmkRpXpFmAIza:22 a=VnNF1IyMAAAA:8 a=4dLYTkTKlH5g-JvQF6AA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE5MDA5NCBTYWx0ZWRfXwtODNm8rHCbw bUbOjwPtx6KeCXLaLzGDPyoj+1a6gBZ7ZVBSI80LbUfWMirzUHwlIk26VnVGv4rsWLCqpEKSCU9 pRW8ug7Mh11eWzaJwZkWPNZ8NRfbpVsiAm02gRXzPxgHWxilLk5MXAPPqJWoYhyfdeBeVRVpiIh rFqzxqL3dOb/c1p6Yzg5wod1zkRsGob4mNCfhJdVxbGrJ5aJ47h0wePO6FqTLpijIAgk527lGPk IiQSVsL2CU5JhAkDMX2BPw3YT165vDUY9hlDpV7ebxRXcCSh1yrXFu9Wm7MS+b+/qsBQtGLgHhV wmnQna6HZllBpn5U1b0LNvw47KMZSgr7vQNa6XEMtLdWprGCnZL8bFzMDDCZ5BstD8Ws6Y5oXhn LtPy7KR/ikuFYRTvwOkKRR3CQfy07OildwBLRr2tjSHK0YABGX+bunwHOqUi3XWiNZXYuRHx/Ai MdCNCKndX8RNR0u2ReA== X-Proofpoint-GUID: rPWNwWVa92mKgaYZ6ZmJ2NiD6sR3zUYL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-19_01,2026-03-19_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 adultscore=0 spamscore=0 malwarescore=0 clxscore=1015 impostorscore=0 bulkscore=0 lowpriorityscore=0 priorityscore=1501 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603190094 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_[and|or]() to make use of the new percpu code section infrastructure. There is no user of this_cpu_and() and only one user of this_cpu_or() within the kernel. Therefore this conversion has hardly any effect, and also removes only preempt_schedule_notrace() function call. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 2479b4748510..510d9ce1ee47 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -208,17 +208,29 @@ =20 #define arch_this_cpu_to_op(pcp, val, op) \ { \ + unsigned long lc_pcpr, lc_pcpo; \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - asm volatile( \ - op " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ : "cc"); \ - preempt_enable_notrace(); \ } =20 #define this_cpu_and_4(pcp, val) arch_this_cpu_to_op(pcp, val, "lan") --=20 2.51.0 From nobody Mon Apr 6 10:31:00 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C42A53CCFAB; Thu, 19 Mar 2026 12:05:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921918; cv=none; b=TaDSPaD3FchcvZZRQVh5SBI7yvTiwIJ0XoMEJ8dSe4/UX24iJniADB/hyzF9eFtRnJ8ulavWUHgG9VC+4m144SAb94GZM+1qYExcZSMw0pxwOMpTAHd4TREFD5hr6Ceah3K/JjZiJN6PQfGdcfzuF6jFQwVAXM+RwYycPhyB+No= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921918; c=relaxed/simple; bh=Oqom8bfGkWsUwigzYFvslJlB2/rFM+gHvI80OX9zH6k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=V+4XXtTyTIzgFaQYbdXydmWdmt/ailh70CLBkSNQB8co9D2ViVofKX0XcdN3aZoSyxUTqbOpgZCHAqKvDVn1cEAmgROXeOF5foB1K73se1cAFDZnRreRgWyCxuNqG0c8izoIdcL8ng5v6aiRLDw6viYPvL9Qq/Vc77h2qAPJFdI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=lPpXjPIN; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="lPpXjPIN" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62J2mEvD1290637; Thu, 19 Mar 2026 12:05:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=jYxXCDONXc96ylJ5G dz3ySZ9k032X8Xsx2ZFcK9zDkg=; b=lPpXjPINZSW/WDYz+EocHb/oEt6y1lY6b UDRWDQkEOE1bK3u8GQcw4Sj3IuQK3QT1EM1VgMdrwdFKNCATLkdr6iQ5WGFvK70e h+MuE531LUcu3NY8xxQTdpgZaME7vgiKAzrItmwjmAxqVmvKIz3qe95Cf0QoRson 69kOVoR1HgQtI/d9kUUOfAU1shYLgNWxTgyMcWh+1GCAu337QEcNjTRZehrCyrgz GdQTRxqCyclTBAm0FRc2TCvQMF9QSbjBKtq/QhLG74Vj06s+5LFtPtwvey4eRwRC 8V9tjIJ+jf6dJGjqfiU9AL6GUba0/TOf+LDHBZFxLeAvd4h2J4pSg== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvyaup000-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:09 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62J874S2005412; Thu, 19 Mar 2026 12:05:08 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwj0sjf89-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:08 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62JC54Il52298048 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 19 Mar 2026 12:05:04 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AE25A20040; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8B8ED2004D; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v2 7/9] s390/percpu: Provide arch_this_cpu_read() implementation Date: Thu, 19 Mar 2026 13:05:01 +0100 Message-ID: <20260319120503.4046659-8-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260319120503.4046659-1-hca@linux.ibm.com> References: <20260319120503.4046659-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE5MDA5NCBTYWx0ZWRfX2A4nlQxm1ByD C78MN/pMh7wnrWWC0eq7ySxYS6dVXASkDu9kE+XrzTiJuKq+TzcePpBsoVvJ/GhiBLqnFdDbvmv 6pOJ8SB8NuEUDl/D9Xcl7GIDDZGvqnR97xa5yyqa5xb3l/u68RYsG4gKi18RCpykXKSQX/WpkBi CnB3E7KWTCRX/DRGEN+kD0UGt0em+pk4rG6gXTi0tzA2+mFrngPqaDgY68oI1Vr2w9uDnLKK+N0 5G57b8xntCnb0f90UItVf+QKXP7vmQBm8JZmv91Ls0WGe59I4ZPH7p1rpwTBSbkOp5wXu4TDcII ecDmfTB+gM8FjwSjqRAURPNdCT3dtWuVWl8RBdMFWFDxI21qt4xDDwFgMmdhf196Y9AftIQ+SAN HyQvysPqj35w3LGA5N844x/P0mC3bipCHxuqjgWVFJSp2loNGpd7ybuMribHbNyJqpNsjMkyCC7 n+6T8XA5Hp4YJncB+KA== X-Authority-Analysis: v=2.4 cv=GIQF0+NK c=1 sm=1 tr=0 ts=69bbe675 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=4bOSrtcVyj7iNoH5sBwA:9 X-Proofpoint-ORIG-GUID: YItMeilmNlWngxygYK66DIslMHpJkhbE X-Proofpoint-GUID: YItMeilmNlWngxygYK66DIslMHpJkhbE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-19_01,2026-03-19_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 bulkscore=0 lowpriorityscore=0 malwarescore=0 phishscore=0 impostorscore=0 suspectscore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603190094 Content-Type: text/plain; charset="utf-8" Provide an s390 specific implementation of arch_this_cpu_read() instead of the generic variant. The generic variant uses preempt_disable() / preempt_enable() pair and READ_ONCE(). Get rid of the preempt_disable() / preempt_enable() pairs by providing an own variant which makes use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~1k (defconfig). Also 87 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 510d9ce1ee47..08c48fa97381 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -240,6 +240,37 @@ =20 #endif /* MARCH_HAS_Z196_FEATURES */ =20 +#define arch_this_cpu_read(pcp, op) \ +({ \ + unsigned long lc_pcpr, lc_pcpo; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ val__, *ptr__; \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [val__] "=3D&d" (val__), [ptr__] "+&a" (ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (*ptr__), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + val__; \ +}) + +#define this_cpu_read_1(pcp) arch_this_cpu_read(pcp, "ic") +#define this_cpu_read_2(pcp) arch_this_cpu_read(pcp, "lh") +#define this_cpu_read_4(pcp) arch_this_cpu_read(pcp, "l") +#define this_cpu_read_8(pcp) arch_this_cpu_read(pcp, "lg") + #define arch_this_cpu_cmpxchg(pcp, oval, nval) \ ({ \ typedef typeof(pcp) pcp_op_T__; \ --=20 2.51.0 From nobody Mon Apr 6 10:31:00 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C42123CCFAA; Thu, 19 Mar 2026 12:05:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921919; cv=none; b=Mo49x/6ck3qrJkiGajEstUNVUIuQC8GjF7f9Mbx9p/po6qlEHa5XbNyOjG0wARAkMWxFAck3S98VpXg/i5pZjNhfSXLz/00q7doZhGkFtuyFB+aUKrqS8c1nt03mghRGTK0Ih4ylJQCMZtDHUabSjh4jZ0/c+ph0pSitI/yM8kg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921919; c=relaxed/simple; bh=U7fp86askpbhtuuJT7RlGBAIrVIPyG5UDU0rTWsQyuo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cZVx7cveu+WRarZK6JZywCZHmOJl8iyvgXQgsuoiA2YoNQZRTssfUEvfzf/C+DCJEGAdM2pITZNDcZaVJr4DYLT6pY9Po0LU6EywF8nMnzACW5YSCHGJBGrtGIXj13s/12lgWdNqWdeulv378S3aEWpLOwIvwPRjLuwGEeKZSAU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=SvIJh+Z1; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="SvIJh+Z1" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62ILRfVj645990; Thu, 19 Mar 2026 12:05:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=6c4i/zhVRm0J1g2HT eHTepeWOP8LfJJiSz/VRN4SxWI=; b=SvIJh+Z16asTkCkduUgnK4srmyUDy3MFw j0IMuotjwkMgRKFReRULg781zSMbe35s2ujeFSsSDEbzbcSVpbGygm9RNBWqcvlI bGH7Js/vZbeGTpWSlFfrzNJXaiSKZlZfI4xd1at14bAWVf2XWMRP54XGE18CDTs4 YvV0o1PT44tXmH8Po4Ijqz0DvDUAl+JGgL4aj1IoZvF6yDrunsPZaiYqLZLbLWrd HIZc7szD2sK6e51Cpb3Q7g7adNkwZB2gMQhqRm0+IcSWnwg1YWg7JmuOWUa3vRBc PDkkC7c5VBtGINNJwH4gMvGNPbALmdK5oUVU/O8ajLEoHeE+b2XvQ== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvyaup001-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:09 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62JA7Qj9032397; Thu, 19 Mar 2026 12:05:08 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwm7k25tr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:08 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62JC545F52298050 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 19 Mar 2026 12:05:05 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D57542004B; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B35F420043; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v2 8/9] s390/percpu: Provide arch_this_cpu_write() implementation Date: Thu, 19 Mar 2026 13:05:02 +0100 Message-ID: <20260319120503.4046659-9-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260319120503.4046659-1-hca@linux.ibm.com> References: <20260319120503.4046659-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE5MDA5NCBTYWx0ZWRfX5qpT/fcysQDs Hd1LJ5pvTkNvonwhxh4xVWyL0ppAaXF1loGtXw1XCH2Q0s/6IXK3n3/FATsKzsMGTNUwJzjRwd5 PnuyL+GqJURVyh38BwMaoIBKZGSfbjP+WNvk1XF9tUCnL+ZQrIw8sifzUI7TechgNaTDOvDL2tf YwQq50Bov0SLWGFfHGJeQPDv/Pw1TGMTKxwB7skdulyiO8qPpnJPpN2pEEJsdfECCKIzVJu8iXI H02hd8McMEprxA40CKAVmeM6/TAD3pay5MEwTBolUW2M7qzkNXkZVjZFyjDqqKqbEhsxsVe4sMe bmUBLm7WVo0WMhCFrrLI+149pdy7FdoR0Vh3OeGYU/KPOtb4TPD0IFtXsIFU7aGcn+ppeKTPllH BEmaRdrbay/AD4cg2hvqhvSrWdeYrHmaDZLWH/OP/F/PLp1SOOpOO9o1T5IahOUvbDbmNaZzT/5 uljrc4z5egI1mMbA3gw== X-Authority-Analysis: v=2.4 cv=GIQF0+NK c=1 sm=1 tr=0 ts=69bbe675 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=ObmbXjF690NQJU6EaucA:9 X-Proofpoint-ORIG-GUID: 4RJ4yB0q0Uraj8G6SfPIHnP8zglvq-zw X-Proofpoint-GUID: 4RJ4yB0q0Uraj8G6SfPIHnP8zglvq-zw X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-19_01,2026-03-19_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 bulkscore=0 lowpriorityscore=0 malwarescore=0 phishscore=0 impostorscore=0 suspectscore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603190094 Content-Type: text/plain; charset="utf-8" Provide an s390 specific implementation of arch_this_cpu_write() instead of the generic variant. The generic variant uses a quite expensive raw_local_irq_save() / raw_local_irq_restore() pair. Get rid of this by providing an own variant which makes use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~1k (defconfig). Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 08c48fa97381..44501a407e6d 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -271,6 +271,38 @@ #define this_cpu_read_4(pcp) arch_this_cpu_read(pcp, "l") #define this_cpu_read_8(pcp) arch_this_cpu_read(pcp, "lg") =20 +#define arch_this_cpu_write(pcp, val, op) \ +{ \ + unsigned long lc_pcpr, lc_pcpo; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ val__ =3D (val); \ + pcp_op_T__ old__, *ptr__; \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "=3Dm" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ +} + +#define this_cpu_write_1(pcp, val) arch_this_cpu_write(pcp, val, "stc") +#define this_cpu_write_2(pcp, val) arch_this_cpu_write(pcp, val, "sth") +#define this_cpu_write_4(pcp, val) arch_this_cpu_write(pcp, val, "st") +#define this_cpu_write_8(pcp, val) arch_this_cpu_write(pcp, val, "stg") + #define arch_this_cpu_cmpxchg(pcp, oval, nval) \ ({ \ typedef typeof(pcp) pcp_op_T__; \ --=20 2.51.0 From nobody Mon Apr 6 10:31:00 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D513B3D16FF; Thu, 19 Mar 2026 12:05:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921921; cv=none; b=QlRj8TshgO+wLo9r9A9tlhaaT/5rfIhdbgtZI1oMs6CndJ5xKyXTTGjoMpPdfGD+27tFBiPMtB5/e2uAoSBgmTtEp/yMbmS6eMwb2hJxIQGtb7aXMXw61g8lrT6s3kx3y4QZZbcPm7KsOB3crmnbWMFvpqZkrfIVNEnpLVeZIUM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773921921; c=relaxed/simple; bh=1kdk+CK6bk84jIsKAayEJ9LpHGdQ5vexBiN3RDJ/cto=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TamgyhNvJivl0gOpbeqTYUuYR2Lzqz7r5sBIVdYwXIllmdl15OluzHKWs2ALQpIlf9DGaEcqOFWFom+NHXjT6NmVamkfOsPih3W4TP1oDyV+T3yzfizBTHlWJOikB96dVToDVPXCjdjnoRtzcoF1TYHf0SM8blam8q6acEf1GmY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=KCMaB5xx; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="KCMaB5xx" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62INusid1368595; Thu, 19 Mar 2026 12:05:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=3HDFlARG0P0h478Bf zN/iHwOctLoiC6OSWAsIU7WS9c=; b=KCMaB5xxUFSdZUowxpbgKx4SUbJdtrHHh kpW9AUIWDG8b24YWvsOSiDuMb3nxdc7wb/60+rvn72QakBlw7JJXP8podZ2BhEfH PjrpuRVBpGgsQTAffPy8cq+RS2THYR2HOmTWtNQARYu5uveEcPD5WKPNyoL4Ql51 uk98M/p89ZM2yqtPG3BcAci7DJT39WHBNrx9afS8Wag15gnKPgyX3O08BgIaqZhU MW4svz3FbCZJ6BcuhyMeXPf2WtFYKbefMPA9c/qfK12ldWuqneKrO+GnaIna+59o Y3nSkdbhoHwijl4dqUtTykv87JFMNi9VVhhebr5/NPvaB9sGydPnA== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvy64xux1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:09 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62JALdRX028471; Thu, 19 Mar 2026 12:05:08 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwmq1j4dk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 19 Mar 2026 12:05:08 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62JC55OF48693688 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 19 Mar 2026 12:05:05 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0899C20043; Thu, 19 Mar 2026 12:05:05 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DA9E520040; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 19 Mar 2026 12:05:04 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v2 9/9] s390/percpu: Remove one and two byte this_cpu operation implementation Date: Thu, 19 Mar 2026 13:05:03 +0100 Message-ID: <20260319120503.4046659-10-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260319120503.4046659-1-hca@linux.ibm.com> References: <20260319120503.4046659-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: khOn5oyK-tvj1JfqDJrmW9CBLMKlMdz9 X-Proofpoint-GUID: khOn5oyK-tvj1JfqDJrmW9CBLMKlMdz9 X-Authority-Analysis: v=2.4 cv=KYnfcAYD c=1 sm=1 tr=0 ts=69bbe675 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=VnNF1IyMAAAA:8 a=Q4reuYiO1UekjhQ8NNAA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE5MDA5NCBTYWx0ZWRfXwHmy8ZwuPuS7 C3Uoc94sDsamoHohMAMcGdoYB6QR05XxnXu4LD0AhbK13ScdQ0/yUG8NmzdjKdjOsHEXe3GOiC/ IduRTnvCqeE77v8aAZgPjCJ/AiwdInoojCS/d25UREK5GwH2rtCE4LNaKdWp+xrMe7liqEAZk8O vBkylNRYqgYouiIRrbZ5Akn74qMTcExpjb0FAMLpO7hs80ZlUcF0EEC+wdlTjB6ANtDXpyo0EDY veQKydDTBCRJlxSfWboMfHI9po8qzn3Cm/db2zuDqe7+Z4e1MLL5FHPVuoRqwn8qvkQVgItfRa9 iE22otg5li/PAmwbNzBfF0OSHFaCkBTLY6lp86W3RQhylU8g2XxGu5HiZDJn1lcKUnz+cppZnC8 TJjXog1+WG0FOEewwbkvc3W7iEDku/Ok2tZgNzpoii8cBp6U6AsbAagTL3+6cV/2HRR6FmF5M71 zGXIqtaFQEFLS2JsoBA== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-19_01,2026-03-19_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 lowpriorityscore=0 priorityscore=1501 suspectscore=0 bulkscore=0 spamscore=0 impostorscore=0 malwarescore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603190094 Content-Type: text/plain; charset="utf-8" There are no one and two byte this_cpu operations within the kernel (defconfig). However even if there would be the s390 implementation, which uses a cmpxchg loop, generates a very large code sequence due to the lack of native one and two byte cmpxchg instructions. Remove the s390 implementation and use the generic implementation. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 9 --------- 1 file changed, 9 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 44501a407e6d..b5b61471439b 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -51,15 +51,6 @@ new__; \ }) =20 -#define this_cpu_add_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) -#define this_cpu_add_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) -#define this_cpu_add_return_1(pcp, val) arch_this_cpu_to_op_simple(pcp, va= l, +) -#define this_cpu_add_return_2(pcp, val) arch_this_cpu_to_op_simple(pcp, va= l, +) -#define this_cpu_and_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, &) -#define this_cpu_and_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, &) -#define this_cpu_or_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) -#define this_cpu_or_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) - /* * Macros to be used for percpu code section based on atomic instructions. * --=20 2.51.0