From nobody Mon Apr 6 21:31:18 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 885A53254BB; Tue, 17 Mar 2026 19:54:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777295; cv=none; b=Z4aA2lEbde3lmT7AgsQbV8n7sAMSWDxpW6Rvdbf4tBFebCfUZmPjFJU1Gut8r+U2RO0Zex0YfOHGLeLQz1LG0ZqqVYUIkGVbA++F8Uee3yKCtBJBu+lCIWSDaamXbaiMgVp8y8Dt/U2G3O88ZO/sGNKZ83Zwbm9guRABZzBzogg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777295; c=relaxed/simple; bh=0RMy2yUPZpOjljYRzamsROmIJDxTeFTzmlSYHsO3eT0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZMyDAiUM2qFCc20QktwZrtwmiD3M1ty1TRJNtEdwf/fEcQplkMzXJT8cAPsuQtS+/67FYLxjr1YfJ1oTp1hF//R/lbbpuokrgGjwAWORC4XAT5dikrH1PSFfex/5KfXUpwDEsUmpAABDunl/gci9ZhjeBTnVPe/EtAi9xl+Ho2c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=k1KufDiO; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="k1KufDiO" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62HJdoIh1093922; Tue, 17 Mar 2026 19:54:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=32nSjMS4mBotDKtzK bPQQzWFpThr7pHDokgqeJVMKUU=; b=k1KufDiOdDBFSsXr67NT63OKvEeCk2zHX pgLMhoIsbaE7MZ3M9eJ70ir5yIS2nABfdxGIX0gtd1/U/DC4VIETT3HBw9Y53qQH c4PWq0Jr5wd+rzVfRcNVSkz0cvqIM42UMR5ez1xdCEk3BJy5uEx5a3RWO1j+Dx4B N4oUh/e9P7mKW7tg37sZTJJRlAtQtIRkFwD8ymvtoXRiwWLvEMJbPKCIpEb2Xj+h SigGwKvf09mVQSvhDnVtXd2SwpcvZdaBsdWRl5tkjF09HEMbE+8RaJjvtjqAIFx5 SB3BTK1ds2oq/aftRBwc3p8h3pKYhyD/oR2NB4NcQJIZYgz1XFqYQ== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvy64ppuc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:42 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62HJeAQQ032327; Tue, 17 Mar 2026 19:54:42 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwm7jtmuh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:41 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62HJsbAg41222422 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 17 Mar 2026 19:54:37 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7031F2004B; Tue, 17 Mar 2026 19:54:37 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 46F1C2004E; Tue, 17 Mar 2026 19:54:37 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 17 Mar 2026 19:54:37 +0000 (GMT) From: Heiko Carstens To: Peter Zijlstra , Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 1/9] s390/percpu: Provide arch_raw_cpu_ptr() Date: Tue, 17 Mar 2026 20:54:28 +0100 Message-ID: <20260317195436.2276810-2-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260317195436.2276810-1-hca@linux.ibm.com> References: <20260317195436.2276810-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 2hvIDpO1c6rE5bkxyU_-GqliHc0Y1w3r X-Proofpoint-GUID: 2hvIDpO1c6rE5bkxyU_-GqliHc0Y1w3r X-Authority-Analysis: v=2.4 cv=KYnfcAYD c=1 sm=1 tr=0 ts=69b9b182 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=VnNF1IyMAAAA:8 a=FJXwAgvRweZ6vCOBEk0A:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE3MDE3NCBTYWx0ZWRfX/Zsq4GwBEgW3 EKKLxHKMHhXk3tqxvrZEQKg9J6aoIClmI1TK92WtOMIEjohRb0txBRyMSI5eLfJwltk7hfKTKoM Q20T1CLwP+VG9HKR11vzW47wE+Ki+wnrxsb6xbh202u8GdGqk3UdiyAPMklD+KccYczhySJFLYz U2jjJWpHXuGQBbUcAW0I1xUDCwLEzKpx19l8xIW3bsPkfWYX6Q4jbYQlJzHeazQCnx8w+pMNODd uAxGIeAD6qfJ2VWKq4uLezzDlgElaRcRzMhsS/hXqttc+ZS2yE5okX6lDWtqwqGkBCrvBHKnr+o ScsvlmS72lHIVCr91/yby/on260bmPLppWBbs5TML5N/v4p0B/mDmyS0o8LktLBGVS48ufEu8yG CdVPO9z1DGlZvCo1TSPwqkjhPXaKwmHLo6zJ7sksAaELWwZ83kYS0uyq7fv9PZfhTKjzNaj6qWE Yo7B1H0vsJ9MQJY7yiQ== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-17_04,2026-03-17_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 lowpriorityscore=0 priorityscore=1501 suspectscore=0 bulkscore=0 spamscore=0 impostorscore=0 malwarescore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603170174 Content-Type: text/plain; charset="utf-8" Provide an s390 specific arch_raw_cpu_ptr() implementation which avoids the detour over get_lowcore() to get the lowcore pointer. The inline assembly is implemented with an alternative so that relocated lowcore (percpu offset is at a different address) is handled correctly. This turns code like this 102f78: a7 39 00 00 lghi %r3,0 102f7c: e3 20 33 b8 00 08 ag %r2,952(%r3) which adds the percpu offset to register r2 with a single instruction 102f7c: e3 20 33 b8 00 08 ag %r2,952(%r0) and also avoids the need of a base register, thus reducing register pressure. With defconfig bloat-o-meter -t provides this result: add/remove: 12/26 grow/shrink: 183/3391 up/down: 14880/-41950 (-27070) Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 5899f57f17d1..b18a96f3a334 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -12,6 +12,24 @@ */ #define __my_cpu_offset get_lowcore()->percpu_offset =20 +#define arch_raw_cpu_ptr(_ptr) \ +({ \ + unsigned long lc_percpu, tcp_ptr__; \ + \ + tcp_ptr__ =3D (__force unsigned long)(_ptr); \ + lc_percpu =3D offsetof(struct lowcore, percpu_offset); \ + asm_inline volatile( \ + ALTERNATIVE("ag %[__ptr__],%[offzero](%%r0)\n", \ + "ag %[__ptr__],%[offalt](%%r0)\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) \ + : [__ptr__] "+d" (tcp_ptr__) \ + : [offzero] "i" (lc_percpu), \ + [offalt] "i" (lc_percpu + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + (TYPEOF_UNQUAL(*(_ptr)) __force __kernel *)tcp_ptr__; \ +}) + /* * We use a compare-and-swap loop since that uses less cpu cycles than * disabling and enabling interrupts like the generic variant would do. --=20 2.51.0 From nobody Mon Apr 6 21:31:18 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D3D834402B; Tue, 17 Mar 2026 19:54:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777295; cv=none; b=jd2qsR+WsyGRnYAueMD7l9Gi/ZzFEWVFbtS2K/0kJSaIKAwhZjpbiGzjvFxA5GXyqBm5ZG9KoUURKnY0oe61zdQ6EHYSPzmrkxNaZe2N7mrmYJNTaBpjCVa5ourRbzqLJ7yoCPFIDJi6dI44SO5FFa8Yu5Q2JCn0xNZg51zA9Yg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777295; c=relaxed/simple; bh=MstLwo7au7DeAAy/jRAj3qMTD68l96MMOZHv2HBF3nM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Nyqsb27tbS2TJKuVavGb3JIRl6CB6Ewu8u3bCPSHwiixYWIwCFE6IhzrhYo7FMCwbgQ8e49nV5S0n1v1GSVlycA5OxkEMmYcbo3WrJAO4JNl8N1ce5uzwqmDkoqTiE3R0EzqA/Hi0IxaEnKCxS2DCkmumsNOrHeYoyUxn0PpuFA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=nTOUdtRn; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="nTOUdtRn" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62HIBtAl1363601; Tue, 17 Mar 2026 19:54:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=a3yY+ErE8l6wYLZxv pVWDNV0yK3nY/2aI0msESMHjxs=; b=nTOUdtRn273Lw/qI+LaQinZFsYMBTdUZA Sy1nu/LPlIV/yN6JbPG6ye17iQ7KBvyLC+I1F6/eV0rU+rkQ8qRUC5GXiQEsJHSV xIW3JdHxkH4tclshjnx49NvJ9XTh2fgrYxlo5OocDlp1bSov0fAOPxJ96LgI8I32 cGkaE+q1wK6Z27wVUohc6pgMKocOKpR50aXY+95XKaOrmkz3ztTc5dgZPQMuh1fq 6Aehe2C8YyHuMfRAm3LPQIwsfv+COKb4Rrq4kGx2CankaRTEnSjH80E/KdwqEuxV aAHqWyAxu6DsKCsb+z8Ie0w7xood+zUaklrdbEQ7ZaPujQULm4UWw== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvy64ppud-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:43 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62HJZ2gE028785; Tue, 17 Mar 2026 19:54:42 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4cwkgkas7s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:42 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62HJsbFX48169370 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 17 Mar 2026 19:54:37 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9DC352004F; Tue, 17 Mar 2026 19:54:37 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 75BF72004D; Tue, 17 Mar 2026 19:54:37 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 17 Mar 2026 19:54:37 +0000 (GMT) From: Heiko Carstens To: Peter Zijlstra , Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 2/9] s390/alternatives: Add new ALT_TYPE_PERCPU type Date: Tue, 17 Mar 2026 20:54:29 +0100 Message-ID: <20260317195436.2276810-3-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260317195436.2276810-1-hca@linux.ibm.com> References: <20260317195436.2276810-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: MP7j4Fn6VoNAqntZqX7u17sPsrNwNPMU X-Proofpoint-GUID: MP7j4Fn6VoNAqntZqX7u17sPsrNwNPMU X-Authority-Analysis: v=2.4 cv=KYnfcAYD c=1 sm=1 tr=0 ts=69b9b183 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=VnNF1IyMAAAA:8 a=XZhbzCa44lJJqUdH0vgA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE3MDE3NCBTYWx0ZWRfXwSGdIZMQowSI 3cCKLrtsVFbMHdpZ7yOYHbIVeYXcwORdNkBAu0KeLsqC0p/F+sqoeaYQOQJlgN2NuKDD9ncxgMg euLnTEAc+F10V7370LNnqCghY6+RNDeu/IIHVYNyinQ19Ud57uhO6IuVr4Dy0Hg/p93bdO5AMON qEVInaUJeHJKcr2pbU87k0eWZB9r9aIrjed0PXO65C/RU3188GRFivUz5fX1yP5mR4LLcQ6tOdr +EldQrCYWJGFxh6ZU0ip1gA1MfofajlrvcGcHBE5WbeeFXoik2YTXYTNf+RdQmpAFis4yWRu3lq GgCyG55RapRyomXTFvvUHGpWDRTPD3yUESGjZ1/Gl3KhILHFRfxKsAYgJ/2IZOAXZiIdE6NEeRb 7udNWaqMrOaUxJyAX1gLibFb16Nu6gMbsClyQTR7p4VF5otGTvslJZhGx8L29RoKLGkoCSfcijx KCHh8KDM2u2+v+IUyUQ== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-17_04,2026-03-17_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 lowpriorityscore=0 priorityscore=1501 suspectscore=0 bulkscore=0 spamscore=0 impostorscore=0 malwarescore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603170174 Content-Type: text/plain; charset="utf-8" The upcoming percpu section code uses two mviy instructions to guard the beginning and end of a percpu code section. The first mviy instruction writes the register number, which contains the percpu address to lowcore. This indicates both the beginning of a percpu code section and which register contains the percpu address. During compile time the mvyi instruction is generated in a way that its base register contains the percpu register, and the immediate field is zero. This needs to be patched so that the base register is zero, and the immediate field contains the register number. For example 101424: eb 00 23 c0 00 52 mviy 960(%r2),0 needs to be patched to 101424: eb 20 03 c0 00 52 mviy 960(%r0),2 Provide a new ALT_TYPE_PERCPU alternative type which handles this specific instruction patching. In addition it also handles the relocated lowcore case, where the displacement of the mviy instruction has a different value. Signed-off-by: Heiko Carstens --- arch/s390/boot/alternative.c | 7 +++++++ arch/s390/include/asm/alternative.h | 5 +++++ arch/s390/kernel/alternative.c | 25 +++++++++++++++++++++++-- 3 files changed, 35 insertions(+), 2 deletions(-) diff --git a/arch/s390/boot/alternative.c b/arch/s390/boot/alternative.c index 19ea7934b918..ad078a2b1192 100644 --- a/arch/s390/boot/alternative.c +++ b/arch/s390/boot/alternative.c @@ -22,6 +22,9 @@ static void alt_debug_all(int type) case ALT_TYPE_SPEC: alt_debug.spec =3D 1; break; + case ALT_TYPE_PERCPU: + alt_debug.percpu =3D 1; + break; } } =20 @@ -115,6 +118,7 @@ void alt_debug_setup(char *str) alt_debug_all(ALT_TYPE_FACILITY); alt_debug_all(ALT_TYPE_FEATURE); alt_debug_all(ALT_TYPE_SPEC); + alt_debug_all(ALT_TYPE_PERCPU); return; } while (*str) { @@ -130,6 +134,9 @@ void alt_debug_setup(char *str) case ALT_TYPE_SPEC: alt_debug_all(ALT_TYPE_SPEC); break; + case ALT_TYPE_PERCPU: + alt_debug_all(ALT_TYPE_PERCPU); + break; } if (*str !=3D ';') break; diff --git a/arch/s390/include/asm/alternative.h b/arch/s390/include/asm/al= ternative.h index 1c56480def9e..9ca2e49338a2 100644 --- a/arch/s390/include/asm/alternative.h +++ b/arch/s390/include/asm/alternative.h @@ -34,6 +34,7 @@ #define ALT_TYPE_FACILITY 0 #define ALT_TYPE_FEATURE 1 #define ALT_TYPE_SPEC 2 +#define ALT_TYPE_PERCPU 3 =20 #define ALT_DATA_SHIFT 0 #define ALT_TYPE_SHIFT 20 @@ -51,6 +52,10 @@ ALT_TYPE_SPEC << ALT_TYPE_SHIFT | \ (facility) << ALT_DATA_SHIFT) =20 +#define ALT_PERCPU(num) (ALT_CTX_EARLY << ALT_CTX_SHIFT | \ + ALT_TYPE_PERCPU << ALT_TYPE_SHIFT | \ + (num) << ALT_DATA_SHIFT) + #ifndef __ASSEMBLER__ =20 #include diff --git a/arch/s390/kernel/alternative.c b/arch/s390/kernel/alternative.c index 02d04ae621ba..e32eb489b899 100644 --- a/arch/s390/kernel/alternative.c +++ b/arch/s390/kernel/alternative.c @@ -28,6 +28,7 @@ struct alt_debug { unsigned long facilities[MAX_FACILITY_BIT / BITS_PER_LONG]; unsigned long mfeatures[MAX_MFEATURE_BIT / BITS_PER_LONG]; int spec; + int percpu; }; =20 static struct alt_debug __bootdata_preserved(alt_debug); @@ -48,8 +49,18 @@ static void alternative_dump(u8 *old, u8 *new, unsigned = int len, unsigned int ty a_debug("[%d/%3d] %016lx: %s -> %s\n", type, data, kptr, oinsn, ninsn); } =20 +struct insn_siy { + u64 opc1 : 8; + u64 i2 : 8; + u64 b1 : 4; + u64 dl1 : 12; + u64 dh1 : 8; + u64 opc2 : 8; +}; + void __apply_alternatives(struct alt_instr *start, struct alt_instr *end, = unsigned int ctx) { + struct insn_siy insn_siy; struct alt_debug *d; struct alt_instr *a; bool debug, replace; @@ -63,6 +74,8 @@ void __apply_alternatives(struct alt_instr *start, struct= alt_instr *end, unsign for (a =3D start; a < end; a++) { if (!(a->ctx & ctx)) continue; + old =3D (u8 *)&a->instr_offset + a->instr_offset; + new =3D (u8 *)&a->repl_offset + a->repl_offset; switch (a->type) { case ALT_TYPE_FACILITY: replace =3D test_facility(a->data); @@ -76,14 +89,22 @@ void __apply_alternatives(struct alt_instr *start, stru= ct alt_instr *end, unsign replace =3D nobp_enabled(); debug =3D d->spec; break; + case ALT_TYPE_PERCPU: + replace =3D true; + insn_siy =3D *(struct insn_siy *)old; + if (test_machine_feature(MFEATURE_LOWCORE)) + insn_siy =3D *(struct insn_siy *)new; + insn_siy.i2 =3D insn_siy.b1; + insn_siy.b1 =3D 0; + new =3D (u8 *)&insn_siy; + debug =3D d->percpu; + break; default: replace =3D false; debug =3D false; } if (!replace) continue; - old =3D (u8 *)&a->instr_offset + a->instr_offset; - new =3D (u8 *)&a->repl_offset + a->repl_offset; if (debug) alternative_dump(old, new, a->instrlen, a->type, a->data); s390_kernel_write(old, new, a->instrlen); --=20 2.51.0 From nobody Mon Apr 6 21:31:18 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 43E7B1A9FAF; Tue, 17 Mar 2026 19:54:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777293; cv=none; b=ECqJcTgE/8lBQidCGUO4jrlmHBAN3y4ld7uWrT89lLTc6NhuqMv2U95q14cnNWgYN1TCHB3xcdtJZWRZJDEXQOtpGK+9EPuTVHKuBhlWY6+3x/DYycw1XYsDAcJF89rTQTJv3tK/rzzhj/sEHsATAkh9b2kn39N27BPHpdx+2Bw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777293; c=relaxed/simple; bh=i1HDNOXvqfPKxyL08nXlqnJFBeHPM2KSJF6rzgUQu4Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dRfAC0vO7rMU+1Wj9Zhrm5JxBh4lyucEa8n6mW6DCLoyPtzvUeAQ8YwjxUSTPatj9O5DzpR3EBkB6fSnhvTynCbKsGe11Mk3UhGxyDOGyTozVQO9vErn4KRWixc0mqbYwqSiG9z5zBQjuFSm+eIhCi3AAWLuZ9camUkKSisCLDM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=XNH1UM5h; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="XNH1UM5h" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62H9wbiw1189478; Tue, 17 Mar 2026 19:54:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=SdN81pYnVwJKbtFS5 YXLDPvFowmPzFIFZ/xCQ+0QJYk=; b=XNH1UM5hWM0NAqvMW9Rhhimu/J40bL80u eEKyBMYWHH27HzvigldS+YWXscFuRVTMUewHg0H06600l0RnoM4TSebEEXa9/ZEt +UTM0/qgFrw1wHl5ZGDvdQU507hu6GbP5gcr4ZZh9Q1c+gZlagJno+A9x67Nm9GK lXXjWbh82ctuQcik4zMoWRHNCobS9IdGHkukC+XoiHz2OiDJGZi0TzKxJokhKKCO jQW0u4utkm5gC2HFyuhN+voWKoQByenbfZbSpNb9Ef+dVjEiDL7fKRWRzPdSn6Fs nCj73YpG6M51Xmp5hLfH2iDEvf4taTK15iHdS0c/Z30+JcDdeWNFg== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cx7vfgw77-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:43 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62HGNOsa028496; Tue, 17 Mar 2026 19:54:42 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwmq1ak3t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:42 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62HJsbJH48169374 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 17 Mar 2026 19:54:38 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D46E820040; Tue, 17 Mar 2026 19:54:37 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A43DF2004B; Tue, 17 Mar 2026 19:54:37 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 17 Mar 2026 19:54:37 +0000 (GMT) From: Heiko Carstens To: Peter Zijlstra , Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 3/9] s390/percpu: Infrastructure for more efficient this_cpu operations Date: Tue, 17 Mar 2026 20:54:30 +0100 Message-ID: <20260317195436.2276810-4-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260317195436.2276810-1-hca@linux.ibm.com> References: <20260317195436.2276810-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: wO3MFv0FNni6nAFSAux-KYVcDWHSsfiY X-Authority-Analysis: v=2.4 cv=KajfcAYD c=1 sm=1 tr=0 ts=69b9b183 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=U7nrCbtTmkRpXpFmAIza:22 a=VnNF1IyMAAAA:8 a=WWXEMnX_Ls2LmYDc0GUA:9 a=RwcZUMTHd8_6jkUI:21 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE3MDE3NCBTYWx0ZWRfXxgn9jLqmkoZ+ e8SUX/76RK4U9vJ/+MYLKDBIuwARiu1I632USIgRQC+FCciEUW7D0jpnMnPcDgvw9lXd0VCGLRr AHVym4Ub1M0zWHA9PWkKQhyuaeTr2qYydstOe7PBBmGxFJNAu6KpNFNUtHt/f3sQL27pXwMzSBD 6Dz3f+T5Yrr+vQ+QlxSv5I2rrBiBHK3eO4Kz3QIZnEF8W7vKF/Vg5Hvi8NXB+HWG2wofSyaPI5Q 1p8nK/UjAVH9NmasztD5U8NGGUO4TZSunLDUxsOTtkQf6fgQNanOYuq4VVXG3jfeTkRA5sgRHig 1MoLymUQkTbcDm21tw7WBOffvGfGZdN1V2dkz/iZ8kpfNVpKP/UrM223axGPtC4hWiJMCAEx3m0 HUNHGz6PTYnHHLAmzsNGVz1eEG3IsEaIbeHFKR9gO8YlorML4JnHjGQAQUhkJlBi0mjB2erSUsX 2k7ooPO+JcAXaav5InA== X-Proofpoint-GUID: wO3MFv0FNni6nAFSAux-KYVcDWHSsfiY X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-17_04,2026-03-17_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 adultscore=0 spamscore=0 malwarescore=0 clxscore=1015 impostorscore=0 bulkscore=0 lowpriorityscore=0 priorityscore=1501 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603170174 Content-Type: text/plain; charset="utf-8" With the intended removal of PREEMPT_NONE this_cpu operations based on atomic instructions, guarded with preempt_disable()/preempt_enable() pairs become more expensive: the preempt_disable() / preempt_enable() pairs are not optimized away anymore during compile time. In particular the conditional call to preempt_schedule_notrace() after preempt_enable() adds additional code and register pressure. E.g. this simple C code sequence DEFINE_PER_CPU(long, foo); long bar(long a) { return this_cpu_add_return(foo, a); } generates this code: 11a976: eb af f0 68 00 24 stmg %r10,%r15,104(%r15) 11a97c: b9 04 00 ef lgr %r14,%r15 11a980: b9 04 00 b2 lgr %r11,%r2 11a984: e3 f0 ff c8 ff 71 lay %r15,-56(%r15) 11a98a: e3 e0 f0 98 00 24 stg %r14,152(%r15) 11a990: eb 01 03 a8 00 6a asi 936,1 <- __preem= pt_count_add(1) 11a996: c0 10 00 d2 ac b5 larl %r1,1b70300 <- address= of percpu var 11a9a0: e3 10 23 b8 00 08 ag %r1,952 <- add per= cpu offset 11a9a6: eb ab 10 00 00 e8 laag %r10,%r11,0(%r1) <- atomic = op 11a9ac: eb ff 03 a8 00 6e alsi 936,-1 <- __preem= pt_count_dec_and_test() 11a9b2: a7 54 00 05 jnhe 11a9bc 11a9b6: c0 e5 00 76 d1 bd brasl %r14,ff4d30 11a9bc: b9 e8 b0 2a agrk %r2,%r10,%r11 11a9c0: eb af f0 a0 00 04 lmg %r10,%r15,160(%r15) 11a9c6 07 fe br %r14 Even though the above example is more or less the worst case, since the branch to preempt_schedule_notrace() requires a stackframe, which otherwise wouldn't be necessary, there is also the conditional jnhe branch instruction. Get rid of the conditional branch with the following code sequence: 11a8e6: c0 30 00 d0 c5 0d larl %r3,1b33300 11a8ec: b9 04 00 43 lgr %r4,%r3 11a8f0: eb 00 43 c0 00 52 mviy 960,4 11a8f6: e3 40 03 b8 00 08 ag %r4,952 11a8fc: eb 52 40 00 00 e8 laag %r5,%r2,0(%r4) 11a902: eb 00 03 c0 00 52 mviy 960,0 11a908: b9 08 00 25 agr %r2,%r5 11a90c 07 fe br %r14 The general idea is that this_cpu operations based on atomic instructions are guarded with mvyi instructions: - The first mvyi instruction writes the register number, which contains the percpu address variable to lowcore. This also indicates that a percpu code section is executed. - The first instruction following the mvyi instruction must be the ag instruction which adds the percpu offset to the percpu address register. - Afterwards the atomic percpu operation follows. - Then a second mvyi instruction writes a zero to lowcore, which indicates the end of the percpu code section. - In case of an interrupt/exception/nmi the register number which was written to lowcore is copied to the exception frame (pt_regs), and a zero is written to lowcore. - On return to the previous context it is checked if a percpu code section was executed (saved register number not zero), and if the process was migrated to a different cpu. If the percpu offset was already added to the percpu address register (instruction address does _not_ point to the ag instruction) the content of the percpu address register is adjusted so it points to percpu variable of the new cpu. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/entry-percpu.h | 54 ++++++++++++++++++++++++++++ arch/s390/include/asm/lowcore.h | 3 +- arch/s390/include/asm/percpu.h | 54 ++++++++++++++++++++++++++++ arch/s390/include/asm/ptrace.h | 2 ++ arch/s390/kernel/irq.c | 5 +++ arch/s390/kernel/nmi.c | 3 ++ arch/s390/kernel/traps.c | 3 ++ 7 files changed, 123 insertions(+), 1 deletion(-) create mode 100644 arch/s390/include/asm/entry-percpu.h diff --git a/arch/s390/include/asm/entry-percpu.h b/arch/s390/include/asm/e= ntry-percpu.h new file mode 100644 index 000000000000..eba795ec0ea0 --- /dev/null +++ b/arch/s390/include/asm/entry-percpu.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef ARCH_S390_ENTRY_PERCPU_H +#define ARCH_S390_ENTRY_PERCPU_H + +#include +#include +#include +#include + +static __always_inline void percpu_entry(struct pt_regs *regs) +{ + struct lowcore *lc =3D get_lowcore(); + + if (user_mode(regs)) + return; + regs->cpu =3D lc->cpu_nr; + regs->percpu_register =3D lc->percpu_register; + lc->percpu_register =3D 0; +} + +static __always_inline void percpu_exit(struct pt_regs *regs) +{ + struct lowcore *lc =3D get_lowcore(); + unsigned int insn, disp; + unsigned char reg; + + reg =3D regs->percpu_register; + if (!reg) + return; + lc->percpu_register =3D reg; + if (regs->cpu =3D=3D lc->cpu_nr) + return; + /* + * Within a percpu code section and process has been migrated to + * a different CPU. Check if the percpu base register needs to be + * updated. This is the case if the PSW does not point to the ADD + * instruction within the section + * - AG %rx,percpu_offset_in_lowcore(%r0,%r0) + * which adds the percpu offset to the percpu base register. + */ + if ((*(u16 *)psw_bits(regs->psw).ia & 0xff0f) !=3D 0xe300) + return; + disp =3D offsetof(struct lowcore, percpu_offset); + if (machine_has_relocated_lowcore()) + disp +=3D LOWCORE_ALT_ADDRESS; + insn =3D (disp & 0xff000) >> 4 | (disp & 0x00fff) << 16 | 0x8; + if (*(u32 *)(psw_bits(regs->psw).ia + 2) !=3D insn) + return; + /* Fixup percpu base register */ + regs->gprs[reg] -=3D __per_cpu_offset[regs->cpu]; + regs->gprs[reg] +=3D lc->percpu_offset; +} + +#endif diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcor= e.h index 50ffe75adeb4..cd1ddfdb5d35 100644 --- a/arch/s390/include/asm/lowcore.h +++ b/arch/s390/include/asm/lowcore.h @@ -165,7 +165,8 @@ struct lowcore { __u32 spinlock_index; /* 0x03b0 */ __u8 pad_0x03b4[0x03b8-0x03b4]; /* 0x03b4 */ __u64 percpu_offset; /* 0x03b8 */ - __u8 pad_0x03c0[0x0400-0x03c0]; /* 0x03c0 */ + __u8 percpu_register; /* 0x03c0 */ + __u8 pad_0x03c1[0x0400-0x03c1]; /* 0x03c1 */ =20 __u32 return_lpswe; /* 0x0400 */ __u32 return_mcck_lpswe; /* 0x0404 */ diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index b18a96f3a334..05eb91428b42 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -60,6 +60,60 @@ #define this_cpu_or_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) #define this_cpu_or_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) =20 +/* + * Macros to be used for percpu code section based on atomic instructions. + * + * Avoid the need to use preempt_disable() / preempt_disable() pairs and t= he + * conditional preempt_schedule_notrace() function calls which come with + * this. The idea is that this_cpu operations based on atomic instructions= are + * guarded with mvyi instructions: + * + * - The first mvyi instruction writes the register number, which contains= the + * percpu address variable to lowcore. This also indicates that a percpu + * code section is executed. + * + * - The first mvyi instruction following the mvyi instruction must be the= ag + * instruction which adds the percpu offset to the percpu address regist= er. + * + * - Afterwards the atomic percpu operation follows. + * + * - Then a second mvyi instruction writes a zero to lowcore, which indica= tes + * the end of the percpu code section. + * + * - In case of an interrupt/exception/nmi the register number which was + * written to lowcore is copied to the exception frame (pt_regs), and a = zero + * is written to lowcore. + * + * - On return to the previous context it is checked if a percpu code sect= ion + * was executed (saved register number not zero), and if the process was + * migrated to a different cpu. If the percpu offset was already added to + * the percpu address register (instruction address does _not_ point to = the + * ag instruction) the content of the percpu address register is adjuste= d so + * it points to percpu variable of the new cpu. + * + * Inline assemblies making use of this typically have a code sequence lik= e: + * + * MVIY_PERCPU(...) <- start of percpu code section + * AG_ALT(...) <- add percpu offset; must be the second instruction + * atomic_op <- atomic op + * MVIY_ALT(...) <- end of percpu code section + */ + +#define MVIY_PERCPU(disp, dispalt, base) \ + ALTERNATIVE(" mviy " disp "(" base " ),0\n", \ + " mviy " dispalt "(" base " ),0\n", \ + ALT_PERCPU(0)) + +#define MVIY_ALT(disp, dispalt, base) \ + ALTERNATIVE(" mviy " disp "(" base " ),0\n", \ + " mviy " dispalt "(" base " ),0\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) + +#define AG_ALT(disp, dispalt, reg) \ + ALTERNATIVE(" ag " reg ", " disp "(%%r0)\n", \ + " ag " reg ", " dispalt "(%%r0)\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) + #ifndef MARCH_HAS_Z196_FEATURES =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) diff --git a/arch/s390/include/asm/ptrace.h b/arch/s390/include/asm/ptrace.h index aaceb1d9110a..495e310c3d6d 100644 --- a/arch/s390/include/asm/ptrace.h +++ b/arch/s390/include/asm/ptrace.h @@ -134,6 +134,8 @@ struct pt_regs { }; unsigned long flags; unsigned long last_break; + unsigned int cpu; + unsigned char percpu_register; }; =20 /* diff --git a/arch/s390/kernel/irq.c b/arch/s390/kernel/irq.c index d10a17e6531d..4843d87aa44d 100644 --- a/arch/s390/kernel/irq.c +++ b/arch/s390/kernel/irq.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "entry.h" =20 DEFINE_PER_CPU_SHARED_ALIGNED(struct irq_stat, irq_stat); @@ -146,6 +147,7 @@ void noinstr do_io_irq(struct pt_regs *regs) struct pt_regs *old_regs =3D set_irq_regs(regs); bool from_idle; =20 + percpu_entry(regs); from_idle =3D test_and_clear_cpu_flag(CIF_ENABLED_WAIT); if (from_idle) update_timer_idle(); @@ -177,6 +179,7 @@ void noinstr do_io_irq(struct pt_regs *regs) =20 if (from_idle) regs->psw.mask &=3D ~(PSW_MASK_EXT | PSW_MASK_IO | PSW_MASK_WAIT); + percpu_exit(regs); } =20 void noinstr do_ext_irq(struct pt_regs *regs) @@ -185,6 +188,7 @@ void noinstr do_ext_irq(struct pt_regs *regs) struct pt_regs *old_regs =3D set_irq_regs(regs); bool from_idle; =20 + percpu_entry(regs); from_idle =3D test_and_clear_cpu_flag(CIF_ENABLED_WAIT); if (from_idle) update_timer_idle(); @@ -212,6 +216,7 @@ void noinstr do_ext_irq(struct pt_regs *regs) =20 if (from_idle) regs->psw.mask &=3D ~(PSW_MASK_EXT | PSW_MASK_IO | PSW_MASK_WAIT); + percpu_exit(regs); } =20 static void show_msi_interrupt(struct seq_file *p, int irq) diff --git a/arch/s390/kernel/nmi.c b/arch/s390/kernel/nmi.c index a55abbf65333..20fd319a3a8e 100644 --- a/arch/s390/kernel/nmi.c +++ b/arch/s390/kernel/nmi.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -374,6 +375,7 @@ void notrace s390_do_machine_check(struct pt_regs *regs) unsigned long mcck_dam_code; int mcck_pending =3D 0; =20 + percpu_entry(regs); irq_state =3D irqentry_nmi_enter(regs); =20 if (user_mode(regs)) @@ -496,6 +498,7 @@ void notrace s390_do_machine_check(struct pt_regs *regs) schedule_mcck_handler(); =20 irqentry_nmi_exit(regs, irq_state); + percpu_exit(regs); } NOKPROBE_SYMBOL(s390_do_machine_check); =20 diff --git a/arch/s390/kernel/traps.c b/arch/s390/kernel/traps.c index 1b5c6fc431cc..095937c2ca72 100644 --- a/arch/s390/kernel/traps.c +++ b/arch/s390/kernel/traps.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -349,6 +350,7 @@ void noinstr __do_pgm_check(struct pt_regs *regs) current->thread.gmap_int_code =3D regs->int_code & 0xffff; return; } + percpu_entry(regs); state =3D irqentry_enter(regs); if (user_mode(regs)) { update_timer_sys(); @@ -386,6 +388,7 @@ void noinstr __do_pgm_check(struct pt_regs *regs) out: local_irq_disable(); irqentry_exit(regs, state); + percpu_exit(regs); } =20 /* --=20 2.51.0 From nobody Mon Apr 6 21:31:18 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 43EEE2D780C; Tue, 17 Mar 2026 19:54:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777293; cv=none; b=CEreiH6Hs3zUp9C0PTuuTEaQn7oaEz8fk1Dn4IuVSsIe+ls19lEIbbR7Bs3ptBf0EW9Ncr4MfivFwszckA3Xv0U07ZS/RACL4+CUOF1CTWOo3sZje5XLe7CgL5P4UKP6plEBtbhHZD10yaStse79MdseQkW2/oFmTYHV9f+2E/4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777293; c=relaxed/simple; bh=3gl86jo+8bIc3OfiiGsNB4ocleZ/BnIClnxir6Y4Ldk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qpvFZR1JGuXUIoGydNgM+OCv45/kqOjFIC30EUDVUPjE+OQ+f49inJu4JPOTgWrSGX64fh2wM/thw7eRjPLrw9EaEBvP5460r9GZo/jsahW2Iu+SlPq53qkaKEjTBKNmaa8vpSiA6BsOnpIRbrr9+yvOHgSpdahfmBgx2PGaB2I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=j6WB2TVI; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="j6WB2TVI" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62HJeIwf949953; Tue, 17 Mar 2026 19:54:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=iNCx15pa7MH7X5LzS pxKLgGYfzmW3lUEW/2ZCCL1/kE=; b=j6WB2TVIWdO5S0P/74GY2dOZeTrA4Fyqr lw5LzHVsoilF5VYK7jZGpVnBtWbUICowrhSNNwUz0XyTYZD0Ddqa+Xiz54d8nyT0 3iX7tKl35MxHDb2UGg2Yim/NKFjGsqM5lLSJGwIwyuccb5wnM1NPlD/r15babRBn 7dWgQ6QF0b8sUh0PTQ/x9mrcaFL504XSrTpeZzdHV00yNpUALoviTT6cloytyTvF 1HQrL4fMQgoIK6NmFcNBhh6ktL42KoH1D59wTWZET/7/p+6J7nQjrPgnfAWYSMFd 2q6iUYe4bCgQt0WWR9B5vIB3jQc/LR//UfhP4hJ/scohJ8+rxmlBQ== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvybs6m85-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:43 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62HJjO3v015676; Tue, 17 Mar 2026 19:54:42 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4cwk0nasq0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:42 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62HJscLe50987490 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 17 Mar 2026 19:54:38 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0DAA52004B; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DA61F2004D; Tue, 17 Mar 2026 19:54:37 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 17 Mar 2026 19:54:37 +0000 (GMT) From: Heiko Carstens To: Peter Zijlstra , Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 4/9] s390/percpu: Use new percpu code section for arch_this_cpu_add() Date: Tue, 17 Mar 2026 20:54:31 +0100 Message-ID: <20260317195436.2276810-5-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260317195436.2276810-1-hca@linux.ibm.com> References: <20260317195436.2276810-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=MMttWcZl c=1 sm=1 tr=0 ts=69b9b183 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=VnNF1IyMAAAA:8 a=SujZmW7ZfIunUtho7CIA:9 X-Proofpoint-ORIG-GUID: T5ZNf9kT-SlQ1bw49jZC965DXJ2AEaUL X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE3MDE3NCBTYWx0ZWRfX5mPInKBvsUkp UR5N+l5zSr92h+zISTvrPOlOVHaFkFTXcATCt51x1Fj3aFkxNFgvxnWQhUxoZztFdOJCbjC209O od8QST2M18+fZCsn34dyfRIeZcH7fQn3qqzNryQki2A2PWOFkSRuhJ/uuVkrzVOL39JW1bC1s0O cRORDiVIK0bg2gqANAcWnmhJZTe7BuNP3Yp2HtQPzal70KBXa9MflEiCmyGHfyAq5cjVxhpDNl9 rhhqL4jBqk4iaLQY6UOZRcADRR46aG79VCeyDaApxSx8PeDvbFLxbUyo/54JZ0kTqtArPe82Ixi 2ABgUoFST9AewHT2aQh1+dpJLaD6eu1d2n6OL+ib5eM5ThDMHl5c/RG4nvByl89tAma2b61/MVZ dxZdrr0q6UGvhOQ1aLSCT7hgKJhDitPLVrGGQt3TmFiXfZwngUElb6FnBde1tl0lMa79yCz7MM9 0tceivmIB/WGxpiV+Jw== X-Proofpoint-GUID: T5ZNf9kT-SlQ1bw49jZC965DXJ2AEaUL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-17_04,2026-03-17_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 lowpriorityscore=0 malwarescore=0 spamscore=0 priorityscore=1501 impostorscore=0 adultscore=0 phishscore=0 clxscore=1015 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603170174 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_add() to make use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~76kb (defconfig). Also more than 5300 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. With: DEFINE_PER_CPU(long, foo); void bar(long a) { this_cpu_add(foo, a); } Old arch_this_cpu_add() looks like this: 00000000000000c0 : c0: c0 04 00 00 00 00 jgnop c0 c6: eb 01 03 a8 00 6a asi 936,1 cc: c4 18 00 00 00 00 lgrl %r1,cc ce: R_390_GOTENT foo+0x2 d2: e3 10 03 b8 00 08 ag %r1,952 d8: eb 22 10 00 00 e8 laag %r2,%r2,0(%r1) de: eb ff 03 a8 00 6e alsi 936,-1 e4: a7 a4 00 05 jhe ee e8: c0 f4 00 00 00 00 jg e8 ea: R_390_PC32DBL __s390_indirect_jump_r14+0x2 ee: c0 f4 00 00 00 00 jg ee f0: R_390_PLT32DBL preempt_schedule_notrace+0x2 New arch_this_cpu_add() looks like this: 00000000000000c0 : c0: c0 04 00 00 00 00 jgnop c0 c6: c4 38 00 00 00 00 lgrl %r3,c6 c8: R_390_GOTENT foo+0x2 cc: b9 04 00 43 lgr %r4,%r3 d0: eb 00 43 c0 00 52 mviy 960(%r4),0 d6: e3 40 03 b8 00 08 ag %r4,952 dc: eb 52 40 00 00 e8 laag %r5,%r2,0(%r4) e2: eb 00 03 c0 00 52 mviy 960,0 e8: c0 f4 00 00 00 00 jg e8 ea: R_390_PC32DBL __s390_indirect_jump_r14+0x2 Note that the conditional function call is removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 65 ++++++++++++++++++++++------------ 1 file changed, 43 insertions(+), 22 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 05eb91428b42..c92b41efbc47 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -127,28 +127,49 @@ =20 #else /* MARCH_HAS_Z196_FEATURES */ =20 -#define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ -{ \ - typedef typeof(pcp) pcp_op_T__; \ - pcp_op_T__ val__ =3D (val); \ - pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - if (__builtin_constant_p(val__) && \ - ((szcast)val__ > -129) && ((szcast)val__ < 128)) { \ - asm volatile( \ - op2 " %[ptr__],%[val__]" \ - : [ptr__] "+Q" (*ptr__) \ - : [val__] "i" ((szcast)val__) \ - : "cc"); \ - } else { \ - asm volatile( \ - op1 " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ - : "cc"); \ - } \ - preempt_enable_notrace(); \ +#define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ +{ \ + unsigned long lc_pcpr, lc_pcpo; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ val__ =3D (val); \ + pcp_op_T__ old__, *ptr__; \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D &(pcp); \ + if (__builtin_constant_p(val__) && \ + ((szcast)val__ > -129) && ((szcast)val__ < 128)) { \ + asm volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op2 " 0(%[ptr__]),%[val__]\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "i" ((szcast)val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + } else { \ + asm volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op1 " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + } \ } =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_add(pcp, val, "laa", "asi",= int) --=20 2.51.0 From nobody Mon Apr 6 21:31:18 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59A01330644; Tue, 17 Mar 2026 19:54:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777293; cv=none; b=G4CX+4J2emz9n8VgMDBD53RWgk/E8NRQB4ldPGT4kdKOLWhe/xU+Ez/40quMkk3PPZbToHiPRw5DQ5avcAVdgacYGcLfEwxz0VFvVXLrzhGEmncPRR+KEs5pvCsVAMhfvns2JzXWyTcPg9tbMDhMubeYM37dAzCvLHT99j86LQU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777293; c=relaxed/simple; bh=bQFMXRKJkbPwxU7aZA3N8XWzRAKDXdBedHHhNU3zBCM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FLEyBxwM+N9bAp4Os4LUo5CFJ8senkcM74gQ8J89DK4hrzXDangD7nEVaP54HXXgWXjudNNNjc0UnNlrJ/qs5kbQWX295HhfLj/xiKhjLB6vVzys4IQTzZ+0qQyhdSeF0MqVS6FawY+Eis1d8pVnVv5qcOqiPa5F7YDdf+QAwLA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=dRzzoHFK; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="dRzzoHFK" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62HIZZTc382519; Tue, 17 Mar 2026 19:54:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=hW2kYde6bQGPyTcsi S8vv+arZjmHxxF6nD/NT5ArOnM=; b=dRzzoHFKlcEATUxSYRGyT069YYgLzRJ3e 2IgiB8wcSXecYZijHSgKbtevzZLDTmBPGKkQFcr+V3bESAszqpA4R2kgAYm22KGE oAK9w7g18cjeJXPNY7oD/8hRQV9lakqBhg5xuRNmIFX1M3Gm75qSK05FHrX3mMdZ ej3iJZCV88d1FySUvoRNGHmRiEuActphUn3VGQshaOi0C/fB2YCqgLmm/I6m2UvQ CJsDPbRGI5cRJUBMQho81QA61mTLfatgRrke65wf5PbSrPi7vEOKG48yEIbGdwzk DQZfJE5npbi+iy5PnnVuokoP/yH5dB854Xo3IU4j2jgQkxtEDVXWg== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvyaue1v5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:43 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62HGX2f1028452; Tue, 17 Mar 2026 19:54:42 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwmq1ak3u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:42 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62HJscBp50987494 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 17 Mar 2026 19:54:38 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3B56D2004B; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 13A0E20040; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) From: Heiko Carstens To: Peter Zijlstra , Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 5/9] s390/percpu: Use new percpu code section for arch_this_cpu_add_return() Date: Tue, 17 Mar 2026 20:54:32 +0100 Message-ID: <20260317195436.2276810-6-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260317195436.2276810-1-hca@linux.ibm.com> References: <20260317195436.2276810-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE3MDE3NCBTYWx0ZWRfX+ugNiiQWQBk/ LwhFfhYfZ3cSqOgwIxC1cJev+cVt9gz2Om0T1Vl6LYoN3fd2H20h0t4KSxqUJm6R71vooWBSH4L 7iuAnqt0VtVoKXfI913cemql667FIPLVkdWeIQPozyYTrmX96OHFAmiD4js6eAoP+QA/FSiDh6a SbWhNfV2JVQBBXNmnC9oEhEfd56WsVkivaL1S4o+gSNxIMBq3XuyINvluAKTz4xYNxCSXecwawF kdxtfLRnUV4ZmLUXgWj7USSPIuF5QDz97vGuOqubaVEM9yHhpmWgInsGeCrQ7uBGb38+anMmjuT iZmvVD6pNZy3nk7P2wonJwnDE+lGt2js6V6f/S1y7uFfpfGiKlSQ+Z4eW0h0oKd3a6FtgykI+c5 xTwJkJaabdfZ1vMF8ZelGbprB4DQ80cyKWrF7R4kynvluJlwUeKamDubJITu72jz743V/dNffBr U+fWAFUYXMjUh9XwsVQ== X-Authority-Analysis: v=2.4 cv=GIQF0+NK c=1 sm=1 tr=0 ts=69b9b183 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=AksqfXeGXF_AYWYsu5gA:9 X-Proofpoint-ORIG-GUID: Yp1qgF_h7ra2zBREn3AzG82d0GI8wB-K X-Proofpoint-GUID: Yp1qgF_h7ra2zBREn3AzG82d0GI8wB-K X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-17_04,2026-03-17_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 bulkscore=0 lowpriorityscore=0 malwarescore=0 phishscore=0 impostorscore=0 suspectscore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603170174 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_add_return() to make use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~4k (defconfig). Also 66 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index c92b41efbc47..6e932f45f128 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -177,17 +177,29 @@ =20 #define arch_this_cpu_add_return(pcp, val, op) \ ({ \ + unsigned long lc_pcpr, lc_pcpo; \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - asm volatile( \ - op " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D &(pcp); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ : "cc"); \ - preempt_enable_notrace(); \ old__ + val__; \ }) =20 --=20 2.51.0 From nobody Mon Apr 6 21:31:18 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D4D6344057; Tue, 17 Mar 2026 19:54:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777295; cv=none; b=NARXPIOK6HwyW7QKA3D9HSIPTgzwECywu3b2Pup/d9Dgq/DTkJZjEdDbfqBt6DeBYxRsMI/0gmcckyiczixswj5HYCYX8K6YD0T/iVz4c1yFSFsVIvqPgdX/YZSNkUi/PQ83shh3r9QoV435y25Wyp691ZHjAcXH+JWJDGy1L/k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777295; c=relaxed/simple; bh=YBxhvxEpN/HTLz4ZjGnc5Y57lSjeh1Jdhu+6fKkajQk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=i51ZyioAljlrTDJYW4AUobkQIua962u7ZmsSNOBj4+vaUQbhST/mAnhtMt9d+MGuvMrVxFCYPYsXn39/qbVwBhXs01ctuGPTNc/kXjIBE06cCmz5UOp/viETdfxQagR2T9rkupMM7zj5mC4mRG78OJEhMEQpfk0W9zcqMy5GjTw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=oKvKwIS/; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="oKvKwIS/" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62HIKcNR617441; Tue, 17 Mar 2026 19:54:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=xG2Ad3gUwaVAO7PyV lzDgxDYYf5LcnXNGAKdUk6zeYA=; b=oKvKwIS/MxDbhVk0x3BHdw+R9pl8I24QZ wYgdKtYML1pYXwDF0eFpFKhk0UdQzfdoZEWdGMRL3fu0quiLxrCfXvH3N5Z7n/HL iNnkaoN2GIHxFOWKGBI/eqACVEOvnduU7egKDMoGbenQR+G7GqPPICNGyaYMqFag H3SF34Q+zEirk+polukHa4LPOUtcTSReNlXXXNmLB9OP2RWcStT7/J1hLLxK6tQg vPGM8wuFltt78vTFlIijwViJKFyorwkQXCfw54XvvQhTU/cELrh2PdG394aGnmUK 8aL+NdKswBLVO9MYJNEWdrUYjgBjYbiuOUMOsUaxXR1bO9WCrcv6Q== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvybs6m86-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:43 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62HIeQQE004563; Tue, 17 Mar 2026 19:54:42 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwj0sawx3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:42 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62HJscsH50987496 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 17 Mar 2026 19:54:38 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 68FF920040; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 414C52004D; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) From: Heiko Carstens To: Peter Zijlstra , Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 6/9] s390/percpu: Use new percpu code section for arch_this_cpu_[and|or]() Date: Tue, 17 Mar 2026 20:54:33 +0100 Message-ID: <20260317195436.2276810-7-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260317195436.2276810-1-hca@linux.ibm.com> References: <20260317195436.2276810-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=MMttWcZl c=1 sm=1 tr=0 ts=69b9b183 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=VnNF1IyMAAAA:8 a=4dLYTkTKlH5g-JvQF6AA:9 X-Proofpoint-ORIG-GUID: GO7A18wmWyi6hnPV3aYCTGmB12z1yBh6 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE3MDE3NCBTYWx0ZWRfX7/za8DarR/E3 2K/X9RtEOn5hBl32zCEYFDAvSNVGE4vztVZk1Qns2SKAqk+ZFcp6JhfoGZukaOkyETtYmZAEnXV UX0PGimrxru1l1LlspW37CjxGnX9T0n6WOl94CFGHP72FrFDwzK7sf2/ONo/8qkWQkkuLArWVbs CZf4fOYbX2fhwcHp1YTmrdCPOoqxAZjd8b0Njpit/cG6KkLq79Ie3sIWACY11u+mpfxtZHP2bP1 Lj9QMXB3qMqyQR6bDlDX2YKth4HCH1C8E0VJMemIJ6e38NuB4CZQfNugtIeWQ9t1BqWVpX+b0Bb LCWCFr64f/EEQ+yV4+P7chWqNHGZ+EMGdx260rC8hVsmpO45/Jp3tTJH0zdSQvVlxW7KAUpN0e1 A2gYDfKRBiTfqOUA9NUZYyNuVTX+r7/Lw9MzJ0vP1JbravdjPBIwJfwUlhtiyN83dv4J+oO8VCp Bqd7T9syifHEmwVwG0Q== X-Proofpoint-GUID: GO7A18wmWyi6hnPV3aYCTGmB12z1yBh6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-17_04,2026-03-17_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 lowpriorityscore=0 malwarescore=0 spamscore=0 priorityscore=1501 impostorscore=0 adultscore=0 phishscore=0 clxscore=1015 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603170174 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_[and|or]() to make use of the new percpu code section infrastructure. There is no user of this_cpu_and() and only one user of this_cpu_or() within the kernel. Therefore this conversion has hardly any effect, and also removes only preempt_schedule_notrace() function call. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 6e932f45f128..f26f03b0889c 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -208,17 +208,29 @@ =20 #define arch_this_cpu_to_op(pcp, val, op) \ { \ + unsigned long lc_pcpr, lc_pcpo; \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - asm volatile( \ - op " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D &(pcp); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ : "cc"); \ - preempt_enable_notrace(); \ } =20 #define this_cpu_and_4(pcp, val) arch_this_cpu_to_op(pcp, val, "lan") --=20 2.51.0 From nobody Mon Apr 6 21:31:18 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A555322B83; Tue, 17 Mar 2026 19:54:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777292; cv=none; b=KjyMuzeM9Lk46fMXW6g6uDWuM4co/4rpwhGH+nbivlKhDosun573ZEfqULwF4jXtWCSHkKHM1Vqv4WffOkD8vD+D365/2vD1MTxYbccn2s6TLkP7cIO4IsSbfpUH9mDTdLHreugEkgFyAondNlAUxionPY0bq50EkZ1FYvUERhA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777292; c=relaxed/simple; bh=zMuIajSoxUlLt+udEfCDgyMqmfbpc/BWh05b1NnASl4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JLR0ZH0pWnxnR5utPrGgSHuEOe/+6CPx72fukQwbhI8v9LoSb6JHrhaDYgs7wv9+9WCt6w62veQ/LkPPFdUk43/fKQBCbQCHNbzgr4LbXhp0py3mHiNBGL448Oe9F/D8XqMJl1th6V8sKZZWiAjtJ2XhVDSHH3unS0nue4yvcrE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=HAzHALBi; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="HAzHALBi" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62HHVE0w532485; Tue, 17 Mar 2026 19:54:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=F21vk/vMMVdHdNApu r+5KHt3UCM5mJcfn8dG2Pxm3FI=; b=HAzHALBil2mdGJL6T4DKBDU3mSTtudasQ fFB4zZ6RpfFdaiYgDBKmynm5jnGRGfsec3Jd7GVRAoLQUrPAFNm21brm7+gQD4S1 fHXPz0bAUwkT9Oh+GqHdkF+GBurlOvqh3mN3PxC7vl2YlWTn8jwuJm2osr97mjv8 w7K7zR5jwN+9CGhaH7FxfiN3jrDW7MYip0bCkdyxF3IhVzfmB43tPjDvfeyn4a0q /+sn2pkFkHusIIxKun3TRAL9B4l95DLTfUttym7ZQDsARDI4RjWi7gzYtDve7ppj 5STXUJVtoqey1vPZ+IHTjj4Fm9V4B9eBWsTqfOCFghWKdW2QYMC3g== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvx3cx497-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:43 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62HJZ2gF028785; Tue, 17 Mar 2026 19:54:43 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4cwkgkas7w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:43 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62HJsc5450987498 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 17 Mar 2026 19:54:38 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 96E692004D; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6F36D2004B; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) From: Heiko Carstens To: Peter Zijlstra , Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 7/9] s390/percpu: Provide arch_this_cpu_read() implementation Date: Tue, 17 Mar 2026 20:54:34 +0100 Message-ID: <20260317195436.2276810-8-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260317195436.2276810-1-hca@linux.ibm.com> References: <20260317195436.2276810-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=arO/yCZV c=1 sm=1 tr=0 ts=69b9b183 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=V8glGbnc2Ofi9Qvn3v5h:22 a=VnNF1IyMAAAA:8 a=4bOSrtcVyj7iNoH5sBwA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE3MDE3NCBTYWx0ZWRfX/DiTJ80GITua PZtiSehD6Tc4nSLxEnim4fm00sHGt7uESFTJXQ35MqQSjNcvUBmjsbD0MG5u6T9cnVJkOqaiKvG IFGha1H64U4nWwSkuXy7YqUFf4CSVReC70Yk5T0X2vbkVpzaigGguIcP4U42JPQaBeOm2ClOXLU nuOHKLEdpeOD1KOoyBTYwaOH8nZyZHfWxGVRcS84EkIz4QAkPTxeSFaH4FbrMwZToW9P2d1U9xq 7Ai8a1kshP2J6+fshr/OycoHYJTAVUItAWgBFh4sY7x/T16f4USKVE75SqIjcXObG6ic2WyVDCL Ag9sK/k0Y6chkXkZQXXZ+HwSkerZDpDvfInxBQw/QoBKpSbDL2tUUeqx90P555mr5Z+SNlOsq3H suW4YJA9Hc3kcnjgWsdCLRrX/0iX7K7AgUIH0gSIQSEiLtZoZAcC6PKilDe5CpoEiTEjyloH1Vf HUsQ65MWWmptXw4NU5A== X-Proofpoint-GUID: a96LXDpE5K0CGuaf59RMbApbgvNJEfGA X-Proofpoint-ORIG-GUID: a96LXDpE5K0CGuaf59RMbApbgvNJEfGA X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-17_04,2026-03-17_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 spamscore=0 lowpriorityscore=0 impostorscore=0 adultscore=0 bulkscore=0 suspectscore=0 malwarescore=0 clxscore=1015 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603170174 Content-Type: text/plain; charset="utf-8" Provide an s390 specific implementation of arch_this_cpu_read() instead of the generic variant. The generic variant uses preempt_disable() / preempt_enable() pair and READ_ONCE(). Get rid of the preempt_disable() / preempt_enable() pairs by providing an own variant which makes use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~1k (defconfig). Also 87 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index f26f03b0889c..35e48d193d99 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -240,6 +240,37 @@ =20 #endif /* MARCH_HAS_Z196_FEATURES */ =20 +#define arch_this_cpu_read(pcp, op) \ +({ \ + unsigned long lc_pcpr, lc_pcpo; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ val__, *ptr__; \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D &(pcp); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [val__] "=3D&d" (val__), [ptr__] "+&a" (ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (*ptr__), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + val__; \ +}) + +#define this_cpu_read_1(pcp) arch_this_cpu_read(pcp, "ic") +#define this_cpu_read_2(pcp) arch_this_cpu_read(pcp, "lh") +#define this_cpu_read_4(pcp) arch_this_cpu_read(pcp, "l") +#define this_cpu_read_8(pcp) arch_this_cpu_read(pcp, "lg") + #define arch_this_cpu_cmpxchg(pcp, oval, nval) \ ({ \ typedef typeof(pcp) pcp_op_T__; \ --=20 2.51.0 From nobody Mon Apr 6 21:31:18 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 653C433F37F; Tue, 17 Mar 2026 19:54:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777295; cv=none; b=QyJRFxhZ6f6qwoCJUThYq1Jtcwn45MKIt3R1bcH8WaYQ691cmutiM61Pr4Uff9bgwtStB+Ke+oneBBSOIKjgfLmo2a/wzq+idfzfF/3Ro9Hmi0SZJ/oIk2oRGg+R6JchombGXqZ/zIoAY8wITuC0W/3OEoSAz90Z2YBeKxoCELg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777295; c=relaxed/simple; bh=W9ULdutoRWAQybVggzg2wgpCF4KcYj1yMLN3VI+bc2g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KtOMtzfc8sS227JhLWd/WkIFvCi7XmNjkGcQJ6hygRigAYJ19kbktwRm0osRDZ+p8gEx8VIqjibqhPP/POArbdMXf5I8aG5TPEbmAfG7rG2qIOUBfMxpMEtiIHH476fVqxuTSdLIrzhwIOFLcLtc3z+UTftiaZ9CKGU8+6XeB/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=SAAbnesV; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="SAAbnesV" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62HChpqA3558410; Tue, 17 Mar 2026 19:54:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=mEx2A5dLN+FO7uPef qdiultlVyfcX81GA6Ne2gPBo5g=; b=SAAbnesVGHhki9JWnxyOU84o6Q/WO5HIn +a9Jum44iJ8fNeWzhVKCUFz9o4ZUCQ3c0q1/aTenKZrGHRtstFkCpBY0KT9zVRrV yfH0hahs4ri6mDKIXirXx1gqgeOk/hcsW0HZwJwyNyxaOdwwJsQyikLd4/PTw13h haPhlfiY8ffl/zqUenIJCgQbAg2Zh5ermAifg2CuOXOWnqEPcdlPevj4ZkyiEjdq 2H4YRyzsFOWA3BJWgl8kLhMXkpy1QLGfhgPSRQiEIT3wapyhKMtdJXa32rIsrXiu pVylRFkUvLFW/AShwQuBYx1qLtInvWU3pjJAdbekP7HMFCkQxHXTg== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvw3hx8v4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:43 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62HJ3V3N015655; Tue, 17 Mar 2026 19:54:43 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4cwk0nasq2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:43 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62HJscDh50987500 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 17 Mar 2026 19:54:38 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C4D392004B; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9CFE720040; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) From: Heiko Carstens To: Peter Zijlstra , Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 8/9] s390/percpu: Provide arch_this_cpu_write() implementation Date: Tue, 17 Mar 2026 20:54:35 +0100 Message-ID: <20260317195436.2276810-9-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260317195436.2276810-1-hca@linux.ibm.com> References: <20260317195436.2276810-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: wVfREDRjqEUGIQ1lGT9BTkT6RIV_e82U X-Proofpoint-ORIG-GUID: wVfREDRjqEUGIQ1lGT9BTkT6RIV_e82U X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE3MDE3NCBTYWx0ZWRfX2WaGt1HuxFix WL1M2GmnU5h0F0MT6hya0DkAKecsJffQ/ZnS/Ik/3xXro4fWGG2k5tt5hE7l3Bqfi9Ydlkv4Lf5 38k/rHHnApuWU2HcqSJiiF2XNbq1XxqwOpwD4QYnNjQmAwBuIlH+j9K2qmfjPlfAuwXDcJE/ZO0 49GTyAXamS90aiI4GQv2gTrDn4zs5faxoz+6fzTMnuQP7deu/VSSidcYUJkpSgDQJcFbIbBV0RW T8xyCLtWpODRHlO3B4dpNeZWmhG1Q2Dg4vejhlSfV4AOhy5aPx4WsA+HXm66tFvdxhm+5vZNNcl Yfrq8TdOuRmzcOlYI5c0zA7AHBZ/IwfqlN/E9DmsSlbOMob+NxzfQ6Jegj2yR1dYr8A4/xJAggB X8zUyDwuB0lja2jnSe4/RFYVMzXjEItirO5BeCXYGZxetIEkW1XZ8GSovva/HiHS/ii2Zua63CV q1zdztQIHVzt+5iG/Kw== X-Authority-Analysis: v=2.4 cv=Hf8ZjyE8 c=1 sm=1 tr=0 ts=69b9b183 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=VnNF1IyMAAAA:8 a=ObmbXjF690NQJU6EaucA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-17_04,2026-03-17_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 lowpriorityscore=0 impostorscore=0 bulkscore=0 suspectscore=0 priorityscore=1501 spamscore=0 adultscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603170174 Content-Type: text/plain; charset="utf-8" Provide an s390 specific implementation of arch_this_cpu_write() instead of the generic variant. The generic variant uses a quite expensive raw_local_irq_save() / raw_local_irq_restore() pair. Get rid of this by providing an own variant which makes use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~1k (defconfig). Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 35e48d193d99..7a411b75bfa6 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -271,6 +271,38 @@ #define this_cpu_read_4(pcp) arch_this_cpu_read(pcp, "l") #define this_cpu_read_8(pcp) arch_this_cpu_read(pcp, "lg") =20 +#define arch_this_cpu_write(pcp, val, op) \ +{ \ + unsigned long lc_pcpr, lc_pcpo; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ val__ =3D (val); \ + pcp_op_T__ old__, *ptr__; \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D &(pcp); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]", "%%r0") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "=3Dm" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ +} + +#define this_cpu_write_1(pcp, val) arch_this_cpu_write(pcp, val, "stc") +#define this_cpu_write_2(pcp, val) arch_this_cpu_write(pcp, val, "sth") +#define this_cpu_write_4(pcp, val) arch_this_cpu_write(pcp, val, "st") +#define this_cpu_write_8(pcp, val) arch_this_cpu_write(pcp, val, "stg") + #define arch_this_cpu_cmpxchg(pcp, oval, nval) \ ({ \ typedef typeof(pcp) pcp_op_T__; \ --=20 2.51.0 From nobody Mon Apr 6 21:31:18 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D45F344046; Tue, 17 Mar 2026 19:54:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777295; cv=none; b=DaLZQb3rFCrZCBJW2o9RCCohiUV4NLI6YWroUkjYFOi3rTB/ue5R6CUKgFzGGBkuhHSThMAgo6Tz2HRvCrMq7ANvhYv7318QWPjjCHp8unwz6zqrhjZIrzkUfrgEJBKk/MYdadlhghmjLbDcbOCz6BJPePZQx4qkMvYn3Aaeimo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773777295; c=relaxed/simple; bh=cyiIEE70OvqU/8K935m4rjmMWeyeJ4lamNcgOtsTcvc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JoR59uax89uMChTyNLiccIsZV23EUEzDy4GMZ4qmkwIDU2wbF9PPdSv9MUeJXX+ra2HSPIlHd/QnailpCbgshs+sKpRCNR5gg9WcUCNKgn3BHhsyDi//nYO8fBnv2UE1jgzph2XmVQ+w8Sh9U/LuUHCyuBwGoJpMRqcnR5U1Oq8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=rxPT7j2j; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="rxPT7j2j" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62HDbcCG2776227; Tue, 17 Mar 2026 19:54:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=TynVgsv2JvJtZX0Ft fpc707Evt8OnZ6jw+3qaze2k+g=; b=rxPT7j2jDWkUUgMGFLker+JfB6WKqaLDG otNdl5p6HT2snn20OTYPzWYdkKw4ov+iLEXPhTFJSfPbrWvs4pv5NPUXqEYIFGuR kWIpNa3EmiFe4TAGRXSn0gx1x2oql8t1+EfaJrQ/ZQIV8V9sXrPezStBvWfqchPx 83QSKTI9R0V+Z9xbe5MJEUc+i9k69bfox8LEQBjSSuM1tl5W3vOWD3q7johOODQS NJZcutJk4yMiPZIVktclzub05/DgEtwN35ItajZ829cFDQ854phVGEOlBXdXDv9V YlcOb2CHMkdYQDYPhlE0dthdFsaGgdXJnT/kZ8ZB2rjaBfSBx/RIQ== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4cvybs6m87-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:44 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62HId3IU005380; Tue, 17 Mar 2026 19:54:43 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cwj0sawx4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 17 Mar 2026 19:54:43 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62HJsdmn31851038 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 17 Mar 2026 19:54:39 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F2B9620040; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CAEE02004D; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 17 Mar 2026 19:54:38 +0000 (GMT) From: Heiko Carstens To: Peter Zijlstra , Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 9/9] s390/percpu: Remove one and two byte this_cpu operation implementation Date: Tue, 17 Mar 2026 20:54:36 +0100 Message-ID: <20260317195436.2276810-10-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260317195436.2276810-1-hca@linux.ibm.com> References: <20260317195436.2276810-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=MMttWcZl c=1 sm=1 tr=0 ts=69b9b184 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=VnNF1IyMAAAA:8 a=Q4reuYiO1UekjhQ8NNAA:9 X-Proofpoint-ORIG-GUID: oL1te38Vtt7yZuJQm4julqVNdWk7rdYS X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzE3MDE3NCBTYWx0ZWRfX16RCjScsw9bM zVSU57SboglzvWOY/cUNYT4gpzf+lZagOpB+ZziFkXx4q83+00x0H0C+lBhg1W8TnmctmlHjfP3 vMO+7T3bmVSKt9qDC6QcDzj5vJEwKkVv3aXYa6WW6ndE9y6ZKh+QMgCd9X4/dvthfcP7V/48R7k +f1Sxqm+ZMw82RSM6p2Ji30NgQEXHZ6W7KwjPvRuuMWjvdjmSZz7F1qGYDXvt7aYzX/JRgTWGax YVk/tKHwN6sK1IRJhd2Sm8Fnm492wLxmHV4cMMSOsIUS3H4qsFKpAlBhf45izGsrM3RbeteL20s mFpflGAdIAuvIwgFMKdfNp9+OrfBfjcokx5sH1E2LgidBZSomO1aSIIPzrXO/GH6pLG/Ngx21pc F5ZRIug5OD5mXi9OIY6mRtMeNXV7+lQ10Ylh7lNiQVeW3hipsdu7G7z4HRcvLcuhMZuK4K11vKh 2q1cQCf1Zd5Fdqq5G2A== X-Proofpoint-GUID: oL1te38Vtt7yZuJQm4julqVNdWk7rdYS X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-17_04,2026-03-17_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 lowpriorityscore=0 malwarescore=0 spamscore=0 priorityscore=1501 impostorscore=0 adultscore=0 phishscore=0 clxscore=1015 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603170174 Content-Type: text/plain; charset="utf-8" There are no one and two byte this_cpu operations within the kernel (defconfig). However even if there would be the s390 implementation, which uses a cmpxchg loop, generates a very large code sequence due to the lack of native one and two byte cmpxchg instructions. Remove the s390 implementation and use the generic implementation. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 9 --------- 1 file changed, 9 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 7a411b75bfa6..11c4d64ba96c 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -51,15 +51,6 @@ new__; \ }) =20 -#define this_cpu_add_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) -#define this_cpu_add_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) -#define this_cpu_add_return_1(pcp, val) arch_this_cpu_to_op_simple(pcp, va= l, +) -#define this_cpu_add_return_2(pcp, val) arch_this_cpu_to_op_simple(pcp, va= l, +) -#define this_cpu_and_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, &) -#define this_cpu_and_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, &) -#define this_cpu_or_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) -#define this_cpu_or_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) - /* * Macros to be used for percpu code section based on atomic instructions. * --=20 2.51.0