From nobody Sun May 24 20:33:27 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F5B73BB9F4; Fri, 22 May 2026 14:13:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459208; cv=none; b=RZyDqyg75NmPvwi2+WYS6cSxQJxYnpzxcBbIOQ27n4msEtmotoj/gt54ZhZ9oMQ+0fgcMrifkVTaCOd5ZSwLW14oxRi7p1mAfjkn81QlgBRvwkSy8fowPFkYiOm2dVQunC3jcqs/8MhpC3till0jc6kdIBNXbebk+YSojOHVBfM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459208; c=relaxed/simple; bh=cFdDtHt2t9Nwc7PkeGHAotk/p/U7VHiOsP/nse8/dzY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rcRigwlAGRKA4Vb8/xAeDFSu7WjXNoZYQrbw4b88hMC4dvIb1WSuDlrMys8FbyOJ+PUhIy0xHru2DJZ9QORLKBgRimryl8jx1pCUo9tXHu5de6P1asOsN2KmI9Mpm2LumTbjoGRxvKbNsYfApB7OJ+JSkASOWtPa7rHSqufrnUE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=WmA8bdV6; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="WmA8bdV6" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64ME3Kcr220727; Fri, 22 May 2026 14:13:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=T+1AB2SUc8Gy4QtAY Boc2rYVnyx3Gryuv3jv9GwWItM=; b=WmA8bdV61K6DKcRPhaQSbMTs6gZSW2Q6F JwZmJEbsnPQN17EuG0WpppP+iGaRlQHfAmkUTpM5HP8UM8T381u3PtIritoCcrd9 TMrTQYRleTQ9UavfmXXvS7J9CE0e6298K7oYOtlTZueQ+fqzNqlI1nWQTQUt9VMl gMXIvdpcbYg1OvAsEntoRjPFWlAWQcZdh+LhRgBdRGU2XO2Z9+yzs7Wn+MnE6nMb n8igp0UVnHYi2bE2d2qYwx0LDdD2W9eaSjnsXP/kEOxQhGC/870wux81Z0nVxkSo 0+7FwLnmf7wp6KgyfNapZAVHwPW3U8SNOFuQl2e3TTS8U+G7Z3IQA== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h8n49nk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:02 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64ME95Sd028436; Fri, 22 May 2026 14:13:02 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4e72wqhea5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:01 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64MECwbD36897144 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 May 2026 14:12:58 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 598C520043; Fri, 22 May 2026 14:12:58 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 186EF2004B; Fri, 22 May 2026 14:12:58 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 May 2026 14:12:58 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , David Laight , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v4 1/8] s390/percpu: Infrastructure for more efficient this_cpu operations Date: Fri, 22 May 2026 16:12:50 +0200 Message-ID: <20260522141257.303617-2-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260522141257.303617-1-hca@linux.ibm.com> References: <20260522141257.303617-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-GUID: Yiee5jYuFPRCYz9iINM8m-m7Er0fJe3O X-Authority-Analysis: v=2.4 cv=GYMnWwXL c=1 sm=1 tr=0 ts=6a10646f cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=VnNF1IyMAAAA:8 a=OodavJ-36c_9b_R7Dl8A:9 a=xWHWdn6JnoG3g-vW:21 X-Proofpoint-ORIG-GUID: dCUfTumLXcCOAYmdifxXYlGkOcSXJBFR X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIyMDE0MSBTYWx0ZWRfX4Xvscv0r2fAl +UlOhArKQ3QsJMPQke6oZhff0p88bDbD7uLmKEAVG6BWVURfFcFz+ZsgD/qTA6qy4e2MWfgUHO5 8BGEabl9Zl5ohrMNCS/TtlaCfn9sWUEzQiqAB5Mr6mX/AumKR5WNzd9XyPeftCOQYa+92Djf/tT kD141D0S57ChOu41F74fg7c7qLVpxHb+hplauOfDFrGLuB0zGZOkZjPGvfZOodsybAS+qx0DNzM wvzy7kMTWDWzFbkYAmg5dEON03BGvu3cUqYFSfsQotdTAiR9fYdJ9kxnfK1iI0nbVDeywlVD8TL fwlRBqzG1kQotA1kCS8TgIKVROtYIQXgZK5uUslt1p8FqyAbD+VHAQ/HTCeIS9YnyilC6Rn8QRH vwcOs78kR/jOzwT+tAqKrBztDxB+sQ20Ymz/wOPxy7LVCif72APeNy+YXtmyNO8vrDSALUbr7q+ FzWYYnrGdMMahofd4Jg== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-22_03,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 clxscore=1015 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605220141 Content-Type: text/plain; charset="utf-8" With the intended removal of PREEMPT_NONE this_cpu operations based on atomic instructions, guarded with preempt_disable()/preempt_enable() pairs become more expensive: the preempt_disable() / preempt_enable() pairs are not optimized away anymore during compile time. In particular the conditional call to preempt_schedule_notrace() after preempt_enable() adds additional code and register pressure. E.g. this simple C code sequence DEFINE_PER_CPU(long, foo); long bar(long a) { return this_cpu_add_return(foo, a); } generates this code: 11a976: eb af f0 68 00 24 stmg %r10,%r15,104(%r15) 11a97c: b9 04 00 ef lgr %r14,%r15 11a980: b9 04 00 b2 lgr %r11,%r2 11a984: e3 f0 ff c8 ff 71 lay %r15,-56(%r15) 11a98a: e3 e0 f0 98 00 24 stg %r14,152(%r15) 11a990: eb 01 03 a8 00 6a asi 936,1 <- __preem= pt_count_add(1) 11a996: c0 10 00 d2 ac b5 larl %r1,1b70300 <- address= of percpu var 11a9a0: e3 10 23 b8 00 08 ag %r1,952 <- add per= cpu offset 11a9a6: eb ab 10 00 00 e8 laag %r10,%r11,0(%r1) <- atomic = op 11a9ac: eb ff 03 a8 00 6e alsi 936,-1 <- __preem= pt_count_dec_and_test() 11a9b2: a7 54 00 05 jnhe 11a9bc 11a9b6: c0 e5 00 76 d1 bd brasl %r14,ff4d30 11a9bc: b9 e8 b0 2a agrk %r2,%r10,%r11 11a9c0: eb af f0 a0 00 04 lmg %r10,%r15,160(%r15) 11a9c6 07 fe br %r14 Even though the above example is more or less the worst case, since the branch to preempt_schedule_notrace() requires a stackframe, which otherwise wouldn't be necessary, there is also the conditional jnhe branch instruction. Get rid of the conditional branch with the following code sequence: 11a8e6: c0 30 00 d0 c5 0d larl %r3,1b33300 11a8ec: b9 04 00 43 lgr %r4,%r3 11a8f0: eb 00 43 c0 00 52 mviy 960,4 11a8f6: e3 40 03 b8 00 08 ag %r4,952 11a8fc: eb 52 40 00 00 e8 laag %r5,%r2,0(%r4) 11a902: eb 00 03 c0 00 52 mviy 960,0 11a908: b9 08 00 25 agr %r2,%r5 11a90c 07 fe br %r14 The general idea is that this_cpu operations based on atomic instructions are guarded with mviy instructions: - The first mviy instruction writes the register number, which contains the percpu address variable to lowcore. This also indicates that a percpu code section is executed. - The first instruction following the mviy instruction must be the ag instruction which adds the percpu offset to the percpu address register. - Afterwards the atomic percpu operation follows. - Then a second mviy instruction writes a zero to lowcore, which indicates the end of the percpu code section. - In case of an interrupt/exception/nmi the register number which was written to lowcore is copied to the exception frame (pt_regs), and a zero is written to lowcore. - On return to the previous context it is checked if a percpu code section was executed (saved register number not zero), and if the process was migrated to a different cpu. If the percpu offset was already added to the percpu address register (instruction address does _not_ point to the ag instruction) the content of the percpu address register is adjusted so it points to percpu variable of the new cpu. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/entry-percpu.h | 78 ++++++++++++++++++++++++++++ arch/s390/include/asm/lowcore.h | 3 +- arch/s390/include/asm/percpu.h | 62 ++++++++++++++++++++++ arch/s390/include/asm/ptrace.h | 2 + arch/s390/kernel/irq.c | 24 ++++++--- arch/s390/kernel/nmi.c | 5 ++ arch/s390/kernel/traps.c | 5 ++ 7 files changed, 171 insertions(+), 8 deletions(-) create mode 100644 arch/s390/include/asm/entry-percpu.h diff --git a/arch/s390/include/asm/entry-percpu.h b/arch/s390/include/asm/e= ntry-percpu.h new file mode 100644 index 000000000000..ea91187b1704 --- /dev/null +++ b/arch/s390/include/asm/entry-percpu.h @@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef ARCH_S390_ENTRY_PERCPU_H +#define ARCH_S390_ENTRY_PERCPU_H + +#include +#include +#include +#include +#include + +static __always_inline void percpu_entry(struct pt_regs *regs) +{ + struct lowcore *lc =3D get_lowcore(); + + if (user_mode(regs)) + return; + regs->cpu =3D lc->cpu_nr; + regs->percpu_register =3D lc->percpu_register; + lc->percpu_register =3D 0; +} + +static __always_inline bool percpu_code_check(struct pt_regs *regs) +{ + unsigned int insn, disp; + struct kprobe *p; + + if (likely(user_mode(regs) || !regs->percpu_register)) + return false; + /* + * Within a percpu code section - check if the percpu base register + * needs to be updated. This is the case if the PSW does not point to + * the ADD instruction within the section. + * - AG %rx,percpu_offset_in_lowcore(%r0,%r0) + * which adds the percpu offset to the percpu base register. + */ + lockdep_assert_preemption_disabled(); +again: + insn =3D READ_ONCE(*(u16 *)psw_bits(regs->psw).ia); + if (unlikely(insn =3D=3D BREAKPOINT_INSTRUCTION)) { + p =3D get_kprobe((void *)psw_bits(regs->psw).ia); + /* + * If the kprobe is concurrently removed on a different CPU + * it might not be found anymore. However text must have + * been restored - try again. + */ + if (!p) + goto again; + insn =3D p->opcode; + } + if ((insn & 0xff0f) !=3D 0xe300) + return true; + disp =3D offsetof(struct lowcore, percpu_offset); + if (machine_has_relocated_lowcore()) + disp +=3D LOWCORE_ALT_ADDRESS; + insn =3D (disp & 0xff000) >> 4 | (disp & 0x00fff) << 16 | 0x8; + if (*(u32 *)(psw_bits(regs->psw).ia + 2) !=3D insn) + return true; + return false; +} + +static __always_inline void percpu_exit(struct pt_regs *regs, bool needs_f= ixup) +{ + struct lowcore *lc =3D get_lowcore(); + unsigned char reg; + + reg =3D regs->percpu_register; + lc->percpu_register =3D reg; + if (likely(!needs_fixup)) + return; + /* Check if process has been migrated to a different CPU. */ + if (regs->cpu =3D=3D lc->cpu_nr) + return; + /* Fixup percpu base register */ + regs->gprs[reg] -=3D __per_cpu_offset[regs->cpu]; + regs->gprs[reg] +=3D lc->percpu_offset; +} + +#endif diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcor= e.h index 50ffe75adeb4..cd1ddfdb5d35 100644 --- a/arch/s390/include/asm/lowcore.h +++ b/arch/s390/include/asm/lowcore.h @@ -165,7 +165,8 @@ struct lowcore { __u32 spinlock_index; /* 0x03b0 */ __u8 pad_0x03b4[0x03b8-0x03b4]; /* 0x03b4 */ __u64 percpu_offset; /* 0x03b8 */ - __u8 pad_0x03c0[0x0400-0x03c0]; /* 0x03c0 */ + __u8 percpu_register; /* 0x03c0 */ + __u8 pad_0x03c1[0x0400-0x03c1]; /* 0x03c1 */ =20 __u32 return_lpswe; /* 0x0400 */ __u32 return_mcck_lpswe; /* 0x0404 */ diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index b18a96f3a334..78602d2f5eba 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -60,6 +60,68 @@ #define this_cpu_or_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) #define this_cpu_or_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) =20 +/* + * Macros to be used for percpu code section based on atomic instructions. + * + * Avoid the need to use preempt_disable() / preempt_disable() pairs and t= he + * conditional preempt_schedule_notrace() function calls which come with + * this. The idea is that this_cpu operations based on atomic instructions= are + * guarded with mviy instructions: + * + * - The first mviy instruction writes the register number, which contains= the + * percpu address variable to lowcore. This also indicates that a percpu + * code section is executed. + * + * - The first mviy instruction following the mviy instruction must be the= ag + * instruction which adds the percpu offset to the percpu address regist= er. + * + * - Afterwards the atomic percpu operation follows. + * + * - Then a second mviy instruction writes a zero to lowcore, which indica= tes + * the end of the percpu code section. + * + * - In case of an interrupt/exception/nmi the register number which was + * written to lowcore is copied to the exception frame (pt_regs), and a = zero + * is written to lowcore. + * + * - On return to the previous context it is checked if a percpu code sect= ion + * was executed (saved register number not zero), and if the process was + * migrated to a different cpu. If the percpu offset was already added to + * the percpu address register (instruction address does _not_ point to = the + * ag instruction) the content of the percpu address register is adjuste= d so + * it points to percpu variable of the new cpu. + * + * Inline assemblies making use of this typically have a code sequence lik= e: + * + * MVIY_PERCPU(...) <- start of percpu code section + * AG_ALT(...) <- add percpu offset; must be the second instruction + * atomic_op <- atomic op + * MVIY_ALT(...) <- end of percpu code section + */ + +#define MVIY_PERCPU(disp, dispalt, reg) \ + ".macro GEN_MVIY disp reg\n" \ + ".irp rs,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15\n" \ + " .ifc \\reg,%%r\\rs\n" \ + " mviy \\disp(%%r0),\\rs\n" \ + " .endif\n" \ + ".endr\n" \ + ".endm\n" \ + ALTERNATIVE("GEN_MVIY " __stringify(disp) " " __stringify(reg) "\n", \ + "GEN_MVIY " __stringify(dispalt) " " __stringify(reg) "\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) \ + ".purgem GEN_MVIY\n" + +#define MVIY_ALT(disp, dispalt) \ + ALTERNATIVE(" mviy " disp "(%%r0),0\n", \ + " mviy " dispalt "(%%r0),0\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) + +#define AG_ALT(disp, dispalt, reg) \ + ALTERNATIVE(" ag " reg ", " disp "(%%r0)\n", \ + " ag " reg ", " dispalt "(%%r0)\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) + #ifndef MARCH_HAS_Z196_FEATURES =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) diff --git a/arch/s390/include/asm/ptrace.h b/arch/s390/include/asm/ptrace.h index aaceb1d9110a..495e310c3d6d 100644 --- a/arch/s390/include/asm/ptrace.h +++ b/arch/s390/include/asm/ptrace.h @@ -134,6 +134,8 @@ struct pt_regs { }; unsigned long flags; unsigned long last_break; + unsigned int cpu; + unsigned char percpu_register; }; =20 /* diff --git a/arch/s390/kernel/irq.c b/arch/s390/kernel/irq.c index d10a17e6531d..efb988833c88 100644 --- a/arch/s390/kernel/irq.c +++ b/arch/s390/kernel/irq.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "entry.h" =20 DEFINE_PER_CPU_SHARED_ALIGNED(struct irq_stat, irq_stat); @@ -142,10 +143,13 @@ static int irq_pending(struct pt_regs *regs) =20 void noinstr do_io_irq(struct pt_regs *regs) { - irqentry_state_t state =3D irqentry_enter(regs); - struct pt_regs *old_regs =3D set_irq_regs(regs); - bool from_idle; + bool from_idle, percpu_needs_fixup; + struct pt_regs *old_regs; + irqentry_state_t state; =20 + percpu_entry(regs); + state =3D irqentry_enter(regs); + old_regs =3D set_irq_regs(regs); from_idle =3D test_and_clear_cpu_flag(CIF_ENABLED_WAIT); if (from_idle) update_timer_idle(); @@ -170,21 +174,25 @@ void noinstr do_io_irq(struct pt_regs *regs) do_irq_async(regs, IO_INTERRUPT); } while (machine_is_lpar() && irq_pending(regs)); =20 + percpu_needs_fixup =3D percpu_code_check(regs); irq_exit_rcu(); - set_irq_regs(old_regs); irqentry_exit(regs, state); =20 if (from_idle) regs->psw.mask &=3D ~(PSW_MASK_EXT | PSW_MASK_IO | PSW_MASK_WAIT); + percpu_exit(regs, percpu_needs_fixup); } =20 void noinstr do_ext_irq(struct pt_regs *regs) { - irqentry_state_t state =3D irqentry_enter(regs); - struct pt_regs *old_regs =3D set_irq_regs(regs); - bool from_idle; + bool from_idle, percpu_needs_fixup; + struct pt_regs *old_regs; + irqentry_state_t state; =20 + percpu_entry(regs); + state =3D irqentry_enter(regs); + old_regs =3D set_irq_regs(regs); from_idle =3D test_and_clear_cpu_flag(CIF_ENABLED_WAIT); if (from_idle) update_timer_idle(); @@ -206,12 +214,14 @@ void noinstr do_ext_irq(struct pt_regs *regs) =20 do_irq_async(regs, EXT_INTERRUPT); =20 + percpu_needs_fixup =3D percpu_code_check(regs); irq_exit_rcu(); set_irq_regs(old_regs); irqentry_exit(regs, state); =20 if (from_idle) regs->psw.mask &=3D ~(PSW_MASK_EXT | PSW_MASK_IO | PSW_MASK_WAIT); + percpu_exit(regs, percpu_needs_fixup); } =20 static void show_msi_interrupt(struct seq_file *p, int irq) diff --git a/arch/s390/kernel/nmi.c b/arch/s390/kernel/nmi.c index 94fbfad49f62..e17a59d4d5a4 100644 --- a/arch/s390/kernel/nmi.c +++ b/arch/s390/kernel/nmi.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -363,6 +364,7 @@ NOKPROBE_SYMBOL(s390_backup_mcck_info); */ void notrace s390_do_machine_check(struct pt_regs *regs) { + bool percpu_needs_fixup; static int ipd_count; static DEFINE_SPINLOCK(ipd_lock); static unsigned long long last_ipd; @@ -374,6 +376,7 @@ void notrace s390_do_machine_check(struct pt_regs *regs) unsigned long mcck_dam_code; int mcck_pending =3D 0; =20 + percpu_entry(regs); irq_state =3D irqentry_nmi_enter(regs); =20 if (user_mode(regs)) @@ -495,7 +498,9 @@ void notrace s390_do_machine_check(struct pt_regs *regs) if (mcck_pending) schedule_mcck_handler(); =20 + percpu_needs_fixup =3D percpu_code_check(regs); irqentry_nmi_exit(regs, irq_state); + percpu_exit(regs, percpu_needs_fixup); } NOKPROBE_SYMBOL(s390_do_machine_check); =20 diff --git a/arch/s390/kernel/traps.c b/arch/s390/kernel/traps.c index 1b5c6fc431cc..564403496a7c 100644 --- a/arch/s390/kernel/traps.c +++ b/arch/s390/kernel/traps.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -329,6 +330,7 @@ static void (*pgm_check_table[128])(struct pt_regs *reg= s); void noinstr __do_pgm_check(struct pt_regs *regs) { struct lowcore *lc =3D get_lowcore(); + bool percpu_needs_fixup; irqentry_state_t state; unsigned int trapnr; union teid teid; @@ -349,6 +351,7 @@ void noinstr __do_pgm_check(struct pt_regs *regs) current->thread.gmap_int_code =3D regs->int_code & 0xffff; return; } + percpu_entry(regs); state =3D irqentry_enter(regs); if (user_mode(regs)) { update_timer_sys(); @@ -385,7 +388,9 @@ void noinstr __do_pgm_check(struct pt_regs *regs) pgm_check_table[trapnr](regs); out: local_irq_disable(); + percpu_needs_fixup =3D percpu_code_check(regs); irqentry_exit(regs, state); + percpu_exit(regs, percpu_needs_fixup); } =20 /* --=20 2.51.0 From nobody Sun May 24 20:33:27 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DAFDE4219F8; Fri, 22 May 2026 14:13:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459204; cv=none; b=fpFazmNzAzGr9Ty9Vw2wfIDsdNKjONRCuO4lolfHJaSkZyxbnkDEK8NWXuo7Q/1nORoyd8HxYAnWT7BufExQAon+yYKSDLl0FmukBXdolxg7bZxhf6kSACAzvbGj86aF0bltr1q3c9qA4f/XyGvl/7dDR64kjd2EVY4AdsAlpEw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459204; c=relaxed/simple; bh=b7FCCZgRhyt9e0vVBfaDC2rx8PAEMyApt/6cQkR9LLM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eHadloApM+ojGty/q7dbkEHZwp5SAeYBKuy6GLQGkmdV4jXp7quMx49S5vU6RmJ3RfkHKfIrkwCfoWyr3RpDLaSxEGSfjSJH+5q2NVvNBUO9cjMP/Zi+Va4bAPceZXNhUCQmw6YcvYESmmySDIsPm+DKhLN9em5BF7OGeIgU6rQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=XtUTGSpx; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="XtUTGSpx" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64MAJglC3901160; Fri, 22 May 2026 14:13:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=WlfIVcJXQcx0zZvV7 ElSdd7SSTWdtlR0tVVA0tV2ph8=; b=XtUTGSpxPJzLBifLSl91iqgxN3sxrYwhi 0UZ34wpMwCJGX8veHDi7ObCPXg0Oicreie/NjMon8VmUdiP+rRnu8rnGlf/SbOmd MUHKvVK7JdH9A4fCcWBLzWh4beKq5r8wZ4Aopp9kYkj4Zm+agqjEvOJ+Sgw8lcgC F4uN8EFxvoAXSg4iX5vHZo2wvLtHfAUlc31dEeSnFevbs6+ktGhSKxZrUe/mn41z 7DphdTDUK1cb1c1WtHKPzQvCAe3PhL+8kaQ+cdTCofL2dVSNK9hvYAj2H9MAlapZ ns5wDUxKtTTFUT09ZVSbklD+LlnQ4Mmpxt6iuaU4ytAoyfxsWIp3w== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h8n49nq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:03 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64ME97JR030310; Fri, 22 May 2026 14:13:02 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4e74dj149t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:02 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64MECwfG62914970 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 May 2026 14:12:58 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 949E02004B; Fri, 22 May 2026 14:12:58 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5FC382004E; Fri, 22 May 2026 14:12:58 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 May 2026 14:12:58 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , David Laight , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v4 2/8] s390/percpu: Add missing do { } while (0) constructs Date: Fri, 22 May 2026 16:12:51 +0200 Message-ID: <20260522141257.303617-3-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260522141257.303617-1-hca@linux.ibm.com> References: <20260522141257.303617-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-GUID: U0AxLtuHKhNm-d6vfc6-W6hYzeH5nB_P X-Authority-Analysis: v=2.4 cv=GYMnWwXL c=1 sm=1 tr=0 ts=6a106470 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=c92rfblmAAAA:8 a=VnNF1IyMAAAA:8 a=VwQbUJbxAAAA:8 a=V-yoBbGPs5LSgb0Mvs4A:9 a=GvGzcOZaWPEFPQC_NcjD:22 X-Proofpoint-ORIG-GUID: RUo6GzenQmhY_3CLC6ZLT9iXHT6RXYNi X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIyMDE0MSBTYWx0ZWRfX9m5Vx+lTjHzR 5xhkIrW3v4ka2Qt2ME/qi0uW1ZxmdtDecGQQdTss4OSDfi+q2WlUrNygMVvRBsTxi+L5ESyCP7A 1Vx9bw0DJKcMfNsjZq47DbnFYPYhRF2vHtv7nHBrBjaomUfXLlNJCQFPpUdnfETVImsXrzFFNkt TL/vMmubJPgq6CV9HMy/62RgGJ5DEdaK65NmTpXywm5SR3Ke10DWtdtvQJfTvzXZ/H+ruIHwyVl n1sWExG7YKxKljB+/siXtS9B0MoErluau/AHtJ5WZ0f/bRk8Eli4aERIVTnelfyepoy5mix1RRg 8JJohaNqVZTg++9btPEguZUFZvK63Loc+60jiN0ETIpiDaQGX9+h1zECWXz9vMLlH3SdBJB8BFu kl96oH/j9dEhWkZA/STdKnWFEjt29m4goPsFrToWE/nH2KmABaXNnqVzF7M0kfV3ppaiFx8iRLq 2IR52jOLttSAZ6c4z6A== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-22_03,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 clxscore=1015 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605220141 Content-Type: text/plain; charset="utf-8" Add missing do { } while (0) constructs in order to avoid potential build failures. Reported-by: Sashiko Closes: https://sashiko.dev/#/patchset/20260319120503.4046659-1-hca%40linux= .ibm.com Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 78602d2f5eba..79d5a4460b18 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -136,7 +136,7 @@ #else /* MARCH_HAS_Z196_FEATURES */ =20 #define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ -{ \ +do { \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ @@ -157,7 +157,7 @@ : "cc"); \ } \ preempt_enable_notrace(); \ -} +} while (0) =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_add(pcp, val, "laa", "asi",= int) #define this_cpu_add_8(pcp, val) arch_this_cpu_add(pcp, val, "laag", "agsi= ", long) @@ -182,7 +182,7 @@ #define this_cpu_add_return_8(pcp, val) arch_this_cpu_add_return(pcp, val,= "laag") =20 #define arch_this_cpu_to_op(pcp, val, op) \ -{ \ +do { \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ @@ -194,7 +194,7 @@ : [val__] "d" (val__) \ : "cc"); \ preempt_enable_notrace(); \ -} +} while (0) =20 #define this_cpu_and_4(pcp, val) arch_this_cpu_to_op(pcp, val, "lan") #define this_cpu_and_8(pcp, val) arch_this_cpu_to_op(pcp, val, "lang") --=20 2.51.0 From nobody Sun May 24 20:33:27 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 979A2402428; Fri, 22 May 2026 14:13:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459198; cv=none; b=f9uh1xrcIKwqGuvlL5puJnEZNK81vLqj6nsgsMli3p2qtRyjhYYUcfoLLItll8HSyVcpCuCoOUn/46DG6/BlrB8fKJmUcWVepNe/Tqdinq3cR488vExAPBwLWR9B+BzxkElyyPsA7E8QVLtVebr0+mmEXFxaeNjCp/sU8gwFE6M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459198; c=relaxed/simple; bh=sGHtPgqRyGymhZBuROEwH1odH0nmdeQ6hAqZerhhFHE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VOOlCScn78+fbbp63t3WsgqXJhTyg7hVDWWJOIRJv/yte+oClsvavwGSawEMfZmPFuw4b/U+KyaMDY2DhXXLuR0KMOPMDmrTmU+LUJlW4ERncq44RkLh+l4h3La5W0PlQ4m8CR1srIMdqVi23iSIVUPydBr59mnXN5l8XAGamAA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=eNURgifk; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="eNURgifk" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64M5vx3D3991718; Fri, 22 May 2026 14:13:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=DdArDXsLaP5gy6DIj K0XhME+n5TSc7pgL2hQ0CfDqcY=; b=eNURgifkyvRGr4K2Ucvrrhuzuipn+aQCA WW0ICkvMlQtz5xvCKXQdxiFdyI122s4yRuErC7VnymWJDkydLTb/DTOJFeP1E1nJ pnaTdXArxZsPYPCZwS11roM8jzkmZlfiBP4qtzJdYBE8znM9EnO4oLKhAAk2Gtff 0V2YWIXY2agh7uct2gYHqFPzJQd+4fhsgBVRm0vbzhDlLb5/ZTpVi+6BuYmV2wxZ vv94AnFA5hVYUCx/b/OlNev7UBYXC/CbjnkGDAyzBeBe6jaAi5fM0L/sVK73fWIW W6b191x6nqWb+v1os+MUL/LlJY9r2yIoV8eiYZJel4lOlcC0xdslw== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6hb8tsp8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:03 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64ME94U0028422; Fri, 22 May 2026 14:13:02 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4e72wqhea6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:02 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64MECw4262914972 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 May 2026 14:12:58 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C5C012004D; Fri, 22 May 2026 14:12:58 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 966BD2004F; Fri, 22 May 2026 14:12:58 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 May 2026 14:12:58 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , David Laight , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v4 3/8] s390/percpu: Use new percpu code section for arch_this_cpu_add() Date: Fri, 22 May 2026 16:12:52 +0200 Message-ID: <20260522141257.303617-4-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260522141257.303617-1-hca@linux.ibm.com> References: <20260522141257.303617-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIyMDE0MSBTYWx0ZWRfXxbWOYpepUwtM MgtUAacbL8fEA1TZgs/3gKPc7FAGYExjYk3P/hM8f5VvK2RfXr0ivNVd7JWH1J88TFYl6cr+Kcw z+qJqgWYF0D9IkxZge0E0eCGsEF2rehPVqs86ggA5rI6XoKORAIeeD12Q+uvufIwBBnOh4MGKs+ pL6PdKh4BR0XiDTiebmIj+gfxoepD+kRTAF5zs9RrpxZew6tFC9hsN79ExdNnrUZLB8HI5DpSeS bYd4Kui/PbmewcG6SreW22p9X7Ksh5IgMgZwb5Yak9MJanBsKyqapzMz8TOkzqSrhnhTmUFINE5 Ctj+hfDcAX/JqJwCF/doAb1dPlPtGXVNy19qrvQ8v9P3FLjgStKT0bD6MITWaUe/EiS7r3LRv4c YWB/AEEyn2ew4KuEF+szHeYrvVfwhNG/gy2p3XzahRYwdi1nH4E7bbG6+Co+ptmPQbxo1tXQguF 0A5s0eATCSeXdcWl5rA== X-Proofpoint-GUID: dj8UldCM0SnvQGqNeoVqmmXkdk9dwmPg X-Proofpoint-ORIG-GUID: veREQf-N8j0MBtuHluveJHQUDTtcM7yf X-Authority-Analysis: v=2.4 cv=aYBRWxot c=1 sm=1 tr=0 ts=6a10646f cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=VnNF1IyMAAAA:8 a=SujZmW7ZfIunUtho7CIA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-22_03,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0 bulkscore=0 suspectscore=0 lowpriorityscore=0 clxscore=1015 spamscore=0 phishscore=0 priorityscore=1501 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605220141 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_add() to make use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~76kb (defconfig). Also more than 5300 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. With: DEFINE_PER_CPU(long, foo); void bar(long a) { this_cpu_add(foo, a); } Old arch_this_cpu_add() looks like this: 00000000000000c0 : c0: c0 04 00 00 00 00 jgnop c0 c6: eb 01 03 a8 00 6a asi 936,1 cc: c4 18 00 00 00 00 lgrl %r1,cc ce: R_390_GOTENT foo+0x2 d2: e3 10 03 b8 00 08 ag %r1,952 d8: eb 22 10 00 00 e8 laag %r2,%r2,0(%r1) de: eb ff 03 a8 00 6e alsi 936,-1 e4: a7 a4 00 05 jhe ee e8: c0 f4 00 00 00 00 jg e8 ea: R_390_PC32DBL __s390_indirect_jump_r14+0x2 ee: c0 f4 00 00 00 00 jg ee f0: R_390_PLT32DBL preempt_schedule_notrace+0x2 New arch_this_cpu_add() looks like this: 00000000000000c0 : c0: c0 04 00 00 00 00 jgnop c0 c6: c4 38 00 00 00 00 lgrl %r3,c6 c8: R_390_GOTENT foo+0x2 cc: b9 04 00 43 lgr %r4,%r3 d0: eb 00 43 c0 00 52 mviy 960(%r0),4 d6: e3 40 03 b8 00 08 ag %r4,952 dc: eb 52 40 00 00 e8 laag %r5,%r2,0(%r4) e2: eb 00 03 c0 00 52 mviy 960,0 e8: c0 f4 00 00 00 00 jg e8 ea: R_390_PC32DBL __s390_indirect_jump_r14+0x2 Note that the conditional function call is removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 65 ++++++++++++++++++++++------------ 1 file changed, 43 insertions(+), 22 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 79d5a4460b18..9140d81b7efc 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -135,28 +135,49 @@ =20 #else /* MARCH_HAS_Z196_FEATURES */ =20 -#define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ -do { \ - typedef typeof(pcp) pcp_op_T__; \ - pcp_op_T__ val__ =3D (val); \ - pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - if (__builtin_constant_p(val__) && \ - ((szcast)val__ > -129) && ((szcast)val__ < 128)) { \ - asm volatile( \ - op2 " %[ptr__],%[val__]" \ - : [ptr__] "+Q" (*ptr__) \ - : [val__] "i" ((szcast)val__) \ - : "cc"); \ - } else { \ - asm volatile( \ - op1 " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ - : "cc"); \ - } \ - preempt_enable_notrace(); \ +#define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ +do { \ + unsigned long lc_pcpr, lc_pcpo; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ val__ =3D (val); \ + pcp_op_T__ old__, *ptr__; \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + if (__builtin_constant_p(val__) && \ + ((szcast)val__ > -129) && ((szcast)val__ < 128)) { \ + asm volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op2 " 0(%[ptr__]),%[val__]\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]") \ + : [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "i" ((szcast)val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + } else { \ + asm volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op1 " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + } \ } while (0) =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_add(pcp, val, "laa", "asi",= int) --=20 2.51.0 From nobody Sun May 24 20:33:27 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93195421F10; Fri, 22 May 2026 14:13:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459197; cv=none; b=oPM0yrPsRK5vkzei1veh3tDHgV2QtyVlSith3wJ7LOF62vKgB9N307siZS9l0WuyaWYCyHae1oBA0T5tkvkt3jO63Y79YeKR3DZ1cFNMwB969CG3Q9Hkk+5dZ2mrC/nyP+mOyWNCf4JHFkkpfUrAc0KRaVqmWViurqokaDNFZMs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459197; c=relaxed/simple; bh=NpPn4ELfcR8XVH/mho/bZeNOVATbqfABoWtMd+c4jTA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GMEkbKWmCEJUEmKIcKjZ+GEZlMm3wuOBovpOBlEyR3FadJa5pdQmYvLVx+CHJZcvESajYU5DhIXA/ZzKUn78o3wJXf8lw5FgeAnUOdKzk+OwReN09nLB3eWbDdYaoFotb/h3xVIGN+TsVgbPEH+eHROfN2OBvytktDrdYZINBI8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=h8xD0kea; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="h8xD0kea" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64M8WfMJ1249248; Fri, 22 May 2026 14:13:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=w4VLruBjFnqGeLxXB 934mHc/udarqvdvuNRwsV+6fx4=; b=h8xD0kears7x5bGmdU3lpUlLfbd6rsb+f ifflWSmE/PhPgDV+DfdZUPhdBiKCfG05QZmuifbKuiGpUS2yNofoeVbhdDDSTTgK vnZArPsH3WtqpFAdFx8PAa1rcwLG927YOsGm3YrhOt7US4i22hVp7+DxY2m3eWa5 bpoxnmIFVpC4YTSHKalOpWdEJj8ygCjDdcXqV9h4nEMs+X9yPn8D6f81a/G1tubo vYzlwGOBigkYUbHW44iMq+cfIRi9Bz5s3eZd/dArSb9JPNIJJ7DVJIXT3eEN7k8F 95h/oOf41EKyXtkp+jkwDCAcVQIXBnnZ8b6wJRqLD0qg7uTYdc5gA== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h88ttnc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:03 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64ME95uf030285; Fri, 22 May 2026 14:13:02 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4e74dj149u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:02 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64MECxn727328920 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 May 2026 14:12:59 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0805320040; Fri, 22 May 2026 14:12:59 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CBBBF2004B; Fri, 22 May 2026 14:12:58 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 May 2026 14:12:58 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , David Laight , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v4 4/8] s390/percpu: Use new percpu code section for arch_this_cpu_add_return() Date: Fri, 22 May 2026 16:12:53 +0200 Message-ID: <20260522141257.303617-5-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260522141257.303617-1-hca@linux.ibm.com> References: <20260522141257.303617-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-ORIG-GUID: 2dh4-TpR8FUbH5IScx-TqpkornKRClnR X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIyMDE0MSBTYWx0ZWRfXwJ5QdVAH42Ov rPDeFF4y2GDqU3PWJDs2j7rK14c8SuPnVtIzozg95QRIONK/SFGz676ZL617uIKbyQ771dfKDvR RzugJoR0DKV45HbAi/ll3dD0D3PSko6tVXbHRgUlpG9uh8kw9ayIBF6HBRqiXWL+uUYXnIqohJX 1O038a/Wa5vjywGQPLBw4hHOsP5EkiqQ19CtpjkYtK6PBMtxHjUdfTNwQUObhHNw7/VPj7nSr71 cuaHFWtazY8BdAbh7AuIwDQSzwplv5QpF82yZ4DkoElsj9+bIan6u4TnlUmWPpiQl1lFvfFWoAS YJEunLoVb7gPyYa80Q2+VU8IrL1VGcCNvsP+YcbuFc4yPCfyk6cAfKdubGktFYqXcZXsMw4Ru/b v90iceplJi74x5QEawSAsUumo3acJpeYljJHnrnl1VHJucKvVlrGeXbkeRf7pyO1i0fyAjKIpUs JpXXsnmMsZTBAfdYY9w== X-Proofpoint-GUID: D2llKi9se_5JyLqMtwqo1k7p5bvT_3aN X-Authority-Analysis: v=2.4 cv=apyCzyZV c=1 sm=1 tr=0 ts=6a10646f cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=V8glGbnc2Ofi9Qvn3v5h:22 a=VnNF1IyMAAAA:8 a=rpl92wOeoJCnW_j-IKQA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-22_03,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501 impostorscore=0 bulkscore=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605220141 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_add_return() to make use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~4k (defconfig). Also 66 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 9140d81b7efc..f2d0e0354582 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -185,17 +185,29 @@ do { \ =20 #define arch_this_cpu_add_return(pcp, val, op) \ ({ \ + unsigned long lc_pcpr, lc_pcpo; \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - asm volatile( \ - op " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ : "cc"); \ - preempt_enable_notrace(); \ old__ + val__; \ }) =20 --=20 2.51.0 From nobody Sun May 24 20:33:27 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 92977421F07; Fri, 22 May 2026 14:13:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459195; cv=none; b=OK5khHFaIPLnsZJV6bFLjGUhv9nXX7CMEFJht56KNF8Vu3eTrbXbxqc+GzbAj3SU9quAPT5p2SYrfHO2H6BaTaz4JYsnBvlV1Xb3nudWIt5U5PAvht+7jOBLBhMRNh5p8HJAXwzPkdRRerB/7SdxZD/I6Z5Z8o2KoYgGYS+cPHI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459195; c=relaxed/simple; bh=GaYj1K6sO65BvmgdyPCtcYf/B0XutCChjv7AN9LJEoQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T8cQHW0l8L02xOt+ZhIqLON/FhNAAXXwrAZjMc4wXo7Cpscnaav7yChJmv5fOFitq+CLv5aC9tmpb5aZBePYCg+LiujBSeWzI2hTLQ6Kr1Rj9wiXzqPo5zEiFMxc1vPxqX7BYoT9p8YqyWB8z9lkWSNST+h5IJ5PNwyWLS2xM7M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=buCdWQx+; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="buCdWQx+" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64M8WfMK1249248; Fri, 22 May 2026 14:13:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=ca4EJwt3Fxr9YM+Gd 7nvh+qnRJop56S/2rZYP22tEpw=; b=buCdWQx+QdMMdEbhP7HkQZVKCPjr57Zpn OQb0tpyQkvLcdGipSBt6z3WuX/5yafttbdDw4z8fzweGGOSvFdogXLx+WK7nOif+ PYp7+AWPkSiQVYMbQGxkQHKv+YRy/piF+qHwdn8L8BKa5Bu57nWyldgZBiVnCP1B O2cI3YoBe0X5dv1VzBHTQ/VA8GPASyAKuwVYHUah5fC9ajHv7O/Bh35lYgdt1Ye9 DTlGNbOxTQHG2X3P0RATZ6RFPFFRQFE8dLqAHh5hfKEMHmSxggGJY5+WRB8JdI4P Hp5DMn8hRUQtEv/I/G3Vud7sTFAZ+6YE3IGU4UquQvh56WCbUju8Q== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h88ttnd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:03 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64ME95h5031298; Fri, 22 May 2026 14:13:02 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4e75kygwk0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:02 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64MECxWH27328922 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 May 2026 14:12:59 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3EB0A20040; Fri, 22 May 2026 14:12:59 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0E30720043; Fri, 22 May 2026 14:12:59 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 May 2026 14:12:59 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , David Laight , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v4 5/8] s390/percpu: Use new percpu code section for arch_this_cpu_[and|or]() Date: Fri, 22 May 2026 16:12:54 +0200 Message-ID: <20260522141257.303617-6-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260522141257.303617-1-hca@linux.ibm.com> References: <20260522141257.303617-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-ORIG-GUID: MedwrGDFBXR9W2Izoo_ruvkqfWu3fwMa X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIyMDE0MSBTYWx0ZWRfX1x+9Kqruw8za GFf2YteRmUdm4U/xfvZXswczvruOVSF48Kksu0rDHKxFM6wetxddwOI+XNHM1vieMrrCIS/k5P+ bHa7LkUxEnk3vFYOVg8wk8PETSul7q6X/PGr9MDKYIUhTB3FGSh6Z3PiI3nE34IhaL2wAT6J58+ 6aTdf0lnKFpEnSIm4i8OJ0noTTJaXuJkX+mPbM4QdQajoUXJ4rjlqV4jaowOSyu6dUBrLj9KC0Q oGcdRVJM9ue9zWEIfyLtgMAonHHTAint5mO5/ksId2joBNQhpmTbs7FUYZIkekHqRrd9lcbcE6C RtVKjSsft6E7xl1iZ/TfOQtE5i6lK7zKKWh+XfTo5Vi0om0QGJhVdK2jbeydXjTSIlWKJ/AsbnG KX/fIwFb7Fjwb5exgwqxyq2c9ZFLvs0nEN4GInOsodgUV7FFN8P3zDsa26xeO70z8VQpvyYoDtp /+l4+L4ma8FQyHeYPrQ== X-Proofpoint-GUID: H8hwbLhgSDwTe-ST0JFaCE78_Y7zgvgl X-Authority-Analysis: v=2.4 cv=apyCzyZV c=1 sm=1 tr=0 ts=6a106470 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=V8glGbnc2Ofi9Qvn3v5h:22 a=VnNF1IyMAAAA:8 a=4dLYTkTKlH5g-JvQF6AA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-22_03,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501 impostorscore=0 bulkscore=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605220141 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_[and|or]() to make use of the new percpu code section infrastructure. There is no user of this_cpu_and() and only one user of this_cpu_or() within the kernel. Therefore this conversion has hardly any effect, and also removes only preempt_schedule_notrace() function call. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index f2d0e0354582..5e0185e5960b 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -216,17 +216,29 @@ do { \ =20 #define arch_this_cpu_to_op(pcp, val, op) \ do { \ + unsigned long lc_pcpr, lc_pcpo; \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - asm volatile( \ - op " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ : "cc"); \ - preempt_enable_notrace(); \ } while (0) =20 #define this_cpu_and_4(pcp, val) arch_this_cpu_to_op(pcp, val, "lan") --=20 2.51.0 From nobody Sun May 24 20:33:27 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72A9F376A02; Fri, 22 May 2026 14:13:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459205; cv=none; b=jlR/e+5YgnNwLjG+cylJczYNhiYt7rYAeRZG31+V+RJ1YZOpwedgOy423ALed4FHiNbFumB4Dm8bTBkupvLmZW0okjSGFMwSo/4iTnCrZB+A273B+taFVFWwJ+NqzL9hQGrYdIOyZ5GVWx5ze4X+vn6RlHjmP7C3iDN+hvD0vGE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459205; c=relaxed/simple; bh=ymvOTQIUMMk/aXoAHGjadV+xaKIR/6VtOmZHZnfV7ds=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=i10nntM7vRMu7KmcaR0xVYANh51+JLswwSfnjr/RtTgZv3R7C5JPmMumBLLi1zbvU8k6irm0GOJjzsUAAzA//so6nxnU+kSJApHGSga+bzwAVUd2x9QqKlv6wclx3+GFLKB+9RYMXLXsymonPJiXKlIsKll6NSDLkhCdmTw8YNc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=mxcrotRD; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="mxcrotRD" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64MAZU6C394602; Fri, 22 May 2026 14:13:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=GcHNS14L5oGtPrEvK Nq+s1QrWpgqQu6oEiiBPNVUaIM=; b=mxcrotRDb4aoUp7GvzQtvaveiXETLUx9N F606SItNkYcTngwqZoDWR/iq6o5frF7UyZJEo6FF0BD12qpGNqo1CnXVhKs5jI/i bAtl6okG73IzDnxwGij6nH7m0vFeYItnPjs3dGbsGPneurZddQnKAit6sXD98hBP qd5IbMc2U8eQlkz13W5Tcn2rJj/JiBnulctJQobiszSXOgYDASyJkQI0PHH4z+Dh 52jqoovCS3Fi7cZDLBQ1NfeYNiNE/tcugMszlJmjAaXJdiSU5bRozJ8BkGNVh2Qi KDNqFufkzql3lYGJd189B+HxnFdWsOR+vK1iSWB7e6YBwrRX7UkRQ== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h9ycegq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:04 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64ME96Ne030307; Fri, 22 May 2026 14:13:03 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4e74dj149x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:03 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64MECxLI30409104 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 May 2026 14:12:59 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 73EE220040; Fri, 22 May 2026 14:12:59 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4497E2004B; Fri, 22 May 2026 14:12:59 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 May 2026 14:12:59 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , David Laight , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v4 6/8] s390/percpu: Provide arch_this_cpu_read() implementation Date: Fri, 22 May 2026 16:12:55 +0200 Message-ID: <20260522141257.303617-7-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260522141257.303617-1-hca@linux.ibm.com> References: <20260522141257.303617-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIyMDE0MSBTYWx0ZWRfX88H+HOQvLCFC adZzs8a1mX+eV8eqSCOFWqkWb2zmAmDNs3pz2HSdttUmK58nP9RfknI2UwgvhS2/fowmlsbR2j+ 9jh9d3WhALBu8UrPk9iJxwWgbDTLPIp7XThrDPHj4T/dNHQ77/iG/sfaOGoETLMYr49Ivm66RoJ qyojxhiScEKYTe0nHvWl1pp36WQHjGAW0DnINbtigRtOuTxXJnL9fflo1bDMJ8TMNjw4hH8D8Xj ASbhPHprCkOP1qHoPc5SmSM5y8ITCwK1ArNzsS8fKJEOF0pMj73ojC+ZEcOlvK2kX6EBGma+hQ2 8Xm+7OnNM1gqjdVzwsxIePDOnzw3UTHz77XW88RySgSN7bmNwJvuhOtmaOupdp1xdIy/kL4EeJQ RP5yz8w+YJB0mMMLrFdwW7HqTxvsxwPajdwVYaxdmXGO5YslErqu/b91JjSnAesqGdRa0HCdwIY UOf3I1sYiQDCe5uJA4Q== X-Authority-Analysis: v=2.4 cv=BNuDalQG c=1 sm=1 tr=0 ts=6a106470 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=VnNF1IyMAAAA:8 a=4bOSrtcVyj7iNoH5sBwA:9 X-Proofpoint-ORIG-GUID: jGfygW6w-RgxBFKIUPPTnBHvqQ4l71VS X-Proofpoint-GUID: RKXgQKclBBQBFxEF3jERM2iumFDWL0U9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-22_03,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 priorityscore=1501 malwarescore=0 impostorscore=0 suspectscore=0 lowpriorityscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605220141 Content-Type: text/plain; charset="utf-8" Provide an s390 specific implementation of arch_this_cpu_read() instead of the generic variant. The generic variant uses preempt_disable() / preempt_enable() pair and READ_ONCE(). Get rid of the preempt_disable() / preempt_enable() pairs by providing an own variant which makes use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~1k (defconfig). Also 87 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 5e0185e5960b..2b6c2c5606b1 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -248,6 +248,37 @@ do { \ =20 #endif /* MARCH_HAS_Z196_FEATURES */ =20 +#define arch_this_cpu_read(pcp, op) \ +({ \ + unsigned long lc_pcpr, lc_pcpo, res__; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ *ptr__; \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[res__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]") \ + : [res__] "=3D&d" (res__), [ptr__] "+&a" (ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (*ptr__), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + (pcp_op_T__)res__; \ +}) + +#define this_cpu_read_1(pcp) arch_this_cpu_read(pcp, "ic") +#define this_cpu_read_2(pcp) arch_this_cpu_read(pcp, "lh") +#define this_cpu_read_4(pcp) arch_this_cpu_read(pcp, "l") +#define this_cpu_read_8(pcp) arch_this_cpu_read(pcp, "lg") + #define arch_this_cpu_cmpxchg(pcp, oval, nval) \ ({ \ typedef typeof(pcp) pcp_op_T__; \ --=20 2.51.0 From nobody Sun May 24 20:33:27 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3AA0B41C314; Fri, 22 May 2026 14:13:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459197; cv=none; b=qjuVLLrda8506X3CqTBIkwHIU49btwXYGF586mzYY50xR/RJD82N1nGXa2iviNcb0X0+r69gMMe/NPy5xyYteIL+l1caGE7h3TARa3VMd4bPcOLVGqBM7UMJKEx43+rcMdTnC9Zvwwuh2BT39xPu7GYxd/BZcfMzb7GrrTiyEFs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459197; c=relaxed/simple; bh=xLUxNolFlbS4tE5XTJ7TIi1oz4ejVGIRGBiVBHuWrdk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uRNYW5Pnr4xl7EtJak7EG2bYhSdkutXsZXOcbQBEe+1xVpztRCbBq+j+ZBnYstD2oae5LFlo0X2XH+z23YF+ssc7mcto9ynnEeDa2utxGEwbr/nf2QN51JJbVijRuC2WcsCMCICJrOneWfuxVrJ8EGi6RpP/TE51bcwX+wzc8a4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=BKVpPapW; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="BKVpPapW" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64M0Ebu02799624; Fri, 22 May 2026 14:13:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=FRAEkWgvcd1absBO4 csZpf4VqU5+MWG23iClBSffMLo=; b=BKVpPapW9ptMjeaAo4E1kw8hJrbSdxGVH HQfjROnCWREW8zCXHoX4IV0XCjrfc6UKh2OpjvdOOqLl4VCDg0GWiVPZKY87LUqm 8OoXjc82R6umaFz0u9HWUTMpIP79SAP5VGH3TPyzP5YsHFxUQz1GaIYwEyfKuk0i cJIrfVH3tJBz3V7o2IPAAFltZKVAa1EjjdJ/8g3Dke6jpwNfT5GvY+Uk6Y3ptT4O AuqJO/xML28l0+YSBITAXKg8DI2M5AS2TWCOyYXEyWcDsezyE4pRFCC9AJS+pI+I QNZy/bIy3XKoYMt2jqGpFG0ecxzkD3m7KxeoaH8VTx4yRv6BKP2gw== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6hawjwbg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:04 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64ME9ExA015396; Fri, 22 May 2026 14:13:03 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4e73wkh6kc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:03 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64MECxsg30409106 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 May 2026 14:12:59 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A925A20040; Fri, 22 May 2026 14:12:59 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 79E1220043; Fri, 22 May 2026 14:12:59 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 May 2026 14:12:59 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , David Laight , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v4 7/8] s390/percpu: Provide arch_this_cpu_write() implementation Date: Fri, 22 May 2026 16:12:56 +0200 Message-ID: <20260522141257.303617-8-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260522141257.303617-1-hca@linux.ibm.com> References: <20260522141257.303617-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIyMDE0MSBTYWx0ZWRfXwu6Srjl5mHNE vlHk1xyCxUjRQlHmy/GRQRNSXVYzKElqDqtiCykE2vE4Rq5uyWPsFYTFE8mkoqilyLff4jhYRgV FmNZoVHsQg8KpPQIZVNStBxjZ7NfMxvGikJEOD2SbBhMu+X7MeIrA0BIq003LTd0MCCNhMxSw7b 8LzsX+g7SnrWYIcDvQRpifcyrjMZQK3SleEofuwYjpZPG2rrqk0qan8cER/Ib2HK1MC1kYavQ39 nyqQOi3d76boMKnPedBjibI+/JSOoy+9jKDBfb1iXq9pS1DDKfKgWRVkG0zsJGtjrRosOuE3sEw I9Ec0zyN7XvOhjjUuYJ0695ZC0c/F5e64jUBEJEwFhNqID4gjgMoWvutGuBNdQA1APUkN1R4SM5 Uf0v2LYg15Kej35x9I6yXjDgNA7lZwdMqoeTb6+G+PjizBxyYTRBeknv9HexJQpZr0YPndxyx5l M0ObH3VObBdydeSqUng== X-Authority-Analysis: v=2.4 cv=Np/htcdJ c=1 sm=1 tr=0 ts=6a106470 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=ObmbXjF690NQJU6EaucA:9 X-Proofpoint-ORIG-GUID: UXZK0JT4O3yXL1L5d4b2mhKGQYuWJ2hF X-Proofpoint-GUID: y4vvrk-xHt8tD_bE3I0BEBMfmg5rgTRq X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-22_03,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 spamscore=0 clxscore=1015 priorityscore=1501 impostorscore=0 lowpriorityscore=0 suspectscore=0 adultscore=0 phishscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605220141 Content-Type: text/plain; charset="utf-8" Provide an s390 specific implementation of arch_this_cpu_write() instead of the generic variant. The generic variant uses a quite expensive raw_local_irq_save() / raw_local_irq_restore() pair. Get rid of this by providing an own variant which makes use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~1k (defconfig). Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 2b6c2c5606b1..667b165b4c0e 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -279,6 +279,36 @@ do { \ #define this_cpu_read_4(pcp) arch_this_cpu_read(pcp, "l") #define this_cpu_read_8(pcp) arch_this_cpu_read(pcp, "lg") =20 +#define arch_this_cpu_write(pcp, val, op) \ +do { \ + unsigned long lc_pcpr, lc_pcpo; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ *ptr__, val__ =3D (val); \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]") \ + : [ptr__] "+&a" (ptr__), "=3Dm" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ +} while (0) + +#define this_cpu_write_1(pcp, val) arch_this_cpu_write(pcp, val, "stc") +#define this_cpu_write_2(pcp, val) arch_this_cpu_write(pcp, val, "sth") +#define this_cpu_write_4(pcp, val) arch_this_cpu_write(pcp, val, "st") +#define this_cpu_write_8(pcp, val) arch_this_cpu_write(pcp, val, "stg") + #define arch_this_cpu_cmpxchg(pcp, oval, nval) \ ({ \ typedef typeof(pcp) pcp_op_T__; \ --=20 2.51.0 From nobody Sun May 24 20:33:27 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEAB8421F17; Fri, 22 May 2026 14:13:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459205; cv=none; b=TJd7Xu4C8rnYbCe5uQK+JF2jYzl2LkO2iH6s5FhqCYd+jPUq5i0YfJBBYLf1wBwFGg4PJiuZKEPihzl8OX4Fovnoj6TTb609yjtNBXh9HK979QR87v3Brwa/d3Enm0nSLTLBCVYnvP+fPeC/qLKIB2TPNdhGWnJi8SC5d6dblXQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779459205; c=relaxed/simple; bh=ef2UwNRzcG9g0NK5tazYND828j/yLBssvJDK5Gg5Xh0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nBO0JXK9AG6eVzkQh89WjA8fsGeHWYDBJNQHgSn5okanb6y2b5BNZMyUqoLpmX6gWyjK2uW9l8zbvMPQqC/tv4EE06W/7cQBkAfDB1bwloYEAXJ+zRn/r7O1SflO1kzeO+8mRIXbUHiRp2ez8b2AGlfsdovoYJ+N+E3DUTyyBeU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=MhLbLbNZ; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="MhLbLbNZ" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64M10OUN1495915; Fri, 22 May 2026 14:13:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=yJkTWInv7qo7a8PXK hXr59zNL9lUcDPJGpABn4QlEu8=; b=MhLbLbNZY6mDWgSSvVKm7k5HhlzF1F9+w oC8+5Vt9gzEL0Y4ZyJl7eFtz7FIyJsYkHs7RGHcWcwHdeaWMY3QL9vib7lIYbiUi I0ELoeaMr3hcmenHkCL4WSZVhrhrwrFnXhPz3Wm/WyMHyjEfoweYACyLONDmvaIF jphtjGwCFCfDfgoQpnLJjiWVFMYQIULnL+A3cwMdcfbI4lvUdSyqCPmuNNQhyHSn /8ze/6bkoWsLVxiXTsYQOgCiLdYBmGwAmJIcbrcsi0YkowEDHcwZZO1gA7OlJHjI KVB8HXx/hwWtnEqypg29IZQARNXLmkyUp0fNaqlG8way2l/JHKz5A== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h9ycegt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:04 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64ME97De024233; Fri, 22 May 2026 14:13:03 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4e739w9anu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 22 May 2026 14:13:03 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64MED0Rt12321152 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 22 May 2026 14:13:00 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DF1ED20040; Fri, 22 May 2026 14:12:59 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AF14F2004B; Fri, 22 May 2026 14:12:59 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 22 May 2026 14:12:59 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: Peter Zijlstra , Yang Shi , Shrikanth Hegde , David Laight , linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v4 8/8] s390/percpu: Remove one and two byte this_cpu operation implementation Date: Fri, 22 May 2026 16:12:57 +0200 Message-ID: <20260522141257.303617-9-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260522141257.303617-1-hca@linux.ibm.com> References: <20260522141257.303617-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTIyMDE0MSBTYWx0ZWRfXywCMk8UAfnHd TKV3gkAelVAsbfJDaZ+qpS044q6Kk8TRRrMCsWE9o72VCEqXS+wT9pwzuEbLusIxokGTQp2clBz yDWkHIg4XW/2lbd7W384ki28jhOTGYwqcB9SwOZvRe89JLy4xkavsvkHFHwu7S2FxrQz/hDq6BZ msFLS4qpF5sFnZUBUbAD5lQuXW0dcwr9UjsZDWiQg6kkvWtKkchW+zndVqZlYIOTbj6fOF8dD3R B8qQ2yBjkDbbkM4L8rnoj8AbtivMFHgktNesTkuwmJrLgvtYvyaRWiA+kyh4jazcyT5xh/TfmwF yHEKrS06wwWR9d2EU5IcbDD4gEBkYoeacl4PKvEUW1SIgiObA8eV9xm+bS4NoFFm5FRyz8rck1X jFUUfIVOLF5olE4vh4mU2yuey55z+rTeKaY4HiRy08kg+FUE9TFg4eL+rKuutvUcAcAadnyTRfN jw1gKNDKb3W+I7y9vog== X-Authority-Analysis: v=2.4 cv=BNuDalQG c=1 sm=1 tr=0 ts=6a106471 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=VnNF1IyMAAAA:8 a=e14ZIUwkndthOOMI8LgA:9 X-Proofpoint-ORIG-GUID: Ksoz4cUCdVxI9OWnSKG4VMSTod0Wb3RN X-Proofpoint-GUID: K2Uig8D3PdNIaOZBGvLxbumqG5vNR92G X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-22_03,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 priorityscore=1501 malwarescore=0 impostorscore=0 suspectscore=0 lowpriorityscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605220141 Content-Type: text/plain; charset="utf-8" There are no one and two byte this_cpu operations within the kernel (defconfig). However even if there would be, the s390 implementation, which uses a cmpxchg loop, generates a very large code sequence due to the lack of native one and two byte cmpxchg instructions. Remove the s390 implementation and use the generic implementation. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/percpu.h | 13 ------------- 1 file changed, 13 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 667b165b4c0e..5c0c061dbd10 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -51,15 +51,6 @@ new__; \ }) =20 -#define this_cpu_add_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) -#define this_cpu_add_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) -#define this_cpu_add_return_1(pcp, val) arch_this_cpu_to_op_simple(pcp, va= l, +) -#define this_cpu_add_return_2(pcp, val) arch_this_cpu_to_op_simple(pcp, va= l, +) -#define this_cpu_and_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, &) -#define this_cpu_and_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, &) -#define this_cpu_or_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) -#define this_cpu_or_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) - /* * Macros to be used for percpu code section based on atomic instructions. * @@ -321,8 +312,6 @@ do { \ ret__; \ }) =20 -#define this_cpu_cmpxchg_1(pcp, oval, nval) arch_this_cpu_cmpxchg(pcp, ova= l, nval) -#define this_cpu_cmpxchg_2(pcp, oval, nval) arch_this_cpu_cmpxchg(pcp, ova= l, nval) #define this_cpu_cmpxchg_4(pcp, oval, nval) arch_this_cpu_cmpxchg(pcp, ova= l, nval) #define this_cpu_cmpxchg_8(pcp, oval, nval) arch_this_cpu_cmpxchg(pcp, ova= l, nval) =20 @@ -353,8 +342,6 @@ do { \ ret__; \ }) =20 -#define this_cpu_xchg_1(pcp, nval) arch_this_cpu_xchg(pcp, nval) -#define this_cpu_xchg_2(pcp, nval) arch_this_cpu_xchg(pcp, nval) #define this_cpu_xchg_4(pcp, nval) arch_this_cpu_xchg(pcp, nval) #define this_cpu_xchg_8(pcp, nval) arch_this_cpu_xchg(pcp, nval) =20 --=20 2.51.0