From nobody Mon Jun 8 22:51:24 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B21EA3BF69D; Tue, 26 May 2026 05:57:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775030; cv=none; b=swP6j0JruxeA+1i4ond3ERRVJxzXWWjrRx+ByJWzNE8aYhExywoQ6gJRGIqo2h82kZoUBzNJ+wD1CPkyQOIbM4gsjuUnmNmDM9zB+TEGA+ZHcOYXgEwdjcqF1rSvtgBMqFhvqCL6xpxveqKPk5Q6hpS5B0+1zRsERf2yZzhB/JQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775030; c=relaxed/simple; bh=NYtxqWiQHgnlDDVuJjx0yxlHSPlDjB0mpSkINAkOiMg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Kfrugat4jd2cFsfxPxDY2eZ4Rs4MleJDrP5AqPRGU7OV/WSfSYW7Hceoho1cPq5hHCu4gKdSGJfNFA4ZzPNTJSJiuancjJSUlFchmD1NiymWxT9CrpX4VvLUnC2ZDAKKiOeOrYvnvk/vgP5yqFzxbQBOUDJv7hNk1bwsoy7SAqU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=qRJKNT5N; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="qRJKNT5N" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64PDJWeb2318206; Tue, 26 May 2026 05:57:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=QpmXCFz8YeXREQBlV LF21Vi1qS3nhlWzS42g0Fx0+74=; b=qRJKNT5N/Ypmypyo0Yah3V2Q0+uG4bY0V tH6JidttefYxLoqSkFHcHF5licAZS/NkV8BxLfiDnI5dm6WntwDXEcwjwVYVrzgh /oaHVfOHcKcJGIlqYQnVdFdFYCK0XSdNYsvHOuvz5wdnfRHRfkU9WNLK1cQVDmQP 1xWPBPrewnPCWlWSTqDoQSqXKMz8drhBqxcmV3vWQ2JZ6iDj+1nc4RI3xv7vZ17n FKW0M9iaG+pi0AbH/NyjtOsTLk1rq5hSHy+h5XRXiDwIBVenBbo536brlTh8/A9+ 1KNbhAgT0VW8LCqJkdVM4mfo1D3s9JRABLM3ySkkJFWobx2cOy++A== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4eb4qbthbg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:07 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64Q5s9Aw016040; Tue, 26 May 2026 05:57:06 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4ebs8y7q8c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:06 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64Q5v3Vd52167016 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 May 2026 05:57:03 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 630F820043; Tue, 26 May 2026 05:57:03 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 34F9C2004D; Tue, 26 May 2026 05:57:03 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 26 May 2026 05:57:03 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v5 1/7] s390/percpu: Infrastructure for more efficient this_cpu operations Date: Tue, 26 May 2026 07:56:56 +0200 Message-ID: <20260526055702.1429061-2-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260526055702.1429061-1-hca@linux.ibm.com> References: <20260526055702.1429061-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTI2MDA0NiBTYWx0ZWRfX6vxUMsq7I3bz 2j9pSV1j3wKTKi7LPPWMCjSPjbQYoz9x6w74EakBJ7JYM/1IMxMMWYzKpFXcRxHaQeQh4BzcWZG 4whfce/Sc0KEA1R6N1TgAzMrSEe1marI3iQPvdwQaPfghMiEXec9Tm6rJeJYyU3ZUgTmDbfjn8s F4qB3OEda/yO7Z59HnlabZe01wWGd/qG5fndopKLUCB8kXYf4Bo7Cc/tg+dzCZN8TbizdedUK2t odMiERM56bpw7za2SjDOON+kyUcB40MXaqN/C6WSamLm5yrNprniTwZUlxGFICFTGvAHLv3p8Y7 ZpgqI9Qq8yD5Y1KT71Q97VNUjPCyoWLOqMu07ZvFOA/1uHsBTOnMQsx9RY+Z3XfxQC7wMq6Nz24 PjUTMlQlHaj8vVdtC+fS0B16yASTidjJMCqBj8h/CliUSg9bhA2qsVvs+bMox0RLSAtgU5DT4VW Le8fscOfYYIju6jBviA== X-Authority-Analysis: v=2.4 cv=KItqylFo c=1 sm=1 tr=0 ts=6a153633 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=VnNF1IyMAAAA:8 a=OodavJ-36c_9b_R7Dl8A:9 a=xWHWdn6JnoG3g-vW:21 X-Proofpoint-ORIG-GUID: jzLGArAGQDHjZTBGK8t37IGbu988l6K2 X-Proofpoint-GUID: jzLGArAGQDHjZTBGK8t37IGbu988l6K2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-26_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 lowpriorityscore=0 spamscore=0 priorityscore=1501 clxscore=1015 adultscore=0 suspectscore=0 bulkscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605260046 Content-Type: text/plain; charset="utf-8" With the intended removal of PREEMPT_NONE this_cpu operations based on atomic instructions, guarded with preempt_disable()/preempt_enable() pairs become more expensive: the preempt_disable() / preempt_enable() pairs are not optimized away anymore during compile time. In particular the conditional call to preempt_schedule_notrace() after preempt_enable() adds additional code and register pressure. E.g. this simple C code sequence DEFINE_PER_CPU(long, foo); long bar(long a) { return this_cpu_add_return(foo, a); } generates this code: 11a976: eb af f0 68 00 24 stmg %r10,%r15,104(%r15) 11a97c: b9 04 00 ef lgr %r14,%r15 11a980: b9 04 00 b2 lgr %r11,%r2 11a984: e3 f0 ff c8 ff 71 lay %r15,-56(%r15) 11a98a: e3 e0 f0 98 00 24 stg %r14,152(%r15) 11a990: eb 01 03 a8 00 6a asi 936,1 <- __preem= pt_count_add(1) 11a996: c0 10 00 d2 ac b5 larl %r1,1b70300 <- address= of percpu var 11a9a0: e3 10 23 b8 00 08 ag %r1,952 <- add per= cpu offset 11a9a6: eb ab 10 00 00 e8 laag %r10,%r11,0(%r1) <- atomic = op 11a9ac: eb ff 03 a8 00 6e alsi 936,-1 <- __preem= pt_count_dec_and_test() 11a9b2: a7 54 00 05 jnhe 11a9bc 11a9b6: c0 e5 00 76 d1 bd brasl %r14,ff4d30 11a9bc: b9 e8 b0 2a agrk %r2,%r10,%r11 11a9c0: eb af f0 a0 00 04 lmg %r10,%r15,160(%r15) 11a9c6 07 fe br %r14 Even though the above example is more or less the worst case, since the branch to preempt_schedule_notrace() requires a stackframe, which otherwise wouldn't be necessary, there is also the conditional jnhe branch instruction. Get rid of the conditional branch with the following code sequence: 11a8e6: c0 30 00 d0 c5 0d larl %r3,1b33300 11a8ec: b9 04 00 43 lgr %r4,%r3 11a8f0: eb 00 43 c0 00 52 mviy 960,4 11a8f6: e3 40 03 b8 00 08 ag %r4,952 11a8fc: eb 52 40 00 00 e8 laag %r5,%r2,0(%r4) 11a902: eb 00 03 c0 00 52 mviy 960,0 11a908: b9 08 00 25 agr %r2,%r5 11a90c 07 fe br %r14 The general idea is that this_cpu operations based on atomic instructions are guarded with mviy instructions: - The first mviy instruction writes the register number, which contains the percpu address variable to lowcore. This also indicates that a percpu code section is executed. - The first instruction following the mviy instruction must be the ag instruction which adds the percpu offset to the percpu address register. - Afterwards the atomic percpu operation follows. - Then a second mviy instruction writes a zero to lowcore, which indicates the end of the percpu code section. - In case of an interrupt/exception/nmi the register number which was written to lowcore is copied to the exception frame (pt_regs), and a zero is written to lowcore. - On return to the previous context it is checked if a percpu code section was executed (saved register number not zero), and if the process was migrated to a different cpu. If the percpu offset was already added to the percpu address register (instruction address does _not_ point to the ag instruction) the content of the percpu address register is adjusted so it points to percpu variable of the new cpu. Signed-off-by: Heiko Carstens Reviewed-by: Alexander Gordeev --- arch/s390/include/asm/entry-percpu.h | 80 ++++++++++++++++++++++++++++ arch/s390/include/asm/lowcore.h | 3 +- arch/s390/include/asm/percpu.h | 62 +++++++++++++++++++++ arch/s390/include/asm/ptrace.h | 2 + arch/s390/kernel/irq.c | 24 ++++++--- arch/s390/kernel/nmi.c | 5 ++ arch/s390/kernel/traps.c | 5 ++ 7 files changed, 173 insertions(+), 8 deletions(-) create mode 100644 arch/s390/include/asm/entry-percpu.h diff --git a/arch/s390/include/asm/entry-percpu.h b/arch/s390/include/asm/e= ntry-percpu.h new file mode 100644 index 000000000000..4ef47265cfce --- /dev/null +++ b/arch/s390/include/asm/entry-percpu.h @@ -0,0 +1,80 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef ARCH_S390_ENTRY_PERCPU_H +#define ARCH_S390_ENTRY_PERCPU_H + +#include +#include +#include +#include +#include + +static __always_inline void percpu_entry(struct pt_regs *regs) +{ + struct lowcore *lc =3D get_lowcore(); + + if (user_mode(regs)) + return; + regs->cpu =3D lc->cpu_nr; + regs->percpu_register =3D lc->percpu_register; + lc->percpu_register =3D 0; +} + +static __always_inline bool percpu_code_check(struct pt_regs *regs) +{ + unsigned int insn, disp; + struct kprobe *p; + + if (likely(user_mode(regs) || !regs->percpu_register)) + return false; + /* + * Within a percpu code section - check if the percpu base register + * needs to be updated. This is the case if the PSW does not point to + * the ADD instruction within the section. + * - AG %rx,percpu_offset_in_lowcore(%r0,%r0) + * which adds the percpu offset to the percpu base register. + */ + lockdep_assert_preemption_disabled(); +again: + insn =3D READ_ONCE(*(u16 *)psw_bits(regs->psw).ia); + if (unlikely(insn =3D=3D BREAKPOINT_INSTRUCTION)) { + p =3D get_kprobe((void *)psw_bits(regs->psw).ia); + /* + * If the kprobe is concurrently removed on a different CPU + * it might not be found anymore. However text must have + * been restored - try again. + */ + if (!p) + goto again; + insn =3D p->opcode; + } + if ((insn & 0xff0f) !=3D 0xe300) + return true; + disp =3D offsetof(struct lowcore, percpu_offset); + if (machine_has_relocated_lowcore()) + disp +=3D LOWCORE_ALT_ADDRESS; + insn =3D (disp & 0xff000) >> 4 | (disp & 0x00fff) << 16 | 0x8; + if (*(u32 *)(psw_bits(regs->psw).ia + 2) !=3D insn) + return true; + return false; +} + +static __always_inline void percpu_exit(struct pt_regs *regs, bool needs_f= ixup) +{ + struct lowcore *lc =3D get_lowcore(); + unsigned char reg; + + if (user_mode(regs)) + return; + reg =3D regs->percpu_register; + lc->percpu_register =3D reg; + if (likely(!needs_fixup)) + return; + /* Check if process has been migrated to a different CPU. */ + if (regs->cpu =3D=3D lc->cpu_nr) + return; + /* Fixup percpu base register */ + regs->gprs[reg] -=3D __per_cpu_offset[regs->cpu]; + regs->gprs[reg] +=3D lc->percpu_offset; +} + +#endif diff --git a/arch/s390/include/asm/lowcore.h b/arch/s390/include/asm/lowcor= e.h index 50ffe75adeb4..cd1ddfdb5d35 100644 --- a/arch/s390/include/asm/lowcore.h +++ b/arch/s390/include/asm/lowcore.h @@ -165,7 +165,8 @@ struct lowcore { __u32 spinlock_index; /* 0x03b0 */ __u8 pad_0x03b4[0x03b8-0x03b4]; /* 0x03b4 */ __u64 percpu_offset; /* 0x03b8 */ - __u8 pad_0x03c0[0x0400-0x03c0]; /* 0x03c0 */ + __u8 percpu_register; /* 0x03c0 */ + __u8 pad_0x03c1[0x0400-0x03c1]; /* 0x03c1 */ =20 __u32 return_lpswe; /* 0x0400 */ __u32 return_mcck_lpswe; /* 0x0404 */ diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index b18a96f3a334..78602d2f5eba 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -60,6 +60,68 @@ #define this_cpu_or_1(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) #define this_cpu_or_2(pcp, val) arch_this_cpu_to_op_simple(pcp, val, |) =20 +/* + * Macros to be used for percpu code section based on atomic instructions. + * + * Avoid the need to use preempt_disable() / preempt_disable() pairs and t= he + * conditional preempt_schedule_notrace() function calls which come with + * this. The idea is that this_cpu operations based on atomic instructions= are + * guarded with mviy instructions: + * + * - The first mviy instruction writes the register number, which contains= the + * percpu address variable to lowcore. This also indicates that a percpu + * code section is executed. + * + * - The first mviy instruction following the mviy instruction must be the= ag + * instruction which adds the percpu offset to the percpu address regist= er. + * + * - Afterwards the atomic percpu operation follows. + * + * - Then a second mviy instruction writes a zero to lowcore, which indica= tes + * the end of the percpu code section. + * + * - In case of an interrupt/exception/nmi the register number which was + * written to lowcore is copied to the exception frame (pt_regs), and a = zero + * is written to lowcore. + * + * - On return to the previous context it is checked if a percpu code sect= ion + * was executed (saved register number not zero), and if the process was + * migrated to a different cpu. If the percpu offset was already added to + * the percpu address register (instruction address does _not_ point to = the + * ag instruction) the content of the percpu address register is adjuste= d so + * it points to percpu variable of the new cpu. + * + * Inline assemblies making use of this typically have a code sequence lik= e: + * + * MVIY_PERCPU(...) <- start of percpu code section + * AG_ALT(...) <- add percpu offset; must be the second instruction + * atomic_op <- atomic op + * MVIY_ALT(...) <- end of percpu code section + */ + +#define MVIY_PERCPU(disp, dispalt, reg) \ + ".macro GEN_MVIY disp reg\n" \ + ".irp rs,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15\n" \ + " .ifc \\reg,%%r\\rs\n" \ + " mviy \\disp(%%r0),\\rs\n" \ + " .endif\n" \ + ".endr\n" \ + ".endm\n" \ + ALTERNATIVE("GEN_MVIY " __stringify(disp) " " __stringify(reg) "\n", \ + "GEN_MVIY " __stringify(dispalt) " " __stringify(reg) "\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) \ + ".purgem GEN_MVIY\n" + +#define MVIY_ALT(disp, dispalt) \ + ALTERNATIVE(" mviy " disp "(%%r0),0\n", \ + " mviy " dispalt "(%%r0),0\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) + +#define AG_ALT(disp, dispalt, reg) \ + ALTERNATIVE(" ag " reg ", " disp "(%%r0)\n", \ + " ag " reg ", " dispalt "(%%r0)\n", \ + ALT_FEATURE(MFEATURE_LOWCORE)) + #ifndef MARCH_HAS_Z196_FEATURES =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_to_op_simple(pcp, val, +) diff --git a/arch/s390/include/asm/ptrace.h b/arch/s390/include/asm/ptrace.h index aaceb1d9110a..495e310c3d6d 100644 --- a/arch/s390/include/asm/ptrace.h +++ b/arch/s390/include/asm/ptrace.h @@ -134,6 +134,8 @@ struct pt_regs { }; unsigned long flags; unsigned long last_break; + unsigned int cpu; + unsigned char percpu_register; }; =20 /* diff --git a/arch/s390/kernel/irq.c b/arch/s390/kernel/irq.c index d10a17e6531d..efb988833c88 100644 --- a/arch/s390/kernel/irq.c +++ b/arch/s390/kernel/irq.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "entry.h" =20 DEFINE_PER_CPU_SHARED_ALIGNED(struct irq_stat, irq_stat); @@ -142,10 +143,13 @@ static int irq_pending(struct pt_regs *regs) =20 void noinstr do_io_irq(struct pt_regs *regs) { - irqentry_state_t state =3D irqentry_enter(regs); - struct pt_regs *old_regs =3D set_irq_regs(regs); - bool from_idle; + bool from_idle, percpu_needs_fixup; + struct pt_regs *old_regs; + irqentry_state_t state; =20 + percpu_entry(regs); + state =3D irqentry_enter(regs); + old_regs =3D set_irq_regs(regs); from_idle =3D test_and_clear_cpu_flag(CIF_ENABLED_WAIT); if (from_idle) update_timer_idle(); @@ -170,21 +174,25 @@ void noinstr do_io_irq(struct pt_regs *regs) do_irq_async(regs, IO_INTERRUPT); } while (machine_is_lpar() && irq_pending(regs)); =20 + percpu_needs_fixup =3D percpu_code_check(regs); irq_exit_rcu(); - set_irq_regs(old_regs); irqentry_exit(regs, state); =20 if (from_idle) regs->psw.mask &=3D ~(PSW_MASK_EXT | PSW_MASK_IO | PSW_MASK_WAIT); + percpu_exit(regs, percpu_needs_fixup); } =20 void noinstr do_ext_irq(struct pt_regs *regs) { - irqentry_state_t state =3D irqentry_enter(regs); - struct pt_regs *old_regs =3D set_irq_regs(regs); - bool from_idle; + bool from_idle, percpu_needs_fixup; + struct pt_regs *old_regs; + irqentry_state_t state; =20 + percpu_entry(regs); + state =3D irqentry_enter(regs); + old_regs =3D set_irq_regs(regs); from_idle =3D test_and_clear_cpu_flag(CIF_ENABLED_WAIT); if (from_idle) update_timer_idle(); @@ -206,12 +214,14 @@ void noinstr do_ext_irq(struct pt_regs *regs) =20 do_irq_async(regs, EXT_INTERRUPT); =20 + percpu_needs_fixup =3D percpu_code_check(regs); irq_exit_rcu(); set_irq_regs(old_regs); irqentry_exit(regs, state); =20 if (from_idle) regs->psw.mask &=3D ~(PSW_MASK_EXT | PSW_MASK_IO | PSW_MASK_WAIT); + percpu_exit(regs, percpu_needs_fixup); } =20 static void show_msi_interrupt(struct seq_file *p, int irq) diff --git a/arch/s390/kernel/nmi.c b/arch/s390/kernel/nmi.c index 94fbfad49f62..e17a59d4d5a4 100644 --- a/arch/s390/kernel/nmi.c +++ b/arch/s390/kernel/nmi.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -363,6 +364,7 @@ NOKPROBE_SYMBOL(s390_backup_mcck_info); */ void notrace s390_do_machine_check(struct pt_regs *regs) { + bool percpu_needs_fixup; static int ipd_count; static DEFINE_SPINLOCK(ipd_lock); static unsigned long long last_ipd; @@ -374,6 +376,7 @@ void notrace s390_do_machine_check(struct pt_regs *regs) unsigned long mcck_dam_code; int mcck_pending =3D 0; =20 + percpu_entry(regs); irq_state =3D irqentry_nmi_enter(regs); =20 if (user_mode(regs)) @@ -495,7 +498,9 @@ void notrace s390_do_machine_check(struct pt_regs *regs) if (mcck_pending) schedule_mcck_handler(); =20 + percpu_needs_fixup =3D percpu_code_check(regs); irqentry_nmi_exit(regs, irq_state); + percpu_exit(regs, percpu_needs_fixup); } NOKPROBE_SYMBOL(s390_do_machine_check); =20 diff --git a/arch/s390/kernel/traps.c b/arch/s390/kernel/traps.c index 1b5c6fc431cc..564403496a7c 100644 --- a/arch/s390/kernel/traps.c +++ b/arch/s390/kernel/traps.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -329,6 +330,7 @@ static void (*pgm_check_table[128])(struct pt_regs *reg= s); void noinstr __do_pgm_check(struct pt_regs *regs) { struct lowcore *lc =3D get_lowcore(); + bool percpu_needs_fixup; irqentry_state_t state; unsigned int trapnr; union teid teid; @@ -349,6 +351,7 @@ void noinstr __do_pgm_check(struct pt_regs *regs) current->thread.gmap_int_code =3D regs->int_code & 0xffff; return; } + percpu_entry(regs); state =3D irqentry_enter(regs); if (user_mode(regs)) { update_timer_sys(); @@ -385,7 +388,9 @@ void noinstr __do_pgm_check(struct pt_regs *regs) pgm_check_table[trapnr](regs); out: local_irq_disable(); + percpu_needs_fixup =3D percpu_code_check(regs); irqentry_exit(regs, state); + percpu_exit(regs, percpu_needs_fixup); } =20 /* --=20 2.51.0 From nobody Mon Jun 8 22:51:24 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B214B3BF695; Tue, 26 May 2026 05:57:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775030; cv=none; b=QJ8HvngnZNQKbHHrQh2SGGqw+KTs8dZuVsYUdqZC8NlCvBhsJSl60ts/wUuD5REVsviZS7JqfGedn1YFUS89KiWK4AD/Qvi5T+A/AOojUrkBliCWml99VUgef3Nqz5cFVkO4nRVb0wzDay6MCzmQU5nKJxkdflXueilMIJSq3GU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775030; c=relaxed/simple; bh=b7FCCZgRhyt9e0vVBfaDC2rx8PAEMyApt/6cQkR9LLM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OTqDqaBe7vm5YRVabQCAf+LCiKnZVzeO0W1jGkg51/8EXsCcjP5HlEXYvXFzyvC8X1tiOxrYp6YqbfFKctHQ2noQKiw6XHwr+XJzUZ2AB+QrvyjzPpqiPhidnyzB1wjl1tggJxw0HNtR0TgYOiqSngUcnjsZAiCE4oMeOBvAyos= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=fhJWDU/u; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="fhJWDU/u" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64Q0KAwI868208; Tue, 26 May 2026 05:57:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=WlfIVcJXQcx0zZvV7 ElSdd7SSTWdtlR0tVVA0tV2ph8=; b=fhJWDU/unSwW6R6jb9hyfXHdLTiGkExXt QmXLS17/ZacnR/5MpBmuWsghdPBWGvwPA2/WtYZ7aiLHT2oAR93PsXb96BJnI/eX i6Nn+Q72ESTJxLEeyHW5RlbDI0/HIt2/G3/RJeJACcRBwlPGihz/cJsJelrFl+gx r7xTiRDR3xnR1nF8htQM2J5Ea0zD72GQ1MvLGSsYWOc+igf+Ror5Qr08ZzaN0/dy 9o/+VksQut32qxJhBDECtOOAdkEer72HSdHNgkD6aQSBa5Md+6VHqAnb+6LfLnpr 0vVg6zoSaTHzrPTrU4/uEDlXW/NUuxbEXhRDq6BJot100o9B4yrcw== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4eb4s2ajh7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:07 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64Q5s7Mu016023; Tue, 26 May 2026 05:57:07 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4ebs8y7q8d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:07 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64Q5v3k053215630 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 May 2026 05:57:03 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8294820040; Tue, 26 May 2026 05:57:03 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 66C242004B; Tue, 26 May 2026 05:57:03 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 26 May 2026 05:57:03 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v5 2/7] s390/percpu: Add missing do { } while (0) constructs Date: Tue, 26 May 2026 07:56:57 +0200 Message-ID: <20260526055702.1429061-3-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260526055702.1429061-1-hca@linux.ibm.com> References: <20260526055702.1429061-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: dxWMZMAXrLVAcOgzTcvpcmW0SsK7N_kb X-Authority-Analysis: v=2.4 cv=Sq2gLvO0 c=1 sm=1 tr=0 ts=6a153634 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=U7nrCbtTmkRpXpFmAIza:22 a=c92rfblmAAAA:8 a=VnNF1IyMAAAA:8 a=VwQbUJbxAAAA:8 a=V-yoBbGPs5LSgb0Mvs4A:9 a=GvGzcOZaWPEFPQC_NcjD:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTI2MDA0NiBTYWx0ZWRfX/GWOz43pv5J+ nnSwOZ8ahvwdCd8rWNi4C25NI5aRn7tjd20KbtHTBmuKvyWz17GH6eNZ/Kc7+Eh1EK0Om+3IEKj nHkQAcfqXY4SZ+T5HoxCP6IYEt/fbI89E/j6IjznaRTqV2+S2t4zV73cxiCzBzuz9PMSVi63rwS CQf/LGTIdDuUk8LreXNzUBa3UOmvB9prNCffVgkGWjzliSoDcqB7TueGzF21EMHMPd2DP36cUir 5xstM08CjQ1aZSe7xdvKLw1FJX4fFqgRC1ScjiLTARd0jIBmPPi0l0KN7ggooBZyaKKMNpzh431 m/QWhhR88pUWRZ0IrYRQWlydnB9JTnc1vhy06XDOzfPieB17Xe2y/xqouHX7pro6BrThONPXD0J Z1MkchsF+qy/CZz1JwNxBT31wYD7vm9R9SqXoNkcPbTXyxtpvrm7LpOwzNBIGddpAiZcsYyDJfy nE92ZbgAL0zT4o/MaQg== X-Proofpoint-ORIG-GUID: dxWMZMAXrLVAcOgzTcvpcmW0SsK7N_kb X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-26_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 priorityscore=1501 phishscore=0 impostorscore=0 malwarescore=0 lowpriorityscore=0 adultscore=0 clxscore=1015 suspectscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605260046 Content-Type: text/plain; charset="utf-8" Add missing do { } while (0) constructs in order to avoid potential build failures. Reported-by: Sashiko Closes: https://sashiko.dev/#/patchset/20260319120503.4046659-1-hca%40linux= .ibm.com Signed-off-by: Heiko Carstens Reviewed-by: Alexander Gordeev --- arch/s390/include/asm/percpu.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 78602d2f5eba..79d5a4460b18 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -136,7 +136,7 @@ #else /* MARCH_HAS_Z196_FEATURES */ =20 #define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ -{ \ +do { \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ @@ -157,7 +157,7 @@ : "cc"); \ } \ preempt_enable_notrace(); \ -} +} while (0) =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_add(pcp, val, "laa", "asi",= int) #define this_cpu_add_8(pcp, val) arch_this_cpu_add(pcp, val, "laag", "agsi= ", long) @@ -182,7 +182,7 @@ #define this_cpu_add_return_8(pcp, val) arch_this_cpu_add_return(pcp, val,= "laag") =20 #define arch_this_cpu_to_op(pcp, val, op) \ -{ \ +do { \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ @@ -194,7 +194,7 @@ : [val__] "d" (val__) \ : "cc"); \ preempt_enable_notrace(); \ -} +} while (0) =20 #define this_cpu_and_4(pcp, val) arch_this_cpu_to_op(pcp, val, "lan") #define this_cpu_and_8(pcp, val) arch_this_cpu_to_op(pcp, val, "lang") --=20 2.51.0 From nobody Mon Jun 8 22:51:24 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E7203BFAD4; Tue, 26 May 2026 05:57:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775031; cv=none; b=rZv29kyheprpx+bQ/izCILRlKEJWlbgI43kiLgrVR9ZFBzAGYroLeS+BJH5V+Gqxa98SCdhU62EWA8taNO3flV3IbPzp1YcNHtImyD3AxTWBVfoX3ZP0rGmfrPIZCuUQnEyW4PkzfNn1ViYESpNuroXWuExuCXISjsncATaCh1Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775031; c=relaxed/simple; bh=sGHtPgqRyGymhZBuROEwH1odH0nmdeQ6hAqZerhhFHE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IgEJH+j1KdxL1XNkYBu7uGCkWJeyOissO1Jiew+webEde5Xe2DnVlGrWqqGLaKrKzGMHd8hbnFI4WTLqs1c+aiRewhii5oEupdUcplB1eb7s6+GTBg8ht+DGYeHH2BXXVaAdpmlM/emRMazOfhxh0VjdlyAFjJvHF3oXOHdYqdM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=lmNy4HVI; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="lmNy4HVI" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64PH5kDP1745959; Tue, 26 May 2026 05:57:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=DdArDXsLaP5gy6DIj K0XhME+n5TSc7pgL2hQ0CfDqcY=; b=lmNy4HVIB8Cy5GqrKWx6W0V0Xx6Dj0j9Z 8sfzedrMeUO2NXt2hMzKLFJN1EI6IDpeIL33AokIwymi2L7kw+IBh1b1MmNsuEKU 1ryTAMdEnGsEw0HxRDE2WFtzPsAlpVERobKcKFtVcoMA11sX/V7HXsoNCnbkclGe EPudguwlHb38xpEkf+5DSV53dv32onRnKCqPygjYClzEtYBQ83+SGZmryqje37c2 MyR1UJPrjGMyBhG+Yzz55wEsePb3Gcmq2UZGnuvnaIymf0G8v3v6UgiX5j8qRHf0 q5LCP0aL7RrIrBHy/CCdVPrKOtUMZG+kmEsaVrPdjD6jMnS3Bse4Q== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4eb4s2ajh8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:08 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64Q5sD19022574; Tue, 26 May 2026 05:57:07 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4ebpxw02xq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:07 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64Q5v3c953215632 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 May 2026 05:57:03 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A195F20043; Tue, 26 May 2026 05:57:03 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 869222004D; Tue, 26 May 2026 05:57:03 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 26 May 2026 05:57:03 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v5 3/7] s390/percpu: Use new percpu code section for arch_this_cpu_add() Date: Tue, 26 May 2026 07:56:58 +0200 Message-ID: <20260526055702.1429061-4-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260526055702.1429061-1-hca@linux.ibm.com> References: <20260526055702.1429061-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 8WremlosdwRyTSQwgOCSUV9jMD2rS4B2 X-Authority-Analysis: v=2.4 cv=Sq2gLvO0 c=1 sm=1 tr=0 ts=6a153634 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=U7nrCbtTmkRpXpFmAIza:22 a=VnNF1IyMAAAA:8 a=SujZmW7ZfIunUtho7CIA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTI2MDA0NiBTYWx0ZWRfX0vkxVPg3PTCg KXuREn8uWycP2THGj681qFox47B3PdWTJlDw58oE9s2sH5TP2IixiPmXYcUpeBixgztOii086f5 wPlW2YtAN9GjVDVShnkR72jV8G2DQ3CXd7Lw4tHgYK4fC/1jMVK0Le68sxfw6rJ2DtVL3IuMAlz xC+kkSDReMid0JevlD85aop/mMmfVg2PnOCvJVI1zVX13QBvGS0fRBuQlQu8tnMzeq09FrmLDYV WS5U35oSm+xtDQ3FkwzXsY0ZSaAxDuQ/uUui0A1xAHAGzBk1NJoQrLCOLAHo+d1NqyndwMGB9YH d1pIRotr2VLXjfCl3//6FJ3CITL7v5YNs3z4MxNk//Tkp7t0VV5PF5Cr9UpifTwfKdOMZwtf86E Z1vBGdHAoDJqIOHEF2tqgjScbR4xdhnjSIAD342ATVljYg6iXeeBVfrHyIzK1FI1/KjSaW6hkeG 140EAUYcGeIl4NNTZsw== X-Proofpoint-ORIG-GUID: 8WremlosdwRyTSQwgOCSUV9jMD2rS4B2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-26_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 priorityscore=1501 phishscore=0 impostorscore=0 malwarescore=0 lowpriorityscore=0 adultscore=0 clxscore=1015 suspectscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605260046 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_add() to make use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~76kb (defconfig). Also more than 5300 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. With: DEFINE_PER_CPU(long, foo); void bar(long a) { this_cpu_add(foo, a); } Old arch_this_cpu_add() looks like this: 00000000000000c0 : c0: c0 04 00 00 00 00 jgnop c0 c6: eb 01 03 a8 00 6a asi 936,1 cc: c4 18 00 00 00 00 lgrl %r1,cc ce: R_390_GOTENT foo+0x2 d2: e3 10 03 b8 00 08 ag %r1,952 d8: eb 22 10 00 00 e8 laag %r2,%r2,0(%r1) de: eb ff 03 a8 00 6e alsi 936,-1 e4: a7 a4 00 05 jhe ee e8: c0 f4 00 00 00 00 jg e8 ea: R_390_PC32DBL __s390_indirect_jump_r14+0x2 ee: c0 f4 00 00 00 00 jg ee f0: R_390_PLT32DBL preempt_schedule_notrace+0x2 New arch_this_cpu_add() looks like this: 00000000000000c0 : c0: c0 04 00 00 00 00 jgnop c0 c6: c4 38 00 00 00 00 lgrl %r3,c6 c8: R_390_GOTENT foo+0x2 cc: b9 04 00 43 lgr %r4,%r3 d0: eb 00 43 c0 00 52 mviy 960(%r0),4 d6: e3 40 03 b8 00 08 ag %r4,952 dc: eb 52 40 00 00 e8 laag %r5,%r2,0(%r4) e2: eb 00 03 c0 00 52 mviy 960,0 e8: c0 f4 00 00 00 00 jg e8 ea: R_390_PC32DBL __s390_indirect_jump_r14+0x2 Note that the conditional function call is removed. Signed-off-by: Heiko Carstens Acked-by: Alexander Gordeev --- arch/s390/include/asm/percpu.h | 65 ++++++++++++++++++++++------------ 1 file changed, 43 insertions(+), 22 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 79d5a4460b18..9140d81b7efc 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -135,28 +135,49 @@ =20 #else /* MARCH_HAS_Z196_FEATURES */ =20 -#define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ -do { \ - typedef typeof(pcp) pcp_op_T__; \ - pcp_op_T__ val__ =3D (val); \ - pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - if (__builtin_constant_p(val__) && \ - ((szcast)val__ > -129) && ((szcast)val__ < 128)) { \ - asm volatile( \ - op2 " %[ptr__],%[val__]" \ - : [ptr__] "+Q" (*ptr__) \ - : [val__] "i" ((szcast)val__) \ - : "cc"); \ - } else { \ - asm volatile( \ - op1 " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ - : "cc"); \ - } \ - preempt_enable_notrace(); \ +#define arch_this_cpu_add(pcp, val, op1, op2, szcast) \ +do { \ + unsigned long lc_pcpr, lc_pcpo; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ val__ =3D (val); \ + pcp_op_T__ old__, *ptr__; \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + if (__builtin_constant_p(val__) && \ + ((szcast)val__ > -129) && ((szcast)val__ < 128)) { \ + asm volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op2 " 0(%[ptr__]),%[val__]\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]") \ + : [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "i" ((szcast)val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + } else { \ + asm volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op1 " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + } \ } while (0) =20 #define this_cpu_add_4(pcp, val) arch_this_cpu_add(pcp, val, "laa", "asi",= int) --=20 2.51.0 From nobody Mon Jun 8 22:51:24 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 953313BFAE1; Tue, 26 May 2026 05:57:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775031; cv=none; b=E6xIZYUTyHfVdNSZmbOz0Et2zrOgGjIoakI52GoQJW8f5by3H76690UrG0+eawVoHoWiY4CvBR4P8zXAgq9eLoSsDDoylI/gcbkj+B/UyZC8rpLsmXCCCeMEKFmx3rAUVdOvno5nrH+nONQ7Tjt5rB9JbHelbzFsNTSSc4EI3uA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775031; c=relaxed/simple; bh=NpPn4ELfcR8XVH/mho/bZeNOVATbqfABoWtMd+c4jTA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=I+uuqdiAAtVgV5jimNy0EKNV+dvOwjkQd2KKR4arHHBBarJ0CD8E8x0N2R1KeSGPj/ltPU681FeKD3wjqpRk0snAYpyj0ZSiIXyTqy0GddhIy0yly5qQf4HkRNZcl5iqoGzJXwPXyJI5oMGmdNcxW+PiVt2Foqm2gy8mKi1IwWo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=nSI4D+xN; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="nSI4D+xN" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64PLQ9sf3804509; Tue, 26 May 2026 05:57:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=w4VLruBjFnqGeLxXB 934mHc/udarqvdvuNRwsV+6fx4=; b=nSI4D+xNCahkpg7ytqNc2/Aut1McuxeST qnYaX9ML62UtQqJVjVY8cOKLOAk0JOodm1S620IedzQ08UW2hao7ooBa8SWksmER irbP9Z3urGoagpYUsiDhdKAEWnTkoUOCjlZKiutLx9T6J5qe+l1kF87FJFgQoyB7 gNSRkV4ng663/PokwxZpEBLAc7fAE2I3KK3lj3HK7SZFgD/ahTonGx4FmvROuSa5 eBelLbvnICh2rMyN4lxoK+8J+9JH25kDb+XGO3ytblWFFRT9AD/ZlET8K3MePpIU 1kHaARrYafmIWZfw65pPhL8uEMVnEAfjjOKNwXpSRv4yn+YeHK2CQ== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4eb4qbthbj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:08 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64Q5sCH4031341; Tue, 26 May 2026 05:57:07 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4ebr2gyvu4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:07 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64Q5v3jh33816920 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 May 2026 05:57:03 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C098020040; Tue, 26 May 2026 05:57:03 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A57F22004B; Tue, 26 May 2026 05:57:03 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 26 May 2026 05:57:03 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v5 4/7] s390/percpu: Use new percpu code section for arch_this_cpu_add_return() Date: Tue, 26 May 2026 07:56:59 +0200 Message-ID: <20260526055702.1429061-5-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260526055702.1429061-1-hca@linux.ibm.com> References: <20260526055702.1429061-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTI2MDA0NiBTYWx0ZWRfX/DBIUf688vhe k07XpiRnHgn6WqtRfwLAOuM7xx3LdRJVzrT4ALIuaLFEF+/Z+cBq13ZW9nwJ7Dess2z+Y0OkQ/t yUT5mbD1c2UQlebp25UMQiejfCwfHB818pj865y0UTJHJquTU5borQxwZ0lgOeh8WM/7ikG9u5I zz3uXgrqZgbPNWy38KvSo6pCA725Z2kE3KSdHQJWtMhbv0s7HRiE05jOPMP0uo95DQBin2FOrIa RUoShJFzjAslBfUiVKaDzlfYGNeDGusq2bPbWWMPtLBdz/zPFQiEIyGdUZ2QfEmHaJCzFM1Szcu FL2z0d+EKifDNXxDWq2Z/B8HHfLqEzGlYQrElHaQLYDTeZaNB6EYAQ9sOpoYaez+k8Pv6uVh0TS HuWf0yI8ZZ8TmA7f3Emp134EFgBDFdKTT1Q41PLrKREechEeZk0s/mHX1Fo/jzk6x361wYoj6R6 Uxv+X4njQMHRS39haBQ== X-Authority-Analysis: v=2.4 cv=KItqylFo c=1 sm=1 tr=0 ts=6a153634 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=VnNF1IyMAAAA:8 a=rpl92wOeoJCnW_j-IKQA:9 X-Proofpoint-ORIG-GUID: 12h1JfXv8IuLUVc8BuafzjQ81vpTPX23 X-Proofpoint-GUID: 12h1JfXv8IuLUVc8BuafzjQ81vpTPX23 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-26_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 lowpriorityscore=0 spamscore=0 priorityscore=1501 clxscore=1015 adultscore=0 suspectscore=0 bulkscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605260046 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_add_return() to make use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~4k (defconfig). Also 66 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. Signed-off-by: Heiko Carstens Acked-by: Alexander Gordeev --- arch/s390/include/asm/percpu.h | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 9140d81b7efc..f2d0e0354582 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -185,17 +185,29 @@ do { \ =20 #define arch_this_cpu_add_return(pcp, val, op) \ ({ \ + unsigned long lc_pcpr, lc_pcpo; \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - asm volatile( \ - op " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ : "cc"); \ - preempt_enable_notrace(); \ old__ + val__; \ }) =20 --=20 2.51.0 From nobody Mon Jun 8 22:51:24 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 63C433BFAD5; Tue, 26 May 2026 05:57:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775031; cv=none; b=RitstomDy/ydnlNuRn9hjX3g6AghLNptgO0wJSHwwEI9Xu6l4ERuOCKn+6+sDMM7y0XdwmTh8EA9Wro2FKqU9NSuTZPoBU3oKuvuKRjnDqaN5U/hnJyTbu4DjjAE1xXzsrX9q0pIlKeiTd6mlnOdq/eCq0F8v7D13G9P9YJPj4Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775031; c=relaxed/simple; bh=GaYj1K6sO65BvmgdyPCtcYf/B0XutCChjv7AN9LJEoQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XtFUcr3ii/Jrd2YHf6lBJjErgoswII0rnDzDZoUfPAorfiCV+fdcpSGJMB9x0iuAWtK8MyCAOvsFuXOYqRFF68DuHyGPZPSubCaN3/BQKX0Kc2eWtdb9jFIOiDmt+lhRS49InbUVIVRS/ofN8WqYBTJWFYJEMRGN/JQPOUv+vqc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=nxnXQ0JK; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="nxnXQ0JK" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64PL0gel645442; Tue, 26 May 2026 05:57:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=ca4EJwt3Fxr9YM+Gd 7nvh+qnRJop56S/2rZYP22tEpw=; b=nxnXQ0JK2pj+/GJ+XZfozb9Ti5dHavwW5 zRGDQyzQlT9/OcPYoHAI2+C4W/UdZIR2vDeRzacMFIGDcBBAd4Zz5gGIaRWvALTh EpwN0EZkJr+V5waX+XBqCtjYnD2iMtd5UvuinOS8/klcFd3DO/RGnFmv95NEZv8B V/eEZSu+rFVbCY5AfPFxQG5hI15CfTNQY1wpL3MUlYYG114YlUoqk3kjGCpHV6xz X/1/UUOM2Vt5RCTtZ/TqXwfqq+5tgG4DgpaCgtXz4FF04XFvs+QOGtH9mXIcSeR3 4UBn0x/FOcmcs0dJOMk24wdKszDsjn3CLvPHxXZhDuNf1nEv6T8Gg== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4eb4nc1ng3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:08 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64Q5sRH5022622; Tue, 26 May 2026 05:57:07 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4ebpxw02xr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:07 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64Q5v3mK52167020 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 May 2026 05:57:04 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DFD5A20043; Tue, 26 May 2026 05:57:03 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C492F2004D; Tue, 26 May 2026 05:57:03 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 26 May 2026 05:57:03 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v5 5/7] s390/percpu: Use new percpu code section for arch_this_cpu_[and|or]() Date: Tue, 26 May 2026 07:57:00 +0200 Message-ID: <20260526055702.1429061-6-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260526055702.1429061-1-hca@linux.ibm.com> References: <20260526055702.1429061-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=VvYTxe2n c=1 sm=1 tr=0 ts=6a153634 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=V8glGbnc2Ofi9Qvn3v5h:22 a=VnNF1IyMAAAA:8 a=4dLYTkTKlH5g-JvQF6AA:9 X-Proofpoint-ORIG-GUID: PSWQ13bMMSalE2ZEI7En-vpjHCh-x9ob X-Proofpoint-GUID: PSWQ13bMMSalE2ZEI7En-vpjHCh-x9ob X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTI2MDA0NiBTYWx0ZWRfX4F5egPZuA0Gn nIvJUqveNFRD0RbXavULVx2dAAD1lKgxK9xqVlWa3YFycQKX5weO5cJPd3a896W1ehcjMem8Du9 neS/W41zLWBu1UQvgi5hS8L4gumLHMQSbp2gAIkr/8TJT1m3EzbdMGvO9oVXjbxUIRZfUginNmb iR7UXIp2CvJJrTc65U9fTQF1fEvA2Id65ECCaw2F5r6NrTi+RFORYfBBO/8s3BdM36FC6B3MWGc PZ8ZD6iK69Fs5CYUJQx1hr9q8sO7RF+XpUTpxu/3m7pG4I39yblgc08NaceBr2xcCyx9RfXYc3C CvbUfsqfcZulAxEQ8p+lB2vh84/FMOC8fD7q6fgayu8eqIswD1cJhFX40CzOheBkZaVygWBDMlW LecalnyL3aNJFgzXgjLxjGVaQHKK6hPDKexaQnoo3+ODHW2O/1S9ZHI8vJGKXHERtDr20c6q48a KV9TCh4WudDwzzwSzrg== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-26_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 phishscore=0 priorityscore=1501 spamscore=0 adultscore=0 lowpriorityscore=0 malwarescore=0 impostorscore=0 clxscore=1015 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605260046 Content-Type: text/plain; charset="utf-8" Convert arch_this_cpu_[and|or]() to make use of the new percpu code section infrastructure. There is no user of this_cpu_and() and only one user of this_cpu_or() within the kernel. Therefore this conversion has hardly any effect, and also removes only preempt_schedule_notrace() function call. Signed-off-by: Heiko Carstens Acked-by: Alexander Gordeev --- arch/s390/include/asm/percpu.h | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index f2d0e0354582..5e0185e5960b 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -216,17 +216,29 @@ do { \ =20 #define arch_this_cpu_to_op(pcp, val, op) \ do { \ + unsigned long lc_pcpr, lc_pcpo; \ typedef typeof(pcp) pcp_op_T__; \ pcp_op_T__ val__ =3D (val); \ pcp_op_T__ old__, *ptr__; \ - preempt_disable_notrace(); \ - ptr__ =3D raw_cpu_ptr(&(pcp)); \ - asm volatile( \ - op " %[old__],%[val__],%[ptr__]" \ - : [old__] "=3Dd" (old__), [ptr__] "+Q" (*ptr__) \ - : [val__] "d" (val__) \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[old__],%[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]") \ + : [old__] "=3D&d" (old__), \ + [ptr__] "+&a" (ptr__), "+m" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ : "cc"); \ - preempt_enable_notrace(); \ } while (0) =20 #define this_cpu_and_4(pcp, val) arch_this_cpu_to_op(pcp, val, "lan") --=20 2.51.0 From nobody Mon Jun 8 22:51:24 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA3983BFAD7; Tue, 26 May 2026 05:57:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775032; cv=none; b=I6FICdMTY65PlphhQEY9BssxVrfXRSrV9SJ0BDekK+5RiOFojlHKd2E0sxL7AXRqd/6SafeeVxesdN2Yxv5uzuiDR/zIlcmg7a2+qQddo247PR6d5H1PuayKz+Q1k12F0sHPPhbIkdoELH3Ei9FHdAeC3DV3mHKyPxMpN3tF4ac= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775032; c=relaxed/simple; bh=8snD0Rp33NTlrJHQS9WPfU5G9iNcR8DL4dp7/u1tK0Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dgw7dideNfqfkPVfHX9rjLrgT7jMTYoqpaoaZ2V4dvrya5ejbW4PokrapbNVXWlA6x/M/z54A2YKrlWYsT4aYe0SNrrgKkg2t6/x4L5koTCLKKF9SkiBfWM4L1EOa1hJfLVK5TxCyUxXcIuaPpldVLLRnWSzBXdHan1arDtYMBM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=XoIJJAgJ; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="XoIJJAgJ" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64PIc9jD3938475; Tue, 26 May 2026 05:57:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=aAiJWe1ptyRsqDIAA Zzjje5rg3egV0JixWRTUR8qANE=; b=XoIJJAgJFHXnOlgdwVasuw90xkyqWF/WK /UldO1BkDuCThZPj65ofqpuenra39TF5aJ8L4mbu7NEMP1IHKX0IC7mHPRbPjtxd +wyATKo2JtaRKq+2Sa1XF1hERbS4Ixuwv2q3yfGbhDT5aKa8oM/aa6B3GzaMTSUH sXnoiiVqjU95z5xciQiorLtEgjU6XnnaJSdZVqHLDi7pasH8gTlk3MDdChGedmpD 1cuddboVrmiIfeNQWkmjSrCMTjQCmD0Tid0t0QMieXvVptHb1Tma2p3qQZ4IlFIi mQUHFbQNjEtZtJ36qr7umooIKCBdRelko3+ULqBczx8l3SP77Jo3A== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4eb4nq1pq8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:08 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64Q5sD1A022574; Tue, 26 May 2026 05:57:08 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4ebpxw02xt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:08 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64Q5v48C21103220 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 May 2026 05:57:04 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0A73320040; Tue, 26 May 2026 05:57:04 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E39372004B; Tue, 26 May 2026 05:57:03 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 26 May 2026 05:57:03 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v5 6/7] s390/percpu: Provide arch_this_cpu_read() implementation Date: Tue, 26 May 2026 07:57:01 +0200 Message-ID: <20260526055702.1429061-7-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260526055702.1429061-1-hca@linux.ibm.com> References: <20260526055702.1429061-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: lX4ZbP0K-bTkLRJGr-4JaWAcvZYmbukT X-Proofpoint-ORIG-GUID: lX4ZbP0K-bTkLRJGr-4JaWAcvZYmbukT X-Authority-Analysis: v=2.4 cv=QIJYgALL c=1 sm=1 tr=0 ts=6a153634 cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=Y2IxJ9c9Rs8Kov3niI8_:22 a=VnNF1IyMAAAA:8 a=4bOSrtcVyj7iNoH5sBwA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTI2MDA0NiBTYWx0ZWRfX/RdYwO/x0dXO ehCDPgFUUbFu5UplGnQvg8wBZdFC02FWh9stk2MPXpK1eSvqWkf3m8Rq7UphjCdX3gwWWSdynCK 8Hnjtr+aSUUEyUGeO+cEsuA6Dm0acGNRzkW47x9K5oqVG8DccMGplaTidXCCJoLeTrvoOKQrHH5 5TG6FyMLYGocbrEtwvTolApfQCRc2tqk4MbKwxXmFqgNY/h/UVB6gDeg8zcN7GgL92WGU4r9qkJ pCXVdJGKjAtc4Qd0UuEicxdOzYz1viIx2D3aRpEeafpAclDVha44kQaWObdjAmm2FWBJz/It2RN RKA0rOSylh659qoVHhBHpV40oo232NJZ9z5MivSqfbqe1ECq04igaVEY/u+FJomRQSu+kTiT6CL DePaNLmFlu6w7j0KPDBXsE/fKfkFmURkOva91L3hIwTHRWe8PAGbyM8KuTXAEjkjxCh7GK+xugB VSFcnEqzvf5d/caJvfA== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-26_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 impostorscore=0 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 adultscore=0 spamscore=0 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605260046 Content-Type: text/plain; charset="utf-8" Provide an s390 specific implementation of arch_this_cpu_read() instead of the generic variant. The generic variant uses preempt_disable() / preempt_enable() pair and READ_ONCE(). Get rid of the preempt_disable() / preempt_enable() pairs by providing an own variant which makes use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~1k (defconfig). Also 87 generated preempt_schedule_notrace() function calls within the kernel image (modules not counted) are removed. Signed-off-by: Heiko Carstens Acked-by: Alexander Gordeev --- arch/s390/include/asm/percpu.h | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 5e0185e5960b..83195a5dc409 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -248,6 +248,37 @@ do { \ =20 #endif /* MARCH_HAS_Z196_FEATURES */ =20 +#define arch_this_cpu_read(pcp, op) \ +({ \ + unsigned long lc_pcpr, lc_pcpo, res__; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ *ptr__; \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[res__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]") \ + : [res__] "=3D&d" (res__), [ptr__] "+&a" (ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (*ptr__), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ + (pcp_op_T__)res__; \ +}) + +#define this_cpu_read_1(pcp) arch_this_cpu_read(pcp, "llgc") +#define this_cpu_read_2(pcp) arch_this_cpu_read(pcp, "llgh") +#define this_cpu_read_4(pcp) arch_this_cpu_read(pcp, "llgf") +#define this_cpu_read_8(pcp) arch_this_cpu_read(pcp, "lg") + #define arch_this_cpu_cmpxchg(pcp, oval, nval) \ ({ \ typedef typeof(pcp) pcp_op_T__; \ --=20 2.51.0 From nobody Mon Jun 8 22:51:24 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBE483BFAE8; Tue, 26 May 2026 05:57:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775031; cv=none; b=qlkDuJwIe+arKGymYqO0+qxGJQot+lzNYtMWIG3dTUNmeafaog/J9wSYqfiJJ754a/AfvyUsQB1wo/lRm3SuhcV3qeatWhzga+q+ceN+eZhFUXbhOiMJFLL9P2wFRmuBbCq1dw03Km6Pq/IFTmyk8OQQgXnJUxYoNvehRmhGWa4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779775031; c=relaxed/simple; bh=43Xl+6ylmbaGi3gLHfwJG+pGBATML3jLmk+Sa/dYv7Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T6VHxhxb85TRxTdfS/bM4oamPUwtbwntLCuppk0Q7tLa6CLwrUMmK+n9oDNqEw/UGAui3RBOuj/AKBhqNb4Kq3vSU0HdAsKjzR7H703fnYoKL7aWenHGN/FwYDh98O+6qzjx9Nji/2P8YrLB3IhoeeX+4N+7ir5VGAclC66ENlo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=c6BPu/aQ; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="c6BPu/aQ" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64PFGRM53026762; Tue, 26 May 2026 05:57:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=P0qd27YAF/bkGFxaW C+Ca8sscSoRItaSoEXNuNGM6sE=; b=c6BPu/aQf09ezW32YpfbaVkrSGjfgy8Wk e+AdfkjjTf+m5M3pcvXnaJIVONf14yZzbEGppntAsF7DeQRMjl/wMxuoFYxX6E3i eZbKN1x2sxHQRAH5q8povWCtqfWbzFhvkv7dsklfiF5GAnukV029lM21BHlBovDU ntBio2UHsapZHtYTcfxyRfaCQjiIiuQWuVmyoD6slgnSG8hcVyle+4dxWMreaZdC 7pYpySHuroUiBVRLy/8M+w4WFUmZhGV5S63/qDpdgx8JMbNO93XL1CiQjBrpkAK2 p9OR/EiqqjNkpIO2AxIHEUYcTvibJFx5dB1B5laCEFPq76ttkc+iQ== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4eb4nc1ng5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:08 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64Q5s9Bc010744; Tue, 26 May 2026 05:57:08 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4ebqjjqyyb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 May 2026 05:57:08 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64Q5v4cN14549250 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 May 2026 05:57:04 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2939020043; Tue, 26 May 2026 05:57:04 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0E3022004D; Tue, 26 May 2026 05:57:04 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Tue, 26 May 2026 05:57:04 +0000 (GMT) From: Heiko Carstens To: Alexander Gordeev , Sven Schnelle , Vasily Gorbik , Christian Borntraeger , Juergen Christ Cc: linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v5 7/7] s390/percpu: Provide arch_this_cpu_write() implementation Date: Tue, 26 May 2026 07:57:02 +0200 Message-ID: <20260526055702.1429061-8-hca@linux.ibm.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260526055702.1429061-1-hca@linux.ibm.com> References: <20260526055702.1429061-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=VvYTxe2n c=1 sm=1 tr=0 ts=6a153634 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=V8glGbnc2Ofi9Qvn3v5h:22 a=VnNF1IyMAAAA:8 a=ObmbXjF690NQJU6EaucA:9 X-Proofpoint-ORIG-GUID: MFsBNTGGsVTzL0XQ9DEVaSYICoZgsJDi X-Proofpoint-GUID: MFsBNTGGsVTzL0XQ9DEVaSYICoZgsJDi X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTI2MDA0NiBTYWx0ZWRfXyo5aS4sfu1G9 2/NBn8u4O390Um7QCutdD6m5sv1U+LcGqgZrzU1hOGfGojgF6Noxum7HUYifk+n44sCG6xPkaie Jr5lt8DLVclY4w4PnujrNGrNIA5gEwK0zJqMOxlbU8L2LhLm0JgHAzRnrs5hms0zuE9UrwsUIr1 zBGB3xxRhfykSqU0cjIIL7Bcyd8QJRDDWUEdGLDd+O+DYX6POfU+39Kkk7dG+356cxksH0FMYL0 xgAJn0cf7Pa3HRJ6ZidxUvaEJY78wJWqybQwCIH9j69v8SENUTo+gTs1tgu0+ipR/b/lXmMBhXr HM7YmxIy5P5W1lP3BCYkZJTXdMP9XO2gchlCYfJINkikXPYZLvOTZdkuDuLGpWRtgrutqE0l0My ey+a+CHACMl8dUWfsw2x+U6siK4rnoX4COF8O/OguLET7nFdw2o2l2b7PMPE30P9RHa9vREGp+m 5UsbTX9MQQYOigyDSxg== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-26_01,2026-05-18_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 phishscore=0 priorityscore=1501 spamscore=0 adultscore=0 lowpriorityscore=0 malwarescore=0 impostorscore=0 clxscore=1015 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605260046 Content-Type: text/plain; charset="utf-8" Provide an s390 specific implementation of arch_this_cpu_write() instead of the generic variant. The generic variant uses a quite expensive raw_local_irq_save() / raw_local_irq_restore() pair. Get rid of this by providing an own variant which makes use of the new percpu code section infrastructure. With this the text size of the kernel image is reduced by ~1k (defconfig). Signed-off-by: Heiko Carstens Acked-by: Alexander Gordeev --- arch/s390/include/asm/percpu.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/arch/s390/include/asm/percpu.h b/arch/s390/include/asm/percpu.h index 83195a5dc409..1d955dd0defa 100644 --- a/arch/s390/include/asm/percpu.h +++ b/arch/s390/include/asm/percpu.h @@ -279,6 +279,36 @@ do { \ #define this_cpu_read_4(pcp) arch_this_cpu_read(pcp, "llgf") #define this_cpu_read_8(pcp) arch_this_cpu_read(pcp, "lg") =20 +#define arch_this_cpu_write(pcp, val, op) \ +do { \ + unsigned long lc_pcpr, lc_pcpo; \ + typedef typeof(pcp) pcp_op_T__; \ + pcp_op_T__ *ptr__, val__ =3D (val); \ + \ + lc_pcpr =3D offsetof(struct lowcore, percpu_register); \ + lc_pcpo =3D offsetof(struct lowcore, percpu_offset); \ + ptr__ =3D PERCPU_PTR(&(pcp)); \ + asm_inline volatile( \ + MVIY_PERCPU("%[disppcpr]", "%[dispaltpcpr]", "%[ptr__]")\ + AG_ALT("%[disppcpo]", "%[dispaltpcpo]", "%[ptr__]") \ + op " %[val__],0(%[ptr__])\n" \ + MVIY_ALT("%[disppcpr]", "%[dispaltpcpr]") \ + : [ptr__] "+&a" (ptr__), "=3Dm" (*ptr__), \ + "=3Dm" (((struct lowcore *)0)->percpu_register) \ + : [val__] "d" (val__), \ + [disppcpr] "i" (lc_pcpr), \ + [disppcpo] "i" (lc_pcpo), \ + [dispaltpcpr] "i" (lc_pcpr + LOWCORE_ALT_ADDRESS), \ + [dispaltpcpo] "i" (lc_pcpo + LOWCORE_ALT_ADDRESS), \ + "m" (((struct lowcore *)0)->percpu_offset) \ + : "cc"); \ +} while (0) + +#define this_cpu_write_1(pcp, val) arch_this_cpu_write(pcp, val, "stc") +#define this_cpu_write_2(pcp, val) arch_this_cpu_write(pcp, val, "sth") +#define this_cpu_write_4(pcp, val) arch_this_cpu_write(pcp, val, "st") +#define this_cpu_write_8(pcp, val) arch_this_cpu_write(pcp, val, "stg") + #define arch_this_cpu_cmpxchg(pcp, oval, nval) \ ({ \ typedef typeof(pcp) pcp_op_T__; \ --=20 2.51.0