From nobody Tue Feb 10 15:43:36 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 980351EA80 for ; Fri, 27 Dec 2024 01:10:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735261835; cv=none; b=Pqr+hoQUc0zw+mGhMyR56WOTHYO6D3aFQm4JvbIEY4zrIRW2lRQtp8PmY1G6CtMfW9ilH3X1oLWKqcaRf8NB+jXrABDJ47wVkAYoYD6w0/J4Sr8PSSO+ZaeYBFeeh46veFOnPNApHfgqnX7PKK8nRGU+vGtGZN/mDkI/ENngljM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735261835; c=relaxed/simple; bh=ZQvAGn/2NlyTLE3ty4U7yyFatMFiuoweHmxM3NYxd9s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=i1YaNpFpE/NUAj/CQ45C2W+DGqTAiYjeZWQOVoTlP1Pkz53jkE0EvibhjQMve+8uIpGLvA94qNp0sbRyf5Bs7Fn+j8fkUgOGtVAdrMFjpQ/uIda8IgfNwz4CL8S9XMSLwT7pxgnEdapjnATGNB+DlxS8lz/36Gn8KbQU9cQ2VX0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PeZ1kEAp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PeZ1kEAp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E7F09C4CEDC; Fri, 27 Dec 2024 01:10:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735261834; bh=ZQvAGn/2NlyTLE3ty4U7yyFatMFiuoweHmxM3NYxd9s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PeZ1kEAp68nyIi/RJ5ZdLkSOeUrgMc/fSVBuT0mLtmilREx+IP91edm7/kKg2N1JH iMVn9kpXvF4fjd/rV49CCfOHwi8bCQ3dU997P4ruWiAg78TmkbOprD3tev8LSCKbk3 RbGmIxM5CvgwhAww2+ntH/FodbVcCpMSWW7vEpXgN2P5PEaxQFT2IApJKntndQRJiM Isgvs+5unaC+O7RMUTlxBLTdoeOiJtUcuNBtFYu6SpdC3H/XzzP/FW3HEJK+XzGHwq UZAbuoS8ZmafCtz4HmO41xtz3732f0HOkTJXdMhq4tEIb1gjgFCa3okQYHMq/2EjF7 Of8pTprIEsLng== From: guoren@kernel.org To: paul.walmsley@sifive.com, palmer@dabbelt.com, guoren@kernel.org, bjorn@rivosinc.com, conor@kernel.org, leobras@redhat.com, peterz@infradead.org, parri.andrea@gmail.com, will@kernel.org, longman@redhat.com, boqun.feng@gmail.com, arnd@arndb.de, alexghiti@rivosinc.com, ajones@ventanamicro.com, rkrcmar@ventanamicro.com, atishp@rivosinc.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Guo Ren Subject: [RFC PATCH V2 1/4] RISC-V: paravirt: Add pvqspinlock KVM backend Date: Thu, 26 Dec 2024 20:10:08 -0500 Message-Id: <20241227011011.2331381-2-guoren@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20241227011011.2331381-1-guoren@kernel.org> References: <20241227011011.2331381-1-guoren@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Guo Ren Add the files functions needed to support the SBI PVLOCK (paravirt qspinlock kick_cpu) extension. Implement kvm_sbi_ext_pvlock_kick_- cpu(), and we only need to call the kvm_vcpu_kick() and bring target_vcpu from the halt state. Reviewed-by: Leonardo Bras Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- arch/riscv/include/asm/kvm_vcpu_sbi.h | 1 + arch/riscv/include/asm/sbi.h | 5 +++ arch/riscv/include/uapi/asm/kvm.h | 1 + arch/riscv/kvm/Makefile | 1 + arch/riscv/kvm/vcpu_sbi.c | 4 ++ arch/riscv/kvm/vcpu_sbi_pvlock.c | 57 +++++++++++++++++++++++++++ 6 files changed, 69 insertions(+) create mode 100644 arch/riscv/kvm/vcpu_sbi_pvlock.c diff --git a/arch/riscv/include/asm/kvm_vcpu_sbi.h b/arch/riscv/include/asm= /kvm_vcpu_sbi.h index b96705258cf9..bc37131938ad 100644 --- a/arch/riscv/include/asm/kvm_vcpu_sbi.h +++ b/arch/riscv/include/asm/kvm_vcpu_sbi.h @@ -88,6 +88,7 @@ extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_d= bcn; extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_sta; extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_experimental; extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_vendor; +extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pvlock; =20 #ifdef CONFIG_RISCV_PMU_SBI extern const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pmu; diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h index 6c82318065cf..03e719f076ad 100644 --- a/arch/riscv/include/asm/sbi.h +++ b/arch/riscv/include/asm/sbi.h @@ -35,6 +35,7 @@ enum sbi_ext_id { SBI_EXT_DBCN =3D 0x4442434E, SBI_EXT_STA =3D 0x535441, SBI_EXT_NACL =3D 0x4E41434C, + SBI_EXT_PVLOCK =3D 0x50564C4B, =20 /* Experimentals extensions must lie within this range */ SBI_EXT_EXPERIMENTAL_START =3D 0x08000000, @@ -401,6 +402,10 @@ enum sbi_ext_nacl_feature { #define SBI_NACL_SHMEM_SRET_X(__i) ((__riscv_xlen / 8) * (__i)) #define SBI_NACL_SHMEM_SRET_X_LAST 31 =20 +enum sbi_ext_pvlock_fid { + SBI_EXT_PVLOCK_KICK_CPU =3D 0, +}; + /* SBI spec version fields */ #define SBI_SPEC_VERSION_DEFAULT 0x1 #define SBI_SPEC_VERSION_MAJOR_SHIFT 24 diff --git a/arch/riscv/include/uapi/asm/kvm.h b/arch/riscv/include/uapi/as= m/kvm.h index 3482c9a73d1b..4590570a8fc3 100644 --- a/arch/riscv/include/uapi/asm/kvm.h +++ b/arch/riscv/include/uapi/asm/kvm.h @@ -198,6 +198,7 @@ enum KVM_RISCV_SBI_EXT_ID { KVM_RISCV_SBI_EXT_VENDOR, KVM_RISCV_SBI_EXT_DBCN, KVM_RISCV_SBI_EXT_STA, + KVM_RISCV_SBI_EXT_PVLOCK, KVM_RISCV_SBI_EXT_MAX, }; =20 diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile index 0fb1840c3e0a..4e1a82a7eeab 100644 --- a/arch/riscv/kvm/Makefile +++ b/arch/riscv/kvm/Makefile @@ -32,6 +32,7 @@ kvm-y +=3D vcpu_sbi_replace.o kvm-y +=3D vcpu_sbi_sta.o kvm-$(CONFIG_RISCV_SBI_V01) +=3D vcpu_sbi_v01.o kvm-y +=3D vcpu_switch.o +kvm-y +=3D vcpu_sbi_pvlock.o kvm-y +=3D vcpu_timer.o kvm-y +=3D vcpu_vector.o kvm-y +=3D vm.o diff --git a/arch/riscv/kvm/vcpu_sbi.c b/arch/riscv/kvm/vcpu_sbi.c index 6e704ed86a83..c770bef51a2f 100644 --- a/arch/riscv/kvm/vcpu_sbi.c +++ b/arch/riscv/kvm/vcpu_sbi.c @@ -82,6 +82,10 @@ static const struct kvm_riscv_sbi_extension_entry sbi_ex= t[] =3D { .ext_idx =3D KVM_RISCV_SBI_EXT_VENDOR, .ext_ptr =3D &vcpu_sbi_ext_vendor, }, + { + .ext_idx =3D KVM_RISCV_SBI_EXT_PVLOCK, + .ext_ptr =3D &vcpu_sbi_ext_pvlock, + }, }; =20 static const struct kvm_riscv_sbi_extension_entry * diff --git a/arch/riscv/kvm/vcpu_sbi_pvlock.c b/arch/riscv/kvm/vcpu_sbi_pvl= ock.c new file mode 100644 index 000000000000..55d889ddc2cd --- /dev/null +++ b/arch/riscv/kvm/vcpu_sbi_pvlock.c @@ -0,0 +1,57 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c), 2024 Alibaba Cloud + * + * Authors: + * Guo Ren + */ + +#include +#include +#include +#include +#include + +static int kvm_sbi_ext_pvlock_kick_cpu(struct kvm_vcpu *vcpu) +{ + struct kvm_cpu_context *cp =3D &vcpu->arch.guest_context; + struct kvm *kvm =3D vcpu->kvm; + struct kvm_vcpu *target; + + target =3D kvm_get_vcpu_by_id(kvm, cp->a0); + if (!target) + return SBI_ERR_INVALID_PARAM; + + kvm_vcpu_kick(target); + + if (READ_ONCE(target->ready)) + kvm_vcpu_yield_to(target); + + return SBI_SUCCESS; +} + +static int kvm_sbi_ext_pvlock_handler(struct kvm_vcpu *vcpu, struct kvm_ru= n *run, + struct kvm_vcpu_sbi_return *retdata) +{ + int ret =3D 0; + struct kvm_cpu_context *cp =3D &vcpu->arch.guest_context; + unsigned long funcid =3D cp->a6; + + switch (funcid) { + case SBI_EXT_PVLOCK_KICK_CPU: + ret =3D kvm_sbi_ext_pvlock_kick_cpu(vcpu); + break; + default: + ret =3D SBI_ERR_NOT_SUPPORTED; + } + + retdata->err_val =3D ret; + + return 0; +} + +const struct kvm_vcpu_sbi_extension vcpu_sbi_ext_pvlock =3D { + .extid_start =3D SBI_EXT_PVLOCK, + .extid_end =3D SBI_EXT_PVLOCK, + .handler =3D kvm_sbi_ext_pvlock_handler, +}; --=20 2.40.1 From nobody Tue Feb 10 15:43:36 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1358035940 for ; Fri, 27 Dec 2024 01:10:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735261838; cv=none; b=h1uTMfn8z+dP/eAcp0S9w3N01jfqDxexleTTGx5VjYqn0MWBoFitDe8nyQV/07ErpvXALwTmhgFU+CFDJFhd9mjFv1SB5ItB5W4BynwvSnkofwW0WldGr4ytcmrAllrh4lSDn5ckKIBBsxLygWgmi35/Tgol04EB8lZxaZxVOh4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735261838; c=relaxed/simple; bh=qVctlfRDKSqvGJJe7RLiJvr7L2bxxrBrfqXVwQ0w6s4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=usyX4rjRbMHgseIEqhMh6+EqxvzjqY7Aq/BLEXOHhGO6GAKVZTWzht5OEWc+fydxNWirfna+uh54CQ41oGsOZXr5u7OUaNIiZi9qdVr4u4J6/aS9vKJZnF3sVQp75rDyDOXWV2CjhNV9I/wbUAoer8vGiqQWIE1FbiRvzq3Mb+c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VkMSXlxC; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VkMSXlxC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 98CBDC4CED1; Fri, 27 Dec 2024 01:10:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735261837; bh=qVctlfRDKSqvGJJe7RLiJvr7L2bxxrBrfqXVwQ0w6s4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VkMSXlxCaYTGsw/MT3j8ixWXlK+goloLcPoChbVn4Hnafknb3V6UdsQiYgCaBNHwi jEIcIA2OgExteYSL9mJYz2q7bTLU3hJ3diS6gWH5eDyMKOd0XJ/frbG0PpEBWdhDdn qoiXxUmSj17RwzF46gto2Fl4Q+WqrioLZMN/M3ubnZ163BCtJRezhqS0VZswzjFsVG HrJIsZYThpgL7PALSVDEPqH1FDmbXetNGEp5pNbfX8osHYvRo6ui+HX4euACDy0PK/ ukQA6vs6ScNs4mSsE0MU3cEvr8sa+YG3i238iwiwmPOV4V1YSbFPOvaC+u3Z99FCcm WHJT/if1sFp3Q== From: guoren@kernel.org To: paul.walmsley@sifive.com, palmer@dabbelt.com, guoren@kernel.org, bjorn@rivosinc.com, conor@kernel.org, leobras@redhat.com, peterz@infradead.org, parri.andrea@gmail.com, will@kernel.org, longman@redhat.com, boqun.feng@gmail.com, arnd@arndb.de, alexghiti@rivosinc.com, ajones@ventanamicro.com, rkrcmar@ventanamicro.com, atishp@rivosinc.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Guo Ren Subject: [RFC PATCH V2 2/4] RISC-V: paravirt: Add pvqspinlock frontend Date: Thu, 26 Dec 2024 20:10:09 -0500 Message-Id: <20241227011011.2331381-3-guoren@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20241227011011.2331381-1-guoren@kernel.org> References: <20241227011011.2331381-1-guoren@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Guo Ren Add an unfair qspinlock virtualization-friendly frontend, by halting the virtual CPU rather than spinning. Using static_call to switch between: native_queued_spin_lock_slowpath() __pv_queued_spin_lock_slowpath() native_queued_spin_unlock() __pv_queued_spin_unlock() Add the pv_wait & pv_kick implementations. Reviewed-by: Leonardo Bras Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- arch/riscv/Kconfig | 12 ++++ arch/riscv/include/asm/Kbuild | 1 - arch/riscv/include/asm/qspinlock.h | 35 +++++++++++ arch/riscv/include/asm/qspinlock_paravirt.h | 28 +++++++++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/qspinlock_paravirt.c | 67 +++++++++++++++++++++ arch/riscv/kernel/setup.c | 4 ++ 7 files changed, 147 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/include/asm/qspinlock.h create mode 100644 arch/riscv/include/asm/qspinlock_paravirt.h create mode 100644 arch/riscv/kernel/qspinlock_paravirt.c diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index d4a7ca0388c0..e241ac39ecd6 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -1071,6 +1071,18 @@ config PARAVIRT_TIME_ACCOUNTING =20 If in doubt, say N here. =20 +config PARAVIRT_SPINLOCKS + bool "Paravirtualization layer for spinlocks" + depends on QUEUED_SPINLOCKS + default y + help + Paravirtualized spinlocks allow a unfair qspinlock to replace the + test-set kvm-guest virt spinlock implementation with something + virtualization-friendly, for example, halt the virtual CPU rather + than spinning. + + If you are unsure how to answer this question, answer Y. + config RELOCATABLE bool "Build a relocatable kernel" depends on MMU && 64BIT && !XIP_KERNEL diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index de13d5a234f8..c726330d2b9f 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -12,6 +12,5 @@ generic-y +=3D spinlock_types.h generic-y +=3D ticket_spinlock.h generic-y +=3D qrwlock.h generic-y +=3D qrwlock_types.h -generic-y +=3D qspinlock.h generic-y +=3D user.h generic-y +=3D vmlinux.lds.h diff --git a/arch/riscv/include/asm/qspinlock.h b/arch/riscv/include/asm/qs= pinlock.h new file mode 100644 index 000000000000..1d9f32334ff1 --- /dev/null +++ b/arch/riscv/include/asm/qspinlock.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c), 2024 Alibaba + * Authors: + * Guo Ren + */ + +#ifndef _ASM_RISCV_QSPINLOCK_H +#define _ASM_RISCV_QSPINLOCK_H + +#ifdef CONFIG_PARAVIRT_SPINLOCKS +#include + +/* How long a lock should spin before we consider blocking */ +#define SPIN_THRESHOLD (1 << 15) + +void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val); +void __pv_init_lock_hash(void); +void __pv_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val); + +static inline void queued_spin_lock_slowpath(struct qspinlock *lock, u32 v= al) +{ + static_call(pv_queued_spin_lock_slowpath)(lock, val); +} + +#define queued_spin_unlock queued_spin_unlock +static inline void queued_spin_unlock(struct qspinlock *lock) +{ + static_call(pv_queued_spin_unlock)(lock); +} +#endif /* CONFIG_PARAVIRT_SPINLOCKS */ + +#include + +#endif /* _ASM_RISCV_QSPINLOCK_H */ diff --git a/arch/riscv/include/asm/qspinlock_paravirt.h b/arch/riscv/inclu= de/asm/qspinlock_paravirt.h new file mode 100644 index 000000000000..a365203dd782 --- /dev/null +++ b/arch/riscv/include/asm/qspinlock_paravirt.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c), 2024 Alibaba Cloud + * Authors: + * Guo Ren + */ + +#ifndef _ASM_RISCV_QSPINLOCK_PARAVIRT_H +#define _ASM_RISCV_QSPINLOCK_PARAVIRT_H + +void pv_wait(u8 *ptr, u8 val); +void pv_kick(int cpu); + +void dummy_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val); +void dummy_queued_spin_unlock(struct qspinlock *lock); + +DECLARE_STATIC_CALL(pv_queued_spin_lock_slowpath, dummy_queued_spin_lock_s= lowpath); +DECLARE_STATIC_CALL(pv_queued_spin_unlock, dummy_queued_spin_unlock); + +void __init pv_qspinlock_init(void); + +void __pv_queued_spin_unlock_slowpath(struct qspinlock *lock, u8 locked); + +bool pv_is_native_spin_unlock(void); + +void __pv_queued_spin_unlock(struct qspinlock *lock); + +#endif /* _ASM_RISCV_QSPINLOCK_PARAVIRT_H */ diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 063d1faf5a53..79f823e0e57d 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -123,3 +123,4 @@ obj-$(CONFIG_COMPAT) +=3D compat_vdso/ obj-$(CONFIG_64BIT) +=3D pi/ obj-$(CONFIG_ACPI) +=3D acpi.o obj-$(CONFIG_ACPI_NUMA) +=3D acpi_numa.o +obj-$(CONFIG_PARAVIRT_SPINLOCKS) +=3D qspinlock_paravirt.o diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qsp= inlock_paravirt.c new file mode 100644 index 000000000000..4ec4765f57f3 --- /dev/null +++ b/arch/riscv/kernel/qspinlock_paravirt.c @@ -0,0 +1,67 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c), 2024 Alibaba Cloud + * Authors: + * Guo Ren + */ + +#include +#include +#include + +void pv_kick(int cpu) +{ + sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU, + cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0); + return; +} + +void pv_wait(u8 *ptr, u8 val) +{ + unsigned long flags; + + if (in_nmi()) + return; + + local_irq_save(flags); + if (READ_ONCE(*ptr) !=3D val) + goto out; + + wait_for_interrupt(); +out: + local_irq_restore(flags); +} + +static void native_queued_spin_unlock(struct qspinlock *lock) +{ + smp_store_release(&lock->locked, 0); +} + +DEFINE_STATIC_CALL(pv_queued_spin_lock_slowpath, native_queued_spin_lock_s= lowpath); +EXPORT_STATIC_CALL(pv_queued_spin_lock_slowpath); + +DEFINE_STATIC_CALL(pv_queued_spin_unlock, native_queued_spin_unlock); +EXPORT_STATIC_CALL(pv_queued_spin_unlock); + +void __init pv_qspinlock_init(void) +{ + if (num_possible_cpus() =3D=3D 1) + return; + + if (!sbi_probe_extension(SBI_EXT_PVLOCK)) + return; + + pr_info("PV qspinlocks enabled\n"); + __pv_init_lock_hash(); + + static_call_update(pv_queued_spin_lock_slowpath, __pv_queued_spin_lock_sl= owpath); + static_call_update(pv_queued_spin_unlock, __pv_queued_spin_unlock); +} + +bool pv_is_native_spin_unlock(void) +{ + if (static_call_query(pv_queued_spin_unlock) =3D=3D native_queued_spin_un= lock) + return true; + else + return false; +} diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c index 45010e71df86..8b51ff5c7300 100644 --- a/arch/riscv/kernel/setup.c +++ b/arch/riscv/kernel/setup.c @@ -278,6 +278,10 @@ static void __init riscv_spinlock_init(void) pr_err("Queued spinlock without Zabha or Ziccrse"); else pr_info("Queued spinlock %s: enabled\n", using_ext); + +#ifdef CONFIG_PARAVIRT_SPINLOCKS + pv_qspinlock_init(); +#endif } =20 extern void __init init_rt_signal_env(void); --=20 2.40.1 From nobody Tue Feb 10 15:43:36 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1058635940 for ; Fri, 27 Dec 2024 01:10:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735261842; cv=none; b=qunIgKJKKEdzurSSos664qIoTTWNbSQV6GC2OWeGi7O6ey5+h6fQ6U+Gi4QNc9elK9aa8tFob1cNpDFVzZ8HhxVAm75SqnRD21b7eupBFLMrmEA8s0HEb21Oap4T/x5HFnN/8BaX5afTxMdrQGgkEwLdMCFW42/KOXd95rSSJ6Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735261842; c=relaxed/simple; bh=fuERFLEm+4/d8Ny2ilkznllQWW+KHxeqo3MyuSlJ5ks=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Bf9clwWWnTJPdwV0H4kGACwXELlVZIStBVvOhMqa44ResczJ25kpnKCLRZHQoCsNLsAaJj6Z5nTNLHO2ghw33bkVM8Ofe2FSz8UkX0ar0uXrOZgzGYKvLOCjJIBNFTK8IMkTZACx4bon6B9Bo8qaQVKu2FsBNxzxSxzd3OZJT4M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WvhNjbqE; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WvhNjbqE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4C341C4CED1; Fri, 27 Dec 2024 01:10:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735261841; bh=fuERFLEm+4/d8Ny2ilkznllQWW+KHxeqo3MyuSlJ5ks=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WvhNjbqEy9asICLql8bu7lqIB9zK2p+NiuQDAK+96oXH5t0q3gjQ0VgYHjndxG93D Hm1IHY7pr9aAfyfrpE9JZDfx9js99q1hJ2Dk67m+Ybgd/bwIv+UfrwbBl1iG0VUpg3 vC9GltSvBlJ9krVNY6khmLXCU9MKO6y+qmhMvb7lJwajuwGby4NRPj96f3UtqgHBsa kEtGMAlca/N4LFw3QyNzFZwxz2jJ1JvjxCmW+W2XTVil5fIMKy4mPvYbKDBPj+RjB8 SFM/RwgPY980fe7r54C2es28e+7iZNWBEaMm4T7yAxwPRTaPimxp/G67qiNyMxq2+l QiDFtSXu5mKlw== From: guoren@kernel.org To: paul.walmsley@sifive.com, palmer@dabbelt.com, guoren@kernel.org, bjorn@rivosinc.com, conor@kernel.org, leobras@redhat.com, peterz@infradead.org, parri.andrea@gmail.com, will@kernel.org, longman@redhat.com, boqun.feng@gmail.com, arnd@arndb.de, alexghiti@rivosinc.com, ajones@ventanamicro.com, rkrcmar@ventanamicro.com, atishp@rivosinc.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Guo Ren Subject: [RFC PATCH V2 3/4] RISC-V: paravirt: pvqspinlock: Add trace point for pv_kick/wait Date: Thu, 26 Dec 2024 20:10:10 -0500 Message-Id: <20241227011011.2331381-4-guoren@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20241227011011.2331381-1-guoren@kernel.org> References: <20241227011011.2331381-1-guoren@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Guo Ren Add trace point for pv_kick&wait, here is the output: ls /sys/kernel/debug/tracing/events/paravirt/ enable filter pv_kick pv_wait cat /sys/kernel/debug/tracing/trace entries-in-buffer/entries-written: 33927/33927 #P:12 _-----=3D> irqs-off/BH-disabled / _----=3D> need-resched | / _---=3D> hardirq/softirq || / _--=3D> preempt-depth ||| / _-=3D> migrate-disable |||| / delay TASK-PID CPU# ||||| TIMESTAMP FUNCTION | | | ||||| | | sh-100 [001] d..2. 28.312294: pv_wait: cpu 1 out of wfi -0 [000] d.h4. 28.322030: pv_kick: cpu 0 kick targe= t cpu 1 sh-100 [001] d..2. 30.982631: pv_wait: cpu 1 out of wfi -0 [000] d.h4. 30.993289: pv_kick: cpu 0 kick targe= t cpu 1 sh-100 [002] d..2. 44.987573: pv_wait: cpu 2 out of wfi -0 [000] d.h4. 44.989000: pv_kick: cpu 0 kick targe= t cpu 2 -0 [003] d.s3. 51.593978: pv_kick: cpu 3 kick targe= t cpu 4 rcu_sched-15 [004] d..2. 51.595192: pv_wait: cpu 4 out of wfi lock_torture_wr-115 [004] ...2. 52.656482: pv_kick: cpu 4 kick targe= t cpu 2 lock_torture_wr-113 [002] d..2. 52.659146: pv_wait: cpu 2 out of wfi lock_torture_wr-114 [008] d..2. 52.659507: pv_wait: cpu 8 out of wfi lock_torture_wr-114 [008] d..2. 52.663503: pv_wait: cpu 8 out of wfi lock_torture_wr-113 [002] ...2. 52.666128: pv_kick: cpu 2 kick targe= t cpu 8 lock_torture_wr-114 [008] d..2. 52.667261: pv_wait: cpu 8 out of wfi lock_torture_wr-114 [009] .n.2. 53.141515: pv_kick: cpu 9 kick targe= t cpu 11 lock_torture_wr-113 [002] d..2. 53.143339: pv_wait: cpu 2 out of wfi lock_torture_wr-116 [007] d..2. 53.143412: pv_wait: cpu 7 out of wfi lock_torture_wr-118 [000] d..2. 53.143457: pv_wait: cpu 0 out of wfi lock_torture_wr-115 [008] d..2. 53.143481: pv_wait: cpu 8 out of wfi lock_torture_wr-117 [011] d..2. 53.143522: pv_wait: cpu 11 out of wfi lock_torture_wr-117 [011] ...2. 53.143987: pv_kick: cpu 11 kick targ= et cpu 8 lock_torture_wr-115 [008] ...2. 53.144269: pv_kick: cpu 8 kick targe= t cpu 7 Reviewed-by: Leonardo Bras Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- arch/riscv/kernel/qspinlock_paravirt.c | 7 +++ .../kernel/trace_events_filter_paravirt.h | 60 +++++++++++++++++++ 2 files changed, 67 insertions(+) create mode 100644 arch/riscv/kernel/trace_events_filter_paravirt.h diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qsp= inlock_paravirt.c index 4ec4765f57f3..781ed2190334 100644 --- a/arch/riscv/kernel/qspinlock_paravirt.c +++ b/arch/riscv/kernel/qspinlock_paravirt.c @@ -9,8 +9,13 @@ #include #include =20 +#define CREATE_TRACE_POINTS +#include "trace_events_filter_paravirt.h" + void pv_kick(int cpu) { + trace_pv_kick(smp_processor_id(), cpu); + sbi_ecall(SBI_EXT_PVLOCK, SBI_EXT_PVLOCK_KICK_CPU, cpuid_to_hartid_map(cpu), 0, 0, 0, 0, 0); return; @@ -28,6 +33,8 @@ void pv_wait(u8 *ptr, u8 val) goto out; =20 wait_for_interrupt(); + + trace_pv_wait(smp_processor_id()); out: local_irq_restore(flags); } diff --git a/arch/riscv/kernel/trace_events_filter_paravirt.h b/arch/riscv/= kernel/trace_events_filter_paravirt.h new file mode 100644 index 000000000000..68570c7602eb --- /dev/null +++ b/arch/riscv/kernel/trace_events_filter_paravirt.h @@ -0,0 +1,60 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c), 2024 Alibaba Cloud + * Authors: + * Guo Ren + */ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM paravirt + +#if !defined(_TRACE_PARAVIRT_H) || defined(TRACE_HEADER_MULTI_READ) +#define _TRACE_PARAVIRT_H + +#include + +TRACE_EVENT(pv_kick, + TP_PROTO(int cpu, int target), + TP_ARGS(cpu, target), + + TP_STRUCT__entry( + __field(int, cpu) + __field(int, target) + ), + + TP_fast_assign( + __entry->cpu =3D cpu; + __entry->target =3D target; + ), + + TP_printk("cpu %d pv_kick target cpu %d", + __entry->cpu, + __entry->target + ) +); + +TRACE_EVENT(pv_wait, + TP_PROTO(int cpu), + TP_ARGS(cpu), + + TP_STRUCT__entry( + __field(int, cpu) + ), + + TP_fast_assign( + __entry->cpu =3D cpu; + ), + + TP_printk("cpu %d out of wfi", + __entry->cpu + ) +); + +#endif /* _TRACE_PARAVIRT_H || TRACE_HEADER_MULTI_READ */ + +#undef TRACE_INCLUDE_PATH +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_PATH ../../../arch/riscv/kernel/ +#define TRACE_INCLUDE_FILE trace_events_filter_paravirt + +/* This part must be outside protection */ +#include --=20 2.40.1 From nobody Tue Feb 10 15:43:36 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB63D1EF01 for ; Fri, 27 Dec 2024 01:10:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735261845; cv=none; b=WDbooFZQ8I1oM0HFDehUeKJOa/pvIcSDXi8rkF7dbHrrsrgR8kyCffF0qqYDs1H3wkIiPPZVc2WPzj056MMIU/jBRt1hiaOdM5dbNE6GsNbdf/1VzQiQyFp2BoHJRewN8wAjFYRu4sYbBZWjANo3ZXCAjghCsDRQACNPC3TqHJs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735261845; c=relaxed/simple; bh=XOA/Re8WRsJ7cF2WRPugYycBZj0hSQujANYi66LdgLQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=L8LEwDTlvKlESsoFcTd7rgV575U6AXPV4VAkb7iqvIvZBP4Vma2PSO9Ac+2nq95lJYS0lGErWYgH/IYs9IvP7ZAA6rrAvjmp1kbXFyFoiXDxa4JDyH8/lUuO5daocOaqXKHC7Ij6/7SAWb1fcHUNcozlp/jJ9lHU0xjolV+SUhw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NdTt6y5R; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NdTt6y5R" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F2151C4CED2; Fri, 27 Dec 2024 01:10:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735261845; bh=XOA/Re8WRsJ7cF2WRPugYycBZj0hSQujANYi66LdgLQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NdTt6y5RCVO++5gWi8KHzzlOpAIlcEaEj0Jzgu2ON69gToMIcfmesBHErEP5cuae1 MEvqF5H+NUiL09MTiF8sgYXzJ+Jnvn+TewnduBTwNGYO48P6REw9nf75jfojJIbOZQ x2t6u2aIbSyG2gDBxZwo7IIK1wv1xntnC+H9rCrcO2phVFeEpcueyThUHlikTQ/RXL eoI6uwqwvShLc3gbM/VrG4YDsrBMf+7pUEt8FB+Aw8f9LXqpeR4SadHqfNDPJsutrr bx+Z6Ewmiak4dvKcWGdFjS/W7xQYPDfpNgPiGpwYmX7uwyCufrYnQodYRSc83ZwDYe HgHURpmhFRS/g== From: guoren@kernel.org To: paul.walmsley@sifive.com, palmer@dabbelt.com, guoren@kernel.org, bjorn@rivosinc.com, conor@kernel.org, leobras@redhat.com, peterz@infradead.org, parri.andrea@gmail.com, will@kernel.org, longman@redhat.com, boqun.feng@gmail.com, arnd@arndb.de, alexghiti@rivosinc.com, ajones@ventanamicro.com, rkrcmar@ventanamicro.com, atishp@rivosinc.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Guo Ren Subject: [RFC PATCH V2 4/4] RISC-V: paravirt: Enable virt_spin_lock() when disable PARAVIRT_SPINLOCKS Date: Thu, 26 Dec 2024 20:10:11 -0500 Message-Id: <20241227011011.2331381-5-guoren@kernel.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20241227011011.2331381-1-guoren@kernel.org> References: <20241227011011.2331381-1-guoren@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Guo Ren The VM guests should fall back to a Test-and-Set spinlock when PARAVIRT_SPINLOCKS disabled, because fair locks have horrible lock 'holder' preemption issues. The virt_spin_lock_key would shortcut for the queued_spin_lock_- slowpath() function that allow virt_spin_lock to hijack it. ref: 43b3f02899f7 ("locking/qspinlock/x86: Fix performance regression under unaccelerated VMs"). Add a static key controlling whether virt_spin_lock() should be called or not. Add nopvspin support as x86. Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- .../admin-guide/kernel-parameters.txt | 2 +- arch/riscv/include/asm/qspinlock.h | 24 +++++++++++++++ arch/riscv/include/asm/sbi.h | 16 ++++++++++ arch/riscv/kernel/qspinlock_paravirt.c | 3 ++ arch/riscv/kernel/sbi.c | 2 +- arch/riscv/kernel/setup.c | 29 +++++++++++++++++++ 6 files changed, 74 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentatio= n/admin-guide/kernel-parameters.txt index 3872bc6ec49d..541202ff0f36 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -4048,7 +4048,7 @@ as generic guest with no PV drivers. Currently support XEN HVM, KVM, HYPER_V and VMWARE guest. =20 - nopvspin [X86,XEN,KVM,EARLY] + nopvspin [X86,RISCV,XEN,KVM,EARLY] Disables the qspinlock slow path using PV optimizations which allow the hypervisor to 'idle' the guest on lock contention. diff --git a/arch/riscv/include/asm/qspinlock.h b/arch/riscv/include/asm/qs= pinlock.h index 1d9f32334ff1..4a62dcb43617 100644 --- a/arch/riscv/include/asm/qspinlock.h +++ b/arch/riscv/include/asm/qspinlock.h @@ -14,6 +14,8 @@ /* How long a lock should spin before we consider blocking */ #define SPIN_THRESHOLD (1 << 15) =20 +extern bool nopvspin; + void native_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val); void __pv_init_lock_hash(void); void __pv_queued_spin_lock_slowpath(struct qspinlock *lock, u32 val); @@ -31,5 +33,27 @@ static inline void queued_spin_unlock(struct qspinlock *= lock) #endif /* CONFIG_PARAVIRT_SPINLOCKS */ =20 #include +#include + +/* + * The KVM guests fall back to a Test-and-Set spinlock, because fair locks + * have horrible lock 'holder' preemption issues. The test_and_set_spinloc= k_key + * would shortcut for the queued_spin_lock_slowpath() function that allow + * virt_spin_lock to hijack it. + */ +DECLARE_STATIC_KEY_FALSE(test_and_set_spinlock_key); + +#define virt_spin_lock test_and_set_spinlock +static inline bool test_and_set_spinlock(struct qspinlock *lock) +{ + if (!static_branch_likely(&test_and_set_spinlock_key)) + return false; + + do { + smp_cond_load_relaxed((s32 *)&lock->val, VAL =3D=3D 0); + } while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) !=3D 0); + + return true; +} =20 #endif /* _ASM_RISCV_QSPINLOCK_H */ diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h index 03e719f076ad..13669c29ead3 100644 --- a/arch/riscv/include/asm/sbi.h +++ b/arch/riscv/include/asm/sbi.h @@ -56,6 +56,21 @@ enum sbi_ext_base_fid { SBI_EXT_BASE_GET_MIMPID, }; =20 +enum sbi_ext_base_impl_id { + SBI_EXT_BASE_IMPL_ID_BBL =3D 0, + SBI_EXT_BASE_IMPL_ID_OPENSBI, + SBI_EXT_BASE_IMPL_ID_XVISOR, + SBI_EXT_BASE_IMPL_ID_KVM, + SBI_EXT_BASE_IMPL_ID_RUSTSBI, + SBI_EXT_BASE_IMPL_ID_DIOSIX, + SBI_EXT_BASE_IMPL_ID_COFFER, + SBI_EXT_BASE_IMPL_ID_XEN, + SBI_EXT_BASE_IMPL_ID_POLARFIRE, + SBI_EXT_BASE_IMPL_ID_COREBOOT, + SBI_EXT_BASE_IMPL_ID_OREBOOT, + SBI_EXT_BASE_IMPL_ID_BHYVE, +}; + enum sbi_ext_time_fid { SBI_EXT_TIME_SET_TIMER =3D 0, }; @@ -449,6 +464,7 @@ static inline int sbi_console_getchar(void) { return -E= NOENT; } long sbi_get_mvendorid(void); long sbi_get_marchid(void); long sbi_get_mimpid(void); +long sbi_get_firmware_id(void); void sbi_set_timer(uint64_t stime_value); void sbi_shutdown(void); void sbi_send_ipi(unsigned int cpu); diff --git a/arch/riscv/kernel/qspinlock_paravirt.c b/arch/riscv/kernel/qsp= inlock_paravirt.c index 781ed2190334..0f2e50754689 100644 --- a/arch/riscv/kernel/qspinlock_paravirt.c +++ b/arch/riscv/kernel/qspinlock_paravirt.c @@ -58,6 +58,9 @@ void __init pv_qspinlock_init(void) if (!sbi_probe_extension(SBI_EXT_PVLOCK)) return; =20 + if (nopvspin) + return; + pr_info("PV qspinlocks enabled\n"); __pv_init_lock_hash(); =20 diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c index 1989b8cade1b..2cbacc345e5d 100644 --- a/arch/riscv/kernel/sbi.c +++ b/arch/riscv/kernel/sbi.c @@ -488,7 +488,7 @@ static inline long sbi_get_spec_version(void) return __sbi_base_ecall(SBI_EXT_BASE_GET_SPEC_VERSION); } =20 -static inline long sbi_get_firmware_id(void) +long sbi_get_firmware_id(void) { return __sbi_base_ecall(SBI_EXT_BASE_GET_IMP_ID); } diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c index 8b51ff5c7300..bb76133e684f 100644 --- a/arch/riscv/kernel/setup.c +++ b/arch/riscv/kernel/setup.c @@ -249,6 +249,32 @@ DEFINE_STATIC_KEY_TRUE(qspinlock_key); EXPORT_SYMBOL(qspinlock_key); #endif =20 +#ifdef CONFIG_QUEUED_SPINLOCKS +DEFINE_STATIC_KEY_FALSE(test_and_set_spinlock_key); + +static bool __init virt_spin_lock_init(void) +{ + if (sbi_get_firmware_id() =3D=3D SBI_EXT_BASE_IMPL_ID_KVM && + !IS_ENABLED(CONFIG_PARAVIRT_SPINLOCKS)) + goto out; + +#ifdef CONFIG_PARAVIRT_SPINLOCKS + if (sbi_probe_extension(SBI_EXT_PVLOCK) && nopvspin) + goto out; +#endif + + return false; +out: + static_branch_enable(&test_and_set_spinlock_key); + return true; +} +#else +static bool __init virt_spin_lock_init(void) +{ + return false; +} +#endif + static void __init riscv_spinlock_init(void) { char *using_ext =3D NULL; @@ -274,6 +300,9 @@ static void __init riscv_spinlock_init(void) } #endif =20 + if (virt_spin_lock_init()) + using_ext =3D "using test and set"; + if (!using_ext) pr_err("Queued spinlock without Zabha or Ziccrse"); else --=20 2.40.1