From nobody Fri Oct 3 23:08:24 2025 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 096BA2EB5CD for ; Sat, 23 Aug 2025 16:39:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755967167; cv=none; b=aBchbdnXiV+5KYpPkODweAKV18zKZt1n2crrqX23538ofC0GkpUWPClMiJj9c4M1/zWfQJ/iVKYqyLm24mdFXib9X1IwUJSgL5RG2S+mJunUjDV2WLzNKnonWjMm185lsBa+mQeNOPPFICf51oLpiqqwsuonMTExoGYRjUJIPdA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755967167; c=relaxed/simple; bh=+xCDiLYk4IxXRI6LCA3i7U2Kcs1SMWnuGNUR4uoW0lY=; h=Message-ID:From:To:Cc:Subject:References:MIME-Version: Content-Type:Date; b=mkDlL3G0czO9RF7zVglYZk7vaXJtwIVHQM4jwip5zn5scVh48pkEL0R4K9L/fvMvjsONOtkr0FEvynB3n2oenc6Q3e3iaCQOMuJ6ne5zAnwg0jD1Fi4hj4wdHtj/ZHZxi+xDz3iITjDjaX5Z9fsRn1mesm3ai3ZGti5qq8bBkWA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=IgOHRXST; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=DRolVXy0; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="IgOHRXST"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="DRolVXy0" Message-ID: <20250823161653.711118277@linutronix.de> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1755967163; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=yWoqwOu/nLPKroY/DD7nEq70Vh5jZEteJAU8HtQMJiM=; b=IgOHRXSTmELTQA2P4tojm0ga59VGVB/Eq8XMbGeKVZJgRHnsGlCkWB8b2dJDzCL11RvsGz EL802XTLpoOARU4Tx1fvyTNssdwAELFaMuAPenscpJjNrX122apw4Sl5HuBibBkueq1CrO CPlY4ynw8yKhp6Br9gplU/B0B4PytIIw7GV5/zg9W8CbO590SeVU4HKahdb5ayc+NOjFp0 No1/EiWUuWZNy5GThN8a7CaGD6Q/LrV8XrYBBE6YE83wCjCd7CJqUuNT5rGd6XSG04x5FY pznWcLVfoHjEUOgaNkSKDbSRSYal4aS0andIv5WkfFSHBQ613QU8hjA+atEHrQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1755967163; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=yWoqwOu/nLPKroY/DD7nEq70Vh5jZEteJAU8HtQMJiM=; b=DRolVXy0MQMPcqDRxeQ+mDyCvcxB5ftnnFajcvAOWOF3035ZJlPhdVskz8tisIYtssmRsf 2c5SZyhfksWmATBg== From: Thomas Gleixner To: LKML Cc: Jens Axboe , Paolo Bonzini , Sean Christopherson , Wei Liu , Dexuan Cui , Mathieu Desnoyers , Peter Zijlstra , "Paul E. McKenney" , Boqun Feng , x86@kernel.org, Arnd Bergmann , Heiko Carstens , Christian Borntraeger , Sven Schnelle , Huacai Chen , Paul Walmsley , Palmer Dabbelt Subject: [patch V2 07/37] rseq, virt: Retrigger RSEQ after vcpu_run() References: <20250823161326.635281786@linutronix.de> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Sat, 23 Aug 2025 18:39:22 +0200 (CEST) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Hypervisors invoke resume_user_mode_work() before entering the guest, which clears TIF_NOTIFY_RESUME. The @regs argument is NULL as there is no user space context available to them, so the rseq notify handler skips inspecting the critical section, but updates the CPU/MM CID values unconditionally so that the eventual pending rseq event is not lost on the way to user space. This is a pointless exercise as the task might be rescheduled before actually returning to user space and it creates unnecessary work in the vcpu_run() loops. It's way more efficient to ignore that invocation based on @regs =3D=3D NULL and let the hypervisors re-raise TIF_NOTIFY_RESUME after returning from the vcpu_run() loop before returning from the ioctl(). This ensures that a pending RSEQ update is not lost and the IDs are updated before returning to user space. Once the RSEQ handling is decoupled from TIF_NOTIFY_RESUME, this turns into a NOOP. Signed-off-by: Thomas Gleixner Cc: Paolo Bonzini Cc: Sean Christopherson Cc: Wei Liu Cc: Dexuan Cui --- drivers/hv/mshv_root_main.c | 2 + include/linux/rseq.h | 17 +++++++++ kernel/rseq.c | 76 +++++++++++++++++++++++----------------= ----- virt/kvm/kvm_main.c | 3 + 4 files changed, 62 insertions(+), 36 deletions(-) --- a/drivers/hv/mshv_root_main.c +++ b/drivers/hv/mshv_root_main.c @@ -585,6 +585,8 @@ static long mshv_run_vp_with_root_schedu } } while (!vp->run.flags.intercept_suspend); =20 + rseq_virt_userspace_exit(); + return ret; } =20 --- a/include/linux/rseq.h +++ b/include/linux/rseq.h @@ -38,6 +38,22 @@ static __always_inline void rseq_exit_to } =20 /* + * KVM/HYPERV invoke resume_user_mode_work() before entering guest mode, + * which clears TIF_NOTIFY_RESUME. To avoid updating user space RSEQ in + * that case just to do it eventually again before returning to user space, + * the entry resume_user_mode_work() invocation is ignored as the register + * argument is NULL. + * + * After returning from guest mode, they have to invoke this function to + * re-raise TIF_NOTIFY_RESUME if necessary. + */ +static inline void rseq_virt_userspace_exit(void) +{ + if (current->rseq_event_pending) + set_tsk_thread_flag(current, TIF_NOTIFY_RESUME); +} + +/* * If parent process has a registered restartable sequences area, the * child inherits. Unregister rseq for a clone with CLONE_VM set. */ @@ -68,6 +84,7 @@ static inline void rseq_execve(struct ta static inline void rseq_handle_notify_resume(struct ksignal *ksig, struct = pt_regs *regs) { } static inline void rseq_signal_deliver(struct ksignal *ksig, struct pt_reg= s *regs) { } static inline void rseq_sched_switch_event(struct task_struct *t) { } +static inline void rseq_virt_userspace_exit(void) { } static inline void rseq_fork(struct task_struct *t, unsigned long clone_fl= ags) { } static inline void rseq_execve(struct task_struct *t) { } static inline void rseq_exit_to_user_mode(void) { } --- a/kernel/rseq.c +++ b/kernel/rseq.c @@ -422,50 +422,54 @@ void __rseq_handle_notify_resume(struct { struct task_struct *t =3D current; int ret, sig; + bool event; + + /* + * If invoked from hypervisors before entering the guest via + * resume_user_mode_work(), then @regs is a NULL pointer. + * + * resume_user_mode_work() clears TIF_NOTIFY_RESUME and re-raises + * it before returning from the ioctl() to user space when + * rseq_event.sched_switch is set. + * + * So it's safe to ignore here instead of pointlessly updating it + * in the vcpu_run() loop. + */ + if (!regs) + return; =20 if (unlikely(t->flags & PF_EXITING)) return; =20 /* - * If invoked from hypervisors or IO-URING, then @regs is a NULL - * pointer, so fixup cannot be done. If the syscall which led to - * this invocation was invoked inside a critical section, then it - * will either end up in this code again or a possible violation of - * a syscall inside a critical region can only be detected by the - * debug code in rseq_syscall() in a debug enabled kernel. + * Read and clear the event pending bit first. If the task + * was not preempted or migrated or a signal is on the way, + * there is no point in doing any of the heavy lifting here + * on production kernels. In that case TIF_NOTIFY_RESUME + * was raised by some other functionality. + * + * This is correct because the read/clear operation is + * guarded against scheduler preemption, which makes it CPU + * local atomic. If the task is preempted right after + * re-enabling preemption then TIF_NOTIFY_RESUME is set + * again and this function is invoked another time _before_ + * the task is able to return to user mode. + * + * On a debug kernel, invoke the fixup code unconditionally + * with the result handed in to allow the detection of + * inconsistencies. */ - if (regs) { - /* - * Read and clear the event pending bit first. If the task - * was not preempted or migrated or a signal is on the way, - * there is no point in doing any of the heavy lifting here - * on production kernels. In that case TIF_NOTIFY_RESUME - * was raised by some other functionality. - * - * This is correct because the read/clear operation is - * guarded against scheduler preemption, which makes it CPU - * local atomic. If the task is preempted right after - * re-enabling preemption then TIF_NOTIFY_RESUME is set - * again and this function is invoked another time _before_ - * the task is able to return to user mode. - * - * On a debug kernel, invoke the fixup code unconditionally - * with the result handed in to allow the detection of - * inconsistencies. - */ - bool event; - - scoped_guard(RSEQ_EVENT_GUARD) { - event =3D t->rseq_event_pending; - t->rseq_event_pending =3D false; - } + scoped_guard(RSEQ_EVENT_GUARD) { + event =3D t->rseq_event_pending; + t->rseq_event_pending =3D false; + } =20 - if (IS_ENABLED(CONFIG_DEBUG_RSEQ) || event) { - ret =3D rseq_ip_fixup(regs, event); - if (unlikely(ret < 0)) - goto error; - } + if (IS_ENABLED(CONFIG_DEBUG_RSEQ) || event) { + ret =3D rseq_ip_fixup(regs, event); + if (unlikely(ret < 0)) + goto error; } + if (unlikely(rseq_update_cpu_node_id(t))) goto error; return; --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -49,6 +49,7 @@ #include #include #include +#include =20 #include #include @@ -4466,6 +4467,8 @@ static long kvm_vcpu_ioctl(struct file * r =3D kvm_arch_vcpu_ioctl_run(vcpu); vcpu->wants_to_run =3D false; =20 + rseq_virt_userspace_exit(); + trace_kvm_userspace_exit(vcpu->run->exit_reason, r); break; }