From nobody Thu Oct 30 23:14:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1751467305; cv=none; d=zohomail.com; s=zohoarc; b=aSGWAZoJnH9EeOhNCaiNORiiD9ewIbNfc7+lD0qu2oFKZ8L6A7w9IZ41HZXdWtmQAqRG5OTAFcxB138OXNHDSnl9NUwCcTnPs2+1u4tD/gKOyVR0EE5mLVEsUpg+WliBhaklXmy0rx7/3na7zUe/kS51x3mV13o+V/ISfX1wahk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1751467305; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ugtucicsaAC7jFXi5T+e68lwTqW8PaUKQa4Xi50TDv8=; b=gZ10Ab2rzsoA/sDRGI2BhEU2nkr8xKfoZ/z0y0qwZxWniGANxnokCx6N4jkecC53EqcO/rY99z8lkCTtZLONKglZHnBl6y9r3hoEJg0+izGhdt70tuKyg7InpJBG312yGSKHxm3ZWd7aqtMRCfYEhJMLuePReAKhbo+EALDymYQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1751467305653278.8088485811088; Wed, 2 Jul 2025 07:41:45 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1031262.1405031 (Exim 4.92) (envelope-from ) id 1uWyeQ-0007qW-8y; Wed, 02 Jul 2025 14:41:30 +0000 Received: by outflank-mailman (output) from mailman id 1031262.1405031; Wed, 02 Jul 2025 14:41:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uWyeQ-0007qP-5x; Wed, 02 Jul 2025 14:41:30 +0000 Received: by outflank-mailman (input) for mailman id 1031262; Wed, 02 Jul 2025 14:41:29 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uWyeP-0007qI-Mg for xen-devel@lists.xenproject.org; Wed, 02 Jul 2025 14:41:29 +0000 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [2a00:1450:4864:20::32e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a2965312-5752-11f0-b894-0df219b8e170; Wed, 02 Jul 2025 16:41:27 +0200 (CEST) Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-4538a2fc7ffso47191685e9.0 for ; Wed, 02 Jul 2025 07:41:27 -0700 (PDT) Received: from localhost.localdomain (host-92-26-98-202.as13285.net. [92.26.98.202]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-453823ad20bsm230946905e9.20.2025.07.02.07.41.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jul 2025 07:41:26 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a2965312-5752-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1751467287; x=1752072087; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ugtucicsaAC7jFXi5T+e68lwTqW8PaUKQa4Xi50TDv8=; b=TCwRUc/UQZdQtRw2qViiFBLTFuWLofehWiBEbHoj4P1mAAI7K/3VKyxtTN03kH9r5I Pg8PTgQvhlhFZZi7vanmFoR4Btac5WuBkxnkNuRvoAGuz+XSQgfs1CzmKtLSj/Ji5o7I fQfM+whferMUZ1otO1yG4oAZqa8ISiBRd3gzw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751467287; x=1752072087; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ugtucicsaAC7jFXi5T+e68lwTqW8PaUKQa4Xi50TDv8=; b=ZLNQvtVY6rPKtuN6OBGbxE+daBVknaax3a1iXC98ldeM/E39CSEE3OTB6de1V3wJvU b/nz1uhpsorTxwtDQ4BWZOifxI2ypD+wYA9YCWeoQQwFDk6ryY7OO70ve1aZu364L8a8 QM/L8N6yeRb3BG2/PvVJ56U1A4zTTNxmJUGtZOdv4/M6Z4uF3xPg2bE0Z4CAJs2qSZzf uyEAehqpGmWiXliiXYRJG4WUFEotmysWYOFbrpvhiOXTWEu8p46M/heYz4O9EcohvW2T 9aJg9dQStinV8X4haDdYZ0tTcuQwVDMuetOsrsW7D3ZftnM3yWIuRHaCjcUb9QNE4RXT 48jA== X-Gm-Message-State: AOJu0YyWxHe4Y11FkUNnNeXNktEae7D4WTxeeQKl1hS1UVuEOXR1pPNQ dRoL6Yx+470n06qTljHCRF6UafLPYvVY9nqGUrvRG1INqga4LhI084iDxlensRBOlbKZZt48X7d pc1XXD/qMqQ== X-Gm-Gg: ASbGncsdcUqW4n7qF+ipza6aq4yV0nEOlieHmmBWMpxB1llzwHfZR53pXeTMiTyZ/Gx 5Bc3K9nd9Orq/mKVjVEmvbbBVRNo5PwqF+MenCChlZbdJ63WTYUscgFbnizDCixumfn7Q7jpv9r ixnRgCgJpoivHGtdZegzHn/VOq7nqKRLJSoyyGYwv6pVwPdjjmYPQyppzw0dfBqIZr7wLk77zoU swpuneIVmbhChFOQGyDKwbj4eHAkw2LTdv9+fC3Pzp2AKFQxflwctsLkCXiDWcvCbOmgyglYNYu 7Le6zSHkLR6pTLUF8MQJnKxwMDsmNqH0m7nKKqvDwhpp0/84U4KRJSZT8Jsv9p8ToPGIAT/snxf 5X9YNIoY6RjobrvDMZmPwlomQo28RGjgmECg= X-Google-Smtp-Source: AGHT+IHkFP1J6kc1yxOu2QxL245KplXrxYDiZd6TOm8wxpelxM2CIysd2q3IjSmWrcMpQPpw+sLl1g== X-Received: by 2002:a05:600c:c176:b0:442:f4a3:a2c0 with SMTP id 5b1f17b1804b1-454a41609efmr32157015e9.13.1751467286716; Wed, 02 Jul 2025 07:41:26 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH 1/6] x86/idle: Remove broken MWAIT implementation Date: Wed, 2 Jul 2025 15:41:16 +0100 Message-Id: <20250702144121.1096448-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250702144121.1096448-1-andrew.cooper3@citrix.com> References: <20250702144121.1096448-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1751467306589116600 cpuidle_wakeup_mwait() is a TOCTOU race. The cpumask_and() sampling cpuidle_mwait_flags can take a arbitrary period of time, and there's no guarantee that the target CPUs are still in MWAIT when writing into mwait_wakeup(cpu). The consequence of the race is that we'll fail to IPI targets. Also, there= 's no guarantee that mwait_idle_with_hints() will raise a TIMER_SOFTIRQ on it's way out. The fundamental bug is that the "in_mwait" variable needs to be in the monitored line in order to do this in a race-free way. Arranging for this while keeping the old implementation is prohibitive, so strip it out in order to implement the new one cleanly. In the interim, th= is causes IPIs to be used unconditionally which is safe albeit suboptimal. Fixes: 3d521e933e1b ("cpuidle: mwait on softirq_pending & remove wakeup ipi= s") Fixes: 1adb34ea846d ("CPUIDLE: re-implement mwait wakeup process") Signed-off-by: Andrew Cooper Reviewed-by: Roger Pau Monn=C3=A9 --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Anthony PERARD CC: Michal Orzel CC: Julien Grall CC: Stefano Stabellini --- xen/arch/x86/acpi/cpu_idle.c | 48 ++++-------------------------- xen/arch/x86/hpet.c | 2 -- xen/arch/x86/include/asm/hardirq.h | 9 +++--- xen/include/xen/cpuidle.h | 2 -- xen/include/xen/irq_cpustat.h | 1 - 5 files changed, 9 insertions(+), 53 deletions(-) diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c index 6c3a10e6fb4e..141f0f0facf6 100644 --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -436,27 +436,6 @@ static int __init cf_check cpu_idle_key_init(void) } __initcall(cpu_idle_key_init); =20 -/* - * The bit is set iff cpu use monitor/mwait to enter C state - * with this flag set, CPU can be waken up from C state - * by writing to specific memory address, instead of sending an IPI. - */ -static cpumask_t cpuidle_mwait_flags; - -void cpuidle_wakeup_mwait(cpumask_t *mask) -{ - cpumask_t target; - unsigned int cpu; - - cpumask_and(&target, mask, &cpuidle_mwait_flags); - - /* CPU is MWAITing on the cpuidle_mwait_wakeup flag. */ - for_each_cpu(cpu, &target) - mwait_wakeup(cpu) =3D 0; - - cpumask_andnot(mask, mask, &target); -} - /* Force sending of a wakeup IPI regardless of mwait usage. */ bool __ro_after_init force_mwait_ipi_wakeup; =20 @@ -465,42 +444,25 @@ bool arch_skip_send_event_check(unsigned int cpu) if ( force_mwait_ipi_wakeup ) return false; =20 - /* - * This relies on softirq_pending() and mwait_wakeup() to access data - * on the same cache line. - */ - smp_mb(); - return !!cpumask_test_cpu(cpu, &cpuidle_mwait_flags); + return false; } =20 void mwait_idle_with_hints(unsigned int eax, unsigned int ecx) { unsigned int cpu =3D smp_processor_id(); - s_time_t expires =3D per_cpu(timer_deadline, cpu); - const void *monitor_addr =3D &mwait_wakeup(cpu); + const unsigned int *this_softirq_pending =3D &softirq_pending(cpu); =20 - monitor(monitor_addr, 0, 0); + monitor(this_softirq_pending, 0, 0); smp_mb(); =20 - /* - * Timer deadline passing is the event on which we will be woken via - * cpuidle_mwait_wakeup. So check it now that the location is armed. - */ - if ( (expires > NOW() || expires =3D=3D 0) && !softirq_pending(cpu) ) + if ( !*this_softirq_pending ) { struct cpu_info *info =3D get_cpu_info(); =20 - cpumask_set_cpu(cpu, &cpuidle_mwait_flags); - spec_ctrl_enter_idle(info); mwait(eax, ecx); spec_ctrl_exit_idle(info); - - cpumask_clear_cpu(cpu, &cpuidle_mwait_flags); } - - if ( expires <=3D NOW() && expires > 0 ) - raise_softirq(TIMER_SOFTIRQ); } =20 static void acpi_processor_ffh_cstate_enter(struct acpi_processor_cx *cx) @@ -901,7 +863,7 @@ void cf_check acpi_dead_idle(void) =20 if ( cx->entry_method =3D=3D ACPI_CSTATE_EM_FFH ) { - void *mwait_ptr =3D &mwait_wakeup(smp_processor_id()); + void *mwait_ptr =3D &softirq_pending(smp_processor_id()); =20 /* * Cache must be flushed as the last operation before sleeping. diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c index 1bca8c8b670d..192de433cfd1 100644 --- a/xen/arch/x86/hpet.c +++ b/xen/arch/x86/hpet.c @@ -188,8 +188,6 @@ static void evt_do_broadcast(cpumask_t *mask) if ( __cpumask_test_and_clear_cpu(cpu, mask) ) raise_softirq(TIMER_SOFTIRQ); =20 - cpuidle_wakeup_mwait(mask); - if ( !cpumask_empty(mask) ) cpumask_raise_softirq(mask, TIMER_SOFTIRQ); } diff --git a/xen/arch/x86/include/asm/hardirq.h b/xen/arch/x86/include/asm/= hardirq.h index 342361cb6fdd..f3e93cc9b507 100644 --- a/xen/arch/x86/include/asm/hardirq.h +++ b/xen/arch/x86/include/asm/hardirq.h @@ -5,11 +5,10 @@ #include =20 typedef struct { - unsigned int __softirq_pending; - unsigned int __local_irq_count; - unsigned int nmi_count; - unsigned int mce_count; - bool __mwait_wakeup; + unsigned int __softirq_pending; + unsigned int __local_irq_count; + unsigned int nmi_count; + unsigned int mce_count; } __cacheline_aligned irq_cpustat_t; =20 #include /* Standard mappings for irq_cpustat_t above = */ diff --git a/xen/include/xen/cpuidle.h b/xen/include/xen/cpuidle.h index 705d0c1135f0..120e354fe340 100644 --- a/xen/include/xen/cpuidle.h +++ b/xen/include/xen/cpuidle.h @@ -92,8 +92,6 @@ extern struct cpuidle_governor *cpuidle_current_governor; bool cpuidle_using_deep_cstate(void); void cpuidle_disable_deep_cstate(void); =20 -extern void cpuidle_wakeup_mwait(cpumask_t *mask); - #define CPUIDLE_DRIVER_STATE_START 1 =20 extern void menu_get_trace_data(u32 *expected, u32 *pred); diff --git a/xen/include/xen/irq_cpustat.h b/xen/include/xen/irq_cpustat.h index b9629f25c266..5f039b4b9a76 100644 --- a/xen/include/xen/irq_cpustat.h +++ b/xen/include/xen/irq_cpustat.h @@ -24,6 +24,5 @@ extern irq_cpustat_t irq_stat[]; /* arch independent irq_stat fields */ #define softirq_pending(cpu) __IRQ_STAT((cpu), __softirq_pending) #define local_irq_count(cpu) __IRQ_STAT((cpu), __local_irq_count) -#define mwait_wakeup(cpu) __IRQ_STAT((cpu), __mwait_wakeup) =20 #endif /* __irq_cpustat_h */ --=20 2.39.5 From nobody Thu Oct 30 23:14:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1751467301; cv=none; d=zohomail.com; s=zohoarc; b=CYOm5upQkxYcN9buRzh7lvMiiCUvsL25luoFlfsxvkWa9TqYsyhcR5mOgP1PiSO8yj5VtjtS86D76toglVpya20RAJb63+5xPAXWvnZU2g7aF7uBT5rySHstFlCYVNNT1Stmjjii8BZFKyj83ww8w3w9EGXiveOEFGMuQ8FH1t4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1751467301; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Y3UdrPpnVz1bdBWjp4Vfz0Zz3kJ6jFSf8VMOzn7gPgo=; b=gp7JwQR5omJ4WAHPrY+7YHaEVZCstw8xUASv50NTB83tzptnZUUPslyhecHvohxxcnF37QHi+Sa3u2o7c9wRg8bUx3pUXbtJyYFh3eLKQzj/eBOs5RiY4uhQ2lWq1gNF++uUndiD6XWeaT1tC1/EDMUSpSwq8R2fokNaei/rx6w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1751467301628413.12647145472226; Wed, 2 Jul 2025 07:41:41 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1031263.1405041 (Exim 4.92) (envelope-from ) id 1uWyeR-00084J-Fn; Wed, 02 Jul 2025 14:41:31 +0000 Received: by outflank-mailman (output) from mailman id 1031263.1405041; Wed, 02 Jul 2025 14:41:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uWyeR-00084C-CX; Wed, 02 Jul 2025 14:41:31 +0000 Received: by outflank-mailman (input) for mailman id 1031263; Wed, 02 Jul 2025 14:41:30 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uWyeQ-0007qI-EB for xen-devel@lists.xenproject.org; Wed, 02 Jul 2025 14:41:30 +0000 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [2a00:1450:4864:20::42d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a35217b9-5752-11f0-b894-0df219b8e170; Wed, 02 Jul 2025 16:41:28 +0200 (CEST) Received: by mail-wr1-x42d.google.com with SMTP id ffacd0b85a97d-3a4f72cba73so5108940f8f.1 for ; Wed, 02 Jul 2025 07:41:28 -0700 (PDT) Received: from localhost.localdomain (host-92-26-98-202.as13285.net. [92.26.98.202]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-453823ad20bsm230946905e9.20.2025.07.02.07.41.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jul 2025 07:41:27 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a35217b9-5752-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1751467288; x=1752072088; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Y3UdrPpnVz1bdBWjp4Vfz0Zz3kJ6jFSf8VMOzn7gPgo=; b=InMruRehK2k/OPtI4mLwzj2kbC2g9nBjavXISjEjJcEBivDRH2QBZtYjmFSOfF5lSv si5Q945h0eAZMigbzqMZ9pwlR7ENNvYPNgbYUQ+sgof0UE8bnDXBqM2SMpjMYjsXqoak WlGAJeyGd6GplSFpcwxYQvOdkibVEYevLNkKg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751467288; x=1752072088; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Y3UdrPpnVz1bdBWjp4Vfz0Zz3kJ6jFSf8VMOzn7gPgo=; b=rimcvDNadWQDEsusvdShU225zJyIthv2NVDpokVi+HdaJS2fBnvwTgc6BrwlnQ/2ZV Kg/B2UvZSQan+tm4sY4qpq1EOt7ikJMQkwmdNhYSFEC6VaoR6D2VsiysqmrW76pH6a6J fERTOR/gg5obDEVcmB+LIFUp896C6DKTY7sFVFXswaQ1GnHIUEIonvbawcSTZ3+2rw94 HPM7SxUAWuEXABj9Aoo4iyxqs3kYsFN2rqd4v7yFgRUSLa7W9XrJM6kNdhI6BIHBAUQq Ik4Fb8g19oaCN6yG7lFUJklrqatYhyRy11p8q63yHfaeV5EeBijwPC/KEz6kZDY88kIY 8dpg== X-Gm-Message-State: AOJu0YxjCkG4FK4+xGOqhYjuVU3PnEDPPiXVQWut8V7k8GJV6r7syCsK cSVCk88LRP5RZM52M91ujG+KtXfUcMi2djcZtyJmt5GX3G7FgvecKKiMz5c+BTS6GrqZOJHTW/W YuY8c++dSeA== X-Gm-Gg: ASbGncs9HhJaT8q+eu7pzrgkQI8mUXQDQCfpa1IUbu4I3nUBrfSOvlsHv/YiroKeQLJ 7jlIRHfwKfC18Ll+8fK0M5hojsKrm8QN7sGTY2XZwjz2IpiyffdSRlyaNTs/gaWYhrgUAr97bFF dJdvE1BBUMvuVHOSs4EDeJmeA/WxNXvvA9ffmJhmfFBXIavzhYKpDMbTzHR66k8e5+IQelG9q7d k3gEvJ1J/X8KfSAvG/meN9QgBbkeZjSJJFO4uK8viE/MUgiRbacf8yfR+/Z/4NnqseAJWARM2pF QsXZgD+gIORcmp/PsER6u+SSnf6VZGx6SioXwk/nEN7cBQ72pXJ0dZVUpYCwgbrXOrGYVR7rD4L utYOVMXIV1KB/Kwcd3H8+COb+tD0HKvNEHxE= X-Google-Smtp-Source: AGHT+IE6XyKnf8Jg78X60Y8B4gr0ajIaNcDAjmJ4x4amxZW4xnl2pfGYH6o6v241bIRolL2YGjBIbw== X-Received: by 2002:a05:6000:4023:b0:3af:1cba:1546 with SMTP id ffacd0b85a97d-3b1f80ed9b5mr2774898f8f.22.1751467287962; Wed, 02 Jul 2025 07:41:27 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH 2/6] x86/idle: Convert force_mwait_ipi_wakeup to X86_BUG_MONITOR Date: Wed, 2 Jul 2025 15:41:17 +0100 Message-Id: <20250702144121.1096448-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250702144121.1096448-1-andrew.cooper3@citrix.com> References: <20250702144121.1096448-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1751467302589116600 We're going to want alternative-patch based on it. No functional change. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Anthony PERARD CC: Michal Orzel CC: Julien Grall CC: Stefano Stabellini --- xen/arch/x86/acpi/cpu_idle.c | 6 ------ xen/arch/x86/cpu/intel.c | 2 +- xen/arch/x86/include/asm/cpufeatures.h | 1 + xen/arch/x86/include/asm/mwait.h | 3 --- 4 files changed, 2 insertions(+), 10 deletions(-) diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c index 141f0f0facf6..dbcb80d81db9 100644 --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -436,14 +436,8 @@ static int __init cf_check cpu_idle_key_init(void) } __initcall(cpu_idle_key_init); =20 -/* Force sending of a wakeup IPI regardless of mwait usage. */ -bool __ro_after_init force_mwait_ipi_wakeup; - bool arch_skip_send_event_check(unsigned int cpu) { - if ( force_mwait_ipi_wakeup ) - return false; - return false; } =20 diff --git a/xen/arch/x86/cpu/intel.c b/xen/arch/x86/cpu/intel.c index 5215b5405c76..f7bd0d777289 100644 --- a/xen/arch/x86/cpu/intel.c +++ b/xen/arch/x86/cpu/intel.c @@ -436,7 +436,7 @@ static void __init probe_mwait_errata(void) { printk(XENLOG_WARNING "Forcing IPI MWAIT wakeup due to CPU erratum\n"); - force_mwait_ipi_wakeup =3D true; + setup_force_cpu_cap(X86_BUG_MONITOR); } } =20 diff --git a/xen/arch/x86/include/asm/cpufeatures.h b/xen/arch/x86/include/= asm/cpufeatures.h index 84c93292c80c..56231b00f15d 100644 --- a/xen/arch/x86/include/asm/cpufeatures.h +++ b/xen/arch/x86/include/asm/cpufeatures.h @@ -53,6 +53,7 @@ XEN_CPUFEATURE(USE_VMCALL, X86_SYNTH(30)) /* Use V= MCALL instead of VMMCAL #define X86_BUG_CLFLUSH_MFENCE X86_BUG( 2) /* MFENCE needed to serialis= e CLFLUSH */ #define X86_BUG_IBPB_NO_RET X86_BUG( 3) /* IBPB doesn't flush the RS= B/RAS */ #define X86_BUG_CLFLUSH_MONITOR X86_BUG( 4) /* MONITOR requires CLFLUSH = */ +#define X86_BUG_MONITOR X86_BUG( 5) /* MONITOR doesn't always no= tice writes (force IPIs) */ =20 #define X86_SPEC_NO_LFENCE_ENTRY_PV X86_BUG(16) /* (No) safety LFENCE for = SPEC_CTRL_ENTRY_PV. */ #define X86_SPEC_NO_LFENCE_ENTRY_INTR X86_BUG(17) /* (No) safety LFENCE fo= r SPEC_CTRL_ENTRY_INTR. */ diff --git a/xen/arch/x86/include/asm/mwait.h b/xen/arch/x86/include/asm/mw= ait.h index c52cd3f51011..000a692f6d19 100644 --- a/xen/arch/x86/include/asm/mwait.h +++ b/xen/arch/x86/include/asm/mwait.h @@ -13,9 +13,6 @@ =20 #define MWAIT_ECX_INTERRUPT_BREAK 0x1 =20 -/* Force sending of a wakeup IPI regardless of mwait usage. */ -extern bool force_mwait_ipi_wakeup; - void mwait_idle_with_hints(unsigned int eax, unsigned int ecx); #ifdef CONFIG_INTEL bool mwait_pc10_supported(void); --=20 2.39.5 From nobody Thu Oct 30 23:14:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1751467309; cv=none; d=zohomail.com; s=zohoarc; b=gJDwD6IyNsAWCZXSseWQY77S1PL24FbT/ocUcb55ESraBaomW0J+jCoNVrOb9duwxqB3BhNOoxM+Gofp4RRmvFWAznoxkWC9b06lbOdkfdtVDJbtQZbOaWFmBgztegmno3rjYi5A4/W31KxDLhs9eJ4A2/kVadw1a6XRudwskxk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1751467309; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ZoxVJO1Cp9G2uE0d1/tqm82YZtH5FBnCWbzeL0LUSJk=; b=iKiW1dojLV/vqY3W0ycEEDhz8Dpn7fgMzRK3LCEtW9T4bPi4jff/hPLdPI4YC7YmypMr1f75wHkgQ+Noi9Pa12oAjUafX8D5MxoeE4gBkI1HGwMQdN60/9KfR+pXew4OV+oy5B5HEj+3bKWFW4T2bkDbgDjRFyj5IVnma1QUtEQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1751467308934854.7091062142582; Wed, 2 Jul 2025 07:41:48 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1031264.1405051 (Exim 4.92) (envelope-from ) id 1uWyeS-0008Ir-N8; Wed, 02 Jul 2025 14:41:32 +0000 Received: by outflank-mailman (output) from mailman id 1031264.1405051; Wed, 02 Jul 2025 14:41:32 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uWyeS-0008Ie-JI; Wed, 02 Jul 2025 14:41:32 +0000 Received: by outflank-mailman (input) for mailman id 1031264; Wed, 02 Jul 2025 14:41:30 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uWyeQ-0007x9-Sx for xen-devel@lists.xenproject.org; Wed, 02 Jul 2025 14:41:30 +0000 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [2a00:1450:4864:20::42f]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a400b35b-5752-11f0-a313-13f23c93f187; Wed, 02 Jul 2025 16:41:30 +0200 (CEST) Received: by mail-wr1-x42f.google.com with SMTP id ffacd0b85a97d-3a588da60dfso2884242f8f.1 for ; Wed, 02 Jul 2025 07:41:30 -0700 (PDT) Received: from localhost.localdomain (host-92-26-98-202.as13285.net. [92.26.98.202]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-453823ad20bsm230946905e9.20.2025.07.02.07.41.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jul 2025 07:41:28 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a400b35b-5752-11f0-a313-13f23c93f187 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1751467289; x=1752072089; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZoxVJO1Cp9G2uE0d1/tqm82YZtH5FBnCWbzeL0LUSJk=; b=dpPF76l0lkDo85jKGOFcY0FpdsPtMcCk3scsPRRGjTn0Fm5yz4vwx4IWKTfKAenvgB ll3QOPB9WI7wxsa1od8lybsFzFzmW37NCrpvVWm7nwds0VLZCDTMqKU5Sr570qgOjR6F QTb5Dg18gl0KlXXRbt5WkWrKOkpq2Io+nwLZs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751467289; x=1752072089; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZoxVJO1Cp9G2uE0d1/tqm82YZtH5FBnCWbzeL0LUSJk=; b=RtR/zaSAZD6Nswj5WeGAX4zHL1L3HeGr6cwDIJcNG25BOigNjkWNuc83q5RqqjXBhQ gh9catBLdVpcQF/H3KIzPTui4yTjGiBXSMZwYlRz4ILeG21cQxzP2tadtu1EwEquoxvo 5CNIaFXRoDEDnbU+V0fXXanKJRbAICyTkAfvnxgIleS0SPAxEWdo786slxdFzOQHyzxs b14EJ6ia2wRb3UWyBl4OIhOvgHRqfoAgwObuCf2TOwrM4nxzcg9ormTlMdc/5vDHsRYs 7buOOYqT4wKOO1unWqn7u2FxnGT5kOX45E4Ih4rLn3+FB71lUY4dZKydnMcnIwhejTdk +3Aw== X-Gm-Message-State: AOJu0Yxexr9CBHrfza0IOxx5MfZWHlo4KNwOH0Tpz2dQXewhMznk+0iY zVCDdeHbCGeGrYORxB5WIYRo5Zja2W5MUapiQSWWUTdA/bPP/1DqwVa3/bKZEx2mc0tF8q7s+Yo hC4NFfv6HMw== X-Gm-Gg: ASbGnct9Z+teJmo+Lom9cAez5CVSW04ivCAEfbpkvrFzr1UmTZtcO70TFo+M6nvqY/p QgS3qY0dTbXdrWGqfyRy9eqPM+QaO/3CrHX3btiOyJ060o+esisyODK7+UzOoEK1hcRmbq1wbxc 19sGCcc9eA1Tf+Q7DzPTU3MDxrR7JPFQ9VBHBO3QbSW8I1sfQ/K2Go8CCZrYmKHvG/rCd0zwyI8 rhnBWLfpQb6W8I+Nc/MP7WkQMT8Ad5XTFe8HscuPky1Gq4mnDjoQXEjllMHig5NB44u/FY7vHRb yz3Lg2U3k3bS50Yds4LO3fg6UpWOucPYbeyPDysZTgTfB9GWbTF6iiYzqJPb0JDxr1FaTjnY9L1 xpKUI5DcobGK0UwsSzeUzTOqkD0dL7XEQQng= X-Google-Smtp-Source: AGHT+IGP5IHWhmosOZevMrRQ/aZBPSTmDtIyCZyYXBNJ508Q4vwzK0RXb2l57iQQJGxlV8oZm7VNBg== X-Received: by 2002:a05:6000:2203:b0:3a5:88e9:a54f with SMTP id ffacd0b85a97d-3b1ff244520mr2789363f8f.1.1751467288963; Wed, 02 Jul 2025 07:41:28 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH 3/6] xen/softirq: Rework arch_skip_send_event_check() into arch_pend_softirq() Date: Wed, 2 Jul 2025 15:41:18 +0100 Message-Id: <20250702144121.1096448-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250702144121.1096448-1-andrew.cooper3@citrix.com> References: <20250702144121.1096448-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1751467310646116600 x86 is the only architecture wanting an optimisation here, but the test_and_set_bit() is a store into the monitored line and is racy with determining whether an IPI can be skipped. Instead, implement a new arch helper with different semantics; to make the softirq pending and decide about IPIs together. For now, implement the default helper. It will be overridden by x86 in a subsequent change. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Roger Pau Monn=C3=A9 --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Anthony PERARD CC: Michal Orzel CC: Julien Grall CC: Stefano Stabellini --- xen/arch/x86/acpi/cpu_idle.c | 5 ----- xen/arch/x86/include/asm/softirq.h | 2 -- xen/common/softirq.c | 8 ++------ xen/include/xen/softirq.h | 16 ++++++++++++++++ 4 files changed, 18 insertions(+), 13 deletions(-) diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c index dbcb80d81db9..caa0fef0b3b1 100644 --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -436,11 +436,6 @@ static int __init cf_check cpu_idle_key_init(void) } __initcall(cpu_idle_key_init); =20 -bool arch_skip_send_event_check(unsigned int cpu) -{ - return false; -} - void mwait_idle_with_hints(unsigned int eax, unsigned int ecx) { unsigned int cpu =3D smp_processor_id(); diff --git a/xen/arch/x86/include/asm/softirq.h b/xen/arch/x86/include/asm/= softirq.h index 415ee866c79d..e4b194f069fb 100644 --- a/xen/arch/x86/include/asm/softirq.h +++ b/xen/arch/x86/include/asm/softirq.h @@ -9,6 +9,4 @@ #define HVM_DPCI_SOFTIRQ (NR_COMMON_SOFTIRQS + 4) #define NR_ARCH_SOFTIRQS 5 =20 -bool arch_skip_send_event_check(unsigned int cpu); - #endif /* __ASM_SOFTIRQ_H__ */ diff --git a/xen/common/softirq.c b/xen/common/softirq.c index 60f344e8425e..0368011f95d1 100644 --- a/xen/common/softirq.c +++ b/xen/common/softirq.c @@ -94,9 +94,7 @@ void cpumask_raise_softirq(const cpumask_t *mask, unsigne= d int nr) raise_mask =3D &per_cpu(batch_mask, this_cpu); =20 for_each_cpu(cpu, mask) - if ( !test_and_set_bit(nr, &softirq_pending(cpu)) && - cpu !=3D this_cpu && - !arch_skip_send_event_check(cpu) ) + if ( !arch_pend_softirq(nr, cpu) && cpu !=3D this_cpu ) __cpumask_set_cpu(cpu, raise_mask); =20 if ( raise_mask =3D=3D &send_mask ) @@ -107,9 +105,7 @@ void cpu_raise_softirq(unsigned int cpu, unsigned int n= r) { unsigned int this_cpu =3D smp_processor_id(); =20 - if ( test_and_set_bit(nr, &softirq_pending(cpu)) - || (cpu =3D=3D this_cpu) - || arch_skip_send_event_check(cpu) ) + if ( arch_pend_softirq(nr, cpu) || cpu =3D=3D this_cpu ) return; =20 if ( !per_cpu(batching, this_cpu) || in_irq() ) diff --git a/xen/include/xen/softirq.h b/xen/include/xen/softirq.h index e9f79ec0ce3e..0814c8e5cd41 100644 --- a/xen/include/xen/softirq.h +++ b/xen/include/xen/softirq.h @@ -23,6 +23,22 @@ enum { =20 #define NR_SOFTIRQS (NR_COMMON_SOFTIRQS + NR_ARCH_SOFTIRQS) =20 +/* + * Ensure softirq @nr is pending on @cpu. Return true if an IPI can be + * skipped, false if the IPI cannot be skipped. + */ +#ifndef arch_pend_softirq +static always_inline bool arch_pend_softirq(unsigned int nr, unsigned int = cpu) +{ + /* + * Try to set the softirq pending. If we set the bit (i.e. the old bit + * was 0), we're responsible to send the IPI. If the softirq was alre= ady + * pending (i.e. the old bit was 1), no IPI is needed. + */ + return test_and_set_bit(nr, &softirq_pending(cpu)); +} +#endif + typedef void (*softirq_handler)(void); =20 void do_softirq(void); --=20 2.39.5 From nobody Thu Oct 30 23:14:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1751467311; cv=none; d=zohomail.com; s=zohoarc; b=MKQqvxh4GMTEHbzqJybe5uhoqnMK+NZ9AQt/OHqAEicEDrhAgtRm0rz1p/YHSx0N2PrsKf2o15+JoEOsYDC56E58j5qqFVqxMZL9vaxf4JVK2+T88uIzLy2Bamoe/V1ECa3wJJ/CDzmko7aEzolvvASp7qKqaBZWCJpkaivkFq4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1751467311; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=EzppBlV46dVW+8Pf+ysgv4D4sf/8Ax0fEdwCrQArIh0=; b=LtFWuL25+IN16OSnwNGHqq7GQZwJmNs6M5NNIWcTmK/jQWfwx+4KB2uUMedJ3YuN08MT835Mj6bOrlSGHcBOuVuxJ48Mupl0gCcHnj3r1ADmHqCa2FiwxeDY7PZtdhD+mmyh7q1gf+0qvBXmGeKUN1KQ93rDe4sYyUQB7R/H9rA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1751467311196834.6149903816256; Wed, 2 Jul 2025 07:41:51 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1031266.1405066 (Exim 4.92) (envelope-from ) id 1uWyeU-00009D-Eg; Wed, 02 Jul 2025 14:41:34 +0000 Received: by outflank-mailman (output) from mailman id 1031266.1405066; Wed, 02 Jul 2025 14:41:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uWyeU-00008X-8j; Wed, 02 Jul 2025 14:41:34 +0000 Received: by outflank-mailman (input) for mailman id 1031266; Wed, 02 Jul 2025 14:41:32 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uWyeS-0007qI-CI for xen-devel@lists.xenproject.org; Wed, 02 Jul 2025 14:41:32 +0000 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [2a00:1450:4864:20::42c]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a4781bba-5752-11f0-b894-0df219b8e170; Wed, 02 Jul 2025 16:41:30 +0200 (CEST) Received: by mail-wr1-x42c.google.com with SMTP id ffacd0b85a97d-3a4f379662cso3846355f8f.0 for ; Wed, 02 Jul 2025 07:41:30 -0700 (PDT) Received: from localhost.localdomain (host-92-26-98-202.as13285.net. [92.26.98.202]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-453823ad20bsm230946905e9.20.2025.07.02.07.41.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jul 2025 07:41:29 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a4781bba-5752-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1751467290; x=1752072090; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EzppBlV46dVW+8Pf+ysgv4D4sf/8Ax0fEdwCrQArIh0=; b=NZzZdyVMLDAPd4irL91kHxDSRaDcr+FMxg/fBZ3gi8lLIcXO6Ow0IoRFhUK5nf2KfD Paxtg6Aqmk5gU8QDVHR09oCXF3jOI+dhCqSZNj8XJlV6W/N/m4xw/P+l0coLV+Yb05Ah Gy0T1bZqNfKUXEa3tFxqiGjAnr+E0G0UsSt44= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751467290; x=1752072090; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EzppBlV46dVW+8Pf+ysgv4D4sf/8Ax0fEdwCrQArIh0=; b=Xip9QuPpbYJvaMIqp8urRTvRdGP3vFnIGpqx2DgZcuXEnfoj6SCEKc7IAsdyyrdTs6 67/Zns73+Dr+Bdj6cWMImmXcGxiASXZzS39w3SU90WQRjZtMU0c0yw40pAfN76vQYam+ c+s1RX3mjg4D9Urd4cWysXk7awNz2NfKVJNsAeV7/V110/yvu9efBP2Ya0E9m8GgNnpy mYIsq5H5b9cD62HXO4HG4VxpuR3TouJsYr1W2Ve+q6vxAWA5AArdPBfV/KWMhWnvElpJ J4vOv3pAsSpgGbfbxICG/bqBeorKyBNN/jW+25fOXBWnRVykKBUC/nPLv30M3WJSRrTJ yBsA== X-Gm-Message-State: AOJu0YxV5gwsIEH5TNvPRQ/vHNZQRnCPZbST+bRmvQQQDLHvHVwBjY+J 6NyXu/ksKR6BZGzRDCNsHe8M0PKb6m4GhidL1mDY9zf2VfUYGkDs1/6L2W1/qEvwV+vepyvgqsD Ui+OIU/7zsg== X-Gm-Gg: ASbGnct2VwFye9/wt1c0wNbfkMdEfJQjP67PkMKIcsiw/4ggjsEF2/HCoVJUeQXAekM eJpZnXfLSlHy8sarl6dKafdwjZF4+JDnR2qRDQG3Yf40xcr0HGJtWClFNkqps2VRTQeRvFEP9So ebqi8npkwDVF/vvgLefZrov9I9New3FIBM9ZpYsqngirVP3Sfq7LG6k8w92NBlUyG68QncNb0KO 8Qrz9HVngviWRTqYyMBC1rq/o1AQNeFlzNutPS0TBj7cXIOcEmh3ZJvrrosPgwGvtc+PEOQV3DL 6y3PheFqaHywC6PWf2DHl7TcvNVrihXRbmYrmi5eL/kmz/GFypUkNMGenb7XvXIxS+q0gSbj044 FV1//cAfMTQHdSkrd5D1Xu+WamipkoOUF0Zk= X-Google-Smtp-Source: AGHT+IEnzXIn+0MqJayMOtG02cyaJCA2PQ9QF7Qz1u3JINoT9MnD91Uzo4J6kaGi59hHeUrD0EpjxA== X-Received: by 2002:a5d:64c4:0:b0:3a4:e502:81e1 with SMTP id ffacd0b85a97d-3b2018c241bmr2665185f8f.52.1751467289832; Wed, 02 Jul 2025 07:41:29 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH 4/6] x86/idle: Implement a new MWAIT IPI-elision algorithm Date: Wed, 2 Jul 2025 15:41:19 +0100 Message-Id: <20250702144121.1096448-5-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250702144121.1096448-1-andrew.cooper3@citrix.com> References: <20250702144121.1096448-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1751467312655116600 In order elide IPIs, we must be able to identify whether a target CPU is in MWAIT at the point it is woken up. i.e. the store to wake it up must also identify the state. Create a new in_mwait variable beside __softirq_pending, so we can use a CMPXCHG to set the softirq while also observing the status safely. Impleme= nt an x86 version of arch_pend_softirq() which does this. In mwait_idle_with_hints(), advertise in_mwait, with an explanation of precisely what it means. X86_BUG_MONITOR can be accounted for simply by not advertising in_mwait. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Anthony PERARD CC: Michal Orzel CC: Julien Grall CC: Stefano Stabellini This is modelled after Linux's TIF_NEED_RESCHED (single bit equivelent of a= ll of __softirq_pending), and TIF_POLLING_NRFLAG (arch-neutral "in_mwait"). In Linux, they're both in the same flags field, which adds complexity. In Xen, __softirq_pending is already unsigned long for everything other than x= 86, so adding an arch-neutral "in_mwait" would need wider changes. --- xen/arch/x86/acpi/cpu_idle.c | 20 +++++++++++++++++- xen/arch/x86/include/asm/hardirq.h | 14 +++++++++++- xen/arch/x86/include/asm/softirq.h | 34 ++++++++++++++++++++++++++++++ 3 files changed, 66 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c index caa0fef0b3b1..13040ef467a0 100644 --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -439,7 +439,21 @@ __initcall(cpu_idle_key_init); void mwait_idle_with_hints(unsigned int eax, unsigned int ecx) { unsigned int cpu =3D smp_processor_id(); - const unsigned int *this_softirq_pending =3D &softirq_pending(cpu); + irq_cpustat_t *stat =3D &irq_stat[cpu]; + const unsigned int *this_softirq_pending =3D &stat->__softirq_pending; + + /* + * By setting in_mwait, we promise to other CPUs that we'll notice cha= nges + * to __softirq_pending without being sent an IPI. We achieve this by + * either not going to sleep, or by having hardware notice on our beha= lf. + * + * Some errata exist where MONITOR doesn't work properly, and the + * workaround is to force the use of an IPI. Cause this to happen by + * simply not advertising outselves as being in_mwait. + */ + alternative_io("movb $1, %[in_mwait]", + "", X86_BUG_MONITOR, + [in_mwait] "=3Dm" (stat->in_mwait)); =20 monitor(this_softirq_pending, 0, 0); smp_mb(); @@ -452,6 +466,10 @@ void mwait_idle_with_hints(unsigned int eax, unsigned = int ecx) mwait(eax, ecx); spec_ctrl_exit_idle(info); } + + alternative_io("movb $0, %[in_mwait]", + "", X86_BUG_MONITOR, + [in_mwait] "=3Dm" (stat->in_mwait)); } =20 static void acpi_processor_ffh_cstate_enter(struct acpi_processor_cx *cx) diff --git a/xen/arch/x86/include/asm/hardirq.h b/xen/arch/x86/include/asm/= hardirq.h index f3e93cc9b507..1647cff04dc8 100644 --- a/xen/arch/x86/include/asm/hardirq.h +++ b/xen/arch/x86/include/asm/hardirq.h @@ -5,7 +5,19 @@ #include =20 typedef struct { - unsigned int __softirq_pending; + /* + * The layout is important. Any CPU can set bits in __softirq_pending, + * but in_mwait is a status bit owned by the CPU. softirq_mwait_raw m= ust + * cover both, and must be in a single cacheline. + */ + union { + struct { + unsigned int __softirq_pending; + bool in_mwait; + }; + uint64_t softirq_mwait_raw; + }; + unsigned int __local_irq_count; unsigned int nmi_count; unsigned int mce_count; diff --git a/xen/arch/x86/include/asm/softirq.h b/xen/arch/x86/include/asm/= softirq.h index e4b194f069fb..069e5716a68d 100644 --- a/xen/arch/x86/include/asm/softirq.h +++ b/xen/arch/x86/include/asm/softirq.h @@ -1,6 +1,8 @@ #ifndef __ASM_SOFTIRQ_H__ #define __ASM_SOFTIRQ_H__ =20 +#include + #define NMI_SOFTIRQ (NR_COMMON_SOFTIRQS + 0) #define TIME_CALIBRATE_SOFTIRQ (NR_COMMON_SOFTIRQS + 1) #define VCPU_KICK_SOFTIRQ (NR_COMMON_SOFTIRQS + 2) @@ -9,4 +11,36 @@ #define HVM_DPCI_SOFTIRQ (NR_COMMON_SOFTIRQS + 4) #define NR_ARCH_SOFTIRQS 5 =20 +/* + * Ensure softirq @nr is pending on @cpu. Return true if an IPI can be + * skipped, false if the IPI cannot be skipped. + * + * We use a CMPXCHG covering both __softirq_pending and in_mwait, in order= to + * set softirq @nr while also observing in_mwait in a race-free way. + */ +static always_inline bool arch_pend_softirq(unsigned int nr, unsigned int = cpu) +{ + uint64_t *ptr =3D &irq_stat[cpu].softirq_mwait_raw; + uint64_t old, new; + unsigned int softirq =3D 1U << nr; + + old =3D ACCESS_ONCE(*ptr); + + do { + if ( old & softirq ) + /* Softirq already pending, nothing to do. */ + return true; + + new =3D old | softirq; + + } while ( (old =3D cmpxchg(ptr, old, new)) !=3D new ); + + /* + * We have caused the softirq to become pending. If in_mwait was set,= the + * target CPU will notice the modification and act on it. + */ + return new & (1UL << 32); +} +#define arch_pend_softirq arch_pend_softirq + #endif /* __ASM_SOFTIRQ_H__ */ --=20 2.39.5 From nobody Thu Oct 30 23:14:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1751467307; cv=none; d=zohomail.com; s=zohoarc; b=NU7hv2JihpaTt+v1aB5MJClB6/74dFt8rt+ppH7rldWFRlOMiqtRDB/3z524s7zc/Rej5oYDkmNI6LUvValM52cBu1F90O5tXLWVLl7YBQi0XeKk7V0ahii42RoTRBz+mnErRHwlpUoIMxX4FCJQwvvhEWLZ2XCJanZUbMe6fCE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1751467307; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=nxIVNczZ1rIA2YAlLtpWMc3jJY/ZNCuwxwnSmoLdJTc=; b=NnBMZtNl2dD6DZ9dnF70E096u0qToXhDy1sRIqdu3eGnikpzUfN4zYM/FH+xf6WEjl7gCwGyaCwEwjtnzMnjvHt+sxdb9Qo1TkF/sIwfsYb4H+g+lsO4A/Eb+XQfwcwcrnqqUR34y2BPH4DQDisuUi5YAicUI0aec7fdVoUS79s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1751467307667839.6983298611775; Wed, 2 Jul 2025 07:41:47 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1031267.1405071 (Exim 4.92) (envelope-from ) id 1uWyeU-0000FO-Sd; Wed, 02 Jul 2025 14:41:34 +0000 Received: by outflank-mailman (output) from mailman id 1031267.1405071; Wed, 02 Jul 2025 14:41:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uWyeU-0000Dt-JD; Wed, 02 Jul 2025 14:41:34 +0000 Received: by outflank-mailman (input) for mailman id 1031267; Wed, 02 Jul 2025 14:41:33 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uWyeT-0007qI-OW for xen-devel@lists.xenproject.org; Wed, 02 Jul 2025 14:41:33 +0000 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [2a00:1450:4864:20::334]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a54f0113-5752-11f0-b894-0df219b8e170; Wed, 02 Jul 2025 16:41:32 +0200 (CEST) Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-4537fdec39fso15338115e9.0 for ; Wed, 02 Jul 2025 07:41:32 -0700 (PDT) Received: from localhost.localdomain (host-92-26-98-202.as13285.net. [92.26.98.202]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-453823ad20bsm230946905e9.20.2025.07.02.07.41.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jul 2025 07:41:30 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a54f0113-5752-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1751467291; x=1752072091; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nxIVNczZ1rIA2YAlLtpWMc3jJY/ZNCuwxwnSmoLdJTc=; b=QQs/R/gGg5vFfbmPazfAM/w8sqCZAiENDbj2vwNdOMM4xoi5cNLbP1DgEZ/9zzIdYJ 3GJlscCOdHW0s7wlht2WPjRavKRuasUNDJn1m1X/Fq7FxUMf3qLrG3QssXA8LjboPAUe guZCxMWmY9cOx4h6r6nfhelGfW977PFze6ZDI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751467291; x=1752072091; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nxIVNczZ1rIA2YAlLtpWMc3jJY/ZNCuwxwnSmoLdJTc=; b=TZ36NIQI092IIikwIrMVBSGhh5NFyNa+rk3dRmbZ737rCorFTWzZ8onDQi5wyI/5Yd UUWz571GgaWACFw9ne8EtjIfnJfjU/Qhpn9Tcd9PKCqHogiFg2mpedfg92giYw87q2jf r9np3U/flCIQu+g39FMS1/EUXvtuvEPiC+mwqxVO3NsvOE2nSykM5EhktOtnoMiX7UXl rWUkTmYYARUghf40vuxytH4mk6Z/nD1E80lWK0DQ5x6ivlqjKJ2nwEwHvaueCIop5Dcq Abf3zysOg/IX3joD0bJcF46dqA49q4WQwJRCXJg/cpg8iures6kt1SJkiBm3pX/1OjDg RRyw== X-Gm-Message-State: AOJu0Yxgi2+jTV4fNJZSnVujY1m5GZEQ5T5czoLiZkqdbvCAnk8HrGP4 145Bd2jYdx8IZKgi8A/nyBbvh3LCE9EUXr5yufnQluweJUcGxTWYRQeRLYz2/2/hZsbZNEsPrDM M+v7lNk+t4w== X-Gm-Gg: ASbGncuWYWR37A/ELPrnyv+1kzeGDo6jyYSh20Tu2Bb6EmgRmu8ts58jZZ+iYCefOZC jQx7LHQeYFgenbG4TTroTqiESpOhh6sbmciOzPogj5fhe2bWJvRzOOsiSSdGNfVLb4GRnh+31As hQehj24Xz9BOnCapAM/FpxMubqR0gUfI2VUSP5Db5xPRMcPzxxuuoxXTfp43eozEi+SP6DypSyw e5deM6mt5qahLBj4gG7SVoc9pdK62LnVBpfay6qaAgwP31QtlZZkzf16PfTYUkILTA8kbn/gXH3 YSak9JuEu/1OynpavCjUUM6irLuFcIg21ISISiAYOZ7nQPcquzaAR7QlfnWYvX2g31ymHWCODMU /Jkt6smNhWIAtuMVINlj2oXkwhRjC6eBjzqwFVSZXVQRWRA== X-Google-Smtp-Source: AGHT+IHPfekVRNza1oHlHHApBPWw80wB98FcLF5N54CE2FrBTKMCr5qBmn3s7N87TlH53I2ia8v4cA== X-Received: by 2002:a05:600c:1d1a:b0:442:dc6f:2f11 with SMTP id 5b1f17b1804b1-454a372e257mr30396825e9.25.1751467291320; Wed, 02 Jul 2025 07:41:31 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH 5/6] x86/idle: Drop incorrect smp_mb() in mwait_idle_with_hints() Date: Wed, 2 Jul 2025 15:41:20 +0100 Message-Id: <20250702144121.1096448-6-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250702144121.1096448-1-andrew.cooper3@citrix.com> References: <20250702144121.1096448-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1751467308472116600 With the recent simplifications, it becomes obvious that smp_mb() isn't the right barrier; all we need is a compiler barrier. Include this in monitor() itself, along with an explantion. No functional change. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Anthony PERARD CC: Michal Orzel CC: Julien Grall CC: Stefano Stabellini --- xen/arch/x86/acpi/cpu_idle.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c index 13040ef467a0..de020dfee87f 100644 --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -66,8 +66,12 @@ static always_inline void monitor( alternative_input("", "clflush (%[addr])", X86_BUG_CLFLUSH_MONITOR, [addr] "a" (addr)); =20 + /* + * The memory clobber is a compiler barrier. Subseqeunt reads from the + * monitored cacheline must not be hoisted over MONITOR. + */ asm volatile ( "monitor" - :: "a" (addr), "c" (ecx), "d" (edx) ); + :: "a" (addr), "c" (ecx), "d" (edx) : "memory" ); } =20 static always_inline void mwait(unsigned int eax, unsigned int ecx) @@ -456,7 +460,6 @@ void mwait_idle_with_hints(unsigned int eax, unsigned i= nt ecx) [in_mwait] "=3Dm" (stat->in_mwait)); =20 monitor(this_softirq_pending, 0, 0); - smp_mb(); =20 if ( !*this_softirq_pending ) { --=20 2.39.5 From nobody Thu Oct 30 23:14:02 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1751467317; cv=none; d=zohomail.com; s=zohoarc; b=Xgdk64JkYSxWrF/4ugoMihRL4h93+U14CsDk9FWFic7KZoxGiEi9EKRq+PKeHg65/LfGfhDc6+7kZ+uUPdQRyCszl0Ot9eXDZNurBhGtLo1TYefp5LiavSPp/9FPw5p5AXX03WbZxZKiivlZq5jATnw6gxRh8jH1QYjpCLqCsHU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1751467317; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=IsaIUoYrgMaZ6LILx+fXrGcgIxtKBujRKKhSNH9JnHQ=; b=W3S64O7NoE3B8qgECExXc+B0e+wQj7xaFpMYY3nHyTD7oZprNyjNXSqbpVlBYLIK+w+Sn6PedETONaRzHwayCmRXHfCrWJY7DJUXTMy9xomQrQ88lBdpPV/dj89EL/Y1DNlDBXMpRFpBJIOzNx+lWegsVSocvR4jqD8wAsE3FWE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1751467317431778.7301888121431; Wed, 2 Jul 2025 07:41:57 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1031268.1405087 (Exim 4.92) (envelope-from ) id 1uWyeW-0000jG-5O; Wed, 02 Jul 2025 14:41:36 +0000 Received: by outflank-mailman (output) from mailman id 1031268.1405087; Wed, 02 Jul 2025 14:41:36 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uWyeW-0000hz-1g; Wed, 02 Jul 2025 14:41:36 +0000 Received: by outflank-mailman (input) for mailman id 1031268; Wed, 02 Jul 2025 14:41:34 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uWyeU-0007qI-Lc for xen-devel@lists.xenproject.org; Wed, 02 Jul 2025 14:41:34 +0000 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [2a00:1450:4864:20::431]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a5d84690-5752-11f0-b894-0df219b8e170; Wed, 02 Jul 2025 16:41:33 +0200 (CEST) Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-3a57ae5cb17so3040309f8f.0 for ; Wed, 02 Jul 2025 07:41:33 -0700 (PDT) Received: from localhost.localdomain (host-92-26-98-202.as13285.net. [92.26.98.202]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-453823ad20bsm230946905e9.20.2025.07.02.07.41.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jul 2025 07:41:31 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a5d84690-5752-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1751467292; x=1752072092; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IsaIUoYrgMaZ6LILx+fXrGcgIxtKBujRKKhSNH9JnHQ=; b=FuwU2zjcf5O96UGtew2sOxrGxXaGWCpLhXvHG1jFQHg1xJkZu+/3c08Ivh1znztCMa 77VHzDzm0a5WamS4m2Bln924oSwOMbX1hPWa6SWzEZGtxh73uqySKrJxG8JY+/762vby gdubz8GeSOqDmYoxOPJ37tRh8RNOrvuSy193w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751467292; x=1752072092; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IsaIUoYrgMaZ6LILx+fXrGcgIxtKBujRKKhSNH9JnHQ=; b=P0TygWY7Z20Nidndh/ndYbxileWSiLrVefp+GsVaU5dv3LQnhYTfyipqZHFmg8Jdnd RFCDhTz75zoEM180px7ox6VuVoZ06UfjNeNyJtkpWveuz6mvns7OfknszPaWy1P+y6+P yq4lOcDBjnNhhWn59SgT/KXQxVXSiD6sBoiH7dqdjSyr9j345X9ea9p7HG5+3b6dOGL3 99BsZx4I0bSmWYQZlumUq3ZU36sID+c4jvrbXTLJ/8RtN6ONfV8NBBHRl9G03mRS14Qz TTvffpy3PBkQtgcPJVA2FtjaRAuRL3vTcUdyZfcImpFTxK0wXfbGRv30LVg3ZKFHiC11 UvpQ== X-Gm-Message-State: AOJu0YzU82UBoxAvI9lWxVR6bIdh+wdNZHn9NnkqRgxSP2TFI5lnAmZT uy9Pz/s9ttbWXvIBou2bPq19SmmHxjHqABDelVaJ3/3okL0XbgXSon22nj1O9da7qRJSC/6ltoW mvj2PxsXp1g== X-Gm-Gg: ASbGncs9l4LHla5aUbAl8MPZOTustRpYSdRcNVqw7hR5KrGRw8KoF3cVdjyIEjdOeKM rXv96kW7lOp2Pq75JUV1GcwmtTSUJAE3VRAxDwkR2Xj8QnbXhO+JlfZe/uLTH51GRQMFkW8lnpB +OMETsiYPlj3JmeikloxtXhNnG6bCJJT2LPTZtgNBebWl2GSIh24dPkbeHrHjQxPDkeIV5YNBwV lc2tu8YaSWVHzH3VTkxuHauf97OrSaINbHbhzlAB52mGbK2HO+UWwJmyIymNAxznWqkO3pfqAdN CAWyV+WBhMf2yP7FJyqRP+8iJQjEVygeeOIlJmgS2mzFYzdNH3QFyjWMuY/YHnExnemFGGE2Yzf SOPB6XB9LV4exX4/WBAlNXvqbEEluWZwOrBCLDWM4SXZgtw== X-Google-Smtp-Source: AGHT+IGgsRlQqTrN48VnadNjRK976IFEOtd7SxWtrGSsHZVPB/eV6P78ZtXvOx3+j5k61B2pskhXRw== X-Received: by 2002:a05:6000:2385:b0:3a4:ee3f:8e1e with SMTP id ffacd0b85a97d-3b201e84a1cmr2523786f8f.39.1751467292250; Wed, 02 Jul 2025 07:41:32 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH 6/6] x86/idle: Fix buggy "x86/mwait-idle: enable interrupts before C1 on Xeons" Date: Wed, 2 Jul 2025 15:41:21 +0100 Message-Id: <20250702144121.1096448-7-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250702144121.1096448-1-andrew.cooper3@citrix.com> References: <20250702144121.1096448-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1751467318699116600 The check of this_softirq_pending must be performed with irqs disabled, but this property was broken by an attempt to optimise entry/exit latency. Commit c227233ad64c in Linux (which we copied into Xen) was fixed up by edc8fc01f608 in Linux, which we have so far missed. Going to sleep without waking on interrupts is nonsensical outside of play_dead(), so overload this to select between two possible MWAITs, the second using the STI shadow to cover MWAIT for exactly the same reason as we do in safe_halt(). Fixes: b17e0ec72ede ("x86/mwait-idle: enable interrupts before C1 on Xeons") Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Anthony PERARD CC: Michal Orzel CC: Julien Grall CC: Stefano Stabellini --- xen/arch/x86/acpi/cpu_idle.c | 16 +++++++++++++++- xen/arch/x86/cpu/mwait-idle.c | 8 ++------ 2 files changed, 17 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c index de020dfee87f..dc8b7111a181 100644 --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -80,6 +80,13 @@ static always_inline void mwait(unsigned int eax, unsign= ed int ecx) :: "a" (eax), "c" (ecx) ); } =20 +static always_inline void sti_mwait_cli(unsigned int eax, unsigned int ecx) +{ + /* STI shadow covers MWAIT. */ + asm volatile ( "sti; mwait; cli" + :: "a" (eax), "c" (ecx) ); +} + #define GET_HW_RES_IN_NS(msr, val) \ do { rdmsrl(msr, val); val =3D tsc_ticks2ns(val); } while( 0 ) #define GET_MC6_RES(val) GET_HW_RES_IN_NS(0x664, val) @@ -461,12 +468,19 @@ void mwait_idle_with_hints(unsigned int eax, unsigned= int ecx) =20 monitor(this_softirq_pending, 0, 0); =20 + ASSERT(!local_irq_is_enabled()); + if ( !*this_softirq_pending ) { struct cpu_info *info =3D get_cpu_info(); =20 spec_ctrl_enter_idle(info); - mwait(eax, ecx); + + if ( ecx & MWAIT_ECX_INTERRUPT_BREAK ) + mwait(eax, ecx); + else + sti_mwait_cli(eax, ecx); + spec_ctrl_exit_idle(info); } =20 diff --git a/xen/arch/x86/cpu/mwait-idle.c b/xen/arch/x86/cpu/mwait-idle.c index 5c16f5ad3a82..b65d6ae9ddc5 100644 --- a/xen/arch/x86/cpu/mwait-idle.c +++ b/xen/arch/x86/cpu/mwait-idle.c @@ -946,12 +946,8 @@ static void cf_check mwait_idle(void) =20 update_last_cx_stat(power, cx, before); =20 - if (cx->irq_enable_early) - local_irq_enable(); - - mwait_idle_with_hints(cx->address, MWAIT_ECX_INTERRUPT_BREAK); - - local_irq_disable(); + mwait_idle_with_hints(cx->address, + cx->irq_enable_early ? 0 : MWAIT_ECX_INTERRU= PT_BREAK); =20 after =3D alternative_call(cpuidle_get_tick); =20 --=20 2.39.5