From nobody Thu Oct 30 23:14:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1751646878; cv=none; d=zohomail.com; s=zohoarc; b=oBKyUSGYxmd78IahJkDIWh0pWHRvo4r4py7lIaSUenen/5OWcCfpzfZmNhjN84Urd/7+pKo61koMXF8itbrG1wzNXis0auduRm3P8NMUXBXO3gdeB1YM9Kste9jyF5ShhCiA/NXSHkytrd0Z5QIztYnbgkSj5BvhV983T62CHZM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1751646878; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=Q5xp3+ysILd0+Q1dhc8jOc5wluMva3PMjYQseqSE5sA=; b=BtJgO1SZVp7MN5q7oshwpbhihCPWeNBow4oDrixJpKPfm4rMKsgRed/p3N+AUWgogpuIw2CqD3G6W0obdnRZIwz64rdxgQexwCFs0Eq+hEvfs4Xnxx5e6XexA4m2jjEnZEJImzxbtvLPvHMY7Xed86E8S/MayXDoL1ktBFUKZmE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 175164687865169.86528719510102; Fri, 4 Jul 2025 09:34:38 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1033539.1406875 (Exim 4.92) (envelope-from ) id 1uXjMh-0005Sq-4p; Fri, 04 Jul 2025 16:34:19 +0000 Received: by outflank-mailman (output) from mailman id 1033539.1406875; Fri, 04 Jul 2025 16:34:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uXjMh-0005Sj-1b; Fri, 04 Jul 2025 16:34:19 +0000 Received: by outflank-mailman (input) for mailman id 1033539; Fri, 04 Jul 2025 16:34:17 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uXjMf-0005Ex-Aw for xen-devel@lists.xenproject.org; Fri, 04 Jul 2025 16:34:17 +0000 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [2a00:1450:4864:20::335]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id b8c04d2f-58f4-11f0-a315-13f23c93f187; Fri, 04 Jul 2025 18:34:15 +0200 (CEST) Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-453749aef9eso4031545e9.3 for ; Fri, 04 Jul 2025 09:34:14 -0700 (PDT) Received: from localhost.localdomain (host-92-26-98-202.as13285.net. [92.26.98.202]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454a9989fcesm59199345e9.16.2025.07.04.09.34.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Jul 2025 09:34:13 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b8c04d2f-58f4-11f0-a315-13f23c93f187 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1751646854; x=1752251654; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q5xp3+ysILd0+Q1dhc8jOc5wluMva3PMjYQseqSE5sA=; b=EUvRRPC7PRXenSHPzPiqmH7Bt4cDYuoNwXF/5cPbvU9E8RWG+IMVhDJZXm2vpYNpV+ Cb7I7bI/tLbVr9qDS0gyOOOweijH6CeJwADYzbsVQpTjptcS/VYn5geK+G797ltGZaD7 HPxcT1+sn/tepAQfRFPHk5lJZoXUlER6S4cQg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751646854; x=1752251654; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q5xp3+ysILd0+Q1dhc8jOc5wluMva3PMjYQseqSE5sA=; b=PkpN75cSyv9J24gToqY0cTiWJ0yMhWtNK51ULlCduUyLd1lrzqJ57fZnaXCgnRfwbm AYuoBe+gJ4gfomMmJBmw2k0n568j+w4eulmqZdhwQcd2fpr+xfms3M6c5zNEgPnv7V62 UXX6Ofmor2ia8sSmBukWRBbHdNaGgPA5yY5RpklWKqn7TI1hG0nc1o/fAYHVNzLaqc0m qdKhMyzS51tyXdrrnFv8FtmBCTf5P5SgFIxGmHORInwOmp+OD9KdwKni3cntLW7s11fB SeAn4p5fCJ6chKF+7iWlPFR2FTluQC1ukoQgj3/l6KspMTPvekM+39LIRfZjT4eZM8iq kTyQ== X-Gm-Message-State: AOJu0YwnYlzEfwqb7g2Zb7KOSz2rbp5RO2rOwWI8pT5eFr8E0qt26ADX maBveCp6/k/44Q6qkcXuOQ+i0KiYfGws0npSqJKOaE/2kir9bJaYYq2Viw0tQsQ7zRjc37ChXmx moU6SNuY0gQ== X-Gm-Gg: ASbGncttg03UnJc9gYpmyR8QQLlj5DPIsF3ZnpWd9typHPDfhYur+1jK+xLn5lsvVGM HW0JreVHSUSRcgk+VeJnyTNsbOq29fHW6SB48a7YCelZMNhDV2q9VPiE8OfqwDdQ8vbRRkzVXom QBBnj4eMUuj5ganvbpnPQoFmw84VnY4065Di3tkox8xR80xnEaOOqbekK3qZqbWy15aXZVGoFIQ nBe19dzH4TUbloSFMN9HNS4U07bRb4BSyt0ECmI/mO6Si9+quaV80zIMTOtNwYTNY2x8PBIlPPk 6eREmzhS7X9uLmTy/scXgW8uh1Nj/xZnWdtovE176aSVibk8zvRksaySXle0LXJeCh6AKP6Hx4U cXVbO/r2A/ePjLV2stkNTw0hKIdzEDNHMQDZ7mZINXSZJcA== X-Google-Smtp-Source: AGHT+IHyMqLpnd28HHVgC5K/nTw5sErQGZLxOcUW3hBZBENFnz0HIx1Ga0NcKCFOj7ksK7q4AJLarQ== X-Received: by 2002:a05:600c:871a:b0:450:d367:c385 with SMTP id 5b1f17b1804b1-454b315e488mr38167375e9.16.1751646853578; Fri, 04 Jul 2025 09:34:13 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH v2 1/6] x86/idle: Remove broken MWAIT implementation Date: Fri, 4 Jul 2025 17:34:05 +0100 Message-Id: <20250704163410.1412367-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250704163410.1412367-1-andrew.cooper3@citrix.com> References: <20250704163410.1412367-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1751646881112116600 cpuidle_wakeup_mwait() is a TOCTOU race. The cpumask_and() sampling cpuidle_mwait_flags can take a arbitrary period of time, and there's no guarantee that the target CPUs are still in MWAIT when writing into mwait_wakeup(cpu). The consequence of the race is that we'll fail to IPI certain targets. Als= o, there's no guarantee that mwait_idle_with_hints() will raise a TIMER_SOFTIRQ on it's way out. The fundamental bug is that the "in_mwait" variable needs to be in the monitored line, and not in a separate cpuidle_mwait_flags variable, in order to do this in a race-free way. Arranging to fix this while keeping the old implementation is prohibitive, = so strip the current one out in order to implement the new one cleanly. In the interim, this causes IPIs to be used unconditionally which is safe albeit suboptimal. Fixes: 3d521e933e1b ("cpuidle: mwait on softirq_pending & remove wakeup ipi= s") Fixes: 1adb34ea846d ("CPUIDLE: re-implement mwait wakeup process") Signed-off-by: Andrew Cooper Reviewed-by: Roger Pau Monn=C3=A9 --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Anthony PERARD CC: Michal Orzel CC: Julien Grall CC: Stefano Stabellini --- xen/arch/x86/acpi/cpu_idle.c | 48 ++++-------------------------- xen/arch/x86/hpet.c | 2 -- xen/arch/x86/include/asm/hardirq.h | 9 +++--- xen/include/xen/cpuidle.h | 2 -- xen/include/xen/irq_cpustat.h | 1 - 5 files changed, 9 insertions(+), 53 deletions(-) diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c index 6c3a10e6fb4e..141f0f0facf6 100644 --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -436,27 +436,6 @@ static int __init cf_check cpu_idle_key_init(void) } __initcall(cpu_idle_key_init); =20 -/* - * The bit is set iff cpu use monitor/mwait to enter C state - * with this flag set, CPU can be waken up from C state - * by writing to specific memory address, instead of sending an IPI. - */ -static cpumask_t cpuidle_mwait_flags; - -void cpuidle_wakeup_mwait(cpumask_t *mask) -{ - cpumask_t target; - unsigned int cpu; - - cpumask_and(&target, mask, &cpuidle_mwait_flags); - - /* CPU is MWAITing on the cpuidle_mwait_wakeup flag. */ - for_each_cpu(cpu, &target) - mwait_wakeup(cpu) =3D 0; - - cpumask_andnot(mask, mask, &target); -} - /* Force sending of a wakeup IPI regardless of mwait usage. */ bool __ro_after_init force_mwait_ipi_wakeup; =20 @@ -465,42 +444,25 @@ bool arch_skip_send_event_check(unsigned int cpu) if ( force_mwait_ipi_wakeup ) return false; =20 - /* - * This relies on softirq_pending() and mwait_wakeup() to access data - * on the same cache line. - */ - smp_mb(); - return !!cpumask_test_cpu(cpu, &cpuidle_mwait_flags); + return false; } =20 void mwait_idle_with_hints(unsigned int eax, unsigned int ecx) { unsigned int cpu =3D smp_processor_id(); - s_time_t expires =3D per_cpu(timer_deadline, cpu); - const void *monitor_addr =3D &mwait_wakeup(cpu); + const unsigned int *this_softirq_pending =3D &softirq_pending(cpu); =20 - monitor(monitor_addr, 0, 0); + monitor(this_softirq_pending, 0, 0); smp_mb(); =20 - /* - * Timer deadline passing is the event on which we will be woken via - * cpuidle_mwait_wakeup. So check it now that the location is armed. - */ - if ( (expires > NOW() || expires =3D=3D 0) && !softirq_pending(cpu) ) + if ( !*this_softirq_pending ) { struct cpu_info *info =3D get_cpu_info(); =20 - cpumask_set_cpu(cpu, &cpuidle_mwait_flags); - spec_ctrl_enter_idle(info); mwait(eax, ecx); spec_ctrl_exit_idle(info); - - cpumask_clear_cpu(cpu, &cpuidle_mwait_flags); } - - if ( expires <=3D NOW() && expires > 0 ) - raise_softirq(TIMER_SOFTIRQ); } =20 static void acpi_processor_ffh_cstate_enter(struct acpi_processor_cx *cx) @@ -901,7 +863,7 @@ void cf_check acpi_dead_idle(void) =20 if ( cx->entry_method =3D=3D ACPI_CSTATE_EM_FFH ) { - void *mwait_ptr =3D &mwait_wakeup(smp_processor_id()); + void *mwait_ptr =3D &softirq_pending(smp_processor_id()); =20 /* * Cache must be flushed as the last operation before sleeping. diff --git a/xen/arch/x86/hpet.c b/xen/arch/x86/hpet.c index 1bca8c8b670d..192de433cfd1 100644 --- a/xen/arch/x86/hpet.c +++ b/xen/arch/x86/hpet.c @@ -188,8 +188,6 @@ static void evt_do_broadcast(cpumask_t *mask) if ( __cpumask_test_and_clear_cpu(cpu, mask) ) raise_softirq(TIMER_SOFTIRQ); =20 - cpuidle_wakeup_mwait(mask); - if ( !cpumask_empty(mask) ) cpumask_raise_softirq(mask, TIMER_SOFTIRQ); } diff --git a/xen/arch/x86/include/asm/hardirq.h b/xen/arch/x86/include/asm/= hardirq.h index 342361cb6fdd..f3e93cc9b507 100644 --- a/xen/arch/x86/include/asm/hardirq.h +++ b/xen/arch/x86/include/asm/hardirq.h @@ -5,11 +5,10 @@ #include =20 typedef struct { - unsigned int __softirq_pending; - unsigned int __local_irq_count; - unsigned int nmi_count; - unsigned int mce_count; - bool __mwait_wakeup; + unsigned int __softirq_pending; + unsigned int __local_irq_count; + unsigned int nmi_count; + unsigned int mce_count; } __cacheline_aligned irq_cpustat_t; =20 #include /* Standard mappings for irq_cpustat_t above = */ diff --git a/xen/include/xen/cpuidle.h b/xen/include/xen/cpuidle.h index 705d0c1135f0..120e354fe340 100644 --- a/xen/include/xen/cpuidle.h +++ b/xen/include/xen/cpuidle.h @@ -92,8 +92,6 @@ extern struct cpuidle_governor *cpuidle_current_governor; bool cpuidle_using_deep_cstate(void); void cpuidle_disable_deep_cstate(void); =20 -extern void cpuidle_wakeup_mwait(cpumask_t *mask); - #define CPUIDLE_DRIVER_STATE_START 1 =20 extern void menu_get_trace_data(u32 *expected, u32 *pred); diff --git a/xen/include/xen/irq_cpustat.h b/xen/include/xen/irq_cpustat.h index b9629f25c266..5f039b4b9a76 100644 --- a/xen/include/xen/irq_cpustat.h +++ b/xen/include/xen/irq_cpustat.h @@ -24,6 +24,5 @@ extern irq_cpustat_t irq_stat[]; /* arch independent irq_stat fields */ #define softirq_pending(cpu) __IRQ_STAT((cpu), __softirq_pending) #define local_irq_count(cpu) __IRQ_STAT((cpu), __local_irq_count) -#define mwait_wakeup(cpu) __IRQ_STAT((cpu), __mwait_wakeup) =20 #endif /* __irq_cpustat_h */ --=20 2.39.5 From nobody Thu Oct 30 23:14:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1751646889; cv=none; d=zohomail.com; s=zohoarc; b=TLF2U89541nMLKBdRj8q9VeDnY6XaqDxuBcLMteoHjMpU8TqCyIMTv2IgC+261owBYNn4INQBpyi3MC9yHGySDbCuaXvtTVCNcl4/vZVR3fW/EoNktNiwVohOjP3cfP4Y5v5GaDRj8DoW5quPogOWgbfTt56GvaVHA/XRvA0uRc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1751646889; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=5iU72UnrXReJr2MaFOqUFKlkLHLyzDHrUJMt+dtpUJU=; b=WOwuVYRgDvg+EwLCA+Ns8nX7/pb5VlNMWT2O9mYD/4/m29Jhc14/KOE183daWMpBZTq2Ehf10/E1X3UsUS/nB9uccE3m2g2lqFjM7yG8ZJf/zvb3CodVI9DOSnUB1p7Elp2eK5IwJLRYMT/gYuL0Ib5pq4l1uNY2G5Nig4FzJ5o= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1751646889507643.9204675624188; Fri, 4 Jul 2025 09:34:49 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1033540.1406884 (Exim 4.92) (envelope-from ) id 1uXjMi-0005gk-Bf; Fri, 04 Jul 2025 16:34:20 +0000 Received: by outflank-mailman (output) from mailman id 1033540.1406884; Fri, 04 Jul 2025 16:34:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uXjMi-0005gd-97; Fri, 04 Jul 2025 16:34:20 +0000 Received: by outflank-mailman (input) for mailman id 1033540; Fri, 04 Jul 2025 16:34:18 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uXjMg-0005Ex-Ax for xen-devel@lists.xenproject.org; Fri, 04 Jul 2025 16:34:18 +0000 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [2a00:1450:4864:20::32f]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id b939275f-58f4-11f0-a315-13f23c93f187; Fri, 04 Jul 2025 18:34:15 +0200 (CEST) Received: by mail-wm1-x32f.google.com with SMTP id 5b1f17b1804b1-451d54214adso7739415e9.3 for ; Fri, 04 Jul 2025 09:34:15 -0700 (PDT) Received: from localhost.localdomain (host-92-26-98-202.as13285.net. [92.26.98.202]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454a9989fcesm59199345e9.16.2025.07.04.09.34.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Jul 2025 09:34:14 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b939275f-58f4-11f0-a315-13f23c93f187 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1751646854; x=1752251654; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5iU72UnrXReJr2MaFOqUFKlkLHLyzDHrUJMt+dtpUJU=; b=RCy3eAlDiA98DtwzwkpruuXBFdIyv7cM2gx9AaKoltWQa17/1WurFZmRVI+bVUlcOV M+pKrgZ0T/RyUCQh712GcFoBYa+9/nMgltUpHtSIIB+qz/L47U8BsiJ9HDlA2Am9a8n0 LyKlfaSIpO1HRn9GYr6s1118AZGnstzjuFV5s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751646854; x=1752251654; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5iU72UnrXReJr2MaFOqUFKlkLHLyzDHrUJMt+dtpUJU=; b=SomP2lqCqhRCFXKIRDOPh5I6/aP6DKnweTmoZirfZKg9WO0HLVembGpUT/KSeA0chM 7H6sCw+Nl79E7l9Q/6USpRjjuF/oAWmGFysLi4MQpcI3x/x/NNl3jFpo/H7swjOJDCDn zhOTi+bE8O2qKPxk4RslJYS2AJFMizhWT25EL91ZBjDYfKaXh8jWGIgaYP5nIrNttuPG LjCjP4JqnokafUJGbqzqhuVodnWFfS8krAqSTIV7DGUWxHLs3NEPq6OI1StSm/ewEI5r S2/fr6sKledrMWmk/ULBYjLKfnR9YK8dJqwXRB4lcorPh9ozMeer63By73FtgW9bfYx9 TZHg== X-Gm-Message-State: AOJu0Yy6KjG+TwTdm7yKg4JQkG8pLD01pH8/mNn5za/gR5a9BQSpJBMO jhisSLmmLVLhyLccuBTvRyW9CbUFFhMI1lKR60mKUR/GqpEndo7WdUm9lnGeA/esNDVKbnfzpo4 p63enhHbJcQ== X-Gm-Gg: ASbGncvNMe28SlafITG0JpDEx49CwMc04CWuZs7XoX+PXMwBILaI0CGleZOZiqMj6+n GL8TYUn+E+fdp5cpcJZh48K+C+5eRd0WNzs03tNaB+bcLN4xJds0/QhA1HhyQTGO3cx+x54LJZf xhR5Cfgoc0NyA0JlGhx1F8Sfliqc9uJ+XOpObIAdPktT715U7cBBmgp1wPhqeA/jr6cX8wq/bGM UVtZtF+P40qmYK1ro/gCOeXBOZFCGeWI/uQ7ejJnGCPZbx2+xULWIX9p4vuI6hEaIIsN0JBJNM7 sFqwlY2FQXrBYMeOA7WJ3gGXVJUB0RXGAfQvX1Mn3gjPZ8eUsuCOHvmDxURxrNQrx5Vo4Wo/P7V VjPIBAj8o004G5OT2xkc78hY8ITxPSOk2Mja2nZ5FHYHXcQ== X-Google-Smtp-Source: AGHT+IGYsipzvN5mN/Qg7mf5hscWFms+YgPJvylnuK0D3RGEYffewhqomHT4x19qpY3Ir/6ka3PTag== X-Received: by 2002:a05:600c:474b:b0:453:8a62:df34 with SMTP id 5b1f17b1804b1-454b3161d07mr28883905e9.21.1751646854444; Fri, 04 Jul 2025 09:34:14 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH v2 2/6] x86/idle: Drop incorrect smp_mb() in mwait_idle_with_hints() Date: Fri, 4 Jul 2025 17:34:06 +0100 Message-Id: <20250704163410.1412367-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250704163410.1412367-1-andrew.cooper3@citrix.com> References: <20250704163410.1412367-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1751646890845116600 With the recent simplifications, it becomes obvious that smp_mb() isn't the right barrier. Strictly speaking, MONITOR is ordered as a load, but smp_rm= b() isn't correct either, as this only pertains to local ordering. All we need= is a compiler barrier(). Merge the barier() into the monitor() itself, along with an explantion. No functional change. Signed-off-by: Andrew Cooper Acked-by: Roger Pau Monn=C3=A9 --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Anthony PERARD CC: Michal Orzel CC: Julien Grall CC: Stefano Stabellini v2: * Move earlier in the series, not that it matters IMO. * Expand the commit message. --- xen/arch/x86/acpi/cpu_idle.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c index 141f0f0facf6..68dd44be5bb0 100644 --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -66,8 +66,12 @@ static always_inline void monitor( alternative_input("", "clflush (%[addr])", X86_BUG_CLFLUSH_MONITOR, [addr] "a" (addr)); =20 + /* + * The memory clobber is a compiler barrier. Subseqeunt reads from the + * monitored cacheline must not be reordered over MONITOR. + */ asm volatile ( "monitor" - :: "a" (addr), "c" (ecx), "d" (edx) ); + :: "a" (addr), "c" (ecx), "d" (edx) : "memory" ); } =20 static always_inline void mwait(unsigned int eax, unsigned int ecx) @@ -453,7 +457,6 @@ void mwait_idle_with_hints(unsigned int eax, unsigned i= nt ecx) const unsigned int *this_softirq_pending =3D &softirq_pending(cpu); =20 monitor(this_softirq_pending, 0, 0); - smp_mb(); =20 if ( !*this_softirq_pending ) { --=20 2.39.5 From nobody Thu Oct 30 23:14:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1751646884; cv=none; d=zohomail.com; s=zohoarc; b=ZHksEUcy8VK7h0aSpUOxsKHPfjo/XDhqVO+Q2sLA5mxoVin2PtLXg6NwkxLawQwjkcYdPBgBdu+B1NnPvESJGug5F0nwBe9foDm4WxVVUlu4DM/+RW0ZqglYLFqStqOCUETpudCiytsPsHeqrCtQatrMRUFmZzXjFWWemjea7iI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1751646884; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=5p9jnTI9OOKO89IBAp1RiVOJgBP+wGqib7qDxuX0FJc=; b=FbH1M9Ap3F24isBg6V0ZrUugioawteLFvyz3A/oUW+VEWa5eorP9OGxF/vZ8tFG5aQURohuOLTuMUk8IdArq0GwSJEY4QmMk9I2G9Z9ZpnDSPvXtJ1nLSpNIBePFHe5wKpGwvbyrqHeC5jUDamJAjSdpoxcxQbmXoGuvtj9lIAk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1751646884478559.9882037599448; Fri, 4 Jul 2025 09:34:44 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1033541.1406892 (Exim 4.92) (envelope-from ) id 1uXjMi-0005kv-Qw; Fri, 04 Jul 2025 16:34:20 +0000 Received: by outflank-mailman (output) from mailman id 1033541.1406892; Fri, 04 Jul 2025 16:34:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uXjMi-0005k9-Kz; Fri, 04 Jul 2025 16:34:20 +0000 Received: by outflank-mailman (input) for mailman id 1033541; Fri, 04 Jul 2025 16:34:19 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uXjMh-0005FA-FQ for xen-devel@lists.xenproject.org; Fri, 04 Jul 2025 16:34:19 +0000 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [2a00:1450:4864:20::430]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id b9c1b8db-58f4-11f0-b894-0df219b8e170; Fri, 04 Jul 2025 18:34:16 +0200 (CEST) Received: by mail-wr1-x430.google.com with SMTP id ffacd0b85a97d-3a52874d593so932367f8f.0 for ; Fri, 04 Jul 2025 09:34:16 -0700 (PDT) Received: from localhost.localdomain (host-92-26-98-202.as13285.net. [92.26.98.202]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454a9989fcesm59199345e9.16.2025.07.04.09.34.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Jul 2025 09:34:14 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b9c1b8db-58f4-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1751646855; x=1752251655; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5p9jnTI9OOKO89IBAp1RiVOJgBP+wGqib7qDxuX0FJc=; b=sxF7fjgd0qdWMpMEALperIkzhZpx001OGSF5TgMYRdijmcQc0XJSAlbwHVGMA+GTLR UF2690Rr26YlUToVUsvlwN8PmTOUy97zwMylJA9xdDiuQ3rleQKDUDryCTmgiUIDwNAI TtIrns6vDw9widtyftPnhqZPwsu/I8h52svcg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751646855; x=1752251655; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5p9jnTI9OOKO89IBAp1RiVOJgBP+wGqib7qDxuX0FJc=; b=U0VbsuoBeVGzf7jBpV7lpuAO3aJaLm4wRu/2ZowmdhKEhnvvmVXFiphf1UVP3orNgf s9MHkaZF1q9+d4EpRVpFzXMCfqfT13/zPOMc/Cv1N1hOAK9BiOl5//xFtZ36Z3F6lAvI 7jEvbeJeUWecBx/7gKdIulLpkWqFCPatwLkImYdknVUYC8sEA4VSn48c7jpDiUDnaiys nilKfrPPCpuJ7bQCQFx8Wlsc9YASnPHL5Yw5x2XIE32Z6a4ZUL840aID7h9AUWlNUBWG iF/F9NdiloD7QiSI9GpwrCQZ6fKzn9Bn5XqvNTcqcQKlam96rgYv3f9ku+yZ4yu2+sFc GSnw== X-Gm-Message-State: AOJu0YzmjP0radV1OLFb89pqpYRzjW56IEq/lBIBTCt1/+ty/GY26s4s IfO4T+n1GHeGTYnWv89P1RH03O+yav04l61MisGfwkzob1p4IXvi0Qhc+JHgT2rfI/YUIOfkKOg jWkqHE0TjpQ== X-Gm-Gg: ASbGncs7t7nZO0cXfK3kDXFf+Lvu1Be4w+TqUoAa7C9ZxzkDaJMyrKfHww+5wrzUEU7 oXvnBxMQKgmeNSyHWfZeIhbTvAzjtr+hMntlgylgO+8zn1ieTsKtiFUQeaDVmh+wBqsCGRNEi5S JaYsA4991J2/ZHOJPa9F+Yx6ItWveUtUkmnHBLLjyAPsDk8Z5+MKnk62n/fN6oq6MsOAwxNAl6w mMBTDm1c6RCgZ06hUXJZt/4/5fE4eHEW3QNhDweyelgCrlT7mSubHDAvKFjIHy1GnyHbG+7ZT4w S2l1a+RSP//QA9+nb82gU7nyQc6NvEQPcIK4kg6sZcc7InqMeEXAra7zR3eqIWP8acWmY7LHyEs ldSpshtWoMLkLMcs/3r1lr6awomxxKInMLqg= X-Google-Smtp-Source: AGHT+IHswtAm/LmWb7CmazSM1x7eZIHusL64Hapm3ly1PC5a9RGKAlsNjY7rapAh+P3ZPIq2zhHLLA== X-Received: by 2002:adf:b643:0:b0:3a4:d4cd:b06 with SMTP id ffacd0b85a97d-3b496607b62mr2103761f8f.34.1751646855260; Fri, 04 Jul 2025 09:34:15 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH v2 3/6] x86/idle: Convert force_mwait_ipi_wakeup to X86_BUG_MONITOR Date: Fri, 4 Jul 2025 17:34:07 +0100 Message-Id: <20250704163410.1412367-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250704163410.1412367-1-andrew.cooper3@citrix.com> References: <20250704163410.1412367-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1751646885004116600 We're going to want alternative-patch based on it. No functional change. Signed-off-by: Andrew Cooper Acked-by: Roger Pau Monn=C3=A9 --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Anthony PERARD CC: Michal Orzel CC: Julien Grall CC: Stefano Stabellini --- xen/arch/x86/acpi/cpu_idle.c | 6 ------ xen/arch/x86/cpu/intel.c | 2 +- xen/arch/x86/include/asm/cpufeatures.h | 1 + xen/arch/x86/include/asm/mwait.h | 3 --- 4 files changed, 2 insertions(+), 10 deletions(-) diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c index 68dd44be5bb0..07056a91a29e 100644 --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -440,14 +440,8 @@ static int __init cf_check cpu_idle_key_init(void) } __initcall(cpu_idle_key_init); =20 -/* Force sending of a wakeup IPI regardless of mwait usage. */ -bool __ro_after_init force_mwait_ipi_wakeup; - bool arch_skip_send_event_check(unsigned int cpu) { - if ( force_mwait_ipi_wakeup ) - return false; - return false; } =20 diff --git a/xen/arch/x86/cpu/intel.c b/xen/arch/x86/cpu/intel.c index 5215b5405c76..f7bd0d777289 100644 --- a/xen/arch/x86/cpu/intel.c +++ b/xen/arch/x86/cpu/intel.c @@ -436,7 +436,7 @@ static void __init probe_mwait_errata(void) { printk(XENLOG_WARNING "Forcing IPI MWAIT wakeup due to CPU erratum\n"); - force_mwait_ipi_wakeup =3D true; + setup_force_cpu_cap(X86_BUG_MONITOR); } } =20 diff --git a/xen/arch/x86/include/asm/cpufeatures.h b/xen/arch/x86/include/= asm/cpufeatures.h index 84c93292c80c..56231b00f15d 100644 --- a/xen/arch/x86/include/asm/cpufeatures.h +++ b/xen/arch/x86/include/asm/cpufeatures.h @@ -53,6 +53,7 @@ XEN_CPUFEATURE(USE_VMCALL, X86_SYNTH(30)) /* Use V= MCALL instead of VMMCAL #define X86_BUG_CLFLUSH_MFENCE X86_BUG( 2) /* MFENCE needed to serialis= e CLFLUSH */ #define X86_BUG_IBPB_NO_RET X86_BUG( 3) /* IBPB doesn't flush the RS= B/RAS */ #define X86_BUG_CLFLUSH_MONITOR X86_BUG( 4) /* MONITOR requires CLFLUSH = */ +#define X86_BUG_MONITOR X86_BUG( 5) /* MONITOR doesn't always no= tice writes (force IPIs) */ =20 #define X86_SPEC_NO_LFENCE_ENTRY_PV X86_BUG(16) /* (No) safety LFENCE for = SPEC_CTRL_ENTRY_PV. */ #define X86_SPEC_NO_LFENCE_ENTRY_INTR X86_BUG(17) /* (No) safety LFENCE fo= r SPEC_CTRL_ENTRY_INTR. */ diff --git a/xen/arch/x86/include/asm/mwait.h b/xen/arch/x86/include/asm/mw= ait.h index c52cd3f51011..000a692f6d19 100644 --- a/xen/arch/x86/include/asm/mwait.h +++ b/xen/arch/x86/include/asm/mwait.h @@ -13,9 +13,6 @@ =20 #define MWAIT_ECX_INTERRUPT_BREAK 0x1 =20 -/* Force sending of a wakeup IPI regardless of mwait usage. */ -extern bool force_mwait_ipi_wakeup; - void mwait_idle_with_hints(unsigned int eax, unsigned int ecx); #ifdef CONFIG_INTEL bool mwait_pc10_supported(void); --=20 2.39.5 From nobody Thu Oct 30 23:14:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1751646880; cv=none; d=zohomail.com; s=zohoarc; b=jCbSrBa0MZpCHx1DMhx5DrCMjiPr3HXeMENm/9TgmDLU2jtk8oswfhPwXfm0P3HNHisp/P+g10PV25GjC3c3Nu99Qe1H2PbPBMIJrKH21Vp1acoUQpifLhONEspVwIkLUL0tC/5A5uK51rWLJsXP9SHv/2iEbj7VOoFog6/yTvM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1751646880; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=HnOR3BkSmDKvwBYwNYDK05tkMwtt9au0xqjngZKJSYs=; b=b2ThidGcO783vjSH/a+CNNQiGzJGgBFjV9yViEuzScxlHwUkN2YiPKzvpkIeuhsbgvAiM1jF7TriosAXYLKX1zPNTNqtxrR+TL3AtibbgusAXsnE8gfHAP3U8WnWxoO4sJyONKBvsWRiqRL/Z39Jj6r5iXmOr3SenOjAJwQBlXw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1751646880911837.0131888903353; Fri, 4 Jul 2025 09:34:40 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1033542.1406898 (Exim 4.92) (envelope-from ) id 1uXjMj-0005qb-4U; Fri, 04 Jul 2025 16:34:21 +0000 Received: by outflank-mailman (output) from mailman id 1033542.1406898; Fri, 04 Jul 2025 16:34:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uXjMi-0005pZ-TD; Fri, 04 Jul 2025 16:34:20 +0000 Received: by outflank-mailman (input) for mailman id 1033542; Fri, 04 Jul 2025 16:34:19 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uXjMh-0005FA-MM for xen-devel@lists.xenproject.org; Fri, 04 Jul 2025 16:34:19 +0000 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [2a00:1450:4864:20::32a]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id ba48e741-58f4-11f0-b894-0df219b8e170; Fri, 04 Jul 2025 18:34:17 +0200 (CEST) Received: by mail-wm1-x32a.google.com with SMTP id 5b1f17b1804b1-450ccda1a6eso8188915e9.2 for ; Fri, 04 Jul 2025 09:34:17 -0700 (PDT) Received: from localhost.localdomain (host-92-26-98-202.as13285.net. [92.26.98.202]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454a9989fcesm59199345e9.16.2025.07.04.09.34.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Jul 2025 09:34:15 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ba48e741-58f4-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1751646856; x=1752251656; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HnOR3BkSmDKvwBYwNYDK05tkMwtt9au0xqjngZKJSYs=; b=lyT0IZaIlx4pouh7/bIhNnRNJARcyXl1zx2l0DOcEfyfKpNh5AvcgfjSbQw3Bkrhoz RavVVfpKPl7CeGGSohH8IyuVCjaXknWmVqTE5H1s81PTMYCbWH5C3Vxhtge4EdKuBZ+z g+/AyY0J/WNEDuaXFofwEnd3920Z1o9O9WYgk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751646856; x=1752251656; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HnOR3BkSmDKvwBYwNYDK05tkMwtt9au0xqjngZKJSYs=; b=YkYXOdEMd68yTRUun+Jf67mZp0y2hM+ULQBUun+IDV8Ng2dwDRjkJDeUTsku/e++1m JebS/dGCxdRG+PKkIdMItCm0U6kUhSwEqHeIAFkWS/HvHH0vHgt4n/WdWlgEmV/Gpzft rD+cKPxDaEJHN1v4gzRno7yd9k603xY4SYDgCiZXf3YMdHhfCq5iXPgeCcOdeSXXGJQI cbIDSeBDD3fKMY64AFtKmIr6w27XDB9ltRX7rq9/XFRAt8zVoevEtnXJ2O/yWM+ep4K1 f2AbZtOWZylm52UmW6A25VEXFQU4k/8iVgxcWYBwh2kTxmpzrUfleVMH7+b2fNi8Ljwx IyMQ== X-Gm-Message-State: AOJu0YyEZClr7aMujRof9RmchVAk6arY+oCpYhdFViHHKewLqv3+/mrS lbPZ94kxpmhf6HXnPvjwj5o5t/aX+3sLKcRXualeqZKs/qbpo3ipJxB/bFebmbgP9vHc1fdvN8C WW2jdaGC+uA== X-Gm-Gg: ASbGncuONGT6Wmb3HVxf76U1K+KI2ROOZiKj6jOHC9ONsSLdH2deAzDSYhypJmlMDbO 4GAlSnBl5xR6+BC55iKHxKYgSiHOpF7zJI8dEgucZx9M1uqk8QvVHiPY1IS+uzI5XQkxS08nXgl +d4FqCZ8/WZZ5OqZF6uK4Fn6y2L84AbFuziTpDo3C8rcojELGnm46+yryAb79IoXH/4hZMzEFH+ st6TJDWQj1r8FSlMnADeYb47De99+lz52cY7LdE76bAeW9s6Mf8/bry3Hr9wAr6NhTXJY+rIrqV jTkiWp1RDta+26A9DiiNcDDR7dfvDyFM/1eJI7WEZqK1EtBJV0q7S9oLgQlc8U9fk0HFUrshMSm 9VlEvhYpJYOca6oTMt+ajR0KeUEiifE5CtKA= X-Google-Smtp-Source: AGHT+IF4CipTklA7sl4Su6Os0xn3SQPS4yR3idnXvFQxMFXlwr+scLWQEmQvF5xXUPhmVHp51GvqVg== X-Received: by 2002:a05:600c:a087:b0:454:ab1a:8c39 with SMTP id 5b1f17b1804b1-454b310d890mr31356655e9.26.1751646856044; Fri, 04 Jul 2025 09:34:16 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Jan Beulich , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH v2 4/6] xen/softirq: Rework arch_skip_send_event_check() into arch_set_softirq() Date: Fri, 4 Jul 2025 17:34:08 +0100 Message-Id: <20250704163410.1412367-5-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250704163410.1412367-1-andrew.cooper3@citrix.com> References: <20250704163410.1412367-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1751646882832116600 x86 is the only architecture wanting an optimisation here, but the test_and_set_bit() is a store into the monitored line (i.e. will wake up the target) and, prior to the removal of the broken IPI-elision algorithm, was racy, causing unnecessary IPIs to be sent. To do this in a race-free way, the store to the monited line needs to also sample the status of the target in one atomic action. Implement a new arch helper with different semantics; to make the softirq pending and decide abo= ut IPIs together. For now, implement the default helper. It will be overridd= en by x86 in a subsequent change. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Roger Pau Monn=C3=A9 --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Anthony PERARD CC: Michal Orzel CC: Julien Grall CC: Stefano Stabellini v2: * Rename new helper to arch_set_softirq() * Expand commit message --- xen/arch/x86/acpi/cpu_idle.c | 5 ----- xen/arch/x86/include/asm/softirq.h | 2 -- xen/common/softirq.c | 8 ++------ xen/include/xen/softirq.h | 16 ++++++++++++++++ 4 files changed, 18 insertions(+), 13 deletions(-) diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c index 07056a91a29e..4f69df5a7438 100644 --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -440,11 +440,6 @@ static int __init cf_check cpu_idle_key_init(void) } __initcall(cpu_idle_key_init); =20 -bool arch_skip_send_event_check(unsigned int cpu) -{ - return false; -} - void mwait_idle_with_hints(unsigned int eax, unsigned int ecx) { unsigned int cpu =3D smp_processor_id(); diff --git a/xen/arch/x86/include/asm/softirq.h b/xen/arch/x86/include/asm/= softirq.h index 415ee866c79d..e4b194f069fb 100644 --- a/xen/arch/x86/include/asm/softirq.h +++ b/xen/arch/x86/include/asm/softirq.h @@ -9,6 +9,4 @@ #define HVM_DPCI_SOFTIRQ (NR_COMMON_SOFTIRQS + 4) #define NR_ARCH_SOFTIRQS 5 =20 -bool arch_skip_send_event_check(unsigned int cpu); - #endif /* __ASM_SOFTIRQ_H__ */ diff --git a/xen/common/softirq.c b/xen/common/softirq.c index 60f344e8425e..dc3aabce3330 100644 --- a/xen/common/softirq.c +++ b/xen/common/softirq.c @@ -94,9 +94,7 @@ void cpumask_raise_softirq(const cpumask_t *mask, unsigne= d int nr) raise_mask =3D &per_cpu(batch_mask, this_cpu); =20 for_each_cpu(cpu, mask) - if ( !test_and_set_bit(nr, &softirq_pending(cpu)) && - cpu !=3D this_cpu && - !arch_skip_send_event_check(cpu) ) + if ( !arch_set_softirq(nr, cpu) && cpu !=3D this_cpu ) __cpumask_set_cpu(cpu, raise_mask); =20 if ( raise_mask =3D=3D &send_mask ) @@ -107,9 +105,7 @@ void cpu_raise_softirq(unsigned int cpu, unsigned int n= r) { unsigned int this_cpu =3D smp_processor_id(); =20 - if ( test_and_set_bit(nr, &softirq_pending(cpu)) - || (cpu =3D=3D this_cpu) - || arch_skip_send_event_check(cpu) ) + if ( arch_set_softirq(nr, cpu) || cpu =3D=3D this_cpu ) return; =20 if ( !per_cpu(batching, this_cpu) || in_irq() ) diff --git a/xen/include/xen/softirq.h b/xen/include/xen/softirq.h index e9f79ec0ce3e..48f17e49efa1 100644 --- a/xen/include/xen/softirq.h +++ b/xen/include/xen/softirq.h @@ -23,6 +23,22 @@ enum { =20 #define NR_SOFTIRQS (NR_COMMON_SOFTIRQS + NR_ARCH_SOFTIRQS) =20 +/* + * Ensure softirq @nr is pending on @cpu. Return true if an IPI can be + * skipped, false if the IPI cannot be skipped. + */ +#ifndef arch_set_softirq +static always_inline bool arch_set_softirq(unsigned int nr, unsigned int c= pu) +{ + /* + * Try to set the softirq pending. If we set the bit (i.e. the old bit + * was 0), we're responsible to send the IPI. If the softirq was alre= ady + * pending (i.e. the old bit was 1), no IPI is needed. + */ + return test_and_set_bit(nr, &softirq_pending(cpu)); +} +#endif + typedef void (*softirq_handler)(void); =20 void do_softirq(void); --=20 2.39.5 From nobody Thu Oct 30 23:14:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1751646883; cv=none; d=zohomail.com; s=zohoarc; b=WJgUIQJ+7n/Xi6ZjPgLi8+TqrQ5niyGjWEYrZvJZ/YkvB0n9BrfrS2FduDcS3SEAPwmu793C3ZtHm8A+zuicEGGNFaQzMeh5+eEsWwzG/Qwm48E3T9bIBG2gOnY8JGDT8RSaU/sF1REzjB4qkEn1/iQZfBXY0tGjvBh1T8dfVWk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1751646883; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=UEIMY77XlHNrZg7VYSte2iM/TK0ot478AJ0j8XeENak=; b=dg7SMjxyaybzFs0ANxkYG9paxiZqd3BNRgysaMynBVhICj/ulEBV/o+wV3zumgXoBtQhuTlJKQg6AQWOhuwlvo9BiXjOSlRqVvFc98U7MJL198jCazGf0QThhid65GXQdSdWvpsXt59aSlsvgLSbF6GszUeQE25ou7IsDVisCns= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1751646883947384.83506265726476; Fri, 4 Jul 2025 09:34:43 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1033543.1406903 (Exim 4.92) (envelope-from ) id 1uXjMj-0005w0-Hu; Fri, 04 Jul 2025 16:34:21 +0000 Received: by outflank-mailman (output) from mailman id 1033543.1406903; Fri, 04 Jul 2025 16:34:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uXjMj-0005tv-8E; Fri, 04 Jul 2025 16:34:21 +0000 Received: by outflank-mailman (input) for mailman id 1033543; Fri, 04 Jul 2025 16:34:19 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uXjMh-0005FA-TG for xen-devel@lists.xenproject.org; Fri, 04 Jul 2025 16:34:19 +0000 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [2a00:1450:4864:20::432]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id baaa5a50-58f4-11f0-b894-0df219b8e170; Fri, 04 Jul 2025 18:34:17 +0200 (CEST) Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-3a4e742dc97so1459899f8f.0 for ; Fri, 04 Jul 2025 09:34:17 -0700 (PDT) Received: from localhost.localdomain (host-92-26-98-202.as13285.net. [92.26.98.202]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454a9989fcesm59199345e9.16.2025.07.04.09.34.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Jul 2025 09:34:16 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: baaa5a50-58f4-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1751646857; x=1752251657; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UEIMY77XlHNrZg7VYSte2iM/TK0ot478AJ0j8XeENak=; b=KGeXL3Oy6ahDyaHm/mMLRB46/hL9/sX+IPgLUJl/uePqqHceGCzx2DnF2celfh99go B4lghzIZ+iaksAD4gXnVi8oaK+apXcPytpZqA36Xft8NDVNeLOmKfGW89RixLWsiSA2q EHHU6XbVU28AwjwM9+d4UMWUsO+L+637ZHIVg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751646857; x=1752251657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UEIMY77XlHNrZg7VYSte2iM/TK0ot478AJ0j8XeENak=; b=ltmKJb40iK8eVYkSYLiTESNM+KxWofubrOJSdlzDy3dPWbvU2Hks0DtzSAHA9L2wWv LNEOAU5jP2Akkerw9XIm60mT+sGE4J1t2w9VS62TuESZ0Uc8XBq+TzsWCpW8v/A3m3Vm r6+Nt2NYHPZXHyep+Hljxgqy2QbOc93boNDWTpkC58sNMnpDuwVelckbaDn8Jnrq2Ujp 7QpeaqDBGPDGHlolkW5vTQRej6JN9NBlYk4NHfkDt3xSQjTLoQ/uPmkZO7kB3jq8v936 1P0baGR0lB2pjgAS3IlQ82IUYyRUaq2zSHD9NG3d2YflUgC2ooVxXTphibEryFFVocP8 9hhg== X-Gm-Message-State: AOJu0YzzWQUvWZvdWfIev23AP8VqEUKPy9ZTY7LMCrN+N6mc9YdEaE8G qt3gFWb1eX8w0aBB3bi4UQxNMDhUgJY8YAkZPDa6J7it8VxrK/XMyMOB5g54hZgyakZC6e7F+n9 +k+QGUhpW1Q== X-Gm-Gg: ASbGnctcvujItHbTXRzGrer6G9M9f/idX5nIljgE8o2xm8/alRfS62RUk1zzco/fdNS FFK1vxlvexephH8bIqiu8oVv/ztSlop1m+/SZO66Ustt+sON5gn/HuDLyuFtMZ88+qr4hHnpA+f IhnjrJT6an11zctR/HFaavoifwtsAuceVIHNAYhyd71sr0yDHpoaJsN34HDoKtDVqEXiikQ+jSR svEMl8biwwYfu4/+y96g/SnrTnNiHNmHoXSbXKX84q9gSTifPeh8hxoEzAXgOR+8ip8myw/pZRJ vw6CJwNO8hiicPH4DrKP5k4KZTPHkGTWmcGpHgz/Rfbft1LzfX2AfCU2KAb0Ubt6/fgW3ytUkGL z71cT9ciqWq/jEkBN8zucjItRD6URK2ptSE0= X-Google-Smtp-Source: AGHT+IG45D0AN4G/SGXixEV06FQbvAvWSyVuh3ig7WEuWYHAvIREVbV++swpxV44w5A0x0m0Gzzyyg== X-Received: by 2002:a05:6000:2a86:b0:3b2:e0ad:758c with SMTP id ffacd0b85a97d-3b492584666mr2266089f8f.3.1751646856779; Fri, 04 Jul 2025 09:34:16 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH v2 5/6] x86/idle: Implement a new MWAIT IPI-elision algorithm Date: Fri, 4 Jul 2025 17:34:09 +0100 Message-Id: <20250704163410.1412367-6-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250704163410.1412367-1-andrew.cooper3@citrix.com> References: <20250704163410.1412367-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1751646885076116600 In order elide IPIs, we must be able to identify whether a target CPU is in MWAIT at the point it is woken up. i.e. the store to wake it up must also identify the state. Create a new in_mwait variable beside __softirq_pending, so we can use a CMPXCHG to set the softirq while also observing the status safely. Impleme= nt an x86 version of arch_pend_softirq() which does this. In mwait_idle_with_hints(), advertise in_mwait, with an explanation of precisely what it means. X86_BUG_MONITOR can be accounted for simply by not advertising in_mwait. Signed-off-by: Andrew Cooper Acked-by: Roger Pau Monn=C3=A9 --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Anthony PERARD CC: Michal Orzel CC: Julien Grall CC: Stefano Stabellini This is modelled after Linux's TIF_NEED_RESCHED (single bit equivelent of a= ll of __softirq_pending), and TIF_POLLING_NRFLAG (arch-neutral "in_mwait"). In Linux, they're both in the same flags field, which adds complexity. In Xen, __softirq_pending is already unsigned long for everything other than x= 86, so adding an arch-neutral "in_mwait" would need wider changes. v2: * Fix cmpxchg() expression. * Use BUILD_BUG_ON()s to check opencoding of in_mwait. TODO: We want to introduce try_cmpxchg() which is better at the C and code-= gen level. --- xen/arch/x86/acpi/cpu_idle.c | 20 ++++++++++++- xen/arch/x86/include/asm/hardirq.h | 14 ++++++++- xen/arch/x86/include/asm/softirq.h | 48 ++++++++++++++++++++++++++++++ 3 files changed, 80 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c index 4f69df5a7438..c5d7a6c6fe2a 100644 --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -443,7 +443,21 @@ __initcall(cpu_idle_key_init); void mwait_idle_with_hints(unsigned int eax, unsigned int ecx) { unsigned int cpu =3D smp_processor_id(); - const unsigned int *this_softirq_pending =3D &softirq_pending(cpu); + irq_cpustat_t *stat =3D &irq_stat[cpu]; + const unsigned int *this_softirq_pending =3D &stat->__softirq_pending; + + /* + * By setting in_mwait, we promise to other CPUs that we'll notice cha= nges + * to __softirq_pending without being sent an IPI. We achieve this by + * either not going to sleep, or by having hardware notice on our beha= lf. + * + * Some errata exist where MONITOR doesn't work properly, and the + * workaround is to force the use of an IPI. Cause this to happen by + * simply not advertising ourselves as being in_mwait. + */ + alternative_io("movb $1, %[in_mwait]", + "", X86_BUG_MONITOR, + [in_mwait] "=3Dm" (stat->in_mwait)); =20 monitor(this_softirq_pending, 0, 0); =20 @@ -455,6 +469,10 @@ void mwait_idle_with_hints(unsigned int eax, unsigned = int ecx) mwait(eax, ecx); spec_ctrl_exit_idle(info); } + + alternative_io("movb $0, %[in_mwait]", + "", X86_BUG_MONITOR, + [in_mwait] "=3Dm" (stat->in_mwait)); } =20 static void acpi_processor_ffh_cstate_enter(struct acpi_processor_cx *cx) diff --git a/xen/arch/x86/include/asm/hardirq.h b/xen/arch/x86/include/asm/= hardirq.h index f3e93cc9b507..1647cff04dc8 100644 --- a/xen/arch/x86/include/asm/hardirq.h +++ b/xen/arch/x86/include/asm/hardirq.h @@ -5,7 +5,19 @@ #include =20 typedef struct { - unsigned int __softirq_pending; + /* + * The layout is important. Any CPU can set bits in __softirq_pending, + * but in_mwait is a status bit owned by the CPU. softirq_mwait_raw m= ust + * cover both, and must be in a single cacheline. + */ + union { + struct { + unsigned int __softirq_pending; + bool in_mwait; + }; + uint64_t softirq_mwait_raw; + }; + unsigned int __local_irq_count; unsigned int nmi_count; unsigned int mce_count; diff --git a/xen/arch/x86/include/asm/softirq.h b/xen/arch/x86/include/asm/= softirq.h index e4b194f069fb..55b65c9747b1 100644 --- a/xen/arch/x86/include/asm/softirq.h +++ b/xen/arch/x86/include/asm/softirq.h @@ -1,6 +1,8 @@ #ifndef __ASM_SOFTIRQ_H__ #define __ASM_SOFTIRQ_H__ =20 +#include + #define NMI_SOFTIRQ (NR_COMMON_SOFTIRQS + 0) #define TIME_CALIBRATE_SOFTIRQ (NR_COMMON_SOFTIRQS + 1) #define VCPU_KICK_SOFTIRQ (NR_COMMON_SOFTIRQS + 2) @@ -9,4 +11,50 @@ #define HVM_DPCI_SOFTIRQ (NR_COMMON_SOFTIRQS + 4) #define NR_ARCH_SOFTIRQS 5 =20 +/* + * Ensure softirq @nr is pending on @cpu. Return true if an IPI can be + * skipped, false if the IPI cannot be skipped. + * + * We use a CMPXCHG covering both __softirq_pending and in_mwait, in order= to + * set softirq @nr while also observing in_mwait in a race-free way. + */ +static always_inline bool arch_set_softirq(unsigned int nr, unsigned int c= pu) +{ + uint64_t *ptr =3D &irq_stat[cpu].softirq_mwait_raw; + uint64_t prev, old, new; + unsigned int softirq =3D 1U << nr; + + old =3D ACCESS_ONCE(*ptr); + + for ( ;; ) + { + if ( old & softirq ) + /* Softirq already pending, nothing to do. */ + return true; + + new =3D old | softirq; + + prev =3D cmpxchg(ptr, old, new); + if ( prev =3D=3D old ) + break; + + old =3D prev; + } + + /* + * We have caused the softirq to become pending. If in_mwait was set,= the + * target CPU will notice the modification and act on it. + * + * We can't access the in_mwait field nicely, so use some BUILD_BUG_ON= ()'s + * to cross-check the (1UL << 32) opencoding. + */ + BUILD_BUG_ON(sizeof(irq_stat[0].softirq_mwait_raw) !=3D 8); + BUILD_BUG_ON((offsetof(irq_cpustat_t, in_mwait) - + offsetof(irq_cpustat_t, softirq_mwait_raw)) !=3D 4); + + return new & (1UL << 32) /* in_mwait */; + +} +#define arch_set_softirq arch_set_softirq + #endif /* __ASM_SOFTIRQ_H__ */ --=20 2.39.5 From nobody Thu Oct 30 23:14:03 2025 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1751646884; cv=none; d=zohomail.com; s=zohoarc; b=Ex1Zgv1lU17Bprn9cypvZn8uZvnA9HAqmz6xxaDRdl6+bzsaxhv+DvxLBZw8dX4ADcaWzdKaOtcSaUagptLhndtJYWotp4hFWinkXyxcGILIElMKDu/qBYS7UwI95gw1H1IXf0geEZBtu1IWLSYHe/UqGied26cap3LUUnO2Omk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1751646884; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=YO+Mvsl7T4wCFNFbuv0DlhNSF3ioM+Cmaq7CZFYbXdE=; b=BeFYIhs8Z1PsNhSH69EuYFf1YqCHt1ABrVu0mbNd7Bxw4yfR3dwO1TyVA88iIkkGa0VlH4BolfgXe3C1KUYEol12nFQ3gplPCOqz83OzXng+phuzOveNJDBLblxkkIFbjEErD3T0llavJruV8ASxuwwZ7fQ9bfELvnk0zMkH8A8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1751646884355939.7351935052034; Fri, 4 Jul 2025 09:34:44 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.1033544.1406923 (Exim 4.92) (envelope-from ) id 1uXjMk-0006Vl-V6; Fri, 04 Jul 2025 16:34:22 +0000 Received: by outflank-mailman (output) from mailman id 1033544.1406923; Fri, 04 Jul 2025 16:34:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uXjMk-0006Tz-Np; Fri, 04 Jul 2025 16:34:22 +0000 Received: by outflank-mailman (input) for mailman id 1033544; Fri, 04 Jul 2025 16:34:20 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1uXjMi-0005FA-Pd for xen-devel@lists.xenproject.org; Fri, 04 Jul 2025 16:34:20 +0000 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [2a00:1450:4864:20::331]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id bb1b1a91-58f4-11f0-b894-0df219b8e170; Fri, 04 Jul 2025 18:34:18 +0200 (CEST) Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-454ac069223so11402595e9.1 for ; Fri, 04 Jul 2025 09:34:18 -0700 (PDT) Received: from localhost.localdomain (host-92-26-98-202.as13285.net. [92.26.98.202]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-454a9989fcesm59199345e9.16.2025.07.04.09.34.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Jul 2025 09:34:17 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: bb1b1a91-58f4-11f0-b894-0df219b8e170 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1751646858; x=1752251658; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YO+Mvsl7T4wCFNFbuv0DlhNSF3ioM+Cmaq7CZFYbXdE=; b=tMNWO0DfFlXx2U0P/0b1rThrfB8ZZmkY5r1eaBJZ1VDIKmPwXL73LqGGxAP/dWFX/p CPurcNZsbPHoYrVIssPHx9sYFz4JGTG5SRLOC6mNB5gZlToyY3q2ZW5z9YtFwwaciQE6 er+BZ5VsMCtgPQTA4VwKJuuHCoIqH7wc1I5To= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751646858; x=1752251658; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YO+Mvsl7T4wCFNFbuv0DlhNSF3ioM+Cmaq7CZFYbXdE=; b=GPdBZt/VzYajvApE00MMTq3y8Ysa5PtitHLWxi0p6ViJKoL8u2Sm6oAR40LqbEtktw 7Kj89KGLG/LqV9KYz9iWtJFgJoLkOFfwNhPof2gEUMlp6WlOZKckT4FynJ0pGRmQk1pR 9+64Z6VTd/Mmwoqpww9AyPviuj70RT/5hliEbvO7RsVVXQQtnNOsVjX3WcvxGcfIZSwE IFzw/+tLrOUi3XN9ODC7NbP9gaIGItDoeoPW6uIX0ChQ6W6N1OP80oB9/fZZFGP5csrl TreS4S1C/rj9fv3SYuleIvhnkm5AOZQR9gO92D7LJT7vV1gqGV7B8KTd0cI0haSZZ2OS TsgQ== X-Gm-Message-State: AOJu0Yzw5GYePPJVpj6+6NZDoWrH3nIUyFjOT9/J1XrZPrV5T9H+s9bC gTqOLC2H/Xandhs6//a+o4tE9p1uD7oPIJ4dgVyIl2LD/G0BgzdTiUNBjHSwqVu8bz01w4PR8TA gigqaxmFehg== X-Gm-Gg: ASbGncszI+tL9I06bqDQV66J1IDDBE6J833BtXq8FwAvgeKX8FGrr5sED2MWvxgKFJF 87Xhy5txXVuXDpQuut30PhuAJKyAkJ1b4qTOcOqXF/bQW34uHcvUUM7jXrGfAYAy1/9TmSVpcYU MOMEX8byXiBqHbdQfbF/40nqkpNGNgPdZn3uD0af0UZv2By587dZXXoSGxzHV0JMsdeUna/MSbD 9entOm7WfpMn6doVUqNVadCQw0Ky4whdZkulXIMnnM5YZrggXXKf/fK2baalWECQmbyxbOYFzTl fQ2BPIet+xlTH5V3qbD31vEUcp49kSl97JQLoPG4xzXHL6O6j+NOTyzBUhsIoY9350xkjQ1V+3y Bg+lwPMv39eJ1J01Vfw99woVcdH0r/7b+z9c= X-Google-Smtp-Source: AGHT+IE2d/OampR6SLtstlBq6hmveCWj/rNyeENkROaUbFT4K4SfTi1e0fXMFWyVMIv6wvHs+240Ew== X-Received: by 2002:a05:600c:1da0:b0:453:835a:db with SMTP id 5b1f17b1804b1-454b1f471fdmr32877685e9.4.1751646857639; Fri, 04 Jul 2025 09:34:17 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Anthony PERARD , Michal Orzel , Julien Grall , Stefano Stabellini Subject: [PATCH v2 6/6] x86/idle: Fix buggy "x86/mwait-idle: enable interrupts before C1 on Xeons" Date: Fri, 4 Jul 2025 17:34:10 +0100 Message-Id: <20250704163410.1412367-7-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250704163410.1412367-1-andrew.cooper3@citrix.com> References: <20250704163410.1412367-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) X-ZM-MESSAGEID: 1751646884995116600 The check of this_softirq_pending must be performed with irqs disabled, but this property was broken by an attempt to optimise entry/exit latency. Commit c227233ad64c in Linux (which we copied into Xen) was fixed up by edc8fc01f608 in Linux, which we have so far missed. Going to sleep without waking on interrupts is nonsensical outside of play_dead(), so overload this to select between two possible MWAITs, the second using the STI shadow to cover MWAIT for exactly the same reason as we do in safe_halt(). Fixes: b17e0ec72ede ("x86/mwait-idle: enable interrupts before C1 on Xeons") Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Anthony PERARD CC: Michal Orzel CC: Julien Grall CC: Stefano Stabellini --- xen/arch/x86/acpi/cpu_idle.c | 16 +++++++++++++++- xen/arch/x86/cpu/mwait-idle.c | 8 ++------ 2 files changed, 17 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c index c5d7a6c6fe2a..e50a9234a0d4 100644 --- a/xen/arch/x86/acpi/cpu_idle.c +++ b/xen/arch/x86/acpi/cpu_idle.c @@ -80,6 +80,13 @@ static always_inline void mwait(unsigned int eax, unsign= ed int ecx) :: "a" (eax), "c" (ecx) ); } =20 +static always_inline void sti_mwait_cli(unsigned int eax, unsigned int ecx) +{ + /* STI shadow covers MWAIT. */ + asm volatile ( "sti; mwait; cli" + :: "a" (eax), "c" (ecx) ); +} + #define GET_HW_RES_IN_NS(msr, val) \ do { rdmsrl(msr, val); val =3D tsc_ticks2ns(val); } while( 0 ) #define GET_MC6_RES(val) GET_HW_RES_IN_NS(0x664, val) @@ -461,12 +468,19 @@ void mwait_idle_with_hints(unsigned int eax, unsigned= int ecx) =20 monitor(this_softirq_pending, 0, 0); =20 + ASSERT(!local_irq_is_enabled()); + if ( !*this_softirq_pending ) { struct cpu_info *info =3D get_cpu_info(); =20 spec_ctrl_enter_idle(info); - mwait(eax, ecx); + + if ( ecx & MWAIT_ECX_INTERRUPT_BREAK ) + mwait(eax, ecx); + else + sti_mwait_cli(eax, ecx); + spec_ctrl_exit_idle(info); } =20 diff --git a/xen/arch/x86/cpu/mwait-idle.c b/xen/arch/x86/cpu/mwait-idle.c index 5c16f5ad3a82..5e98011bfd0c 100644 --- a/xen/arch/x86/cpu/mwait-idle.c +++ b/xen/arch/x86/cpu/mwait-idle.c @@ -946,12 +946,8 @@ static void cf_check mwait_idle(void) =20 update_last_cx_stat(power, cx, before); =20 - if (cx->irq_enable_early) - local_irq_enable(); - - mwait_idle_with_hints(cx->address, MWAIT_ECX_INTERRUPT_BREAK); - - local_irq_disable(); + mwait_idle_with_hints(cx->address, + cx->irq_enable_early ? 0 : MWAIT_ECX_INTERRUPT_BREAK); =20 after =3D alternative_call(cpuidle_get_tick); =20 --=20 2.39.5