From nobody Tue Dec 16 14:36:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F6A31C68B4; Wed, 29 May 2024 20:28:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717014516; cv=none; b=bZLX9mNgaV2gNzx0VjxuQpdZ0CG1aiOPfAgMwApAe8WT85Vhc6YOi2+E/b3urhcgqzZYMbGAql73iUD65Gy6kKaqwHlhpGXtWSvIml1bsaYfSs5b/pmWuJHrsZeR+ovndAQflzqfbXa4RWIQFr8tQqvaLJT9kxo6MKhWsNh3kJg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717014516; c=relaxed/simple; bh=dSqou1KOvj6VEChE+4XJ6rK/xivwkSVK7t0PEao7PtQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=N8+NvCpBXr2Yqx3P9VQ00EJLpgcbKS8iFi1PgUGtOe/xOV8TeAaSNKiOIoWWLBuBPKCj0i3jukE4xcrHrk9PuWeFLSYBaQN+H9szSipE3tjTB53TtXQ0FmIasC8k9G8GhgAcaNKgcihIplVvEoHoRwZIBLDDCh2z4Pp/C851wFk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=WvmBMDnE; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WvmBMDnE" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717014515; x=1748550515; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dSqou1KOvj6VEChE+4XJ6rK/xivwkSVK7t0PEao7PtQ=; b=WvmBMDnExqAbG9WrhC/lfNvA7d9xkqVCnkfd6r4qj8pHhyv8GiuIRKoA fqh003oC3x0tdCnOHMO3ep107qalpo38TS9xMij6I7q1/b3+cOoq86GcL 8mhu0y6EVtC0mNCF89cSFs39Okf/SR2PwEO3dFQrSwoQKGPG6+87yPPeF QCKhNMzpoVriLer8reyeNpFlp51oB2gp5HelAKyTje7atXk27faK8vc+e htgeZPUZySWvqqfweE3JJlSL/8o1VhYtXKY6eFuTEzkKPU+0Y6qxnvrzr B/jEn3cXYMzQw5OgpmkAgAZz3tUgu2QXxBvMHBynnSnhbcs3gwhHb70es w==; X-CSE-ConnectionGUID: fsgGmAEdQSiUq9r1flJNJQ== X-CSE-MsgGUID: bM9p2GDxT1+OEM6sZoyNaQ== X-IronPort-AV: E=McAfee;i="6600,9927,11087"; a="13574527" X-IronPort-AV: E=Sophos;i="6.08,199,1712646000"; d="scan'208";a="13574527" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 May 2024 13:28:32 -0700 X-CSE-ConnectionGUID: CoBr3p4CRT2E4CYboQLtWA== X-CSE-MsgGUID: Ln1ZCyiKTjO2S3SDc3NfrA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,199,1712646000"; d="scan'208";a="35491264" Received: from jacob-builder.jf.intel.com ([10.54.39.125]) by fmviesa007.fm.intel.com with ESMTP; 29 May 2024 13:28:32 -0700 From: Jacob Pan To: X86 Kernel , LKML , Thomas Gleixner , Dave Hansen , "H. Peter Anvin" , "Ingo Molnar" , "Borislav Petkov" , linux-perf-users@vger.kernel.org, Peter Zijlstra Cc: Andi Kleen , "Xin Li" , Jacob Pan Subject: [PATCH 1/6] x86/irq: Add enumeration of NMI source reporting CPU feature Date: Wed, 29 May 2024 13:33:20 -0700 Message-Id: <20240529203325.3039243-2-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240529203325.3039243-1-jacob.jun.pan@linux.intel.com> References: <20240529203325.3039243-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The lack of a mechanism to pinpoint the origins of Non-Maskable Interrupts (NMIs) necessitates that the NMI vector 2 handler consults each NMI source handler individually. This approach leads to inefficiencies, delays, and the occurrence of unnecessary NMIs, thereby also constraining the potential applications of NMIs. A new CPU feature, known as NMI source reporting, has been introduced as part of the Flexible Return and Event Delivery (FRED) spec. This feature enables the NMI vector 2 handler to directly obtain information about the NMI source from the FRED event data. The functionality of NMI source reporting is tied to the FRED. Although it is enumerated by a unique CPUID feature bit, it cannot be turned off independently once FRED is activated. Signed-off-by: Jacob Pan --- arch/x86/Kconfig | 9 +++++++++ arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/disabled-features.h | 8 +++++++- arch/x86/kernel/cpu/cpuid-deps.c | 1 + arch/x86/kernel/traps.c | 4 +++- 5 files changed, 21 insertions(+), 2 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 1d7122a1883e..b8b15f20b94e 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -511,12 +511,21 @@ config X86_CPU_RESCTRL config X86_FRED bool "Flexible Return and Event Delivery" depends on X86_64 + select X86_NMI_SOURCE help When enabled, try to use Flexible Return and Event Delivery instead of the legacy SYSCALL/SYSENTER/IDT architecture for ring transitions and exception/interrupt handling if the system supports it. =20 +config X86_NMI_SOURCE + def_bool n + help + Once enabled, information on NMI originator/source can be provided + via FRED event data. This makes NMI processing more efficient in that + NMI handler does not need to check for every possible source at + runtime when NMI is delivered. + config X86_BIGSMP bool "Support for big SMP systems with more than 8 CPUs" depends on SMP && X86_32 diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpuf= eatures.h index 3c7434329661..ec78d361e685 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -327,6 +327,7 @@ #define X86_FEATURE_FRED (12*32+17) /* Flexible Return and Event Delivery= */ #define X86_FEATURE_LKGS (12*32+18) /* "" Load "kernel" (userspace) GS */ #define X86_FEATURE_WRMSRNS (12*32+19) /* "" Non-serializing WRMSR */ +#define X86_FEATURE_NMI_SOURCE (12*32+20) /* NMI source reporting */ #define X86_FEATURE_AMX_FP16 (12*32+21) /* "" AMX fp16 Support */ #define X86_FEATURE_AVX_IFMA (12*32+23) /* "" Support for VPMAD= D52[H,L]UQ */ #define X86_FEATURE_LAM (12*32+26) /* Linear Address Masking */ diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/as= m/disabled-features.h index c492bdc97b05..3856c4737d65 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -123,6 +123,12 @@ # define DISABLE_FRED (1 << (X86_FEATURE_FRED & 31)) #endif =20 +#ifdef CONFIG_X86_NMI_SOURCE +# define DISABLE_NMI_SOURCE 0 +#else +# define DISABLE_NMI_SOURCE (1 << (X86_FEATURE_NMI_SOURCE & 31)) +#endif + #ifdef CONFIG_KVM_AMD_SEV #define DISABLE_SEV_SNP 0 #else @@ -145,7 +151,7 @@ #define DISABLED_MASK10 0 #define DISABLED_MASK11 (DISABLE_RETPOLINE|DISABLE_RETHUNK|DISABLE_UNRET| \ DISABLE_CALL_DEPTH_TRACKING|DISABLE_USER_SHSTK) -#define DISABLED_MASK12 (DISABLE_FRED|DISABLE_LAM) +#define DISABLED_MASK12 (DISABLE_FRED|DISABLE_LAM|DISABLE_NMI_SOURCE) #define DISABLED_MASK13 0 #define DISABLED_MASK14 0 #define DISABLED_MASK15 0 diff --git a/arch/x86/kernel/cpu/cpuid-deps.c b/arch/x86/kernel/cpu/cpuid-d= eps.c index b7d9f530ae16..3f1a1a1961fa 100644 --- a/arch/x86/kernel/cpu/cpuid-deps.c +++ b/arch/x86/kernel/cpu/cpuid-deps.c @@ -84,6 +84,7 @@ static const struct cpuid_dep cpuid_deps[] =3D { { X86_FEATURE_SHSTK, X86_FEATURE_XSAVES }, { X86_FEATURE_FRED, X86_FEATURE_LKGS }, { X86_FEATURE_FRED, X86_FEATURE_WRMSRNS }, + { X86_FEATURE_FRED, X86_FEATURE_NMI_SOURCE}, {} }; =20 diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 4fa0b17e5043..465f04e4a79f 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1427,8 +1427,10 @@ early_param("fred", fred_setup); =20 void __init trap_init(void) { - if (cpu_feature_enabled(X86_FEATURE_FRED) && !enable_fred) + if (cpu_feature_enabled(X86_FEATURE_FRED) && !enable_fred) { setup_clear_cpu_cap(X86_FEATURE_FRED); + setup_clear_cpu_cap(X86_FEATURE_NMI_SOURCE); + } =20 /* Init cpu_entry_area before IST entries are set up */ setup_cpu_entry_areas(); --=20 2.25.1 From nobody Tue Dec 16 14:36:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3AD601C68AD; Wed, 29 May 2024 20:28:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717014517; cv=none; b=gqu39cP2UTMtlSF9mLFd/9uRHGYmGAdfEXXDaQ7871/hfe0a2Y4RFwcKojOYi55b3dhg/wDQWHIJv+BLH865hKi/tY2FHoy2SdS/6WRy/utsok5iPFb2ciyoXga1bd1I8TY5HJUu92USD7YVQdZkgs1Y14a8oNxVYISfu0YPzr0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717014517; c=relaxed/simple; bh=7TFhy41O3STQzIxoLjTsrqMb6t2iMacBEvka8LKbO+o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pzZveinAm8mHMxUf/EZvnVEtO/yMUC+9KYXOfav6YVo40G21cKEyEzUPmmL6lxUGqs8lTqYU4j3vlR3R92wLGW/c/EmoFblMni2CqXi/iu7utTZJgDXbd2EbDNGk0Wc21brQOj+MObbyuAxApgludaweD3Gde1ZwCYm8+ZXlYa0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=kda1ebIq; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="kda1ebIq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717014515; x=1748550515; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7TFhy41O3STQzIxoLjTsrqMb6t2iMacBEvka8LKbO+o=; b=kda1ebIqNSIw7BtnhRTLouq4qDqUKrP7gF2hGCu+jTkMbouuTt4FD31H fJhdaAwnWTtQ8TFfpxvibIFIrGuGhPe6X0SMpcXwbxOrf4ylXrh3VrN0P hltkJXVhceFyrMX95nX3JiafE05usbj306pf+UlegCzqQJJoIXDZonHOy jFf+kd3gLBSN6c8MUbi8IKLjc3KqyKgf0MCVt4SYAmSSi0QG/FaBaLTF4 2NUzRahqaXHeSnpg7pJHUr4DF1B93wrMC7R9JS9/kgsN4SpCV+Q3UVmzX SlOSoJBVJm7HJJVnhGxEJm4NDLFCa/eLU694z/ogEQVgFRuwfq6MQrSvj A==; X-CSE-ConnectionGUID: GQqHLB56SOiqJuYZW/RFCQ== X-CSE-MsgGUID: 7Thlda8SQf+6CNXr9kDjBg== X-IronPort-AV: E=McAfee;i="6600,9927,11087"; a="13574536" X-IronPort-AV: E=Sophos;i="6.08,199,1712646000"; d="scan'208";a="13574536" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 May 2024 13:28:33 -0700 X-CSE-ConnectionGUID: 9Twz8PXJQOWu3QFzWwgloA== X-CSE-MsgGUID: 76iEZdFZRFCKdtybyDLAhA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,199,1712646000"; d="scan'208";a="35491267" Received: from jacob-builder.jf.intel.com ([10.54.39.125]) by fmviesa007.fm.intel.com with ESMTP; 29 May 2024 13:28:33 -0700 From: Jacob Pan To: X86 Kernel , LKML , Thomas Gleixner , Dave Hansen , "H. Peter Anvin" , "Ingo Molnar" , "Borislav Petkov" , linux-perf-users@vger.kernel.org, Peter Zijlstra Cc: Andi Kleen , "Xin Li" , Jacob Pan Subject: [PATCH 2/6] x86/irq: Extend NMI handler registration interface to include source Date: Wed, 29 May 2024 13:33:21 -0700 Message-Id: <20240529203325.3039243-3-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240529203325.3039243-1-jacob.jun.pan@linux.intel.com> References: <20240529203325.3039243-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a source vector argument to register_nmi_handler() such that designated NMI originators can leverage NMI source reporting feature. For those who do not use NMI source reporting, 0 (unknown) is used as the source vector. = NMI source vectors (up to 16) are pre-defined. Signed-off-by: Jacob Pan --- arch/x86/events/amd/ibs.c | 2 +- arch/x86/events/core.c | 3 ++- arch/x86/include/asm/irq_vectors.h | 21 +++++++++++++++++++++ arch/x86/include/asm/nmi.h | 4 +++- arch/x86/kernel/apic/hw_nmi.c | 2 +- arch/x86/kernel/cpu/mce/inject.c | 2 +- arch/x86/kernel/cpu/mshyperv.c | 2 +- arch/x86/kernel/kgdb.c | 4 ++-- arch/x86/kernel/nmi.c | 16 ++++++++++++++++ arch/x86/kernel/nmi_selftest.c | 5 +++-- arch/x86/kernel/reboot.c | 2 +- arch/x86/kernel/smp.c | 2 +- arch/x86/platform/uv/uv_nmi.c | 4 ++-- drivers/acpi/apei/ghes.c | 2 +- drivers/char/ipmi/ipmi_watchdog.c | 2 +- drivers/edac/igen6_edac.c | 2 +- drivers/watchdog/hpwdt.c | 6 +++--- 17 files changed, 61 insertions(+), 20 deletions(-) diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index e91970b01d62..20989071f59a 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -1246,7 +1246,7 @@ static __init int perf_event_ibs_init(void) if (ret) goto err_op; =20 - ret =3D register_nmi_handler(NMI_LOCAL, perf_ibs_nmi_handler, 0, "perf_ib= s"); + ret =3D register_nmi_handler(NMI_LOCAL, perf_ibs_nmi_handler, 0, "perf_ib= s", 0); if (ret) goto err_nmi; =20 diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 5b0dd07b1ef1..1ef2201e48ac 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -2100,7 +2100,8 @@ static int __init init_hw_perf_events(void) x86_pmu.intel_ctrl =3D (1 << x86_pmu.num_counters) - 1; =20 perf_events_lapic_init(); - register_nmi_handler(NMI_LOCAL, perf_event_nmi_handler, 0, "PMI"); + + register_nmi_handler(NMI_LOCAL, perf_event_nmi_handler, 0, "PMI", NMI_SOU= RCE_VEC_PMI); =20 unconstrained =3D (struct event_constraint) __EVENT_CONSTRAINT(0, (1ULL << x86_pmu.num_counters) - 1, diff --git a/arch/x86/include/asm/irq_vectors.h b/arch/x86/include/asm/irq_= vectors.h index 13aea8fc3d45..b8388bc00cde 100644 --- a/arch/x86/include/asm/irq_vectors.h +++ b/arch/x86/include/asm/irq_vectors.h @@ -105,6 +105,27 @@ =20 #define NR_VECTORS 256 =20 +/* + * The NMI senders specify the NMI source vector as an 8bit integer in the= ir + * vector field with NMI delivery mode. A local APIC receiving an NMI will + * set the corresponding bit in a 16bit bitmask, which is accumulated until + * the NMI is delivered. + * When a sender didn't specify an NMI source vector the source vector will + * be 0, which will result in bit 0 of the bitmask being set. For out of + * bounds vectors >=3D 16 bit 0 will also be set. + * When bit 0 is set, system software must invoke all registered NMI handl= ers + * as if NMI source feature is not enabled. + */ +#define NMI_SOURCE_VEC_UNKNOWN 0 +#define NMI_SOURCE_VEC_PMI 1 /* PerfMon counters */ +#define NMI_SOURCE_VEC_IPI_BT 2 /* CPU backtrace */ +#define NMI_SOURCE_VEC_IPI_MCE 3 /* MCE injection */ +#define NMI_SOURCE_VEC_IPI_KGDB 4 +#define NMI_SOURCE_VEC_IPI_REBOOT 5 /* Crash reboot */ +#define NMI_SOURCE_VEC_IPI_SMP_STOP 6 /* Panic stop CPU */ +#define NMI_SOURCE_VEC_IPI_TEST 7 /* For remote and local IPIs*/ +#define NR_NMI_SOURCE_VECTORS 8 + #ifdef CONFIG_X86_LOCAL_APIC #define FIRST_SYSTEM_VECTOR POSTED_MSI_NOTIFICATION_VECTOR #else diff --git a/arch/x86/include/asm/nmi.h b/arch/x86/include/asm/nmi.h index 41a0ebb699ec..6fe26fea30eb 100644 --- a/arch/x86/include/asm/nmi.h +++ b/arch/x86/include/asm/nmi.h @@ -39,15 +39,17 @@ struct nmiaction { u64 max_duration; unsigned long flags; const char *name; + unsigned int source_vec; }; =20 -#define register_nmi_handler(t, fn, fg, n, init...) \ +#define register_nmi_handler(t, fn, fg, n, src, init...) \ ({ \ static struct nmiaction init fn##_na =3D { \ .list =3D LIST_HEAD_INIT(fn##_na.list), \ .handler =3D (fn), \ .name =3D (n), \ .flags =3D (fg), \ + .source_vec =3D (src), \ }; \ __register_nmi_handler((t), &fn##_na); \ }) diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c index 45af535c44a0..9f0125d3b8b0 100644 --- a/arch/x86/kernel/apic/hw_nmi.c +++ b/arch/x86/kernel/apic/hw_nmi.c @@ -54,7 +54,7 @@ NOKPROBE_SYMBOL(nmi_cpu_backtrace_handler); static int __init register_nmi_cpu_backtrace_handler(void) { register_nmi_handler(NMI_LOCAL, nmi_cpu_backtrace_handler, - 0, "arch_bt"); + 0, "arch_bt", NMI_SOURCE_VEC_IPI_BT); return 0; } early_initcall(register_nmi_cpu_backtrace_handler); diff --git a/arch/x86/kernel/cpu/mce/inject.c b/arch/x86/kernel/cpu/mce/inj= ect.c index 94953d749475..365a03f11d06 100644 --- a/arch/x86/kernel/cpu/mce/inject.c +++ b/arch/x86/kernel/cpu/mce/inject.c @@ -769,7 +769,7 @@ static int __init inject_init(void) =20 debugfs_init(); =20 - register_nmi_handler(NMI_LOCAL, mce_raise_notify, 0, "mce_notify"); + register_nmi_handler(NMI_LOCAL, mce_raise_notify, 0, "mce_notify", NMI_SO= URCE_VEC_IPI_MCE); mce_register_injector_chain(&inject_nb); =20 setup_inj_struct(&i_mce); diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index e0fd57a8ba84..2fb9408a8ba9 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -486,7 +486,7 @@ static void __init ms_hyperv_init_platform(void) } =20 register_nmi_handler(NMI_UNKNOWN, hv_nmi_unknown, NMI_FLAG_FIRST, - "hv_nmi_unknown"); + "hv_nmi_unknown", 0); #endif =20 #ifdef CONFIG_X86_IO_APIC diff --git a/arch/x86/kernel/kgdb.c b/arch/x86/kernel/kgdb.c index 9c9faa1634fb..d167eb23cf13 100644 --- a/arch/x86/kernel/kgdb.c +++ b/arch/x86/kernel/kgdb.c @@ -603,12 +603,12 @@ int kgdb_arch_init(void) goto out; =20 retval =3D register_nmi_handler(NMI_LOCAL, kgdb_nmi_handler, - 0, "kgdb"); + 0, "kgdb", NMI_SOURCE_VEC_IPI_KGDB); if (retval) goto out1; =20 retval =3D register_nmi_handler(NMI_UNKNOWN, kgdb_nmi_handler, - 0, "kgdb"); + 0, "kgdb", 0); =20 if (retval) goto out2; diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index ed163c8c8604..1ff4f7c9f182 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -86,6 +86,12 @@ static DEFINE_PER_CPU(struct nmi_stats, nmi_stats); =20 static int ignore_nmis __read_mostly; =20 +/* + * Contains all actions registered by originators with source vector, + * excluding UNKNOWN NMI source vector 0. + */ +static struct nmiaction *nmiaction_src_table[NR_NMI_SOURCE_VECTORS - 1]; + int unknown_nmi_panic; /* * Prevent NMI reason port (0x61) being accessed simultaneously, can @@ -163,6 +169,12 @@ static int nmi_handle(unsigned int type, struct pt_reg= s *regs) } NOKPROBE_SYMBOL(nmi_handle); =20 +static inline bool use_nmi_source(unsigned int type, struct nmiaction *a) +{ + return (cpu_feature_enabled(X86_FEATURE_NMI_SOURCE) && + type =3D=3D NMI_LOCAL && a->source_vec); +} + int __register_nmi_handler(unsigned int type, struct nmiaction *action) { struct nmi_desc *desc =3D nmi_to_desc(type); @@ -173,6 +185,8 @@ int __register_nmi_handler(unsigned int type, struct nm= iaction *action) =20 raw_spin_lock_irqsave(&desc->lock, flags); =20 + if (use_nmi_source(type, action)) + rcu_assign_pointer(nmiaction_src_table[action->source_vec], action); /* * Indicate if there are multiple registrations on the * internal NMI handler call chains (SERR and IO_CHECK). @@ -210,6 +224,8 @@ void unregister_nmi_handler(unsigned int type, const ch= ar *name) if (!strcmp(n->name, name)) { WARN(in_nmi(), "Trying to free NMI (%s) from NMI context!\n", n->name); + if (use_nmi_source(type, n)) + rcu_assign_pointer(nmiaction_src_table[n->source_vec], NULL); list_del_rcu(&n->list); found =3D n; break; diff --git a/arch/x86/kernel/nmi_selftest.c b/arch/x86/kernel/nmi_selftest.c index e93a8545c74d..f014c8a66b0c 100644 --- a/arch/x86/kernel/nmi_selftest.c +++ b/arch/x86/kernel/nmi_selftest.c @@ -44,7 +44,7 @@ static void __init init_nmi_testsuite(void) { /* trap all the unknown NMIs we may generate */ register_nmi_handler(NMI_UNKNOWN, nmi_unk_cb, 0, "nmi_selftest_unk", - __initdata); + 0, __initdata); } =20 static void __init cleanup_nmi_testsuite(void) @@ -67,7 +67,8 @@ static void __init test_nmi_ipi(struct cpumask *mask) unsigned long timeout; =20 if (register_nmi_handler(NMI_LOCAL, test_nmi_ipi_callback, - NMI_FLAG_FIRST, "nmi_selftest", __initdata)) { + NMI_FLAG_FIRST, "nmi_selftest", NMI_SOURCE_VEC_IPI_TEST, + __initdata)) { nmi_fail =3D FAILURE; return; } diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index f3130f762784..acc19c1d3b4f 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -910,7 +910,7 @@ void nmi_shootdown_cpus(nmi_shootdown_cb callback) atomic_set(&waiting_for_crash_ipi, num_online_cpus() - 1); /* Would it be better to replace the trap vector here? */ if (register_nmi_handler(NMI_LOCAL, crash_nmi_callback, - NMI_FLAG_FIRST, "crash")) + NMI_FLAG_FIRST, "crash", NMI_SOURCE_VEC_IPI_REBOOT)) return; /* Return what? */ /* * Ensure the new callback function is set before sending diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c index 18266cc3d98c..f27469e40141 100644 --- a/arch/x86/kernel/smp.c +++ b/arch/x86/kernel/smp.c @@ -143,7 +143,7 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_reboot) static int register_stop_handler(void) { return register_nmi_handler(NMI_LOCAL, smp_stop_nmi_callback, - NMI_FLAG_FIRST, "smp_stop"); + NMI_FLAG_FIRST, "smp_stop", NMI_SOURCE_VEC_IPI_SMP_STOP); } =20 static void native_stop_other_cpus(int wait) diff --git a/arch/x86/platform/uv/uv_nmi.c b/arch/x86/platform/uv/uv_nmi.c index 5c50e550ab63..473c34eb264c 100644 --- a/arch/x86/platform/uv/uv_nmi.c +++ b/arch/x86/platform/uv/uv_nmi.c @@ -1029,10 +1029,10 @@ static int uv_handle_nmi_ping(unsigned int reason, = struct pt_regs *regs) =20 static void uv_register_nmi_notifier(void) { - if (register_nmi_handler(NMI_UNKNOWN, uv_handle_nmi, 0, "uv")) + if (register_nmi_handler(NMI_UNKNOWN, uv_handle_nmi, 0, "uv", 0)) pr_warn("UV: NMI handler failed to register\n"); =20 - if (register_nmi_handler(NMI_LOCAL, uv_handle_nmi_ping, 0, "uvping")) + if (register_nmi_handler(NMI_LOCAL, uv_handle_nmi_ping, 0, "uvping", 0)) pr_warn("UV: PING NMI handler failed to register\n"); } =20 diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c index 623cc0cb4a65..393dca95d2b3 100644 --- a/drivers/acpi/apei/ghes.c +++ b/drivers/acpi/apei/ghes.c @@ -1318,7 +1318,7 @@ static void ghes_nmi_add(struct ghes *ghes) { mutex_lock(&ghes_list_mutex); if (list_empty(&ghes_nmi)) - register_nmi_handler(NMI_LOCAL, ghes_notify_nmi, 0, "ghes"); + register_nmi_handler(NMI_LOCAL, ghes_notify_nmi, 0, "ghes", 0); list_add_rcu(&ghes->list, &ghes_nmi); mutex_unlock(&ghes_list_mutex); } diff --git a/drivers/char/ipmi/ipmi_watchdog.c b/drivers/char/ipmi/ipmi_wat= chdog.c index 9a459257489f..61bb5dcade5a 100644 --- a/drivers/char/ipmi/ipmi_watchdog.c +++ b/drivers/char/ipmi/ipmi_watchdog.c @@ -1272,7 +1272,7 @@ static void check_parms(void) } if (do_nmi && !nmi_handler_registered) { rv =3D register_nmi_handler(NMI_UNKNOWN, ipmi_nmi, 0, - "ipmi"); + "ipmi", 0); if (rv) { pr_warn("Can't register nmi handler\n"); return; diff --git a/drivers/edac/igen6_edac.c b/drivers/edac/igen6_edac.c index cdd8480e7368..e672b38f2c2b 100644 --- a/drivers/edac/igen6_edac.c +++ b/drivers/edac/igen6_edac.c @@ -1321,7 +1321,7 @@ static int register_err_handler(void) } =20 rc =3D register_nmi_handler(NMI_SERR, ecclog_nmi_handler, - 0, IGEN6_NMI_NAME); + 0, IGEN6_NMI_NAME, 0); if (rc) { igen6_printk(KERN_ERR, "Failed to register NMI handler\n"); return rc; diff --git a/drivers/watchdog/hpwdt.c b/drivers/watchdog/hpwdt.c index ae30e394d176..5246706afcf6 100644 --- a/drivers/watchdog/hpwdt.c +++ b/drivers/watchdog/hpwdt.c @@ -242,13 +242,13 @@ static int hpwdt_init_nmi_decoding(struct pci_dev *de= v) /* * Only one function can register for NMI_UNKNOWN */ - retval =3D register_nmi_handler(NMI_UNKNOWN, hpwdt_pretimeout, 0, "hpwdt"= ); + retval =3D register_nmi_handler(NMI_UNKNOWN, hpwdt_pretimeout, 0, "hpwdt"= , 0); if (retval) goto error; - retval =3D register_nmi_handler(NMI_SERR, hpwdt_pretimeout, 0, "hpwdt"); + retval =3D register_nmi_handler(NMI_SERR, hpwdt_pretimeout, 0, "hpwdt", 0= ); if (retval) goto error1; - retval =3D register_nmi_handler(NMI_IO_CHECK, hpwdt_pretimeout, 0, "hpwdt= "); + retval =3D register_nmi_handler(NMI_IO_CHECK, hpwdt_pretimeout, 0, "hpwdt= ", 0); if (retval) goto error2; =20 --=20 2.25.1 From nobody Tue Dec 16 14:36:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 161211C8FA8; Wed, 29 May 2024 20:28:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717014518; cv=none; b=DQiZYm8itfVWJ0aJKsozveC2QRV0qjl8KjNLlQoPzM40hqBYMGyj48MW6rd9Dxhdhd+LgpZCI3O2rBhq0SUMyreBlCmuvBzoD74fFVnrNiv2Dk2GiyKFpQCa2iOnFT2KYDw+Ei90VbQxFfgA/1pS082USweMrjNDsoQWvIWKJgM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717014518; c=relaxed/simple; bh=5Ss+qVKsW4nTXNmqKzHj51EVHt4aOtz+UFZAROKBrOg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MBxvI4NSQ0B9xXoot8rkBOXVZzhFzeJXN6SkbRE32oD0/n+sflsd71ViKuWDucRmYDF+X8lTZc+4JKEtqKchD2LoKBSp5A3xe27+87BQjCJJQNvC+0oUE03lkVGJBE4pCEJvmKKOIpzWUkOfBs+w2eJcH7O4OuR4RETqgl4v5HI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=esE4oyEX; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="esE4oyEX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717014517; x=1748550517; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5Ss+qVKsW4nTXNmqKzHj51EVHt4aOtz+UFZAROKBrOg=; b=esE4oyEX0f7gM0o9S7N2T4uiXapAXDYyddUS4IfaP/wRJSvyoRqvzTHp FeX3KTVfBs8f0mq4v7DfJwim8LnsBoBpC0jE3HwpnZs8D+Xz+5nBICN2b zLmCI3/MVqW4Vn99Wy7kz9IO60JEmvj5I/gO9Aans49Q9AaTfB1J1HWm5 mtrfF+360cEvL0WnFJezd7fVFmyPcDz4+XAG1+87j4c96iSdyXqZwOExr 62WEg0PbPlrjTXCKXWnG7Uva0nhW+iBsgvIGCUCIoHkpyM+GKHHd1LUpw +XxE6t1LEq3ECx0jHEvltHDxoi7GbgejBYNjuwgdhuWTVt3tXQHApXjyS w==; X-CSE-ConnectionGUID: jDHljMDXTw+3r7zwGuO2KA== X-CSE-MsgGUID: 5FrWnwNHQvSq/xklSbbSpw== X-IronPort-AV: E=McAfee;i="6600,9927,11087"; a="13574545" X-IronPort-AV: E=Sophos;i="6.08,199,1712646000"; d="scan'208";a="13574545" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 May 2024 13:28:34 -0700 X-CSE-ConnectionGUID: JeoMxu0zR6K9zS/MN1X+bQ== X-CSE-MsgGUID: Ub56xL9GSC6+vcroKOFI/Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,199,1712646000"; d="scan'208";a="35491278" Received: from jacob-builder.jf.intel.com ([10.54.39.125]) by fmviesa007.fm.intel.com with ESMTP; 29 May 2024 13:28:34 -0700 From: Jacob Pan To: X86 Kernel , LKML , Thomas Gleixner , Dave Hansen , "H. Peter Anvin" , "Ingo Molnar" , "Borislav Petkov" , linux-perf-users@vger.kernel.org, Peter Zijlstra Cc: Andi Kleen , "Xin Li" , Jacob Pan Subject: [PATCH 3/6] x86/irq: Factor out common NMI handling code Date: Wed, 29 May 2024 13:33:22 -0700 Message-Id: <20240529203325.3039243-4-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240529203325.3039243-1-jacob.jun.pan@linux.intel.com> References: <20240529203325.3039243-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In preparation for handling NMIs with explicit source reporting, factor out common code for reuse. Signed-off-by: Jacob Pan --- arch/x86/kernel/nmi.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index 1ff4f7c9f182..e2122ec9313c 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -135,6 +135,20 @@ static void nmi_check_duration(struct nmiaction *actio= n, u64 duration) action->handler, duration, decimal_msecs); } =20 +static inline int do_handle_nmi(struct nmiaction *a, struct pt_regs *regs,= unsigned int type) +{ + int thishandled; + u64 delta; + + delta =3D sched_clock(); + thishandled =3D a->handler(type, regs); + delta =3D sched_clock() - delta; + trace_nmi_handler(a->handler, (int)delta, thishandled); + nmi_check_duration(a, delta); + + return thishandled; +} + static int nmi_handle(unsigned int type, struct pt_regs *regs) { struct nmi_desc *desc =3D nmi_to_desc(type); @@ -149,18 +163,8 @@ static int nmi_handle(unsigned int type, struct pt_reg= s *regs) * can be latched at any given time. Walk the whole list * to handle those situations. */ - list_for_each_entry_rcu(a, &desc->head, list) { - int thishandled; - u64 delta; - - delta =3D sched_clock(); - thishandled =3D a->handler(type, regs); - handled +=3D thishandled; - delta =3D sched_clock() - delta; - trace_nmi_handler(a->handler, (int)delta, thishandled); - - nmi_check_duration(a, delta); - } + list_for_each_entry_rcu(a, &desc->head, list) + handled +=3D do_handle_nmi(a, regs, type); =20 rcu_read_unlock(); =20 --=20 2.25.1 From nobody Tue Dec 16 14:36:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5A7031C8FBC; Wed, 29 May 2024 20:28:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717014518; cv=none; b=KhsfS7BPfisu6jaKWdpWt2aYsvSzA+lkwEbpcM+A9ydTRZJBE/dUmuycTJtPpXIfs1bJZtgzibHWSsZaPOSuhbXlDomjVyNcRbccdgjCxaVfecfMtf98RuJhyiO6KvFqC2VCPH/BQNyOPWVLdC2MmTEA4e43rtbO3v4rtSJgsPQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717014518; c=relaxed/simple; bh=TSM1haBCr0WSs8yC/z+/tCZvjsQB/xQTffI8AknEb/o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=D+wX5Y15TwI3kvejGZWNNxojJYwG+RT0S4rCo+WkP/XzR+IF6lMwjyUPALhq/zSWzX6Fr/egL1kLwzz6XlKYR68EZlTpzkWjF3Wbf3Mjd0Kdzwl1fOS4n44FDhq9Z9waiNw9MpjwnsBB3ugBFZqfNVIJ1BFGexA6cjBeHmluPao= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ZqDmzgsG; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ZqDmzgsG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717014517; x=1748550517; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TSM1haBCr0WSs8yC/z+/tCZvjsQB/xQTffI8AknEb/o=; b=ZqDmzgsG3tUOTiyRHIP6Ce/dla8LiEKyRSrGUVrSmGjkg27v7o0uSSkd kJc3B8wlMQu/+qnKyiRdWJ/s5Lz/gwVGQkFC6LraC1B5pZiYeFFaPri7u BYzARMq0Z+vSo1k2xQUsuamFSgJ7Ay/tquk3xwQIr0JIUVs0VdAUrFRDa YO1YoKQDo80xRqI4uEX+Z6IbQrWpYeqfOLGz1YReyDGily0eYQL97j4Lt 3Mm+8oN+Pc24rTzsTjmuwCxzG0HNFApHAONVlE5nUqfCwKiVkotQOpUlX Q/ptYo5pnMqb+sc63FcXS25lpdTq57T8fNjy6nFmmng9WslTdD+zgFBaA Q==; X-CSE-ConnectionGUID: 5sktuyZMQ6meP3vn84A3Cg== X-CSE-MsgGUID: KJQ1xXqqSzadOichZXvkIQ== X-IronPort-AV: E=McAfee;i="6600,9927,11087"; a="13574554" X-IronPort-AV: E=Sophos;i="6.08,199,1712646000"; d="scan'208";a="13574554" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 May 2024 13:28:35 -0700 X-CSE-ConnectionGUID: OsUWBup4TiazNZmPXxdMvw== X-CSE-MsgGUID: XkoolWFiR/q4uGx6lFpkYA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,199,1712646000"; d="scan'208";a="35491284" Received: from jacob-builder.jf.intel.com ([10.54.39.125]) by fmviesa007.fm.intel.com with ESMTP; 29 May 2024 13:28:34 -0700 From: Jacob Pan To: X86 Kernel , LKML , Thomas Gleixner , Dave Hansen , "H. Peter Anvin" , "Ingo Molnar" , "Borislav Petkov" , linux-perf-users@vger.kernel.org, Peter Zijlstra Cc: Andi Kleen , "Xin Li" , Jacob Pan Subject: [PATCH 4/6] x86/irq: Process nmi sources in NMI handler Date: Wed, 29 May 2024 13:33:23 -0700 Message-Id: <20240529203325.3039243-5-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240529203325.3039243-1-jacob.jun.pan@linux.intel.com> References: <20240529203325.3039243-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" With NMI source reporting enabled, NMI handler can prioritize the handling of sources reported explicitly. If the source is unknown, then resume the existing processing flow. i.e. invoke all NMI handlers. Signed-off-by: Jacob Pan --- arch/x86/kernel/nmi.c | 48 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index e2122ec9313c..32c285722734 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -149,12 +149,60 @@ static inline int do_handle_nmi(struct nmiaction *a, = struct pt_regs *regs, unsig return thishandled; } =20 +static inline int nmi_handle_src(unsigned int type, struct pt_regs *regs) +{ + unsigned long source_bitmask; + struct nmiaction *a; + int handled =3D 0; + int vec =3D 1; + + if (!cpu_feature_enabled(X86_FEATURE_NMI_SOURCE) || type !=3D NMI_LOCAL) + return 0; + + source_bitmask =3D fred_event_data(regs); + if (!source_bitmask) { + pr_warn_ratelimited("NMI received without source information!\n"); + return 0; + } + + /* + * Per NMI source specification, there is no guarantee that a valid + * NMI vector is always delivered, even when the source specified + * one. It is software's responsibility to check all available NMI + * sources when bit 0 is set in the NMI source bitmap. i.e. we have + * to call every handler as if we have no NMI source. + * On the other hand, if we do get non-zero vectors, we know exactly + * what the sources are. So we only call the handlers with the bit set. + */ + if (source_bitmask & BIT(NMI_SOURCE_VEC_UNKNOWN)) { + pr_warn_ratelimited("NMI received with unknown source\n"); + return 0; + } + + rcu_read_lock(); + /* Bit 0 is for unknown NMI sources, skip it. */ + for_each_set_bit_from(vec, &source_bitmask, NR_NMI_SOURCE_VECTORS) { + a =3D rcu_dereference(nmiaction_src_table[vec]); + if (!a) { + pr_warn_ratelimited("NMI received %d no handler", vec); + continue; + } + handled +=3D do_handle_nmi(a, regs, type); + } + rcu_read_unlock(); + return handled; +} + static int nmi_handle(unsigned int type, struct pt_regs *regs) { struct nmi_desc *desc =3D nmi_to_desc(type); struct nmiaction *a; int handled=3D0; =20 + handled =3D nmi_handle_src(type, regs); + if (handled) + return handled; + rcu_read_lock(); =20 /* --=20 2.25.1 From nobody Tue Dec 16 14:36:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 840861C9EA1; Wed, 29 May 2024 20:28:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717014519; cv=none; b=kDp1SLFe+AIniobQtlRbAdP5glOJShn4xINbvxYS1e0PuI05NOzSuAfaMn+fA59ixIlopqOt2/H4f917rly/OfA89O6xKs0/IKuptbzUzv98Bl1krZVi+A993oERH137m4hyK0tbIjODpxFhIPYzbFv9PqVovguAKiyANSk37/8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717014519; c=relaxed/simple; bh=rcS88EGxEczwKZqGchK9BbjqykpecfivATqJO505QHU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ad9fBdaIJxItoMyHq0NVfaGICw94+pSE4lWy4/iUC4/g6ENtg9PGaIL8zNB/CCUIkLA2/N2GeZmXNIGvMNRzpTNgRemakD8aV7rUihuos63QV6BywikOj0ItYtKbpbrhQsD7ruf40rWqUAABtxId+EEPtY4XMibz9KO4aZh9l2k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=QYx+SDwx; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="QYx+SDwx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717014518; x=1748550518; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rcS88EGxEczwKZqGchK9BbjqykpecfivATqJO505QHU=; b=QYx+SDwxaodHPyZ9krqdZVd9eL4eI5Pg4HvKQethhKGkzUB9wNvqzR/I 7kznkb9fFLmQWY+JQuvd0zzNhlV/SDkbb8YIqW8HC94nwUqpGXiAwu+VR 9vmD4VAnekjuXjDV8hcrYi4pvKtGel7ycDoufchWD8XWF0ZpL8AV7eXzn WrmMtdC0bf33OjjOsYTlzsDe7mTZANJnkZA5ctBJiTSsb7D01cb/cuZ8s +Qy3adqFTT/oQEdhHlFPSGEzXA4wdT6VwUWYcwdZ/TJ3u2G82GalBRGrz kBRCA1YbomKBGfEetDMY92ergyBt7OFl+2kvPow4WprDYxAIYZq72QwJl A==; X-CSE-ConnectionGUID: 1kxOJGOFQRiPhNKPmx4w9g== X-CSE-MsgGUID: B6ajrsXtRHGRgm38E/Vj2A== X-IronPort-AV: E=McAfee;i="6600,9927,11087"; a="13574565" X-IronPort-AV: E=Sophos;i="6.08,199,1712646000"; d="scan'208";a="13574565" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 May 2024 13:28:35 -0700 X-CSE-ConnectionGUID: XKmrw6dtReyWeL+mmZtZ4w== X-CSE-MsgGUID: IpntXb8LSOyRhdXzO8EvoA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,199,1712646000"; d="scan'208";a="35491292" Received: from jacob-builder.jf.intel.com ([10.54.39.125]) by fmviesa007.fm.intel.com with ESMTP; 29 May 2024 13:28:34 -0700 From: Jacob Pan To: X86 Kernel , LKML , Thomas Gleixner , Dave Hansen , "H. Peter Anvin" , "Ingo Molnar" , "Borislav Petkov" , linux-perf-users@vger.kernel.org, Peter Zijlstra Cc: Andi Kleen , "Xin Li" , Jacob Pan Subject: [PATCH 5/6] perf/x86: Enable NMI source reporting for perfmon Date: Wed, 29 May 2024 13:33:24 -0700 Message-Id: <20240529203325.3039243-6-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240529203325.3039243-1-jacob.jun.pan@linux.intel.com> References: <20240529203325.3039243-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Program the designated NMI source vector into the performance monitoring interrupt (PMI) of the local vector table. PMI handler will be directly invoked when its NMI is generated. This avoids the latency of calling all NMI handlers blindly. Signed-off-by: Jacob Pan --- arch/x86/events/core.c | 8 ++++++-- arch/x86/events/intel/core.c | 6 +++--- arch/x86/include/asm/apic.h | 1 + 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 1ef2201e48ac..db8c30881f5c 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -46,6 +46,7 @@ =20 struct x86_pmu x86_pmu __read_mostly; static struct pmu pmu; +u32 apic_perfmon_ctr =3D APIC_DM_NMI; =20 DEFINE_PER_CPU(struct cpu_hw_events, cpu_hw_events) =3D { .enabled =3D 1, @@ -1680,7 +1681,7 @@ int x86_pmu_handle_irq(struct pt_regs *regs) * This generic handler doesn't seem to have any issues where the * unmasking occurs so it was left at the top. */ - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, apic_perfmon_ctr); =20 for (idx =3D 0; idx < x86_pmu.num_counters; idx++) { if (!test_bit(idx, cpuc->active_mask)) @@ -1723,7 +1724,10 @@ void perf_events_lapic_init(void) /* * Always use NMI for PMU */ - apic_write(APIC_LVTPC, APIC_DM_NMI); + if (cpu_feature_enabled(X86_FEATURE_NMI_SOURCE)) + apic_perfmon_ctr |=3D NMI_SOURCE_VEC_PMI; + + apic_write(APIC_LVTPC, apic_perfmon_ctr); } =20 static int diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index 38c1b1f1deaa..b4a70457c678 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -3093,7 +3093,7 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) * NMI handler. */ if (!late_ack && !mid_ack) - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, apic_perfmon_ctr); intel_bts_disable_local(); cpuc->enabled =3D 0; __intel_pmu_disable_all(true); @@ -3130,7 +3130,7 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) =20 done: if (mid_ack) - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, apic_perfmon_ctr); /* Only restore PMU state when it's active. See x86_pmu_disable(). */ cpuc->enabled =3D pmu_enabled; if (pmu_enabled) @@ -3143,7 +3143,7 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) * Haswell CPUs. */ if (late_ack) - apic_write(APIC_LVTPC, APIC_DM_NMI); + apic_write(APIC_LVTPC, apic_perfmon_ctr); return handled; } =20 diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h index 9327eb00e96d..062a6edd36d3 100644 --- a/arch/x86/include/asm/apic.h +++ b/arch/x86/include/asm/apic.h @@ -59,6 +59,7 @@ extern int local_apic_timer_c2_ok; =20 extern bool apic_is_disabled; extern unsigned int lapic_timer_period; +extern u32 apic_perfmon_ctr; =20 extern enum apic_intr_mode_id apic_intr_mode; enum apic_intr_mode_id { --=20 2.25.1 From nobody Tue Dec 16 14:36:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BEE01C8FB7; Wed, 29 May 2024 20:28:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717014518; cv=none; b=GABv32gei74P0lXjrgEsRa1o2O8Vn1YkzeWRkW0nC1zVpJTMK2P4xF9YGqyzAGzBko68rNzZyO67qI7HqGctT87tpWj4V7FwZ08Q3N2P9JMRHKLa0ryxVAjChCqkh9Q4emv0Me1rZe/yU1nD9Semsh9Ga90b29xN7IOX4LHjQz8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717014518; c=relaxed/simple; bh=OzDhCGOxOXfNQnK7LQxh+1TRrj9b9K7AgFE2duCxNjs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eJzACIxtjNG1d6WcXqvdARl21B2qE729WebFfQ4nFjTd67JCqbEVeOGKPl8yMmmvvu16rzMs6Tby9/C190HTrbAwfXFfMgi5uLngJ6cEo/8uhsGvQrKaHca5zKpnOY3uylp+8K5+VoHRKmpgjIu1LK0G8OFFN+y1iI/80uT/CZo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=YyZMyHzf; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="YyZMyHzf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717014517; x=1748550517; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=OzDhCGOxOXfNQnK7LQxh+1TRrj9b9K7AgFE2duCxNjs=; b=YyZMyHzfiX9Jh0HUAiNNOMnQmdP3IO9Xy/nJs5iBFZp7/DgnXdyfOQvX ++LMLIO7RLyO5kIWaQx97YqRBvpNBEDJr9tPxY4WPE+JJbVEoLUvdlG8S JrYgZbCewIMSAOuNF0HKJc38457LPe/evVhpl3zNULvZvVfckWehLkawV lhGE/exqCMkWczMmA5SgWLF4/Q4w0E7djQ19kXW4eUysEWVDhhEgvkvk/ fV7KrCfmzpnuRktvHdy1aKEOB948IqKUI268lKD8Q4QtSuYJy4QVg8v5V 23+3y3vMIrBZ/wMgut7+oBW+G8VOTZFqUqySp9s+VnciEf+bFTh33OgMz Q==; X-CSE-ConnectionGUID: AdGIypunSa2mG82IbUnIUQ== X-CSE-MsgGUID: Y2t8jaDaTTOLk/gW3w+U9w== X-IronPort-AV: E=McAfee;i="6600,9927,11087"; a="13574568" X-IronPort-AV: E=Sophos;i="6.08,199,1712646000"; d="scan'208";a="13574568" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 May 2024 13:28:35 -0700 X-CSE-ConnectionGUID: 3ztwErLcQl+m1YgkHjaImw== X-CSE-MsgGUID: 5DX4HpxrRemJsikK5emZKg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,199,1712646000"; d="scan'208";a="35491299" Received: from jacob-builder.jf.intel.com ([10.54.39.125]) by fmviesa007.fm.intel.com with ESMTP; 29 May 2024 13:28:35 -0700 From: Jacob Pan To: X86 Kernel , LKML , Thomas Gleixner , Dave Hansen , "H. Peter Anvin" , "Ingo Molnar" , "Borislav Petkov" , linux-perf-users@vger.kernel.org, Peter Zijlstra Cc: Andi Kleen , "Xin Li" , Jacob Pan Subject: [PATCH 6/6] x86/irq: Enable NMI source on IPIs delivered as NMI Date: Wed, 29 May 2024 13:33:25 -0700 Message-Id: <20240529203325.3039243-7-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240529203325.3039243-1-jacob.jun.pan@linux.intel.com> References: <20240529203325.3039243-1-jacob.jun.pan@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Program designated NMI source vectors for all NMI delivered IPIs such that their handlers can be selectively invoked. Signed-off-by: Jacob Pan --- arch/x86/include/asm/irq_vectors.h | 10 ++++++++++ arch/x86/kernel/apic/hw_nmi.c | 3 ++- arch/x86/kernel/apic/ipi.c | 4 ++-- arch/x86/kernel/apic/local.h | 18 ++++++++++++------ arch/x86/kernel/cpu/mce/inject.c | 2 +- arch/x86/kernel/kgdb.c | 2 +- arch/x86/kernel/nmi_selftest.c | 2 +- arch/x86/kernel/reboot.c | 2 +- arch/x86/kernel/smp.c | 2 +- 9 files changed, 31 insertions(+), 14 deletions(-) diff --git a/arch/x86/include/asm/irq_vectors.h b/arch/x86/include/asm/irq_= vectors.h index b8388bc00cde..a13ce6e96542 100644 --- a/arch/x86/include/asm/irq_vectors.h +++ b/arch/x86/include/asm/irq_vectors.h @@ -126,6 +126,16 @@ #define NMI_SOURCE_VEC_IPI_TEST 7 /* For remote and local IPIs*/ #define NR_NMI_SOURCE_VECTORS 8 =20 +/* + * When programming the local APIC, IDT NMI vector and NMI source vector + * are encoded in a single 32 bit variable. The top 16 bits contain + * the NMI source vector and the bottom 16 bits contain NMI_VECTOR (2) + * The top 16 bits are always zero when NMI source feature is not enabled + * or the caller does not use NMI source. + */ +#define NMI_VECTOR_WITH_SOURCE(src) (NMI_VECTOR | (src << 16)) +#define NMI_SOURCE_VEC_MASK GENMASK(15, 0) + #ifdef CONFIG_X86_LOCAL_APIC #define FIRST_SYSTEM_VECTOR POSTED_MSI_NOTIFICATION_VECTOR #else diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c index 9f0125d3b8b0..f73ca95d961e 100644 --- a/arch/x86/kernel/apic/hw_nmi.c +++ b/arch/x86/kernel/apic/hw_nmi.c @@ -20,6 +20,7 @@ #include #include #include +#include =20 #include "local.h" =20 @@ -33,7 +34,7 @@ u64 hw_nmi_get_sample_period(int watchdog_thresh) #ifdef arch_trigger_cpumask_backtrace static void nmi_raise_cpu_backtrace(cpumask_t *mask) { - __apic_send_IPI_mask(mask, NMI_VECTOR); + __apic_send_IPI_mask(mask, NMI_VECTOR_WITH_SOURCE(NMI_SOURCE_VEC_IPI_BT)); } =20 void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu) diff --git a/arch/x86/kernel/apic/ipi.c b/arch/x86/kernel/apic/ipi.c index 5da693d633b7..9d2b18e58758 100644 --- a/arch/x86/kernel/apic/ipi.c +++ b/arch/x86/kernel/apic/ipi.c @@ -157,7 +157,7 @@ static void __default_send_IPI_shortcut(unsigned int sh= ortcut, int vector) * issues where otherwise the system hangs when the panic CPU tries * to stop the others before launching the kdump kernel. */ - if (unlikely(vector =3D=3D NMI_VECTOR)) + if (unlikely(is_nmi_vector(vector))) apic_mem_wait_icr_idle_timeout(); else apic_mem_wait_icr_idle(); @@ -174,7 +174,7 @@ void __default_send_IPI_dest_field(unsigned int dest_ma= sk, int vector, unsigned int dest_mode) { /* See comment in __default_send_IPI_shortcut() */ - if (unlikely(vector =3D=3D NMI_VECTOR)) + if (unlikely(is_nmi_vector(vector))) apic_mem_wait_icr_idle_timeout(); else apic_mem_wait_icr_idle(); diff --git a/arch/x86/kernel/apic/local.h b/arch/x86/kernel/apic/local.h index 842fe28496be..60e90b7bf058 100644 --- a/arch/x86/kernel/apic/local.h +++ b/arch/x86/kernel/apic/local.h @@ -12,6 +12,7 @@ =20 #include #include +#include =20 /* X2APIC */ void __x2apic_send_IPI_dest(unsigned int apicid, int vector, unsigned int = dest); @@ -26,19 +27,24 @@ extern u32 x2apic_max_apicid; =20 DECLARE_STATIC_KEY_FALSE(apic_use_ipi_shorthand); =20 +static inline bool is_nmi_vector(int vector) +{ + return (vector & NMI_SOURCE_VEC_MASK) =3D=3D NMI_VECTOR; +} + static inline unsigned int __prepare_ICR(unsigned int shortcut, int vector, unsigned int dest) { unsigned int icr =3D shortcut | dest; =20 - switch (vector) { - default: - icr |=3D APIC_DM_FIXED | vector; - break; - case NMI_VECTOR: + if (is_nmi_vector(vector)) { icr |=3D APIC_DM_NMI; - break; + if (cpu_feature_enabled(X86_FEATURE_NMI_SOURCE)) + icr |=3D vector >> 16; + } else { + icr |=3D APIC_DM_FIXED | vector; } + return icr; } =20 diff --git a/arch/x86/kernel/cpu/mce/inject.c b/arch/x86/kernel/cpu/mce/inj= ect.c index 365a03f11d06..07bc6c29bd83 100644 --- a/arch/x86/kernel/cpu/mce/inject.c +++ b/arch/x86/kernel/cpu/mce/inject.c @@ -270,7 +270,7 @@ static void __maybe_unused raise_mce(struct mce *m) mce_irq_ipi, NULL, 0); preempt_enable(); } else if (m->inject_flags & MCJ_NMI_BROADCAST) - __apic_send_IPI_mask(mce_inject_cpumask, NMI_VECTOR); + __apic_send_IPI_mask(mce_inject_cpumask, NMI_VECTOR_WITH_SOURCE(NMI_SO= URCE_VEC_IPI_MCE)); } start =3D jiffies; while (!cpumask_empty(mce_inject_cpumask)) { diff --git a/arch/x86/kernel/kgdb.c b/arch/x86/kernel/kgdb.c index d167eb23cf13..02198cf9fe21 100644 --- a/arch/x86/kernel/kgdb.c +++ b/arch/x86/kernel/kgdb.c @@ -416,7 +416,7 @@ static void kgdb_disable_hw_debug(struct pt_regs *regs) */ void kgdb_roundup_cpus(void) { - apic_send_IPI_allbutself(NMI_VECTOR); + apic_send_IPI_allbutself(NMI_VECTOR_WITH_SOURCE(NMI_SOURCE_VEC_IPI_KGDB)); } #endif =20 diff --git a/arch/x86/kernel/nmi_selftest.c b/arch/x86/kernel/nmi_selftest.c index f014c8a66b0c..5aa122d3368c 100644 --- a/arch/x86/kernel/nmi_selftest.c +++ b/arch/x86/kernel/nmi_selftest.c @@ -76,7 +76,7 @@ static void __init test_nmi_ipi(struct cpumask *mask) /* sync above data before sending NMI */ wmb(); =20 - __apic_send_IPI_mask(mask, NMI_VECTOR); + __apic_send_IPI_mask(mask, NMI_VECTOR_WITH_SOURCE(NMI_SOURCE_VEC_IPI_TEST= )); =20 /* Don't wait longer than a second */ timeout =3D USEC_PER_SEC; diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c index acc19c1d3b4f..fb63bc0d6a0f 100644 --- a/arch/x86/kernel/reboot.c +++ b/arch/x86/kernel/reboot.c @@ -918,7 +918,7 @@ void nmi_shootdown_cpus(nmi_shootdown_cb callback) */ wmb(); =20 - apic_send_IPI_allbutself(NMI_VECTOR); + apic_send_IPI_allbutself(NMI_VECTOR_WITH_SOURCE(NMI_SOURCE_VEC_IPI_REBOOT= )); =20 /* Kick CPUs looping in NMI context. */ WRITE_ONCE(crash_ipi_issued, 1); diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c index f27469e40141..b79e78762a73 100644 --- a/arch/x86/kernel/smp.c +++ b/arch/x86/kernel/smp.c @@ -217,7 +217,7 @@ static void native_stop_other_cpus(int wait) pr_emerg("Shutting down cpus with NMI\n"); =20 for_each_cpu(cpu, &cpus_stop_mask) - __apic_send_IPI(cpu, NMI_VECTOR); + __apic_send_IPI(cpu, NMI_VECTOR_WITH_SOURCE(NMI_SOURCE_VEC_IPI_SMP_STO= P)); } /* * Don't wait longer than 10 ms if the caller didn't --=20 2.25.1