From nobody Thu Apr 2 01:08:41 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 36DEB369234 for ; Tue, 31 Mar 2026 02:14:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923298; cv=none; b=DbHFiurAVjKXJFbRVfrVTXJwuS+D4kguFO8swym/Xc1hNfo70Fhp8rCtxafFMsvmGeDo1lZ0LFD5cy74pUVQqXKBTRtPgQvg1G2FokSWNGlJq5eAnnYrXmUDT/zw28k9xzvMvknhHBYox/X4LTE5D2RLisiy4L+5SObG/CZV9ZQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923298; c=relaxed/simple; bh=jVIM7EIFyMMrxh4UN+c1WxZeiLXcm6oRhS6lRuk9LUk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IE1CISkWJ3hWSYQZ+1Z34UI0sMxpu8l7qvkCOGTXIYykvJkU7f9X3+3WQ6uHycyPlG8Gn1xAz5mX7JJ+9XTe5I6JwEh1qy0wQevB0XmQmy4n9dv1bHFPUEDNJcbDGQOsgxY+iHg5LxsirT2KHKco/a5wWmULJqvgNq49P0bVn9A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=TcYesPFj; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="TcYesPFj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774923296; x=1806459296; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jVIM7EIFyMMrxh4UN+c1WxZeiLXcm6oRhS6lRuk9LUk=; b=TcYesPFjTPVAgPfMs5RDmxkxNwpZqNQwCbg6YZTNU6mrXAjRXiEQZnnX J8yJzwAfzccSiYTXG6xgaTq5EB3WOw30P35k+MckzBh0occ+5faM6yExU 89ay9/SBIgVc7qAMMDnGbHlo6W/mPJl92s06etZl9VtT/3aAysMsLl37d K7CAkbFi41YoigNtBiJUCnM13AElAT2EAgiYuV1GVZwBv1NFT6jRFqXvf JETSewQsnNHKNN9YU+QE/rEwDr7HfLcdB29lrGkMaLFZiS0+eKrJb8GUc PoHUSKrJs5b197yUEOgcjnO3vTpdyIiJUOKWt+9aL8K29bdfMWUpURKB5 A==; X-CSE-ConnectionGUID: X1UW6qbgRAm4aJL2AwhSnA== X-CSE-MsgGUID: aWIxYLDYQtuME6cLqGzUYQ== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93508052" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93508052" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 19:07:40 -0700 X-CSE-ConnectionGUID: TdGIrLcjSuCgOcur/XqCzg== X-CSE-MsgGUID: VD/FogoKSquGYn7STjBRkw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="221358669" Received: from chang-linux-3.sc.intel.com (HELO chang-linux-3) ([172.25.66.172]) by fmviesa006.fm.intel.com with ESMTP; 30 Mar 2026 19:07:34 -0700 From: "Chang S. Bae" To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, peterz@infradead.org, david.kaplan@amd.com, chang.seok.bae@intel.com Subject: [PATCH v2 01/11] stop_machine: Clarify @cpus == NULL semantics Date: Tue, 31 Mar 2026 01:42:39 +0000 Message-ID: <20260331014251.86353-2-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260331014251.86353-1-chang.seok.bae@intel.com> References: <20260331014251.86353-1-chang.seok.bae@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The stop-machine API description currently mentions that @cpus =3D=3D NULL means running on "each cpu_online_mask". Previously, it was described as "any cpu_online_mask" before commit: fc6f89dc707 ("stop_machine: Improve kernel-doc function-header comments") In fact, multi_cpu_stop() selects the first CPU in cpu_online_mask when @cpus is NULL. Right now cpumask_any() is defined as cpumask_first(). So the previous description was closer but apparently it was not clear enough either. Fix those comments for clarity. Signed-off-by: Chang S. Bae --- V1 -> V2: New patch Considering stop_machine_nmi_cpuslocked() and its another cpumask, I could realize this nullptr implication is not clear enough. Then, it ended up with this fix. I also considered just saying CPU0, but still 'cpumask_first(cpu_online_mas= k)' are there. So, leave it like that, instead of converting them aggressively. --- include/linux/stop_machine.h | 6 ++++-- kernel/stop_machine.c | 4 +++- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/include/linux/stop_machine.h b/include/linux/stop_machine.h index 72820503514c..c753dd53e79d 100644 --- a/include/linux/stop_machine.h +++ b/include/linux/stop_machine.h @@ -99,7 +99,8 @@ static inline void print_stop_info(const char *log_lvl, s= truct task_struct *task * stop_machine: freeze the machine on all CPUs and run this function * @fn: the function to run * @data: the data ptr to pass to @fn() - * @cpus: the cpus to run @fn() on (NULL =3D run on each online CPU) + * @cpus: the CPUs to run @fn() on. If NULL, @fn() runs on a single + * (arbitrary) CPU from cpu_online_mask. * * Description: This causes a thread to be scheduled on every CPU, which * will run with interrupts disabled. Each CPU specified by @cpus will @@ -133,7 +134,8 @@ int stop_machine(cpu_stop_fn_t fn, void *data, const st= ruct cpumask *cpus); * stop_machine_cpuslocked: freeze the machine on all CPUs and run this fu= nction * @fn: the function to run * @data: the data ptr to pass to @fn() - * @cpus: the cpus to run @fn() on (NULL =3D run on each online CPU) + * @cpus: the CPUs to run @fn() on. If NULL, @fn() runs on a single + * (arbitrary) CPU from cpu_online_mask. * * Same as above. Avoids nested calls to cpus_read_lock(). * diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c index 3fe6b0c99f3d..822cf56fdc81 100644 --- a/kernel/stop_machine.c +++ b/kernel/stop_machine.c @@ -655,9 +655,11 @@ EXPORT_SYMBOL_GPL(stop_core_cpuslocked); =20 /** * stop_machine_from_inactive_cpu - stop_machine() from inactive CPU + * * @fn: the function to run * @data: the data ptr for the @fn() - * @cpus: the cpus to run the @fn() on (NULL =3D any online cpu) + * @cpus: the CPUs to run the @fn() on. If NULL, @fn() runs on a single + * (arbitrary) CPU from cpu_online_mask. * * This is identical to stop_machine() but can be called from a CPU which * is not active. The local CPU is in the process of hotplug (so no other --=20 2.51.0 From nobody Thu Apr 2 01:08:41 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CEC5936AB5A for ; Tue, 31 Mar 2026 02:14:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923299; cv=none; b=fAjjgf5KTYnKYLhJ4xs3ehZkRcxX19/EO42lQPNWAXhkNM8Ai/nc3MprvF74EFDsFOpOOIPaYrf28RFiC/JE2lug08mbQoz7/NeP6fWwwogrlTirfWrC39YgOOcYMmfmuUdJsTUOM9DlMqCxv/tUmgiTb5bZ/WGe0QS4G7XS1i0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923299; c=relaxed/simple; bh=bWPUTIHzB77pCmbOwvOUZ3WJ7CbJ0BxyXczy1UKJjsg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=A95Tcde0Gv9I0Xx6MGE4NNkeOwFfyY8xiSKCwXws66DtcwKzjMIByFB7xEY4rPMsIpjIuVUT4m7RqqsmT4GY5/FHpSxtjd77bnC/FkSnanq6WKdgGsqj45mo46ZrDKJvBjb/ZurKpZdDJ9uISuCFVj1GR6fCxIrBH8+lLqjGQHA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=c0e9x30K; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="c0e9x30K" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774923298; x=1806459298; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bWPUTIHzB77pCmbOwvOUZ3WJ7CbJ0BxyXczy1UKJjsg=; b=c0e9x30KgExc6kFWX6JDbl2YmhYdL1J80iyuyuTEnsu6oNe73WjVhNCN 8vbt7lvEmrjRyj79ViHSwEpis+R0pFTv33k0EbToAwrZ0gJQL+8k0N6D9 0Q8L+4U1Eg20xEbNN7L/L3MZ5oPGjZswpDGSZBg8fIYa7JU7Kq+K4/fiY 1ioiXFfbzML6f4To0yeW0g/w1bRNGDoKkv1H6S3Jk8uc3ggCf4D+Uugmd 1KFZw6GbDuPhnxTvOHRGyb9IZZYSUPAptDK5G2xDcJ4+mxYNC95R0mgny fuF1RdNHSlzIV++1nbCBaUKPVOvTLGAB5vINS6gvXMvVWUWPVKPNq+GyX Q==; X-CSE-ConnectionGUID: HsQ3GAHgRsChfCUPGyhmIw== X-CSE-MsgGUID: FNVT36acTP6WY6EYsck/Yg== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93508064" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93508064" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 19:07:40 -0700 X-CSE-ConnectionGUID: dlu+euLvSz6zEMmgP1EjoA== X-CSE-MsgGUID: 7c/reNQITbCGBNzQ8V5/Ew== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="221358677" Received: from chang-linux-3.sc.intel.com (HELO chang-linux-3) ([172.25.66.172]) by fmviesa006.fm.intel.com with ESMTP; 30 Mar 2026 19:07:37 -0700 From: "Chang S. Bae" To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, peterz@infradead.org, david.kaplan@amd.com, chang.seok.bae@intel.com Subject: [RFC][PATCH v2 02/11] stop_machine: Accumulate error code rather than overwrite Date: Tue, 31 Mar 2026 01:42:40 +0000 Message-ID: <20260331014251.86353-3-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260331014251.86353-1-chang.seok.bae@intel.com> References: <20260331014251.86353-1-chang.seok.bae@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" cpu_stopper_thread() invokes a stop function and collects its error code in struct cpu_stop_done. In the multi stop-machine case, it is shared data, but currently an arbitrary error is recorded as overwriting. With different errors, accumulating error code instead can distinguish a multi-error condition as bits are cumulatively set. Convert the error recoding to accumulate return values. Suggested-by: Borislav Petkov Signed-off-by: Chang S. Bae Link: https://lore.kernel.org/lkml/20260304163335.GDaahe3wdnqxSC2yfw@fat_cr= ate.local --- V1 -> V2: New patch While tried to explain its benefit here, I considered this change deserves more discussions to ensure its impact, so RFC. --- include/linux/stop_machine.h | 12 ++++++------ kernel/stop_machine.c | 10 +++++----- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/include/linux/stop_machine.h b/include/linux/stop_machine.h index c753dd53e79d..2f986555113a 100644 --- a/include/linux/stop_machine.h +++ b/include/linux/stop_machine.h @@ -124,9 +124,9 @@ static inline void print_stop_info(const char *log_lvl,= struct task_struct *task * the possibility of blocking in cpus_read_lock() means that the caller * cannot usefully rely on this serialization. * - * Return: 0 if all invocations of @fn return zero. Otherwise, the - * value returned by an arbitrarily chosen member of the set of calls to - * @fn that returned non-zero. + * Return: 0 if all invocations of @fn return zero. Otherwise, an + * accumulated return value from all invocation of @fn that returned + * non-zero. */ int stop_machine(cpu_stop_fn_t fn, void *data, const struct cpumask *cpus); =20 @@ -154,9 +154,9 @@ int stop_machine_cpuslocked(cpu_stop_fn_t fn, void *dat= a, const struct cpumask * * * Context: Must be called from within a cpus_read_lock() protected region. * - * Return: 0 if all invocations of @fn return zero. Otherwise, the - * value returned by an arbitrarily chosen member of the set of calls to - * @fn that returned non-zero. + * Return: 0 if all invocations of @fn return zero. Otherwise, an + * accumulated return value from all invocation of @fn that returned + * non-zero. */ int stop_core_cpuslocked(unsigned int cpu, cpu_stop_fn_t fn, void *data); =20 diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c index 822cf56fdc81..15268f1207e9 100644 --- a/kernel/stop_machine.c +++ b/kernel/stop_machine.c @@ -459,7 +459,7 @@ static int __stop_cpus(const struct cpumask *cpumask, * RETURNS: * -ENOENT if @fn(@arg) was not executed at all because all cpus in * @cpumask were offline; otherwise, 0 if all executions of @fn - * returned 0, any non zero return value if any returned non zero. + * returned 0, the accumulated value of all non-zero @fn returns. */ static int stop_cpus(const struct cpumask *cpumask, cpu_stop_fn_t fn, void= *arg) { @@ -512,7 +512,7 @@ static void cpu_stopper_thread(unsigned int cpu) ret =3D fn(arg); if (done) { if (ret) - done->ret =3D ret; + done->ret |=3D ret; cpu_stop_signal_done(done); } preempt_count_dec(); @@ -674,8 +674,8 @@ EXPORT_SYMBOL_GPL(stop_core_cpuslocked); * Local CPU is inactive. Temporarily stops all active CPUs. * * RETURNS: - * 0 if all executions of @fn returned 0, any non zero return value if any - * returned non zero. + * 0 if all executions of @fn returned 0, otherwise the accumulated value + * of all non-zero @fn returns. */ int stop_machine_from_inactive_cpu(cpu_stop_fn_t fn, void *data, const struct cpumask *cpus) @@ -705,5 +705,5 @@ int stop_machine_from_inactive_cpu(cpu_stop_fn_t fn, vo= id *data, cpu_relax(); =20 mutex_unlock(&stop_cpus_mutex); - return ret ?: done.ret; + return ret | done.ret; } --=20 2.51.0 From nobody Thu Apr 2 01:08:41 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BBE91368272 for ; Tue, 31 Mar 2026 02:14:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923297; cv=none; b=XLFJhi3/Q+zRP42Z6t8snabjIjc40p4B4C1kXG+V3T58Qcmq7OcBaCB44vjvYJK/AyWknUmSJtF4OgQxw80YAbNdgHLZOUleSRY+YI4hqtoBXb8eulGwaAFULsmLTYXaun0xHehWBD2+WD3YJc3EpvRyr+Cn/YwK1jsVf3uBZeg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923297; c=relaxed/simple; bh=lq3b7BEifEyLFAFMNfdpLJ7nzFwn9qYMHE1xHq/129M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tax3/0nN91rHgo7oGKaZ3qYs0RxAYc4VM7D/v/IGOdnrVcqnZnuu7mdsei/fAvLDIJN8qeYnAQ/o4g6S1/Fksk6Zjomc1871tvjsr4n3rj6j7EBf4q665z1GAst3Y+fySwAfC2ru/EGcpK9egv5k0mZoey3KKLynhIx0ezNkiOI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=WL7ce7jc; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WL7ce7jc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774923296; x=1806459296; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=lq3b7BEifEyLFAFMNfdpLJ7nzFwn9qYMHE1xHq/129M=; b=WL7ce7jcjZP8sSBHHCIUXXBBgmtGu4C7cKNTMBI/ZntOu454V9M+DkwH 6ZC4D8sFfiVdeHtZGjIugWiN8QO+drx58u2RsKgSSSmm+3UFXoBW/2ibC 2OlO+L7fWMyDg3PHW130XiS1dHp/I/PKfE/V7bYlANIip0pDukySfFTqH La3gBZ5tW1L2x3Cpn1/6yy2m9DzsOn9Pdw2fTXAvPJNlX8EEnzUJc5SJ9 szGgklHa4tVH2c3IqQs7WLITO5RcSYVoyYqO+l3vWzaz+w7/vq0Yt/54Y aaz45bfHtA8BJYTJtfT4c7VVOZvcL0fmUSeFkNOrpunsmZfUARpcNDAlm A==; X-CSE-ConnectionGUID: J9b0o9hzSQunXPqtXqdGTg== X-CSE-MsgGUID: SUozWIRWTeKkIphWnSlpZQ== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93508058" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93508058" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 19:07:40 -0700 X-CSE-ConnectionGUID: zCkvLtEYQUqy2DAGDRrKLw== X-CSE-MsgGUID: 5SJR8udMQaC6CmTMIYne5Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="221358680" Received: from chang-linux-3.sc.intel.com (HELO chang-linux-3) ([172.25.66.172]) by fmviesa006.fm.intel.com with ESMTP; 30 Mar 2026 19:07:40 -0700 From: "Chang S. Bae" To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, peterz@infradead.org, david.kaplan@amd.com, chang.seok.bae@intel.com Subject: [PATCH v2 03/11] stop_machine: Refactor multi-CPU stop glue code Date: Tue, 31 Mar 2026 01:42:41 +0000 Message-ID: <20260331014251.86353-4-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260331014251.86353-1-chang.seok.bae@intel.com> References: <20260331014251.86353-1-chang.seok.bae@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" stop_machine_cpuslocked() currently configures struct multi_stop_data and invokes the multi-stop operation. An upcoming stop_machine() variant will have different configurations but the latter part will be shareable. Extract the common part into stop_multi_cpus() to highlight each unique front-end. No functional change. Signed-off-by: Chang S. Bae --- V1 -> V2: New patch --- kernel/stop_machine.c | 35 ++++++++++++++++++++--------------- 1 file changed, 20 insertions(+), 15 deletions(-) diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c index 15268f1207e9..092c65c002ff 100644 --- a/kernel/stop_machine.c +++ b/kernel/stop_machine.c @@ -584,18 +584,8 @@ static int __init cpu_stop_init(void) } early_initcall(cpu_stop_init); =20 -int stop_machine_cpuslocked(cpu_stop_fn_t fn, void *data, - const struct cpumask *cpus) +static int stop_multi_cpus(struct multi_stop_data *msdata) { - struct multi_stop_data msdata =3D { - .fn =3D fn, - .data =3D data, - .num_threads =3D num_online_cpus(), - .active_cpus =3D cpus, - }; - - lockdep_assert_cpus_held(); - if (!stop_machine_initialized) { /* * Handle the case where stop_machine() is called @@ -605,19 +595,34 @@ int stop_machine_cpuslocked(cpu_stop_fn_t fn, void *d= ata, unsigned long flags; int ret; =20 - WARN_ON_ONCE(msdata.num_threads !=3D 1); + WARN_ON_ONCE(msdata->num_threads !=3D 1); =20 local_irq_save(flags); hard_irq_disable(); - ret =3D (*fn)(data); + ret =3D msdata->fn(msdata->data); local_irq_restore(flags); =20 return ret; } =20 /* Set the initial state and stop all online cpus. */ - set_state(&msdata, MULTI_STOP_PREPARE); - return stop_cpus(cpu_online_mask, multi_cpu_stop, &msdata); + set_state(msdata, MULTI_STOP_PREPARE); + return stop_cpus(cpu_online_mask, multi_cpu_stop, msdata); +} + +int stop_machine_cpuslocked(cpu_stop_fn_t fn, void *data, + const struct cpumask *cpus) +{ + struct multi_stop_data msdata =3D { + .fn =3D fn, + .data =3D data, + .num_threads =3D num_online_cpus(), + .active_cpus =3D cpus, + }; + + lockdep_assert_cpus_held(); + + return stop_multi_cpus(&msdata); } =20 int stop_machine(cpu_stop_fn_t fn, void *data, const struct cpumask *cpus) --=20 2.51.0 From nobody Thu Apr 2 01:08:41 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 791AF368269 for ; Tue, 31 Mar 2026 02:14:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923312; cv=none; b=ZmEcH2kxdCApduMykhnVyZ6BrW8zY94+YFOE7Oc9RUp6MhFrxWyiKM/3NXJqc+tXjJtPyVVYDMm+pSfAcA/2QzZbwny4M9cwSTUpuSkkwUSUnEQebtBJuqlJwxXMK/nj/Inw0rcmv+lJUFZswD5tYvtquz/Em87ZLkGayKLTLuE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923312; c=relaxed/simple; bh=29kx2ZjLTgH1i/16gpKyzlKrskEdV5uHiROvzlKKw38=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=paJQ7Nx/1LQj3ZJ7tkDbSZadWJP6NU8fJFZTr0RidenElopuDSXcJx3swOt239yJJpdq6ic+/ftTA02GCJ5GnkFlU4O3SmELs9Wnky1ZcRMprKooFlW6jKuZSPKs3ZLTPCAtW+SKJVUACZbxLxz3o0zqUkwwIwPPqTevZFvv7rw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=T54EODzL; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="T54EODzL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774923302; x=1806459302; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=29kx2ZjLTgH1i/16gpKyzlKrskEdV5uHiROvzlKKw38=; b=T54EODzLxwmkKPPpgPsRUG/8FTfaT4N4jwOL95nEbSzZe1dGfOGpm8Ie WCRug3CFB7XhAR5FmY9voCRXQqqh7I4FUMTwMBrEMSi+v39T/EVDez6fh 2dahnGvOgVGkwSmaIwH9VpcoyGD1iAPmvF0rmdhcE4extsc4BRMnJRo2k WweH+TQUx7l6p6ntwgCo/5zjuOBaJd5L6IJocSs+uRY9uemjyhG2GzXN5 nivDd9DZoce3KKtID7wN7EvOIeTQE9yOX/rHoJ2VsbZ3xFHkV2X3Bkler q7eWI7qM0/FNr19iZalVIEBRyABLIRftw0z5ptFCxPyHTJa1s2tMZrzH6 w==; X-CSE-ConnectionGUID: eEnnporIR3+G4nbo/M/pMA== X-CSE-MsgGUID: fEIdNXGjSciDPEw/+KDEKQ== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93508090" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93508090" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 19:07:46 -0700 X-CSE-ConnectionGUID: Tc7S211yTde7Pf/LNtLkfQ== X-CSE-MsgGUID: 2zk1LiPXTAGnBxE6XfJWJw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="221358685" Received: from chang-linux-3.sc.intel.com (HELO chang-linux-3) ([172.25.66.172]) by fmviesa006.fm.intel.com with ESMTP; 30 Mar 2026 19:07:42 -0700 From: "Chang S. Bae" To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, peterz@infradead.org, david.kaplan@amd.com, chang.seok.bae@intel.com Subject: [PATCH v2 04/11] stop_machine: Add NMI-based execution path Date: Tue, 31 Mar 2026 01:42:42 +0000 Message-ID: <20260331014251.86353-5-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260331014251.86353-1-chang.seok.bae@intel.com> References: <20260331014251.86353-1-chang.seok.bae@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently multi_cpu_stop() executes the stop function in the MULTI_STOP_RUN state. But NMIs can still preempt the execution. For use cases requiring stricter isolation from them, provide an option to run the stop function from NMI context. Then, the NMI stop function must be built without instrumentation, as exceptions may re-enable NMIs on return via IRET. Annotate the NMI handler entirely with noinstr. However, objtool cannot reliably determine whether the indirect call target is located in a noinstr section, so it may emit false positives. To avoid this, temporarily lift the instrumentation restriction around the indirect call site and document the intention. The x86 microcode loader is currently the primary user for this. But other architectures are not expected to use it. Add a build option to make it opt-in. Originally-by: David Kaplan Suggested-by: Borislav Petkov Suggested-by: Thomas Gleixner Signed-off-by: Chang S. Bae Link: https://lore.kernel.org/lkml/20260129121729.GRaXtP2aeWkQKegxC2@fat_cr= ate.local Link: https://lore.kernel.org/lkml/87wm0zl8p2.ffs@tglx --- V1 -> V2: Multiple changes * Fix racy return value read [**] * Add Kconfig option (Thomas) * Add cpumask to track NMI delivery (Boris) * Rework implementation. Consolidate under ifdef [**]: Observed delay in IPI/NMI delivery when testing error return paths --- arch/Kconfig | 3 ++ include/linux/stop_machine.h | 16 ++++++ kernel/stop_machine.c | 96 +++++++++++++++++++++++++++++++++++- 3 files changed, 114 insertions(+), 1 deletion(-) diff --git a/arch/Kconfig b/arch/Kconfig index 102ddbd4298e..f84fd528aae7 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1841,4 +1841,7 @@ config ARCH_WANTS_PRE_LINK_VMLINUX config ARCH_HAS_CPU_ATTACK_VECTORS bool =20 +config STOP_MACHINE_NMI + bool + endmenu diff --git a/include/linux/stop_machine.h b/include/linux/stop_machine.h index 2f986555113a..9424d363ab38 100644 --- a/include/linux/stop_machine.h +++ b/include/linux/stop_machine.h @@ -19,6 +19,12 @@ */ typedef int (*cpu_stop_fn_t)(void *arg); =20 +/* + * Stop function variant runnable from NMI context. This makes the + * noinstr requirement explicit at the type level. + */ +typedef int (*cpu_stop_nmisafe_fn_t)(void *arg); + #ifdef CONFIG_SMP =20 struct cpu_stop_work { @@ -189,4 +195,14 @@ stop_machine_from_inactive_cpu(cpu_stop_fn_t fn, void = *data, } =20 #endif /* CONFIG_SMP || CONFIG_HOTPLUG_CPU */ + +#ifdef CONFIG_STOP_MACHINE_NMI + +void arch_send_self_nmi(void); +bool noinstr stop_machine_nmi_handler(void); + +#else +static inline bool stop_machine_nmi_handler(void) { return false; } +#endif /* CONFIG_STOP_MACHINE_NMI */ + #endif /* _LINUX_STOP_MACHINE */ diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c index 092c65c002ff..45ea62f1b2b5 100644 --- a/kernel/stop_machine.c +++ b/kernel/stop_machine.c @@ -22,6 +22,7 @@ #include #include #include +#include =20 /* * Structure to determine completion condition and record errors. May @@ -174,6 +175,15 @@ struct multi_stop_data { =20 enum multi_stop_state state; atomic_t thread_ack; + +#ifdef CONFIG_STOP_MACHINE_NMI + /* Used in the NMI stop_machine variant */ + bool use_nmi; + /* A separate function type to highlight noinstr requirement */ + cpu_stop_nmisafe_fn_t nmisafe_fn; + /* cpumask of CPUs on which to raise an NMI */ + cpumask_var_t nmi_cpus; +#endif }; =20 static void set_state(struct multi_stop_data *msdata, @@ -197,6 +207,8 @@ notrace void __weak stop_machine_yield(const struct cpu= mask *cpumask) cpu_relax(); } =20 +static int multi_stop_run(struct multi_stop_data *msdata); + /* This is the cpu_stop function which stops the CPU. */ static int multi_cpu_stop(void *data) { @@ -235,7 +247,7 @@ static int multi_cpu_stop(void *data) break; case MULTI_STOP_RUN: if (is_active) - err =3D msdata->fn(msdata->data); + err =3D multi_stop_run(msdata); break; default: break; @@ -712,3 +724,85 @@ int stop_machine_from_inactive_cpu(cpu_stop_fn_t fn, v= oid *data, mutex_unlock(&stop_cpus_mutex); return ret | done.ret; } + +#ifdef CONFIG_STOP_MACHINE_NMI + +struct nmi_stop { + struct multi_stop_data *data; + int ret; + bool done; +}; + +static DEFINE_PER_CPU(struct nmi_stop, nmi_stop); + +/* + * Instrumentation may trigger nested exceptions such as #INT3, #DB, + * or #PF. IRET from those would re-enable NMIs, which opposes the goal + * of this NMI stop-machine facility. + */ +bool noinstr stop_machine_nmi_handler(void) +{ + struct multi_stop_data *msdata =3D raw_cpu_read(nmi_stop.data); + unsigned int cpu =3D smp_processor_id(); + int ret; + + if (!msdata || !cpumask_test_and_clear_cpu(cpu, msdata->nmi_cpus)) + return false; + + /* + * The indirect call to @nmisafe_fn() is indistinguishable at + * post-compilation. Temporarily enabling instrumentation avoids + * objtool false positives. + */ + instrumentation_begin(); + ret =3D msdata->nmisafe_fn(msdata->data); + instrumentation_end(); + + raw_cpu_write(nmi_stop.ret, ret); + raw_cpu_write(nmi_stop.done, true); + raw_cpu_write(nmi_stop.data, NULL); + + return true; +} + +static bool wait_for_nmi_handler(void) +{ + /* Conservative timeout */ + unsigned long timeout =3D USEC_PER_SEC; + + while (!this_cpu_read(nmi_stop.done) && timeout--) + udelay(1); + + return this_cpu_read(nmi_stop.done); +} + +static int nmi_stop_run(struct multi_stop_data *msdata) +{ + /* + * Save per-CPU state accessible from NMI context and raise a + * self-NMI to execute the stop function from the NMI handler + */ + this_cpu_write(nmi_stop.data, msdata); + this_cpu_write(nmi_stop.done, false); + arch_send_self_nmi(); + + /* Ensure the handler went through before reading the result */ + if (!wait_for_nmi_handler()) + return -ETIMEDOUT; + + return this_cpu_read(nmi_stop.ret); +} + +static int multi_stop_run(struct multi_stop_data *msdata) +{ + return msdata->use_nmi ? nmi_stop_run(msdata) : msdata->fn(msdata->data); +} + +#else + +static int multi_stop_run(struct multi_stop_data *msdata) +{ + return msdata->fn(msdata->data); +} + +#endif /* CONFIG_STOP_MACHINE_NMI */ --=20 2.51.0 From nobody Thu Apr 2 01:08:41 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2137436BCF2 for ; Tue, 31 Mar 2026 02:14:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923305; cv=none; b=MO6N+PmebtesgIKAVxJGdoJMfVKYaclAayasMWovSrbk8wK+Jmt4AR0Tanx499VXCLElVtBQjsTRV7KwdKPX9E1VKSwYDAJVlgoVrUWrK1jKqkLvXyAHJhV1uLTCzhYLCB1kYlYxj4FkpQVcSzdOI2Kgg9zmfm/+o8psltBG0Mw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923305; c=relaxed/simple; bh=pTRggg7XBjD2T4wKwJT9pAS3nvh+xTpTxzElTaSZrrw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IpQb9xCX7zmix3B325ZLoZQNwI06mRJsRBvvS4DsDa5mANdy/Scpb/G3T1AgB/N+b1JeWL7+b0m7HVvcVOgY8rQaMWMFcIEERGEF6X9XhKOY87U17Ed80gsDUw1wsQPTVueaV1kF+BmUX/HS6u44SOvrmHk1X7wYHeTtibNFXhg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=VD8577Ta; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="VD8577Ta" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774923299; x=1806459299; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pTRggg7XBjD2T4wKwJT9pAS3nvh+xTpTxzElTaSZrrw=; b=VD8577TabU88bUUErWcmzy0HGt2Fgstasis2jN8xp28wgJIQts6is/Jd SE9Iz9NbSmTG7AD6eoUSfnNo3NmmpAewxlRKtIemRlOcYlboCI2xvBNzG lN+I8IPdiLs5QOMD3CvoXQjyGPTsiwkLnjJ1cX1wIUicCjCiaiwj1eNQ+ VAFGjqp6oePHn8VxvwEbNBpDsavfN1+airoClSNN+zTRA/xZzN4VY1kTP k6asaobWViRNqedgWZQKIyRj3xCdUyM2gvNVNb/GH2/qIyHQBFLVS4LXd sP3+nsqzpM6kuI9jjyHjOpfhXPMID6xtk+GFs/PQ9IDW9ISjf1UmL2esh A==; X-CSE-ConnectionGUID: pCV9g4QBRGauSvxFakTAkA== X-CSE-MsgGUID: gJrsWZ+ER5WMUDRaPFXZdg== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93508096" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93508096" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 19:07:46 -0700 X-CSE-ConnectionGUID: am2yAA1LTrqni8TI9uO9ig== X-CSE-MsgGUID: zJtlH5vQTY+PMZHixwVQhw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="221358694" Received: from chang-linux-3.sc.intel.com (HELO chang-linux-3) ([172.25.66.172]) by fmviesa006.fm.intel.com with ESMTP; 30 Mar 2026 19:07:43 -0700 From: "Chang S. Bae" To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, peterz@infradead.org, david.kaplan@amd.com, chang.seok.bae@intel.com Subject: [PATCH v2 05/11] stop_machine: Introduce stop_machine_nmi_cpuslocked() Date: Tue, 31 Mar 2026 01:42:43 +0000 Message-ID: <20260331014251.86353-6-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260331014251.86353-1-chang.seok.bae@intel.com> References: <20260331014251.86353-1-chang.seok.bae@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" With the NMI control logic in place, introduce an API to run the target function from NMI context. Originally-by: David Kaplan Suggested-by: Borislav Petkov Signed-off-by: Chang S. Bae Link: https://lore.kernel.org/lkml/20260202105411.GVaYCCUygtEUNrMUtG@fat_cr= ate.local --- V1 -> V2: * Support nmi_cpus mask (Boris), including @cpus=3DNULL cases * Split out API introduction * Drop out stop_machine_nmi() [**] [**] could be added but no user yet in this series --- include/linux/stop_machine.h | 24 ++++++++++++++++++++ kernel/stop_machine.c | 43 ++++++++++++++++++++++++++++++++++++ 2 files changed, 67 insertions(+) diff --git a/include/linux/stop_machine.h b/include/linux/stop_machine.h index 9424d363ab38..2da9aa0ec3d3 100644 --- a/include/linux/stop_machine.h +++ b/include/linux/stop_machine.h @@ -201,6 +201,30 @@ stop_machine_from_inactive_cpu(cpu_stop_fn_t fn, void = *data, void arch_send_self_nmi(void); bool noinstr stop_machine_nmi_handler(void); =20 +/** + * stop_machine_nmi_cpuslocked() - Freeze CPUs and run a function in NMI c= ontext + * + * @nmisafe_fn: The function to run + * @data: The data pointer for @nmisafe_fn() + * @cpus: A cpumask containing the CPUs to run @nmisafe_fn() on. If NULL, + * @nmisafe_fn() runs on a single (arbitrary) CPU from + * cpu_online_mask. + * + * Description: This stop_machine() variant runs @nmisafe_fn() from NMI co= ntext + * to prevent preemption by other NMIs. The callback must be built with no= instr. + * Other than that, the semantics match stop_machine_cpuslocked(). + * + * Context: Must be called from within a cpus_read_lock() protected region. + * Avoid nested calls to cpus_read_lock(). + * + * Return: 0 if all invocations of @nmisafe_fn return zero, -ENOMEM if cpu= mask + * allocation fails, -EINVAL if any target CPU failed to receive NMI. Othe= rwise, + * an accumulated return value from all invocation of @nmisafe_fn that ret= urned + * non-zero. + */ +int stop_machine_nmi_cpuslocked(cpu_stop_nmisafe_fn_t nmisafe_fn, void *da= ta, + const struct cpumask *cpus); + #else static inline bool stop_machine_nmi_handler(void) { return false; } #endif /* CONFIG_STOP_MACHINE_NMI */ diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c index 45ea62f1b2b5..e20e4d3e7b16 100644 --- a/kernel/stop_machine.c +++ b/kernel/stop_machine.c @@ -798,6 +798,49 @@ static int multi_stop_run(struct multi_stop_data *msda= ta) return msdata->use_nmi ? nmi_stop_run(msdata) : msdata->fn(msdata->data); } =20 +int stop_machine_nmi_cpuslocked(cpu_stop_nmisafe_fn_t nmisafe_fn, void *da= ta, + const struct cpumask *cpus) +{ + struct multi_stop_data msdata =3D { + .nmisafe_fn =3D nmisafe_fn, + .data =3D data, + .num_threads =3D num_online_cpus(), + .active_cpus =3D cpus, + .use_nmi =3D true, + }; + int ret; + + if (!zalloc_cpumask_var(&msdata.nmi_cpus, GFP_KERNEL)) + return -ENOMEM; + + /* + * NMI CPUs should be exactly those 'active' CPUs executing the + * stop function. Follow the selection logic in multi_cpu_stop() + * if not provided. + */ + if (!msdata.active_cpus) + cpumask_set_cpu(cpumask_first(cpu_online_mask), msdata.nmi_cpus); + else + cpumask_copy(msdata.nmi_cpus, msdata.active_cpus); + + lockdep_assert_cpus_held(); + + ret =3D stop_multi_cpus(&msdata); + + /* + * The NMI handler clears each CPU bit. If any of those NMIs were + * ever missed out, return error clearly. + */ + if (!cpumask_empty(msdata.nmi_cpus)) { + pr_err("CPUs %*pbl didn't run the stop_machine NMI handler.\n", + cpumask_pr_args(msdata.nmi_cpus)); + ret =3D -EINVAL; + } + + free_cpumask_var(msdata.nmi_cpus); + return ret; +} + #else =20 static int multi_stop_run(struct multi_stop_data *msdata) --=20 2.51.0 From nobody Thu Apr 2 01:08:41 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3847A36655F for ; Tue, 31 Mar 2026 02:14:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923312; cv=none; b=FFKC0aV1qX8h+zDnegxljzGch5gUpGEg4u7CpfvtWSyPbVZmTq0Se3DcX/DUMRckZFb/wMh/Gdl1qq/P7UmvTFu4Jod0KiJ5QPY5w53gsAgiDdQKlEgQAbg4dnrRhxcUO1bjgzWW6gmWNw7TpAAoD/Eh9z1l24eitI8oB883QB0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923312; c=relaxed/simple; bh=TfjFXIHkuW170vXqfJPDenhxgs1bqGIqJUs4bTky7pc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fvdQBiVWJYAGorTM4rVhn/9ccShYFQo6O66wA1cRXvUnE7DKuu6VF+9V2DnyvEV4/x4FgwVDPhPrkS08Ofi8PYe9FuWkj1IfxHcmooV6XSdIqHsyx3eNuigk0iGN7bRVjD7R/O3Yg5JS/UPv9y9bVrDiD7dgWv60yMnQNhZpDec= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=KLn6oQfN; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="KLn6oQfN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774923302; x=1806459302; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TfjFXIHkuW170vXqfJPDenhxgs1bqGIqJUs4bTky7pc=; b=KLn6oQfNqAdX5HlHVCmXh0RnL9fgyVioBGqgfBig2WSuQvQXdo6jf7WL YUdE9h6zmdaVsW24lWUmdhjC5iwcfnj5isxJGKySjds7L9R2rV7WIjT15 CqcGczEp9HNowb85xKJm0sT7RSem/625uEgB9fqnizLr1oWj56SnogzVO vuxZmCNHoiGeNLhrVLhGCuhWSg6hddixyPOOpmAIxSmTUybZML4w3bVNJ 20sy3uayeIBMt64F5U9tpm0NN21iHRGBXUrifWnUBoNCr+SQJXXWl4WsF wh2og9Dx9QZ+/IUgGTFSUUJ9ahle50T8xzx8To+m+Fiu23zNjjhk72VcU A==; X-CSE-ConnectionGUID: o6q50zK+SBO302CjlGz03w== X-CSE-MsgGUID: nJMFC8PZSZyVWCr2DDvAIg== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93508102" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93508102" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 19:07:46 -0700 X-CSE-ConnectionGUID: s1bAR/G7SOy/vkSfawh/LQ== X-CSE-MsgGUID: 47v05ZWqSjOIpmuw5dtuCw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="221358700" Received: from chang-linux-3.sc.intel.com (HELO chang-linux-3) ([172.25.66.172]) by fmviesa006.fm.intel.com with ESMTP; 30 Mar 2026 19:07:45 -0700 From: "Chang S. Bae" To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, peterz@infradead.org, david.kaplan@amd.com, chang.seok.bae@intel.com Subject: [PATCH v2 06/11] x86/apic: Implement self-NMI support Date: Tue, 31 Mar 2026 01:42:44 +0000 Message-ID: <20260331014251.86353-7-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260331014251.86353-1-chang.seok.bae@intel.com> References: <20260331014251.86353-1-chang.seok.bae@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Kaplan Add a function to send an NMI-to-self which is needed for stop-machine NMI facility. Note that send_IPI_self() cannot be used to send an NMI so send_IPI() is used. Signed-off-by: David Kaplan Signed-off-by: Chang S. Bae --- V1 -> V2: Fix a typo (Thomas) and wrap it under the new build option --- arch/x86/kernel/apic/ipi.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/x86/kernel/apic/ipi.c b/arch/x86/kernel/apic/ipi.c index 98a57cb4aa86..597f03192a49 100644 --- a/arch/x86/kernel/apic/ipi.c +++ b/arch/x86/kernel/apic/ipi.c @@ -4,6 +4,7 @@ #include #include #include +#include =20 #include =20 @@ -248,6 +249,15 @@ void default_send_IPI_self(int vector) __default_send_IPI_shortcut(APIC_DEST_SELF, vector); } =20 +#ifdef CONFIG_STOP_MACHINE_NMI + +void arch_send_self_nmi(void) +{ + apic->send_IPI(smp_processor_id(), NMI_VECTOR); +} + +#endif + #ifdef CONFIG_X86_32 void default_send_IPI_mask_sequence_logical(const struct cpumask *mask, in= t vector) { --=20 2.51.0 From nobody Thu Apr 2 01:08:41 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ADB9A365A12 for ; Tue, 31 Mar 2026 02:15:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923330; cv=none; b=u2joBGJvhGFKK4sUuEKCBIXs/7ir2Hme24saudrkUaThMOiYhPNRRURfQskaJVbmJYCc68zUrgaM+M+AJbGFHJv0K2RewjxgxkC1J16nZZb/asFh3+6/eihfYjyaiT2B0fIOKfORm5SIRVvhLcmdrN6hB5USl3mUqC2aG6NpIxo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923330; c=relaxed/simple; bh=izHRzXJgV7CxACyoH4KHjGpZsKjWF7DOxEVlUNzWCvQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Nwa//zVZ8tXH2xmLPwJZPwgEbhlvIf9Cww+sh881k3vNoC84t8O8ke9t2hum4gn5GPljlkjkebhO6OEKSeSLCiNzU8nKfKYfA0J/4BpijrGnmNN1R/MR1P0mealcMb6/chghiOC8zh9nuQdDWmWDQUCP4Fy01hIIn2rEge1aeOU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Xv+LKnDC; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Xv+LKnDC" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774923330; x=1806459330; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=izHRzXJgV7CxACyoH4KHjGpZsKjWF7DOxEVlUNzWCvQ=; b=Xv+LKnDC6NBOEiFold+6cg3DHOdo52xfepccxto4tKLOlsZN2ykqMrLd k5ydfY+yoC7xt9J1whMt4iArX5nvBzcbBe+5FbBdFwSBEvUtdZ8VQiB2q gi+49+XbWZWlaguftOElH2YV49gdMvSENu0sHEhqaec3ttDDJIbDMOdL0 5Z3g5TcqeV+E/fPP6N7OleZlvlA8PiQ4ddxldc+rF6x9th6z88AgTNmWU wmtNlCQJSGyuCmssJkgBT5hWFmn4VnbyaJFpwyN0DNHyAlM7Ei5S/Hgpu s5jTHhHZvqs+bXw2hdcuUj9kj4WXL2OiHW/UQA5tv0nlLH/Fji8QnZkGH A==; X-CSE-ConnectionGUID: 1lhNo2V0T8mra59k2Mlm/g== X-CSE-MsgGUID: 6pugh9uSQxaqQJ8J3S+86g== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93508158" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93508158" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 19:07:52 -0700 X-CSE-ConnectionGUID: Qs7rGYSyTJapr48pwcQ7KQ== X-CSE-MsgGUID: ksu7KxOhRNq9wvixAMV8WQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="221358707" Received: from chang-linux-3.sc.intel.com (HELO chang-linux-3) ([172.25.66.172]) by fmviesa006.fm.intel.com with ESMTP; 30 Mar 2026 19:07:47 -0700 From: "Chang S. Bae" To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, peterz@infradead.org, david.kaplan@amd.com, chang.seok.bae@intel.com Subject: [PATCH v2 07/11] x86/nmi: Support NMI stop-machine handler Date: Tue, 31 Mar 2026 01:42:45 +0000 Message-ID: <20260331014251.86353-8-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260331014251.86353-1-chang.seok.bae@intel.com> References: <20260331014251.86353-1-chang.seok.bae@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Kaplan Call the stop-machine handler from the NMI path in order to support the NMI stop-machine. Signed-off-by: David Kaplan Signed-off-by: Chang S. Bae --- V1 -> V2: Switch away from static key reference --- arch/x86/kernel/nmi.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index 3d239ed12744..b7ea2907142c 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -24,6 +24,7 @@ #include #include #include +#include #include =20 #include @@ -382,6 +383,9 @@ static noinstr void default_do_nmi(struct pt_regs *regs) =20 instrumentation_begin(); =20 + if (stop_machine_nmi_handler()) + goto out; + if (microcode_nmi_handler_enabled() && microcode_nmi_handler()) goto out; =20 --=20 2.51.0 From nobody Thu Apr 2 01:08:41 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8FEF366DA5 for ; Tue, 31 Mar 2026 02:15:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923331; cv=none; b=NJx8MCde3iNUqtOT9iOtOynARl5i+Y8BinSV7esHUcLCGe8uxzbGw9au+md9V4x9tURSP3aVZWO+MnfsdMpSxTcY1ASmk/afLwWoFLUV9XhF2w/2ZdhoZRznaA17wg8R7nEaoF6usveYXbgwMD3Yl+XoH2JFv0TXEcjAADNaSA0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923331; c=relaxed/simple; bh=8jYbdlHHDRN/ZXZosS7i9KR7L3WkD3vpxCBE6MNpgLg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nDVIa7+KW6n6EhJTUWRNMUEVYInE0zQO3sJ/HLLYd4puEImS8vx7wblZGWiByBXis19lHcrY+Xky8DtPL8wVN8uuVUqF/j9ei3Hj8a3Sv9e1gjQkz8y6BW8d1WHAesZkUrxSxHO4Jc8ty9r+BZT5peFWVD/xfSVOMEqr5v8/h0A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=gaSpggwv; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="gaSpggwv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774923330; x=1806459330; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8jYbdlHHDRN/ZXZosS7i9KR7L3WkD3vpxCBE6MNpgLg=; b=gaSpggwvxItJ6UUyM7BzCYbjVz8a0p3Ht/1331nHXuf7qwnWJTurPp9J /WnTIkyAMao4ezsdCjDSdIA8H+wPO2dRy0takIXHHc+PuNxZDgwzcd/65 BkhS8xHZfYYz3JnfvxNqVgcE0f9hygCOdwN10S6xEL++x3BHQCkjbHvli k0ZGSs1Hv2pxsSAuWIKLkOirh6r9XzEX6lBzWbrI33tWTxuKDOHT4Tr+P ry1QEkoDqZ6CVPs7bybRo2YcBQ9Wped/0NRYX+okGAEQ+eejGoBR0IRIf zauTipgrl/QYIEjoKS0Ia/7XXOPnUXs24UI0BZvH4tw/ZlVu/jh5XGmnW g==; X-CSE-ConnectionGUID: 36WKEGxWQeeo96Xo9tB/pA== X-CSE-MsgGUID: NJa4QR4hQOKmdiJj84i7FA== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93508151" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93508151" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 19:07:52 -0700 X-CSE-ConnectionGUID: 1WP/eizETq2pJT/Zr/LijQ== X-CSE-MsgGUID: JrtncTTbRUqWJX45vx60hw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="221358712" Received: from chang-linux-3.sc.intel.com (HELO chang-linux-3) ([172.25.66.172]) by fmviesa006.fm.intel.com with ESMTP; 30 Mar 2026 19:07:48 -0700 From: "Chang S. Bae" To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, peterz@infradead.org, david.kaplan@amd.com, chang.seok.bae@intel.com Subject: [PATCH v2 08/11] x86/microcode: Distinguish NMI control path on stop-machine callback Date: Tue, 31 Mar 2026 01:42:46 +0000 Message-ID: <20260331014251.86353-9-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260331014251.86353-1-chang.seok.bae@intel.com> References: <20260331014251.86353-1-chang.seok.bae@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" load_cpus_stopped() currently centralizes the stop_machine() callback for both NMI and NMI-less rendezvous. microcode_update_handler() alone is enough for the latter. While the NMI-based rendezvous finally reaches the same update handler, it requires additional logic to trigger and process NMIs. That machinery will be replaced by stop-machine facility. As preparation for that conversion, split the callback path to make NMI-specific steps explicit and clear. Rename the function to align with the change. Signed-off-by: Chang S. Bae --- V1 -> V2: Rename load_cpus_stopped() (Thomas) --- arch/x86/kernel/cpu/microcode/core.c | 30 ++++++++++++---------------- 1 file changed, 13 insertions(+), 17 deletions(-) diff --git a/arch/x86/kernel/cpu/microcode/core.c b/arch/x86/kernel/cpu/mic= rocode/core.c index 651202e6fefb..abd640b1d286 100644 --- a/arch/x86/kernel/cpu/microcode/core.c +++ b/arch/x86/kernel/cpu/microcode/core.c @@ -532,7 +532,7 @@ void noinstr microcode_offline_nmi_handler(void) wait_for_ctrl(); } =20 -static noinstr bool microcode_update_handler(void) +static noinstr int microcode_update_handler(void *unused) { unsigned int cpu =3D raw_smp_processor_id(); =20 @@ -548,7 +548,7 @@ static noinstr bool microcode_update_handler(void) touch_nmi_watchdog(); instrumentation_end(); =20 - return true; + return 0; } =20 /* @@ -569,19 +569,15 @@ bool noinstr microcode_nmi_handler(void) return false; =20 raw_cpu_write(ucode_ctrl.nmi_enabled, false); - return microcode_update_handler(); + return microcode_update_handler(NULL) =3D=3D 0; } =20 -static int load_cpus_stopped(void *unused) +static int stop_cpu_in_nmi(void *unused) { - if (microcode_ops->use_nmi) { - /* Enable the NMI handler and raise NMI */ - this_cpu_write(ucode_ctrl.nmi_enabled, true); - apic->send_IPI(smp_processor_id(), NMI_VECTOR); - } else { - /* Just invoke the handler directly */ - microcode_update_handler(); - } + /* Enable the NMI handler and raise NMI */ + this_cpu_write(ucode_ctrl.nmi_enabled, true); + apic->send_IPI(smp_processor_id(), NMI_VECTOR); + return 0; } =20 @@ -618,13 +614,13 @@ static int load_late_stop_cpus(bool is_safe) */ store_cpu_caps(&prev_info); =20 - if (microcode_ops->use_nmi) + if (microcode_ops->use_nmi) { static_branch_enable_cpuslocked(µcode_nmi_handler_enable); - - stop_machine_cpuslocked(load_cpus_stopped, NULL, cpu_online_mask); - - if (microcode_ops->use_nmi) + stop_machine_cpuslocked(stop_cpu_in_nmi, NULL, cpu_online_mask); static_branch_disable_cpuslocked(µcode_nmi_handler_enable); + } else { + stop_machine_cpuslocked(microcode_update_handler, NULL, cpu_online_mask); + } =20 /* Analyze the results */ for_each_cpu_and(cpu, cpu_present_mask, &cpus_booted_once_mask) { --=20 2.51.0 From nobody Thu Apr 2 01:08:41 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D0EA271A71 for ; Tue, 31 Mar 2026 02:15:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923329; cv=none; b=CGX9+uNCf5PWsQa+UvqOCJYlaf9lPveOn3qbMH7Tolk4ubdiV37sCDvO3c5x+fU7l4iS0QyBFgSNDZST5zDB60Soh4lNTchPGbTRnPLlt1gK2bQ8T3M56bQELcxcrfqB+vCDECvxXFEzMd661xTer3lQ/P+9TqIitKYM0g/Wses= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923329; c=relaxed/simple; bh=Yt3P3BSNSYGhqiOGU8DHhre+e+biKLJ/omkG6TP4WFQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KOSDnfnQ0dKZBqdpfILxpFk9teDBOg70Etmc+AZUrdA9bdDjrmB10prSyBeiLERYLPLau97LIyJW8CmJg4Z6/a2K72WK+5YHKG0Ys2EANOXfxL1SUSIPYlKR7EBb4SEPU3JRn6Ue/AKWqvMaChDkf6+PniiXzjHl64jJ9BnfF3U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=OaI22pIr; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="OaI22pIr" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774923328; x=1806459328; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Yt3P3BSNSYGhqiOGU8DHhre+e+biKLJ/omkG6TP4WFQ=; b=OaI22pIrFWXjSCx30E1V/qzNE46JgQTea7P2pFseQQnye5vUWBVVb2wQ znrnKeZrqytwoNSRCm8/vPYAfbaVfHfoNO57ki8HhAMQ0rZR6NojIE7S+ 0tTv4PexVrAIDl0Xjgwlt5povmUH4b/GJbBlV5a1BYFlNZVc4HfhE3w7t ajo+37YE4wVvPnUWntmWVLN1TVMegtrEOhxhvLYVJB9Gy0vfaGHg5aj9q 3i05LK9yGgv+h4Dvxu6yCKt4YN8wiGc8VLPiViXNaB8qXelvpVuG8usSK eMfhXYvHul1xPE69DZ1kNMDB2Vw6AKlvtew9Q0n4xvtsQqA1tms5VVYDA A==; X-CSE-ConnectionGUID: ZiqF3P1XSkOHH2S0nWF+Ag== X-CSE-MsgGUID: SPtNssMDSEOA8kHXAOglkg== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93508145" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93508145" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 19:07:52 -0700 X-CSE-ConnectionGUID: QmyqTaBQS4qhAojc8JWowA== X-CSE-MsgGUID: 7oAwMREMTQWMBF6CubiO0A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="221358718" Received: from chang-linux-3.sc.intel.com (HELO chang-linux-3) ([172.25.66.172]) by fmviesa006.fm.intel.com with ESMTP; 30 Mar 2026 19:07:50 -0700 From: "Chang S. Bae" To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, peterz@infradead.org, david.kaplan@amd.com, chang.seok.bae@intel.com Subject: [PATCH v2 09/11] x86/microcode: Use stop-machine NMI facility Date: Tue, 31 Mar 2026 01:42:47 +0000 Message-ID: <20260331014251.86353-10-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260331014251.86353-1-chang.seok.bae@intel.com> References: <20260331014251.86353-1-chang.seok.bae@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The existing NMI-based loading logic explicitly sends NMIs to online CPUs and invokes microcode_update_handler() from the NMI context. The stop-machine NMI variant already provides the mechanism on x86. Replace the custom NMI control logic with stop_machine_nmi_cpuslocked(). Signed-off-by: Chang S. Bae --- V1 -> V2: Select that stop-machine build option --- arch/x86/Kconfig | 1 + arch/x86/include/asm/microcode.h | 1 - arch/x86/kernel/cpu/microcode/core.c | 19 +++---------------- arch/x86/kernel/nmi.c | 3 --- 4 files changed, 4 insertions(+), 20 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 0b5f30d769ff..0f7e88ba7433 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1338,6 +1338,7 @@ config MICROCODE_LATE_LOADING bool "Late microcode loading (DANGEROUS)" default n depends on MICROCODE && SMP + select STOP_MACHINE_NMI help Loading microcode late, when the system is up and executing instructions is a tricky business and should be avoided if possible. Just the sequen= ce diff --git a/arch/x86/include/asm/microcode.h b/arch/x86/include/asm/microc= ode.h index 3c317d155771..62d10c43da9c 100644 --- a/arch/x86/include/asm/microcode.h +++ b/arch/x86/include/asm/microcode.h @@ -79,7 +79,6 @@ static inline u32 intel_get_microcode_revision(void) } #endif /* !CONFIG_CPU_SUP_INTEL */ =20 -bool microcode_nmi_handler(void); void microcode_offline_nmi_handler(void); =20 #ifdef CONFIG_MICROCODE_LATE_LOADING diff --git a/arch/x86/kernel/cpu/microcode/core.c b/arch/x86/kernel/cpu/mic= rocode/core.c index abd640b1d286..ebcc73e67af1 100644 --- a/arch/x86/kernel/cpu/microcode/core.c +++ b/arch/x86/kernel/cpu/microcode/core.c @@ -563,22 +563,9 @@ static noinstr int microcode_update_handler(void *unus= ed) * path which must be NMI safe until the primary thread completed the * update. */ -bool noinstr microcode_nmi_handler(void) +static noinstr int microcode_nmi_handler(void *data) { - if (!raw_cpu_read(ucode_ctrl.nmi_enabled)) - return false; - - raw_cpu_write(ucode_ctrl.nmi_enabled, false); - return microcode_update_handler(NULL) =3D=3D 0; -} - -static int stop_cpu_in_nmi(void *unused) -{ - /* Enable the NMI handler and raise NMI */ - this_cpu_write(ucode_ctrl.nmi_enabled, true); - apic->send_IPI(smp_processor_id(), NMI_VECTOR); - - return 0; + return microcode_update_handler(data); } =20 static int load_late_stop_cpus(bool is_safe) @@ -616,7 +603,7 @@ static int load_late_stop_cpus(bool is_safe) =20 if (microcode_ops->use_nmi) { static_branch_enable_cpuslocked(µcode_nmi_handler_enable); - stop_machine_cpuslocked(stop_cpu_in_nmi, NULL, cpu_online_mask); + stop_machine_nmi_cpuslocked(microcode_nmi_handler, NULL, cpu_online_mask= ); static_branch_disable_cpuslocked(µcode_nmi_handler_enable); } else { stop_machine_cpuslocked(microcode_update_handler, NULL, cpu_online_mask); diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index b7ea2907142c..324f4353be88 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -386,9 +386,6 @@ static noinstr void default_do_nmi(struct pt_regs *regs) if (stop_machine_nmi_handler()) goto out; =20 - if (microcode_nmi_handler_enabled() && microcode_nmi_handler()) - goto out; - /* * CPU-specific NMI must be processed before non-CPU-specific * NMI, otherwise we may lose it, because the CPU-specific --=20 2.51.0 From nobody Thu Apr 2 01:08:41 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21C7036A030 for ; Tue, 31 Mar 2026 02:15:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923332; cv=none; b=tM2XBMlFWFcVNOH43jKkinq6NqpowHV/c0/pBdD3jZlPLUgAr9gXZcM/MQYVCmW6ijXs3y4M8FWmHWwXB/DPjBC6NwACS1udUyC4gv0vzsQpIPhwhOVzZ7nFX+XOpxVmV5ZTJU94paBa4GYspkYdBc3xFb06SJzJ8uZ7cgk+Y1U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923332; c=relaxed/simple; bh=ZW6pGC9Nm5eFl1qOtWzebKKSgLIJNOayuOAQAQVGwGk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Bhcv+Rs7dgvsrOnk9SlO67t4MIcakii9RHHt4BZBVQMGbGlHY6TolcfETUYkEwJHm+/WqM9/FC0v1PGdErulI2uyB0tJDMfZuC2BSgjjuIxyWD3EtKYvAfTxJe5A0Y+G+Iw2jtu4Osbps6WQT/H6HmM3kbpzjnAGItaXPAqFByg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=TrrQzfgu; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="TrrQzfgu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774923331; x=1806459331; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZW6pGC9Nm5eFl1qOtWzebKKSgLIJNOayuOAQAQVGwGk=; b=TrrQzfguNV3GfOc2fwDlEsrWz1EWSa6a3jtyHCQ1/wG2XoL5y50MihKK YHMbcYCGnEFZqmIwvVgNFY5nCLmmKuyejeZbdt9YeNvxQ8rgvK2qPkY6j LMeF37EHG4eCj+uIY7BoJITOJFzxFo8flGMoc9yLzDUel8qFzzHVWOvFe d145MiAIMnBD548HbXJGQL/HFfUxebWHHkBfUen6ee7meIziojBrEzb2h T9xHrUoHwewSsTlJAeP6xMSfnXmyubcya0K9bnVMSXrf4q572qtfFF3Wd xcK/IeTG82lOJkR1vhEiisOr2YEbue86zJ5nGlM2AwYUqx3+zDthCPX4f A==; X-CSE-ConnectionGUID: XgA60q6bTYGkU0bjkQSZAA== X-CSE-MsgGUID: Pf1yW5G/RsaYFE/pjiza/Q== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93508186" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93508186" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 19:07:58 -0700 X-CSE-ConnectionGUID: 7gYVqMrHTtG71EqVEykQmQ== X-CSE-MsgGUID: YSXsK7ucRtunKuYd+bLrMg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="221358722" Received: from chang-linux-3.sc.intel.com (HELO chang-linux-3) ([172.25.66.172]) by fmviesa006.fm.intel.com with ESMTP; 30 Mar 2026 19:07:52 -0700 From: "Chang S. Bae" To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, peterz@infradead.org, david.kaplan@amd.com, chang.seok.bae@intel.com Subject: [PATCH v2 10/11] x86/nmi: Simplify offline microcode handler invocation Date: Tue, 31 Mar 2026 01:42:48 +0000 Message-ID: <20260331014251.86353-11-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260331014251.86353-1-chang.seok.bae@intel.com> References: <20260331014251.86353-1-chang.seok.bae@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The microcode loader now uses the stop-machine facility for NMI-based loading. The offline handler is the only remaining user of the microcode NMI static key. But the handler already has internal guarding. Offline CPU doesn't matter how fast it handles anyway, so remove the static key. Instead, check the build option. Signed-off-by: Chang S. Bae --- V1 -> V2: Switch away from static key reference --- arch/x86/kernel/nmi.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c index 324f4353be88..4ca891f7075f 100644 --- a/arch/x86/kernel/nmi.c +++ b/arch/x86/kernel/nmi.c @@ -548,7 +548,7 @@ DEFINE_IDTENTRY_RAW(exc_nmi) raw_atomic_long_inc(&nsp->idt_calls); =20 if (arch_cpu_is_offline(smp_processor_id())) { - if (microcode_nmi_handler_enabled()) + if (IS_ENABLED(CONFIG_MICROCODE_LATE_LOADING)) microcode_offline_nmi_handler(); return; } @@ -711,7 +711,7 @@ DEFINE_FREDENTRY_NMI(exc_nmi) irqentry_state_t irq_state; =20 if (arch_cpu_is_offline(smp_processor_id())) { - if (microcode_nmi_handler_enabled()) + if (IS_ENABLED(CONFIG_MICROCODE_LATE_LOADING)) microcode_offline_nmi_handler(); return; } --=20 2.51.0 From nobody Thu Apr 2 01:08:41 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13BFB36A013 for ; Tue, 31 Mar 2026 02:15:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923332; cv=none; b=TxpJaJiNb0gYmdrl/DSiy8k54HDsUVI/E9y++KBQISvyKftFjRUc76fMGu0MzTF++jWX5ZGytDn9rLJY/M+OwQ+5GmPkuZZr+DThtPD0jYXmc6TJow7iamW7g45SlL77zTcGH1t5C4VL061KeTd8Z8jddPkMfyXUIawuR3Jx1Bk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923332; c=relaxed/simple; bh=2GpCiZfU6WmNzDqR8CNiGs0Ar8OXPA4pk1VSGr1xYKM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=M6WafcMheOvDzr2Ig2CRSF3bJLdRwzHgLuyuuM/XIDzSXaVAp98tWB0BlZM+OUBPvcVZwxJK6i0YDdwArm2LqnS/9PQBaW/q5a4WEUzFbHZgnVWdSKTkGXalY8pquZ23K7YYbaNDWxbY+rlVvrvX31HfqNx1azCAdA/e0pqXfD8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ctOzFm2k; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ctOzFm2k" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774923331; x=1806459331; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2GpCiZfU6WmNzDqR8CNiGs0Ar8OXPA4pk1VSGr1xYKM=; b=ctOzFm2kDu/RPhbwu3deAZOO/yfyFywpHq6Z/YIKKN/ilqp275H9fDmL EpCGYuvVsfMiDqfEbfB/46/Xwg6GVtVx2Fyv0dmfvIi2QB/ERsZu0URrE kt2D0beXB9bOtQoWZKjs0/PvZQ4yWJQuf9UnQez1r7o6Y8oPt9EP8YuPJ QYd1QWs3S8Qh7CsiHS3j9Qmf5Yfeyc9nOoPORNA4bW1CsUqDK05kVdbDf 7F7oIis/ogH0adBOAA91rnZqGsfDp59w3T2h8f6ok6faFG7/+LMTjejji ntBRjYNJQOg4tINxfQKFaBvwMX/iwbp3kjD6wWFFdAQjGeatLk6moxtIN A==; X-CSE-ConnectionGUID: fGx5sJAIQdGJfe+urcuxcw== X-CSE-MsgGUID: SZbyAopiQDW3B2vgoQM/eA== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93508180" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93508180" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 19:07:58 -0700 X-CSE-ConnectionGUID: ycdmb2VNTB6BAMrDEfrLQg== X-CSE-MsgGUID: fA52ps0IQuW1UlHX4U8BfQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="221358728" Received: from chang-linux-3.sc.intel.com (HELO chang-linux-3) ([172.25.66.172]) by fmviesa006.fm.intel.com with ESMTP; 30 Mar 2026 19:07:54 -0700 From: "Chang S. Bae" To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, peterz@infradead.org, david.kaplan@amd.com, chang.seok.bae@intel.com Subject: [PATCH v2 11/11] x86/microcode: Remove microcode_nmi_handler_enable Date: Tue, 31 Mar 2026 01:42:49 +0000 Message-ID: <20260331014251.86353-12-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260331014251.86353-1-chang.seok.bae@intel.com> References: <20260331014251.86353-1-chang.seok.bae@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove it, as there is no more user. Signed-off-by: Chang S. Bae --- arch/x86/include/asm/microcode.h | 10 ---------- arch/x86/kernel/cpu/microcode/core.c | 8 ++------ 2 files changed, 2 insertions(+), 16 deletions(-) diff --git a/arch/x86/include/asm/microcode.h b/arch/x86/include/asm/microc= ode.h index 62d10c43da9c..a250a8849168 100644 --- a/arch/x86/include/asm/microcode.h +++ b/arch/x86/include/asm/microcode.h @@ -81,14 +81,4 @@ static inline u32 intel_get_microcode_revision(void) =20 void microcode_offline_nmi_handler(void); =20 -#ifdef CONFIG_MICROCODE_LATE_LOADING -DECLARE_STATIC_KEY_FALSE(microcode_nmi_handler_enable); -static __always_inline bool microcode_nmi_handler_enabled(void) -{ - return static_branch_unlikely(µcode_nmi_handler_enable); -} -#else -static __always_inline bool microcode_nmi_handler_enabled(void) { return f= alse; } -#endif - #endif /* _ASM_X86_MICROCODE_H */ diff --git a/arch/x86/kernel/cpu/microcode/core.c b/arch/x86/kernel/cpu/mic= rocode/core.c index ebcc73e67af1..775233c069c7 100644 --- a/arch/x86/kernel/cpu/microcode/core.c +++ b/arch/x86/kernel/cpu/microcode/core.c @@ -324,7 +324,6 @@ struct microcode_ctrl { bool nmi_enabled; }; =20 -DEFINE_STATIC_KEY_FALSE(microcode_nmi_handler_enable); static DEFINE_PER_CPU(struct microcode_ctrl, ucode_ctrl); static atomic_t late_cpus_in, offline_in_nmi; static unsigned int loops_per_usec; @@ -601,13 +600,10 @@ static int load_late_stop_cpus(bool is_safe) */ store_cpu_caps(&prev_info); =20 - if (microcode_ops->use_nmi) { - static_branch_enable_cpuslocked(µcode_nmi_handler_enable); + if (microcode_ops->use_nmi) stop_machine_nmi_cpuslocked(microcode_nmi_handler, NULL, cpu_online_mask= ); - static_branch_disable_cpuslocked(µcode_nmi_handler_enable); - } else { + else stop_machine_cpuslocked(microcode_update_handler, NULL, cpu_online_mask); - } =20 /* Analyze the results */ for_each_cpu_and(cpu, cpu_present_mask, &cpus_booted_once_mask) { --=20 2.51.0