From nobody Mon Apr 6 09:11:39 2026 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2137436BCF2 for ; Tue, 31 Mar 2026 02:14:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923305; cv=none; b=MO6N+PmebtesgIKAVxJGdoJMfVKYaclAayasMWovSrbk8wK+Jmt4AR0Tanx499VXCLElVtBQjsTRV7KwdKPX9E1VKSwYDAJVlgoVrUWrK1jKqkLvXyAHJhV1uLTCzhYLCB1kYlYxj4FkpQVcSzdOI2Kgg9zmfm/+o8psltBG0Mw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774923305; c=relaxed/simple; bh=pTRggg7XBjD2T4wKwJT9pAS3nvh+xTpTxzElTaSZrrw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IpQb9xCX7zmix3B325ZLoZQNwI06mRJsRBvvS4DsDa5mANdy/Scpb/G3T1AgB/N+b1JeWL7+b0m7HVvcVOgY8rQaMWMFcIEERGEF6X9XhKOY87U17Ed80gsDUw1wsQPTVueaV1kF+BmUX/HS6u44SOvrmHk1X7wYHeTtibNFXhg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=VD8577Ta; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="VD8577Ta" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1774923299; x=1806459299; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pTRggg7XBjD2T4wKwJT9pAS3nvh+xTpTxzElTaSZrrw=; b=VD8577TabU88bUUErWcmzy0HGt2Fgstasis2jN8xp28wgJIQts6is/Jd SE9Iz9NbSmTG7AD6eoUSfnNo3NmmpAewxlRKtIemRlOcYlboCI2xvBNzG lN+I8IPdiLs5QOMD3CvoXQjyGPTsiwkLnjJ1cX1wIUicCjCiaiwj1eNQ+ VAFGjqp6oePHn8VxvwEbNBpDsavfN1+airoClSNN+zTRA/xZzN4VY1kTP k6asaobWViRNqedgWZQKIyRj3xCdUyM2gvNVNb/GH2/qIyHQBFLVS4LXd sP3+nsqzpM6kuI9jjyHjOpfhXPMID6xtk+GFs/PQ9IDW9ISjf1UmL2esh A==; X-CSE-ConnectionGUID: pCV9g4QBRGauSvxFakTAkA== X-CSE-MsgGUID: gJrsWZ+ER5WMUDRaPFXZdg== X-IronPort-AV: E=McAfee;i="6800,10657,11744"; a="93508096" X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="93508096" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Mar 2026 19:07:46 -0700 X-CSE-ConnectionGUID: am2yAA1LTrqni8TI9uO9ig== X-CSE-MsgGUID: zJtlH5vQTY+PMZHixwVQhw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.23,151,1770624000"; d="scan'208";a="221358694" Received: from chang-linux-3.sc.intel.com (HELO chang-linux-3) ([172.25.66.172]) by fmviesa006.fm.intel.com with ESMTP; 30 Mar 2026 19:07:43 -0700 From: "Chang S. Bae" To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, tglx@kernel.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, peterz@infradead.org, david.kaplan@amd.com, chang.seok.bae@intel.com Subject: [PATCH v2 05/11] stop_machine: Introduce stop_machine_nmi_cpuslocked() Date: Tue, 31 Mar 2026 01:42:43 +0000 Message-ID: <20260331014251.86353-6-chang.seok.bae@intel.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260331014251.86353-1-chang.seok.bae@intel.com> References: <20260331014251.86353-1-chang.seok.bae@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" With the NMI control logic in place, introduce an API to run the target function from NMI context. Originally-by: David Kaplan Suggested-by: Borislav Petkov Signed-off-by: Chang S. Bae Link: https://lore.kernel.org/lkml/20260202105411.GVaYCCUygtEUNrMUtG@fat_cr= ate.local --- V1 -> V2: * Support nmi_cpus mask (Boris), including @cpus=3DNULL cases * Split out API introduction * Drop out stop_machine_nmi() [**] [**] could be added but no user yet in this series --- include/linux/stop_machine.h | 24 ++++++++++++++++++++ kernel/stop_machine.c | 43 ++++++++++++++++++++++++++++++++++++ 2 files changed, 67 insertions(+) diff --git a/include/linux/stop_machine.h b/include/linux/stop_machine.h index 9424d363ab38..2da9aa0ec3d3 100644 --- a/include/linux/stop_machine.h +++ b/include/linux/stop_machine.h @@ -201,6 +201,30 @@ stop_machine_from_inactive_cpu(cpu_stop_fn_t fn, void = *data, void arch_send_self_nmi(void); bool noinstr stop_machine_nmi_handler(void); =20 +/** + * stop_machine_nmi_cpuslocked() - Freeze CPUs and run a function in NMI c= ontext + * + * @nmisafe_fn: The function to run + * @data: The data pointer for @nmisafe_fn() + * @cpus: A cpumask containing the CPUs to run @nmisafe_fn() on. If NULL, + * @nmisafe_fn() runs on a single (arbitrary) CPU from + * cpu_online_mask. + * + * Description: This stop_machine() variant runs @nmisafe_fn() from NMI co= ntext + * to prevent preemption by other NMIs. The callback must be built with no= instr. + * Other than that, the semantics match stop_machine_cpuslocked(). + * + * Context: Must be called from within a cpus_read_lock() protected region. + * Avoid nested calls to cpus_read_lock(). + * + * Return: 0 if all invocations of @nmisafe_fn return zero, -ENOMEM if cpu= mask + * allocation fails, -EINVAL if any target CPU failed to receive NMI. Othe= rwise, + * an accumulated return value from all invocation of @nmisafe_fn that ret= urned + * non-zero. + */ +int stop_machine_nmi_cpuslocked(cpu_stop_nmisafe_fn_t nmisafe_fn, void *da= ta, + const struct cpumask *cpus); + #else static inline bool stop_machine_nmi_handler(void) { return false; } #endif /* CONFIG_STOP_MACHINE_NMI */ diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c index 45ea62f1b2b5..e20e4d3e7b16 100644 --- a/kernel/stop_machine.c +++ b/kernel/stop_machine.c @@ -798,6 +798,49 @@ static int multi_stop_run(struct multi_stop_data *msda= ta) return msdata->use_nmi ? nmi_stop_run(msdata) : msdata->fn(msdata->data); } =20 +int stop_machine_nmi_cpuslocked(cpu_stop_nmisafe_fn_t nmisafe_fn, void *da= ta, + const struct cpumask *cpus) +{ + struct multi_stop_data msdata =3D { + .nmisafe_fn =3D nmisafe_fn, + .data =3D data, + .num_threads =3D num_online_cpus(), + .active_cpus =3D cpus, + .use_nmi =3D true, + }; + int ret; + + if (!zalloc_cpumask_var(&msdata.nmi_cpus, GFP_KERNEL)) + return -ENOMEM; + + /* + * NMI CPUs should be exactly those 'active' CPUs executing the + * stop function. Follow the selection logic in multi_cpu_stop() + * if not provided. + */ + if (!msdata.active_cpus) + cpumask_set_cpu(cpumask_first(cpu_online_mask), msdata.nmi_cpus); + else + cpumask_copy(msdata.nmi_cpus, msdata.active_cpus); + + lockdep_assert_cpus_held(); + + ret =3D stop_multi_cpus(&msdata); + + /* + * The NMI handler clears each CPU bit. If any of those NMIs were + * ever missed out, return error clearly. + */ + if (!cpumask_empty(msdata.nmi_cpus)) { + pr_err("CPUs %*pbl didn't run the stop_machine NMI handler.\n", + cpumask_pr_args(msdata.nmi_cpus)); + ret =3D -EINVAL; + } + + free_cpumask_var(msdata.nmi_cpus); + return ret; +} + #else =20 static int multi_stop_run(struct multi_stop_data *msdata) --=20 2.51.0