When microcode staging is initiated, operations are carried out through
an MMIO interface. Each package has a unique interface specified by the
IA32_MCU_STAGING_MBOX_ADDR MSR, which maps to a set of 32-bit registers.
Prepare staging with the following steps:
1. Ensure the microcode image is 32-bit aligned to match the MMIO
register size.
2. Identify each MMIO interface based on its per-package scope.
3. Invoke the staging function for each identified interface, which
will be implemented separately.
Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com>
Link: https://lore.kernel.org/all/871pznq229.ffs@tglx
---
V1 -> V2:
* Adjust to reference the staging_state struct.
RFC-V1 -> V1:
* Simplify code by leveraging the architectural per-package staging scope
(Thomas).
* Fix MSR read code (Boris and Dave).
* Rename the staging function: staging_work() -> do_stage() (Boris).
* Polish the result messages (Boris).
* Add a prototype for builds without CONFIG_CPU_SUP_INTEL (Boris).
* Massage the changelog.
Note:
1. Using a direct reference to 'cpu_primary_thread_mask' in
for_each_cpu(...) causes a build error when !CONFIG_SMP. Instead, use
the wrapper function topology_is_primary_thread() to avoid it.
2. Ideally, the do_stage() function would be as simple as a single WRMSR
execution. If this were the case, the staging flow could be completed
with this patch.
---
arch/x86/include/asm/msr-index.h | 2 +
arch/x86/kernel/cpu/microcode/intel.c | 57 ++++++++++++++++++++++++++-
2 files changed, 57 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index bc6d2de109b5..f123abfdffcb 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -891,6 +891,8 @@
#define MSR_IA32_UCODE_WRITE 0x00000079
#define MSR_IA32_UCODE_REV 0x0000008b
+#define MSR_IA32_MCU_STAGING_MBOX_ADDR 0x000007a5
+
/* Intel SGX Launch Enclave Public Key Hash MSRs */
#define MSR_IA32_SGXLEPUBKEYHASH0 0x0000008C
#define MSR_IA32_SGXLEPUBKEYHASH1 0x0000008D
diff --git a/arch/x86/kernel/cpu/microcode/intel.c b/arch/x86/kernel/cpu/microcode/intel.c
index 57ed5d414cd1..14c20b53f14d 100644
--- a/arch/x86/kernel/cpu/microcode/intel.c
+++ b/arch/x86/kernel/cpu/microcode/intel.c
@@ -64,8 +64,11 @@ struct extended_sigtable {
* @bytes_sent: Total bytes transmitted so far
* @offset: Current offset in the microcode image
* @state: Current state of the staging process
+ *
+ * Staging is performed sequentially per package, so concurrent access is
+ * not expected.
*/
-struct staging_state {
+static struct staging_state {
void __iomem *mmio_base;
void *ucode_ptr;
unsigned int ucode_len;
@@ -73,7 +76,7 @@ struct staging_state {
unsigned int bytes_sent;
unsigned int offset;
enum ucode_state state;
-};
+} staging;
#define DEFAULT_UCODE_TOTALSIZE (DEFAULT_UCODE_DATASIZE + MC_HEADER_SIZE)
#define EXT_HEADER_SIZE (sizeof(struct extended_sigtable))
@@ -320,6 +323,55 @@ static __init struct microcode_intel *scan_microcode(void *data, size_t size,
return size ? NULL : patch;
}
+/*
+ * Handle the staging process using the mailbox MMIO interface. The
+ * caller is expected to check the result in staging.state.
+ */
+static void do_stage(u64 mmio_pa)
+{
+ pr_debug_once("Staging implementation is pending.\n");
+ staging.state = UCODE_ERROR;
+}
+
+static void stage_microcode(void)
+{
+ unsigned int pkg_id = UINT_MAX;
+ u64 mmio_pa;
+ int cpu;
+
+ staging.ucode_ptr = ucode_patch_late;
+ staging.ucode_len = get_totalsize(&ucode_patch_late->hdr);
+ if (!IS_ALIGNED(staging.ucode_len, sizeof(u32)))
+ return;
+
+ lockdep_assert_cpus_held();
+
+ /*
+ * The MMIO address is unique per package, and all the SMT
+ * primary threads are online here. Find each MMIO space by
+ * their package ids to avoid duplicate staging.
+ */
+ for_each_cpu(cpu, cpu_online_mask) {
+ if (!topology_is_primary_thread(cpu) ||
+ topology_logical_package_id(cpu) == pkg_id)
+ continue;
+ pkg_id = topology_logical_package_id(cpu);
+
+ rdmsrl_on_cpu(cpu, MSR_IA32_MCU_STAGING_MBOX_ADDR, &mmio_pa);
+
+ do_stage(mmio_pa);
+ if (staging.state != UCODE_OK) {
+ pr_err("Error: staging failed with %s for CPU%d at package %u.\n",
+ staging.state == UCODE_TIMEOUT ? "timeout" : "error state",
+ cpu, pkg_id);
+ return;
+ }
+ }
+
+ pr_info("Staging of patch revision 0x%x succeeded.\n",
+ ((struct microcode_header_intel *)ucode_patch_late)->rev);
+}
+
static enum ucode_state __apply_microcode(struct ucode_cpu_info *uci,
struct microcode_intel *mc,
u32 *cur_rev)
@@ -648,6 +700,7 @@ static struct microcode_ops microcode_intel_ops = {
.collect_cpu_info = collect_cpu_info,
.apply_microcode = apply_microcode_late,
.finalize_late_load = finalize_late_load,
+ .stage_microcode = stage_microcode,
.use_nmi = IS_ENABLED(CONFIG_X86_64),
};
--
2.45.2When microcode staging is initiated, operations are carried out through
an MMIO interface. Each package has a unique interface specified by the
IA32_MCU_STAGING_MBOX_ADDR MSR, which maps to a set of 32-bit registers.
Prepare staging with the following steps:
1. Ensure the microcode image is 32-bit aligned to match the MMIO
register size.
2. Identify each MMIO interface based on its per-package scope.
3. Invoke the staging function for each identified interface, which
will be implemented separately.
Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com>
Link: https://lore.kernel.org/all/871pznq229.ffs@tglx
---
V2 -> V2a:
* Remove a global variable and adjust stage_microcode() (Dave [1]).
Note: this quick revision is just intended to ensure that the feedback
has been properly addressed.
[1]: https://lore.kernel.org/lkml/b01224ee-c935-4b08-a76f-5dc49341182d@intel.com/
---
arch/x86/include/asm/msr-index.h | 2 ++
arch/x86/kernel/cpu/microcode/intel.c | 49 +++++++++++++++++++++++++++
2 files changed, 51 insertions(+)
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index bc6d2de109b5..f123abfdffcb 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -891,6 +891,8 @@
#define MSR_IA32_UCODE_WRITE 0x00000079
#define MSR_IA32_UCODE_REV 0x0000008b
+#define MSR_IA32_MCU_STAGING_MBOX_ADDR 0x000007a5
+
/* Intel SGX Launch Enclave Public Key Hash MSRs */
#define MSR_IA32_SGXLEPUBKEYHASH0 0x0000008C
#define MSR_IA32_SGXLEPUBKEYHASH1 0x0000008D
diff --git a/arch/x86/kernel/cpu/microcode/intel.c b/arch/x86/kernel/cpu/microcode/intel.c
index 57ed5d414cd1..5d0216e9aee5 100644
--- a/arch/x86/kernel/cpu/microcode/intel.c
+++ b/arch/x86/kernel/cpu/microcode/intel.c
@@ -320,6 +320,54 @@ static __init struct microcode_intel *scan_microcode(void *data, size_t size,
return size ? NULL : patch;
}
+/*
+ * Handle the staging process using the mailbox MMIO interface.
+ * Return the result state.
+ */
+static enum ucode_state do_stage(u64 mmio_pa)
+{
+ pr_debug_once("Staging implementation is pending.\n");
+ return UCODE_ERROR;
+}
+
+static void stage_microcode(void)
+{
+ unsigned int pkg_id = UINT_MAX;
+ enum ucode_state ret;
+ u64 mmio_pa;
+ int cpu;
+
+ if (!IS_ALIGNED(get_totalsize(&ucode_patch_late->hdr), sizeof(u32)))
+ return;
+
+ lockdep_assert_cpus_held();
+
+ /*
+ * The MMIO address is unique per package, and all the SMT
+ * primary threads are online here. Find each MMIO space by
+ * their package ids to avoid duplicate staging.
+ */
+ for_each_cpu(cpu, cpu_online_mask) {
+ if (!topology_is_primary_thread(cpu) ||
+ topology_logical_package_id(cpu) == pkg_id)
+ continue;
+ pkg_id = topology_logical_package_id(cpu);
+
+ rdmsrl_on_cpu(cpu, MSR_IA32_MCU_STAGING_MBOX_ADDR, &mmio_pa);
+
+ ret = do_stage(mmio_pa);
+ if (ret != UCODE_OK) {
+ pr_err("Error: staging failed with %s for CPU%d at package %u.\n",
+ ret == UCODE_TIMEOUT ? "timeout" : "error state",
+ cpu, pkg_id);
+ return;
+ }
+ }
+
+ pr_info("Staging of patch revision 0x%x succeeded.\n",
+ ((struct microcode_header_intel *)ucode_patch_late)->rev);
+}
+
static enum ucode_state __apply_microcode(struct ucode_cpu_info *uci,
struct microcode_intel *mc,
u32 *cur_rev)
@@ -648,6 +696,7 @@ static struct microcode_ops microcode_intel_ops = {
.collect_cpu_info = collect_cpu_info,
.apply_microcode = apply_microcode_late,
.finalize_late_load = finalize_late_load,
+ .stage_microcode = stage_microcode,
.use_nmi = IS_ENABLED(CONFIG_X86_64),
};
--
2.45.2>+static void stage_microcode(void)
>+{
>+ unsigned int pkg_id = UINT_MAX;
>+ enum ucode_state ret;
>+ u64 mmio_pa;
>+ int cpu;
>+
>+ if (!IS_ALIGNED(get_totalsize(&ucode_patch_late->hdr), sizeof(u32)))
>+ return;
>+
>+ lockdep_assert_cpus_held();
>+
>+ /*
>+ * The MMIO address is unique per package, and all the SMT
>+ * primary threads are online here. Find each MMIO space by
>+ * their package ids to avoid duplicate staging.
>+ */
>+ for_each_cpu(cpu, cpu_online_mask) {
for_each_online_cpu(cpu)?
>+ if (!topology_is_primary_thread(cpu) ||
>+ topology_logical_package_id(cpu) == pkg_id)
>+ continue;
Documentation/arch/x86/topology.rst states:
- topology_core_cpumask():
The cpumask contains all online threads in the package to which a thread
belongs.
The number of online threads is also printed in /proc/cpuinfo "siblings."
So, how about:
if (cpu != cpumask_first(topology_core_cpumask(cpu)))
continue;
and dropping the pkg_id?
>+ pkg_id = topology_logical_package_id(cpu);
>+
>+ rdmsrl_on_cpu(cpu, MSR_IA32_MCU_STAGING_MBOX_ADDR, &mmio_pa);
Note rdmsrl_on_cpu() may return an error. please consider adding
error-handling. Is it possible that somehow one package doesn't support
this staging feature while others do?
>+
>+ ret = do_stage(mmio_pa);
>+ if (ret != UCODE_OK) {
>+ pr_err("Error: staging failed with %s for CPU%d at package %u.\n",
>+ ret == UCODE_TIMEOUT ? "timeout" : "error state",
>+ cpu, pkg_id);
Shall we print a message somewhere showing "Continuing updates without
staging"?
It could be confusing for users to see a success message following an error
message that states "Error: staging failed ..."
>+ return;
>+ }
>+ }
>+
>+ pr_info("Staging of patch revision 0x%x succeeded.\n",
>+ ((struct microcode_header_intel *)ucode_patch_late)->rev);
>+}
>+
> static enum ucode_state __apply_microcode(struct ucode_cpu_info *uci,
> struct microcode_intel *mc,
> u32 *cur_rev)
>@@ -648,6 +696,7 @@ static struct microcode_ops microcode_intel_ops = {
> .collect_cpu_info = collect_cpu_info,
> .apply_microcode = apply_microcode_late,
> .finalize_late_load = finalize_late_load,
>+ .stage_microcode = stage_microcode,
> .use_nmi = IS_ENABLED(CONFIG_X86_64),
> };
>
>--
>2.45.2
>
On 3/26/2025 12:35 AM, Chao Gao wrote:
>> + for_each_cpu(cpu, cpu_online_mask) {
>
> for_each_online_cpu(cpu)?
Yes, it looks equivalent but shorter.
> So, how about:
>
> if (cpu != cpumask_first(topology_core_cpumask(cpu)))
> continue;
>
> and dropping the pkg_id?
No, the pkg_id check is intentional to prevent duplicate staging within
a package. As noted in the comment: "The MMIO address is unique per
package."
>> + rdmsrl_on_cpu(cpu, MSR_IA32_MCU_STAGING_MBOX_ADDR, &mmio_pa);
>
> Note rdmsrl_on_cpu() may return an error. please consider adding
> error-handling. Is it possible that somehow one package doesn't support
> this staging feature while others do?
rdmsrl_on_cpu() -> smp_call_function_single() -> generic_exec_single():
if ((unsigned)cpu >= nr_cpu_ids || !cpu_online(cpu)) {
csd_unlock(csd);
return -ENXIO;
}
This error condition applies to an invalid cpu, but since the function
is guarded by cpu_online_mask, it should not occur.
That said though, ignoring the return value may appear to be incorrect.
Perhaps,
err = rdmsrl_on_cpu(cpu, MSR_IA32_MCU_STAGING_MBOX_ADDR, &mmio_pa);
if (WARN_ON_ONCE(err))
return;
> Shall we print a message somewhere showing "Continuing updates without
> staging"?
>
> It could be confusing for users to see a success message following an error
> message that states "Error: staging failed ..."
This function already prints either a success or failure message based
on staging results which are variable.
But this behavior follows the established policy that loading should
continue even if staging fails, which is a known and invariant behavior
at runtime.
So, explicitly stating that updates will proceed without staging seems
redundant and could be considered noise.
Thanks,
Chang
On Wed, Mar 26, 2025 at 11:43:58AM -0700, Chang S. Bae wrote:
>On 3/26/2025 12:35 AM, Chao Gao wrote:
>> > + for_each_cpu(cpu, cpu_online_mask) {
>>
>> for_each_online_cpu(cpu)?
>
>Yes, it looks equivalent but shorter.
>
>> So, how about:
>>
>> if (cpu != cpumask_first(topology_core_cpumask(cpu)))
>> continue;
>>
>> and dropping the pkg_id?
>
>No, the pkg_id check is intentional to prevent duplicate staging within a
>package. As noted in the comment: "The MMIO address is unique per package."
The check I suggested can also achieve the goal and is simpler, right?
>
>> > + rdmsrl_on_cpu(cpu, MSR_IA32_MCU_STAGING_MBOX_ADDR, &mmio_pa);
>>
>> Note rdmsrl_on_cpu() may return an error. please consider adding
>> error-handling. Is it possible that somehow one package doesn't support
>> this staging feature while others do?
>
>rdmsrl_on_cpu() -> smp_call_function_single() -> generic_exec_single():
>
> if ((unsigned)cpu >= nr_cpu_ids || !cpu_online(cpu)) {
> csd_unlock(csd);
> return -ENXIO;
> }
>
>This error condition applies to an invalid cpu, but since the function is
>guarded by cpu_online_mask, it should not occur.
Ok. I misread rdmsrl_on_cpu(). I thought it would return an error if the
MSR doesn't exist on that CPU. But that's not the case.
>
>That said though, ignoring the return value may appear to be incorrect.
>Perhaps,
>
> err = rdmsrl_on_cpu(cpu, MSR_IA32_MCU_STAGING_MBOX_ADDR, &mmio_pa);
> if (WARN_ON_ONCE(err))
> return;
Looks good.
On 3/26/2025 6:44 PM, Chao Gao wrote:
>
> The check I suggested can also achieve the goal and is simpler, right?
Okay, I’d (finally) define this mask compilable for CONFIG_SMP=n, in
topology.h:
#define cpu_primary_thread_mask cpu_none_mask
Then,
for_each_cpu(cpu, cpu_primary_thread_mask) {
if (topology_logical_package_id(cpu) == pkg_id)
continue;
...
}
This still looks simpler. Plus, while timing isn’t a major concern on
this staging path, cpumask_first() on every CPU appears costly. On my
measurements, this takes about a third of the time compared to yours.
Thanks,
Chang
© 2016 - 2025 Red Hat, Inc.