VMware registers each per-CPU steal-time GPA with the host. An encrypted
guest must first convert that storage to shared memory, but the existing
setup publishes the address without conversion.

Patch 1 makes the decrypted per-CPU section available with
CONFIG_X86_MEM_ENCRYPT, including TDX-only configurations. Patch 2 defers
encrypted-guest setup until allocator-backed page-table splitting is
available, converts every possible CPU's storage before publishing any
GPA, and attempts to roll back all conversions on failure.

TDX's conversion callback uses __pa(), so patch 2 preflights every possible
CPU and leaves steal time disabled if a TDX guest uses vmalloc-backed
per-CPU storage. This covers percpu_alloc=page and automatic allocator
fallback. AMD encrypted guests support those mappings and are not rejected.
Supporting them in TDX would require a separate conversion-API change.

Conversion need not preserve zeroes, and the host initializes only the
8-byte counter. Patch 2 therefore clears each 64-byte object after
conversion and before registration, without disturbing other decrypted
objects that can share its page.

The intended merge path is tip's x86/vmware branch, following
commit ac26963a1175 ("percpu: Introduce DEFINE_PER_CPU_DECRYPTED").
Per-CPU and asm-generic maintainer Acks are requested for patch 1.

This replaces patch 4 of Alexey's v2 posting:
https://lore.kernel.org/all/20260309235250.2611115-1-alexey.makhalov@broadcom.com/

Zack Rusin (2):
  percpu: Use X86_MEM_ENCRYPT for decrypted per-CPU data
  x86/vmware: Decrypt steal-time storage before sharing it

 arch/x86/kernel/cpu/vmware.c      | 96 ++++++++++++++++++++++++++++++-
 include/asm-generic/vmlinux.lds.h |  2 +-
 include/linux/percpu-defs.h       |  2 +-
 3 files changed, 97 insertions(+), 3 deletions(-)


base-commit: fd73f4a6659897191fa0d40695fe370925dd3780
-- 
2.53.0

On Wed, Sep 16, 2026 at 01:05:38AM -0400, Zack Rusin wrote:
> VMware registers each per-CPU steal-time GPA with the host. An encrypted
> guest must first convert that storage to shared memory, but the existing
> setup publishes the address without conversion.
> 
> Patch 1 makes the decrypted per-CPU section available with
> CONFIG_X86_MEM_ENCRYPT, including TDX-only configurations. Patch 2 defers
> encrypted-guest setup until allocator-backed page-table splitting is
> available, converts every possible CPU's storage before publishing any
> GPA, and attempts to roll back all conversions on failure.

I acked 1/2, but I don't like where 2/2 does the conversion.

A variable declared with DEFINE_PER_CPU_DECRYPTED() ends up in a
dedicated, page-aligned linker section. The point of the section is that
one place converts it. Instead every user does it itself: KVM in
sev_map_percpu_data(), and now VMware in vmware_decrypt_steal_time(),
each with its own vendor checks and failure handling. The macro today
only buys page isolation, not the shared mapping its name promises.

The underlying problem is that the whole "decrypted section"
infrastructure is built around SME/SEV and was never generalized.
__bss_decrypted, early_set_memory_decrypted() and
mem_encrypt_free_decrypted_mem() are all under CONFIG_AMD_MEM_ENCRYPT
and implemented in mem_encrypt_amd.c. sme_postprocess_startup() converts
.bss..decrypted only if sme_get_me_mask() is set, so a TDX guest never
shares it. 1/2 moves the per-CPU linker section to X86_MEM_ENCRYPT, but
nothing that would act on that section follows.

Rather than have every TDX user reinvent the conversion, I would rather
see the infrastructure made vendor-neutral: boundary symbols for the
per-CPU decrypted section like the ones .bss..decrypted has, an early
conversion primitive that works on TDX as well as SEV, and a single
conversion of both sections at boot. Then sev_map_percpu_data() and
this driver's loop go away.

> TDX's conversion callback uses __pa(), so patch 2 preflights every possible
> CPU and leaves steal time disabled if a TDX guest uses vmalloc-backed
> per-CPU storage. This covers percpu_alloc=page and automatic allocator
> fallback. AMD encrypted guests support those mappings and are not rejected.
> Supporting them in TDX would require a separate conversion-API change.

Refusing vmalloc-backed storage on TDX is the right call, and not because
of the __pa() in the callback. Converting a vmalloc alias means either
fracturing the direct map or leaving a private direct-map alias to a
shared page, and the latter is a guest shutdown the moment
load_unaligned_zeropad() steps into it. See the comment in
tdx_early_init() and the earlier discussion of the same idea:

https://lore.kernel.org/all/xqi2bkulnhen2vax5msbzczlaywx3dsc7ezpn7oo5qn7u7xzap@xmaseinov7tf/

But that decision belongs in the same central place as the conversion.
If the first per-CPU chunk is vmalloc-backed on TDX, the decrypted
section is simply not shared, and users see that, instead of every
driver re-deriving it.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

On Wed, Sep 16, 2026 at 8:18 AM Kiryl Shutsemau <kas@kernel.org> wrote:
>
> On Wed, Sep 16, 2026 at 01:05:38AM -0400, Zack Rusin wrote:
> > VMware registers each per-CPU steal-time GPA with the host. An encrypted
> > guest must first convert that storage to shared memory, but the existing
> > setup publishes the address without conversion.
> >
> > Patch 1 makes the decrypted per-CPU section available with
> > CONFIG_X86_MEM_ENCRYPT, including TDX-only configurations. Patch 2 defers
> > encrypted-guest setup until allocator-backed page-table splitting is
> > available, converts every possible CPU's storage before publishing any
> > GPA, and attempts to roll back all conversions on failure.
>
> I acked 1/2, but I don't like where 2/2 does the conversion.

I saw that, thank you. Since I'll be respinning v2 of this I can go
ahead and drop the unused DECLARE_PER_CPU_DECRYPTED in that one if you
want as well.

> A variable declared with DEFINE_PER_CPU_DECRYPTED() ends up in a
> dedicated, page-aligned linker section. The point of the section is that
> one place converts it. Instead every user does it itself: KVM in
> sev_map_percpu_data(), and now VMware in vmware_decrypt_steal_time(),
> each with its own vendor checks and failure handling. The macro today
> only buys page isolation, not the shared mapping its name promises.
>
> The underlying problem is that the whole "decrypted section"
> infrastructure is built around SME/SEV and was never generalized.
> __bss_decrypted, early_set_memory_decrypted() and
> mem_encrypt_free_decrypted_mem() are all under CONFIG_AMD_MEM_ENCRYPT
> and implemented in mem_encrypt_amd.c. sme_postprocess_startup() converts
> .bss..decrypted only if sme_get_me_mask() is set, so a TDX guest never
> shares it. 1/2 moves the per-CPU linker section to X86_MEM_ENCRYPT, but
> nothing that would act on that section follows.
>
> Rather than have every TDX user reinvent the conversion, I would rather
> see the infrastructure made vendor-neutral: boundary symbols for the
> per-CPU decrypted section like the ones .bss..decrypted has, an early
> conversion primitive that works on TDX as well as SEV, and a single
> conversion of both sections at boot. Then sev_map_percpu_data() and
> this driver's loop go away.
>
> > TDX's conversion callback uses __pa(), so patch 2 preflights every possible
> > CPU and leaves steal time disabled if a TDX guest uses vmalloc-backed
> > per-CPU storage. This covers percpu_alloc=page and automatic allocator
> > fallback. AMD encrypted guests support those mappings and are not rejected.
> > Supporting them in TDX would require a separate conversion-API change.
>
> Refusing vmalloc-backed storage on TDX is the right call, and not because
> of the __pa() in the callback. Converting a vmalloc alias means either
> fracturing the direct map or leaving a private direct-map alias to a
> shared page, and the latter is a guest shutdown the moment
> load_unaligned_zeropad() steps into it. See the comment in
> tdx_early_init() and the earlier discussion of the same idea:
>
> https://lore.kernel.org/all/xqi2bkulnhen2vax5msbzczlaywx3dsc7ezpn7oo5qn7u7xzap@xmaseinov7tf/
>
> But that decision belongs in the same central place as the conversion.
> If the first per-CPU chunk is vmalloc-backed on TDX, the decrypted
> section is simply not shared, and users see that, instead of every
> driver re-deriving it.

I went through the linked thread, including the dropped 2022
Quote-buffer alias approach and the 2024 netvsc work's kexec
regression, fix and re-review discussion. I will keep TDX vmalloc
support out of this series and correct the explanation to cover alias
consistency, direct-map splitting and load_unaligned_zeropad(),
including removing the suggestion that per-page GPA lookup alone
resolves it.

After tracing the boot paths, I think there are two distinct pieces
here. The SEV boot GHCB needs .bss..decrypted before BSS is cleared,
whereas the SMP per-CPU instances do not exist until
setup_per_cpu_areas(). KVM then registers its boot-CPU per-CPU buffers
in smp_prepare_boot_cpu(), so the common conversion needs the early
primitive you proposed. A shared early_initcall would be too late for
KVM. The UP registration path probably also needs handling separately

My suggestion would be to do one of two things (or at least I think
those two options are realistic to me, given that I'm no expert here):
- add the central per-CPU infrastructure and migrate KVM/VMware
together, removing both loops and putting the TDX vmalloc rejection
and readiness state in common code. Would you and the KVM maintainers
be happy with that first, and generalizing .bss..decrypted separately
while preserving its earliest SEV setup? afaict the latter also needs
to broaden the annotation and linker guards, and changes kvmclock
storage and sharing on TDX-only builds.
- a smaller VMware fix using a separately allocated, direct-mapped
page per CPU while the common infrastructure is developed. Using the
normal allocator would also defer ordinary-guest registration, unless
I kept their current static storage as a second path. This option
would retain caller-managed sharing and buffer lifetime handling.

The second option is, of course, easier for me, but I'm happy to do
the first to get steal-time storage working for us. Do you have any
thoughts on this?

z

On Wed, Sep 16, 2026 at 11:41:19AM -0400, Zack Rusin wrote:
> > I acked 1/2, but I don't like where 2/2 does the conversion.
> 
> I saw that, thank you. Since I'll be respinning v2 of this I can go
> ahead and drop the unused DECLARE_PER_CPU_DECRYPTED in that one if you
> want as well.

Yes, please.

> I went through the linked thread, including the dropped 2022
> Quote-buffer alias approach and the 2024 netvsc work's kexec
> regression, fix and re-review discussion. I will keep TDX vmalloc
> support out of this series and correct the explanation to cover alias
> consistency, direct-map splitting and load_unaligned_zeropad(),
> including removing the suggestion that per-page GPA lookup alone
> resolves it.
> 
> After tracing the boot paths, I think there are two distinct pieces
> here. The SEV boot GHCB needs .bss..decrypted before BSS is cleared,
> whereas the SMP per-CPU instances do not exist until
> setup_per_cpu_areas(). KVM then registers its boot-CPU per-CPU buffers
> in smp_prepare_boot_cpu(), so the common conversion needs the early
> primitive you proposed. A shared early_initcall would be too late for
> KVM. The UP registration path probably also needs handling separately

Agreed on all of that. Leave .bss..decrypted alone in this series; the
SEV boot GHCB pins its conversion to the startup code and nothing on
TDX needs it today.

With the primitive called before smp_prepare_boot_cpu() there is no
readiness state to track.

For the vmalloc-backed case, I would rather not handle it at all. Force
the embed allocator when memory encryption is on: ignore
percpu_alloc=page with a warning and don't fall back to page mode if
embed fails, just let it hit the existing panic. On 64-bit embed only
fails on memblock exhaustion at boot or when the NUMA groups spread over
more than 3/4 of vmalloc space. Neither happens in a guest.

> My suggestion would be to do one of two things (or at least I think
> those two options are realistic to me, given that I'm no expert here):
> - add the central per-CPU infrastructure and migrate KVM/VMware
> together, removing both loops and putting the TDX vmalloc rejection
> and readiness state in common code. Would you and the KVM maintainers
> be happy with that first, and generalizing .bss..decrypted separately
> while preserving its earliest SEV setup? afaict the latter also needs
> to broaden the annotation and linker guards, and changes kvmclock
> storage and sharing on TDX-only builds.
> - a smaller VMware fix using a separately allocated, direct-mapped
> page per CPU while the common infrastructure is developed. Using the
> normal allocator would also defer ordinary-guest registration, unless
> I kept their current static storage as a second path. This option
> would retain caller-managed sharing and buffer lifetime handling.
> 
> The second option is, of course, easier for me, but I'm happy to do
> the first to get steal-time storage working for us. Do you have any
> thoughts on this?

The first one, please. It is the better long-term shape and it removes
the KVM loop as well, so the per-CPU section finally does what its
name says.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov