arch/x86/boot/string.c | 13 ++------- arch/x86/include/asm/cpuid/api.h | 2 +- arch/x86/include/asm/shared/string.h | 52 ++++++++++++++++++++++++++++++++++++ arch/x86/include/asm/string.h | 21 +-------------- arch/x86/platform/pvh/enlighten.c | 3 ++- 5 files changed, 58 insertions(+), 33 deletions(-)
The issue of unbootable VMs with CONFIG_PVH due to CONFIG_KASAN is back.
Booting directly from vmlinux (instead of bzImage) now fails with gcc-14/15
(but works with gcc-12/13) if CONFIG_KASAN_GENERIC is set, on Ubuntu 25.10.
The PVH code is required/supposed not to use the KASAN memory access check
in the kernel entry point as KASAN has not yet been setup, or an exception
is hit and the boot fails.
This was previously described and addressed with __builtin_mem{cmp,set}():
- commit 661362e3dcab ("xen, pvh: fix unbootable VMs (PVH + KASAN - AMD_MEM_ENCRYPT)")
- commit 416a33c9afce ("x86/cpu: fix unbootable VMs by inlining memcmp() in hypervisor_cpuid_base()")
- commit fbe5a6dfe492 ("xen, pvh: fix unbootable VMs by inlining memset() in xen_prepare_pvh()")
However, even with __builtin the compiler may decide to use the out of line
function instead of the inline implementation. So, that does not really fix
the issue unconditionally; see details below.
In order to address this, it's required to switch to inline implementations
that do not depend on the compiler.
There's such a memset() in <asm/string.h> and memcmp() in 'boot/string.c'.
Use them instead of builtins in PVH entry.
Testing:
- Booting from vmlinux (fixed) and bzImage (still works) using
allnoconfig + CONFIG_PVH + CONFIG_KASAN with gcc-12/13/14/15.
- Building with CONFIG_KEXEC_FILE, CONFIG_CFI and !CONFIG_KASAN with LLVM 20
(check for a build error not caught previously).
Details/Debugging:
- Only CONFIG_PVH (works):
make allnoconfig
./scripts/config \
-e 64BIT -e HYPERVISOR_GUEST -e PVH \
-e SERIAL_8250 -e SERIAL_8250_CONSOLE
make olddefconfig
make -j$(nproc) vmlinux
qemu-system-x86_64 \
-accel kvm -nodefaults -nographic -serial stdio \
-kernel vmlinux -append 'console=ttyS0'
...
SeaBIOS (version ...)
Booting from ROM...
Linux version ...
...
<Ctrl-C>
- With CONFIG_KASAN (fails)
./scripts/config -e KASAN
make olddefconfig
make -j$(nproc) vmlinux
qemu-system-x86_64 \
-accel kvm -nodefaults -nographic -serial stdio \
-kernel vmlinux -append 'console=ttyS0'
...
SeaBIOS (version ...)
Booting from ROM...
<QEMU reboot loop, flashing the text above>
- Debugging:
Enable debug info and rebuild.
QEMU: enable and wait for GDB, stop rebooting, remain running.
qemu-system-x86_64 \
-s -S -no-reboot -no-shutdown \
<other options>
gdb vmlinux
(gdb) target remote localhost:1234
...
(gdb) c
...
Thread 2 received signal SIGQUIT, Quit.
...
(gdb) info threads
Id Target Id Frame
1 Thread 1.1 (CPU#0 [running]) bytes_is_nonzero (
start=0xfffffbfff031eebe <error: Cannot access memory at address 0xfffffbfff031eebe>, size=1)
at .../linux/mm/kasan/generic.c:98
* 2 Thread 1.2 (CPU#1 [halted ]) 0x00000000000fd0a9 in ?? ()
...
(gdb) thr 1
...
(gdb) bt
#0 bytes_is_nonzero (start=0xfffffbfff031eebe <error: Cannot access memory at address 0xfffffbfff031eebe>, size=1)
at .../linux/mm/kasan/generic.c:98
#1 memory_is_nonzero (start=0xfffffbfff031eebe, end=0xfffffbfff031eebf) at .../linux/mm/kasan/generic.c:115
#2 memory_is_poisoned_n (addr=0xffffffff818f75f0, size=8) at .../linux/mm/kasan/generic.c:140
#3 memory_is_poisoned (addr=0xffffffff818f75f0, size=8) at .../linux/mm/kasan/generic.c:172
#4 check_region_inline (addr=0xffffffff818f75f0, size=8, write=false, ret_ip=18446744071585002062)
at .../linux/mm/kasan/generic.c:191
#5 kasan_check_range (addr=addr@entry=0xffffffff818f75f0, size=size@entry=8, write=write@entry=false,
ret_ip=18446744071585002062) at .../linux/mm/kasan/generic.c:200
#6 0xffffffff813eb283 in __asan_loadN (addr=addr@entry=0xffffffff818f75f0, size=size@entry=8)
at .../linux/mm/kasan/generic.c:278
#7 0xffffffff815df24e in memcmp (cs=cs@entry=0xffffffff818f75f0, ct=ct@entry=0x1be2fe4, count=<optimized out>,
count@entry=12) at .../linux/lib/string.c:683
#8 0xffffffff81ba2323 in cpuid_base_hypervisor (sig=0xffffffff818f75f0 "XenVMMXenVMM", leaves=2)
at .../linux/arch/x86/include/asm/cpuid/api.h:206
#9 xen_cpuid_base () at .../linux/arch/x86/include/asm/xen/hypervisor.h:46
#10 xen_prepare_pvh () at .../linux/arch/x86/platform/pvh/enlighten.c:119
#11 0x0000000001ba2588 in ?? ()
#12 0x0000000000000000 in ?? ()
(gdb)
Frames #7-#8 show the non-builtin memcmp() (lib/string.c) was called
even with __builtin_memcmp() being used in cpuid_base_hypervisor().
Signed-off-by: Mauricio Faria de Oliveira <mfo@igalia.com>
---
Changes in v10:
- Patch 1:
- retain the "cc" clobber commented out (Borislav Petkov)
- explain based on compiler implementation, not documentation (Borislav Petkov, Michael Matz)
- Patch 2:
- remove single quotes from filenames in commit message (Borislav Petkov)
- simplify function comment (Borislav Petkov)
- Patch 4:
- combine Patch 5 from v9 (Borislav Petkov)
- update function name: s/hypervisor_base_cpuid/cpuid_base_hypervisor/g
- slightly modify the commit message.
- Link to v9: https://lore.kernel.org/r/20260822-pvh-kasan-inline-v9-0-e70ef3b75b6a@igalia.com
Changes in v9:
- Patch 1: new patch in v9 to fix the patch submitted separately in v8.
- Rebased to next-20260821.
- Link to v8: https://lore.kernel.org/r/20260723-pvh-kasan-inline-v8-0-c1f62c156f52@igalia.com
Changes in v8:
- Patch 2 in v7 was submitted separately as requested, and the
rest of this series was rebased on top of it (Borislav Petkov).
Link: https://lore.kernel.org/all/20260723-x86-memcmp-asm-v2-1-d93ecb43797f@igalia.com/
- Patch 2 in v8:
- Mention 'No functional changes' (Borislav Petkov).
- Remove comment at the top of the header (Borislav Petkov).
- Link to v7: https://lore.kernel.org/r/20260721-pvh-kasan-inline-v7-0-38979a50cef0@igalia.com
Changes in v7:
- Patch 2 (added):
- Address pre-existing issues in 'asm' (Borislav Petkov, Sashiko).
- Link to v6: https://lore.kernel.org/r/20260701-pvh-kasan-inline-v6-0-ba99045dfa9f@igalia.com
Changes in v6:
- Patch 1:
- Explain the return value difference between __inline_memcmp() and memcmp().
- Patch 2 (added):
- Group __inline string functions in <asm/shared/string.h>.
- Link to v5: https://lore.kernel.org/r/20260630-pvh-kasan-inline-v5-0-52afc979be81@igalia.com
Changes in v5:
- Create a minimal separate header in <asm/shared/string.h> instead,
to be used by 'boot/setup.c' and <asm/string.h> (Borislav Petkov).
- Patch 1 (in v4/v3) is no longer needed; removed.
- Patch 1 (in v5):
- Briefly mention there are issues with <asm/string.h>.
- Remove 'Reviewed-by: Jurgen Gross' to be conservative
(same code change and result, but the means changed).
- Link to v4: https://lore.kernel.org/r/20260526-pvh-kasan-inline-v4-0-a310e6a25ecd@igalia.com
Changes in v4:
- Patch 1: address Juergen's feedback:
- s/In next patch/In a future patch/.
- Move footnote (Reasons not to include...) after "---".
- Add 'Reviewed-by: Juergen Gross' in patches 1 and 2 as well.
- Link to v3: https://lore.kernel.org/r/20260520-pvh-kasan-inline-v3-0-bede769c6ec7@igalia.com
Changes in v3:
- Create and use a separate header for inline string functions
to fix a build error reported by kernel test robot (patch 1).
- That also removes '#ifndef _SETUP/#endif' in <asm/string.h>.
- Link to v2: https://lore.kernel.org/r/20260427-pvh-kasan-inline-v2-0-2c57b8dcff6a@igalia.com
Changes in v2:
- Add comment about the return value of __inline_memcmp() in patch 1. (v3: now 2)
- Add 'Reviewed-by: Juergen Gross' in patches 2 and 3 (v3: now 3 and 4).
- Link to v1: https://lore.kernel.org/r/20260422-pvh-kasan-inline-v1-0-7e6194344c92@igalia.com
---
Mauricio Faria de Oliveira (4):
x86/boot: comment out and document redundant "cc" clobber in memcmp()
x86/asm, x86/boot: expose inline memcmp()
x86/asm: group inline string functions
x86/cpuid: fix unbootable VMs by really inlining memcmp() in cpuid_base_hypervisor() and xen_prepare_pvh()
arch/x86/boot/string.c | 13 ++-------
arch/x86/include/asm/cpuid/api.h | 2 +-
arch/x86/include/asm/shared/string.h | 52 ++++++++++++++++++++++++++++++++++++
arch/x86/include/asm/string.h | 21 +--------------
arch/x86/platform/pvh/enlighten.c | 3 ++-
5 files changed, 58 insertions(+), 33 deletions(-)
---
base-commit: 5c4d4169604b335c38bbc79bc1fc03042981fc6f
change-id: 20260422-pvh-kasan-inline-6efac77f1b27
Best regards,
--
Mauricio Faria de Oliveira <mfo@igalia.com>
On Mon, Sep 21, 2026 at 10:36:31PM -0300, Mauricio Faria de Oliveira wrote:
> The issue of unbootable VMs with CONFIG_PVH due to CONFIG_KASAN is back.
>
> Booting directly from vmlinux (instead of bzImage) now fails with gcc-14/15
> (but works with gcc-12/13) if CONFIG_KASAN_GENERIC is set, on Ubuntu 25.10.
>
> The PVH code is required/supposed not to use the KASAN memory access check
> in the kernel entry point as KASAN has not yet been setup, or an exception
> is hit and the boot fails.
>
> This was previously described and addressed with __builtin_mem{cmp,set}():
> - commit 661362e3dcab ("xen, pvh: fix unbootable VMs (PVH + KASAN - AMD_MEM_ENCRYPT)")
> - commit 416a33c9afce ("x86/cpu: fix unbootable VMs by inlining memcmp() in hypervisor_cpuid_base()")
> - commit fbe5a6dfe492 ("xen, pvh: fix unbootable VMs by inlining memset() in xen_prepare_pvh()")
>
> However, even with __builtin the compiler may decide to use the out of line
> function instead of the inline implementation. So, that does not really fix
> the issue unconditionally; see details below.
So, this whole deal doesn't sound to me like we need to backport it to stable
- it rather looks more like fixing some configs which want to enable KASAN on
PVH guests.
In that case, I'll queue this for 7.4.
If this needs to go to stable, then there better be a pretty good reason for
it.
Right?
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
On 2026-09-22 00:58, Borislav Petkov wrote:
> On Mon, Sep 21, 2026 at 10:36:31PM -0300, Mauricio Faria de Oliveira wrote:
>> The issue of unbootable VMs with CONFIG_PVH due to CONFIG_KASAN is back.
>>
>> Booting directly from vmlinux (instead of bzImage) now fails with gcc-14/15
>> (but works with gcc-12/13) if CONFIG_KASAN_GENERIC is set, on Ubuntu 25.10.
>>
>> The PVH code is required/supposed not to use the KASAN memory access check
>> in the kernel entry point as KASAN has not yet been setup, or an exception
>> is hit and the boot fails.
>>
>> This was previously described and addressed with __builtin_mem{cmp,set}():
>> - commit 661362e3dcab ("xen, pvh: fix unbootable VMs (PVH + KASAN - AMD_MEM_ENCRYPT)")
>> - commit 416a33c9afce ("x86/cpu: fix unbootable VMs by inlining memcmp() in hypervisor_cpuid_base()")
>> - commit fbe5a6dfe492 ("xen, pvh: fix unbootable VMs by inlining memset() in xen_prepare_pvh()")
>>
>> However, even with __builtin the compiler may decide to use the out of line
>> function instead of the inline implementation. So, that does not really fix
>> the issue unconditionally; see details below.
>
> So, this whole deal doesn't sound to me like we need to backport it to stable
> - it rather looks more like fixing some configs which want to enable KASAN on
> PVH guests.
>
> In that case, I'll queue this for 7.4.
>
> If this needs to go to stable, then there better be a pretty good reason for
> it.
>
> Right?
I think that is fine, yes. Even though it's a boot failure, actually
hitting it depends on all of: CONFIG_KASAN, CONFIG_PVH, booting from the
PVH entry point, _plus_ a compiler version that triggers it.
Nonetheless, this may be picked up for stable due to the Fixes: tags,
but I can provide backports as needed.
BTW, I just realized that the title of patch 4/4 is missing memset().
Would you mind adding it, please? Or I can send v11.
-x86/cpuid: fix unbootable VMs by really inlining memcmp() in
cpuid_base_hypervisor() and xen_prepare_pvh()
+x86/cpuid: fix unbootable VMs by really inlining memcmp() and memset()
in cpuid_base_hypervisor() and xen_prepare_pvh()
Thanks,
--
Mauricio
On Tue, Sep 22, 2026 at 07:13:35PM -0300, Mauricio Faria de Oliveira wrote:
> I think that is fine, yes. Even though it's a boot failure, actually
> hitting it depends on all of: CONFIG_KASAN, CONFIG_PVH, booting from the
> PVH entry point, _plus_ a compiler version that triggers it.
>
> Nonetheless, this may be picked up for stable due to the Fixes: tags,
> but I can provide backports as needed.
Right, just consider all the bandwidth of the folks involved downstream:
stable team and all distros backporting stuff every day. This sounds like an
exotic thing so let's be conservative here pls.
> BTW, I just realized that the title of patch 4/4 is missing memset().
> Would you mind adding it, please? Or I can send v11.
> -x86/cpuid: fix unbootable VMs by really inlining memcmp() in
> cpuid_base_hypervisor() and xen_prepare_pvh()
> +x86/cpuid: fix unbootable VMs by really inlining memcmp() and memset()
> in cpuid_base_hypervisor() and xen_prepare_pvh()
Sure, np.
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
On 2026-09-22 22:12, Borislav Petkov wrote: > On Tue, Sep 22, 2026 at 07:13:35PM -0300, Mauricio Faria de Oliveira wrote: >> I think that is fine, yes. Even though it's a boot failure, actually >> hitting it depends on all of: CONFIG_KASAN, CONFIG_PVH, booting from the >> PVH entry point, _plus_ a compiler version that triggers it. >> >> Nonetheless, this may be picked up for stable due to the Fixes: tags, >> but I can provide backports as needed. > > Right, just consider all the bandwidth of the folks involved downstream: > stable team and all distros backporting stuff every day. This sounds like an > exotic thing so let's be conservative here pls. Absolutely. I worked for a long time with distro kernels and backporting. I just mentioned that it is possible for this to be picked up, and that if it is, then I can help with it if needed; not pushing. :) >> BTW, I just realized that the title of patch 4/4 is missing memset(). >> Would you mind adding it, please? Or I can send v11. >> -x86/cpuid: fix unbootable VMs by really inlining memcmp() in >> cpuid_base_hypervisor() and xen_prepare_pvh() >> +x86/cpuid: fix unbootable VMs by really inlining memcmp() and memset() >> in cpuid_base_hypervisor() and xen_prepare_pvh() > > Sure, np. Thanks! > > Thx. -- Mauricio
© 2016 - 2026 Red Hat, Inc.