arch/x86/boot/string.c | 6 ++---- arch/x86/include/asm/cpuid/api.h | 2 +- arch/x86/include/asm/string.h | 21 +------------------- arch/x86/include/asm/string_inline.h | 37 ++++++++++++++++++++++++++++++++++++ arch/x86/platform/pvh/enlighten.c | 3 ++- 5 files changed, 43 insertions(+), 26 deletions(-)
The issue of unbootable VMs with CONFIG_PVH due to CONFIG_KASAN is back.
Booting directly from vmlinux (instead of bzImage) now fails with gcc-14/15
(but works with gcc-12/13) if CONFIG_KASAN_GENERIC is set, on Ubuntu 25.10.
The PVH code is required/supposed not to use the KASAN memory access check
in the kernel entry point as KASAN has not yet been setup, or an exception
is hit and the boot fails.
This was previously described and addressed with __builtin_mem{cmp,set}():
- commit 661362e3dcab ("xen, pvh: fix unbootable VMs (PVH + KASAN - AMD_MEM_ENCRYPT)")
- commit 416a33c9afce ("x86/cpu: fix unbootable VMs by inlining memcmp() in hypervisor_cpuid_base()")
- commit fbe5a6dfe492 ("xen, pvh: fix unbootable VMs by inlining memset() in xen_prepare_pvh()")
However, even with __builtin the compiler may decide to use the out of line
function instead of the inline implementation. So, that does not really fix
the issue unconditionally; see details below.
In order to address this, it's required to switch to inline implementations
that do not depend on the compiler.
There's such a memset() in <asm/string.h> and memcmp() in 'boot/string.c'.
Use them instead of builtins in PVH entry.
Testing:
- Booting from vmlinux (fixed) and bzImage (still works) using
allnoconfig + CONFIG_PVH + CONFIG_KASAN with gcc-12/13/14/15.
- Building with CONFIG_KEXEC_FILE, CONFIG_CFI and !CONFIG_KASAN with LLVM 20
(check for a build error related to <asm/string.h>, not caught previously).
Details/Debugging:
- Only CONFIG_PVH (works):
make allnoconfig
./scripts/config \
-e 64BIT -e HYPERVISOR_GUEST -e PVH \
-e SERIAL_8250 -e SERIAL_8250_CONSOLE
make olddefconfig
make -j$(nproc) vmlinux
qemu-system-x86_64 \
-accel kvm -nodefaults -nographic -serial stdio \
-kernel vmlinux -append 'console=ttyS0'
...
SeaBIOS (version ...)
Booting from ROM...
Linux version ...
...
<Ctrl-C>
- With CONFIG_KASAN (fails)
./scripts/config -e KASAN
make olddefconfig
make -j$(nproc) vmlinux
qemu-system-x86_64 \
-accel kvm -nodefaults -nographic -serial stdio \
-kernel vmlinux -append 'console=ttyS0'
...
SeaBIOS (version ...)
Booting from ROM...
<QEMU reboot loop, flashing the text above>
- Debugging:
Enable debug info and rebuild.
QEMU: enable and wait for GDB, stop rebooting, remain running.
qemu-system-x86_64 \
-s -S -no-reboot -no-shutdown \
<other options>
gdb vmlinux
(gdb) target remote localhost:1234
...
(gdb) c
...
Thread 2 received signal SIGQUIT, Quit.
...
(gdb) info threads
Id Target Id Frame
1 Thread 1.1 (CPU#0 [running]) bytes_is_nonzero (
start=0xfffffbfff031eebe <error: Cannot access memory at address 0xfffffbfff031eebe>, size=1)
at .../linux/mm/kasan/generic.c:98
* 2 Thread 1.2 (CPU#1 [halted ]) 0x00000000000fd0a9 in ?? ()
...
(gdb) thr 1
...
(gdb) bt
#0 bytes_is_nonzero (start=0xfffffbfff031eebe <error: Cannot access memory at address 0xfffffbfff031eebe>, size=1)
at .../linux/mm/kasan/generic.c:98
#1 memory_is_nonzero (start=0xfffffbfff031eebe, end=0xfffffbfff031eebf) at .../linux/mm/kasan/generic.c:115
#2 memory_is_poisoned_n (addr=0xffffffff818f75f0, size=8) at .../linux/mm/kasan/generic.c:140
#3 memory_is_poisoned (addr=0xffffffff818f75f0, size=8) at .../linux/mm/kasan/generic.c:172
#4 check_region_inline (addr=0xffffffff818f75f0, size=8, write=false, ret_ip=18446744071585002062)
at .../linux/mm/kasan/generic.c:191
#5 kasan_check_range (addr=addr@entry=0xffffffff818f75f0, size=size@entry=8, write=write@entry=false,
ret_ip=18446744071585002062) at .../linux/mm/kasan/generic.c:200
#6 0xffffffff813eb283 in __asan_loadN (addr=addr@entry=0xffffffff818f75f0, size=size@entry=8)
at .../linux/mm/kasan/generic.c:278
#7 0xffffffff815df24e in memcmp (cs=cs@entry=0xffffffff818f75f0, ct=ct@entry=0x1be2fe4, count=<optimized out>,
count@entry=12) at .../linux/lib/string.c:683
#8 0xffffffff81ba2323 in cpuid_base_hypervisor (sig=0xffffffff818f75f0 "XenVMMXenVMM", leaves=2)
at .../linux/arch/x86/include/asm/cpuid/api.h:206
#9 xen_cpuid_base () at .../linux/arch/x86/include/asm/xen/hypervisor.h:46
#10 xen_prepare_pvh () at .../linux/arch/x86/platform/pvh/enlighten.c:119
#11 0x0000000001ba2588 in ?? ()
#12 0x0000000000000000 in ?? ()
(gdb)
Frames #7-#8 show the non-builtin memcmp() (lib/string.c) was called
even with __builtin_memcmp() being used in cpuid_base_hypervisor().
Signed-off-by: Mauricio Faria de Oliveira <mfo@igalia.com>
---
Changes in v4:
- Patch 1: address Juergen's feedback:
- s/In next patch/In a future patch/.
- Move footnote (Reasons not to include...) after "---".
- Add 'Reviewed-by: Juergen Gross' in patches 1 and 2 as well.
- Link to v3: https://lore.kernel.org/r/20260520-pvh-kasan-inline-v3-0-bede769c6ec7@igalia.com
Changes in v3:
- Create and use a separate header for inline string functions
to fix a build error reported by kernel test robot (patch 1).
- That also removes '#ifndef _SETUP/#endif' in <asm/string.h>.
- Link to v2: https://lore.kernel.org/r/20260427-pvh-kasan-inline-v2-0-2c57b8dcff6a@igalia.com
Changes in v2:
- Add comment about the return value of __inline_memcmp() in patch 1. (v3: now 2)
- Add 'Reviewed-by: Juergen Gross' in patches 2 and 3 (v3: now 3 and 4).
- Link to v1: https://lore.kernel.org/r/20260422-pvh-kasan-inline-v1-0-7e6194344c92@igalia.com
---
Mauricio Faria de Oliveira (4):
x86/asm: move inline string functions to <asm/string_inline.h>
x86/asm, x86/boot: expose inline memcmp
x86/cpuid: fix unbootable VMs by really inlining memcmp() in hypervisor_cpuid_base()
x86/pvh: fix unbootable VMs by really inlining memset() in xen_prepare_pvh()
arch/x86/boot/string.c | 6 ++----
arch/x86/include/asm/cpuid/api.h | 2 +-
arch/x86/include/asm/string.h | 21 +-------------------
arch/x86/include/asm/string_inline.h | 37 ++++++++++++++++++++++++++++++++++++
arch/x86/platform/pvh/enlighten.c | 3 ++-
5 files changed, 43 insertions(+), 26 deletions(-)
---
base-commit: d387b06f7c15b4639244ad66b4b0900c6a02b430
change-id: 20260422-pvh-kasan-inline-6efac77f1b27
Best regards,
--
Mauricio Faria de Oliveira <mfo@igalia.com>
On 2026-05-26 11:52, Mauricio Faria de Oliveira wrote: > The issue of unbootable VMs with CONFIG_PVH due to CONFIG_KASAN is back. > > Booting directly from vmlinux (instead of bzImage) now fails with gcc-14/15 > (but works with gcc-12/13) if CONFIG_KASAN_GENERIC is set, on Ubuntu 25.10. v5: https://lore.kernel.org/all/20260601-pvh-kasan-inline-v5-0-c996e92dfe9a@igalia.com -- Mauricio
On Tue, May 26, 2026 at 11:52:32AM -0300, Mauricio Faria de Oliveira wrote:
> The issue of unbootable VMs with CONFIG_PVH due to CONFIG_KASAN is back.
... and we care about that particular configuration because...?
> arch/x86/boot/string.c | 6 ++----
> arch/x86/include/asm/cpuid/api.h | 2 +-
> arch/x86/include/asm/string.h | 21 +-------------------
> arch/x86/include/asm/string_inline.h | 37 ++++++++++++++++++++++++++++++++++++
> arch/x86/platform/pvh/enlighten.c | 3 ++-
> 5 files changed, 43 insertions(+), 26 deletions(-)
I fail to see justification for those gymnastics.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
Thanks for having a look at this. On 2026-05-27 17:47, Borislav Petkov wrote: > On Tue, May 26, 2026 at 11:52:32AM -0300, Mauricio Faria de Oliveira wrote: >> The issue of unbootable VMs with CONFIG_PVH due to CONFIG_KASAN is back. > > ... and we care about that particular configuration because...? PVH and KASAN can be enabled independently for their own purposes (startup entry point and debugging), but when combined, KASAN might break booting via the PVH entry point. There are precedents for disabling KASAN instrumentation in specific code paths, including this one (see listed commits), so addressing this issue/regression seems reasonable. In this instance, the use case is to speed up the dev-test cycle when debugging on VMs: PVH allows booting directly from vmlinux, avoiding the need to build/compress and decompress bzImage, while KASAN helps detect issues early. > >> arch/x86/boot/string.c | 6 ++---- >> arch/x86/include/asm/cpuid/api.h | 2 +- >> arch/x86/include/asm/string.h | 21 +------------------- >> arch/x86/include/asm/string_inline.h | 37 ++++++++++++++++++++++++++++++++++++ >> arch/x86/platform/pvh/enlighten.c | 3 ++- >> 5 files changed, 43 insertions(+), 26 deletions(-) > > I fail to see justification for those gymnastics. Briefly, this is a side effect of reusing code from 'boot/string.c' through <asm/string.h>, which introduces additional complications; please see patch 1 for details. These complications could be avoided by not reusing that code and instead adding slightly duplicated code [1] directly to <asm/string.h>, if that is preferred. Please let me know your thoughts. Thanks, [1] https://lore.kernel.org/all/324ef97b16f52e0ccc72f6381d1b5dd2@igalia.com/ -- Mauricio
On Sat, May 30, 2026 at 03:25:25PM -0300, Mauricio Faria de Oliveira wrote:
> In this instance, the use case is to speed up the dev-test cycle when
> debugging on VMs: PVH allows booting directly from vmlinux, avoiding the
> need to build/compress and decompress bzImage, while KASAN helps detect
> issues early.
Makes sense in itself. And you can't do your use case with KVM? It has to be
Xen?
> Briefly, this is a side effect of reusing code from 'boot/string.c'
> through <asm/string.h>, which introduces additional complications;
> please see patch 1 for details.
>
> These complications could be avoided by not reusing that code and
> instead adding slightly duplicated code [1] directly to <asm/string.h>,
> if that is preferred.
I'll take a look.
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
On 2026-05-30 19:04, Borislav Petkov wrote: > On Sat, May 30, 2026 at 03:25:25PM -0300, Mauricio Faria de Oliveira wrote: >> In this instance, the use case is to speed up the dev-test cycle when >> debugging on VMs: PVH allows booting directly from vmlinux, avoiding the >> need to build/compress and decompress bzImage, while KASAN helps detect >> issues early. > > Makes sense in itself. And you can't do your use case with KVM? It has to be > Xen? It uses KVM, actually. IIUIC, the PVH entry point was originally developed for Xen, but later QEMU support was added for KVM as well. (found this blog post [1] with a nice summary and links about it.) >> Briefly, this is a side effect of reusing code from 'boot/string.c' >> through <asm/string.h>, which introduces additional complications; >> please see patch 1 for details. >> >> These complications could be avoided by not reusing that code and >> instead adding slightly duplicated code [1] directly to <asm/string.h>, >> if that is preferred. > > I'll take a look. Ok, thanks! > > Thx. -- Mauricio
On 2026-05-30 20:01, Mauricio Faria de Oliveira wrote: > On 2026-05-30 19:04, Borislav Petkov wrote: >> On Sat, May 30, 2026 at 03:25:25PM -0300, Mauricio Faria de Oliveira wrote: >>> In this instance, the use case is to speed up the dev-test cycle when >>> debugging on VMs: PVH allows booting directly from vmlinux, avoiding the >>> need to build/compress and decompress bzImage, while KASAN helps detect >>> issues early. >> >> Makes sense in itself. And you can't do your use case with KVM? It has to be >> Xen? > > It uses KVM, actually. > > IIUIC, the PVH entry point was originally developed for Xen, but later > QEMU support was added for KVM as well. > (found this blog post [1] with a nice summary and links about it.) [1] https://stefano-garzarella.github.io/posts/2019-08-23-qemu-linux-kernel-pvh/ >>> Briefly, this is a side effect of reusing code from 'boot/string.c' >>> through <asm/string.h>, which introduces additional complications; >>> please see patch 1 for details. >>> >>> These complications could be avoided by not reusing that code and >>> instead adding slightly duplicated code [1] directly to <asm/string.h>, >>> if that is preferred. >> >> I'll take a look. > > Ok, thanks! > >> >> Thx. -- Mauricio
© 2016 - 2026 Red Hat, Inc.