[PATCH v9 10/19] KVM: selftests: Set up TDX boot code region

Sagi Shahar posted 19 patches 1 month, 1 week ago
There is a newer version of this series
[PATCH v9 10/19] KVM: selftests: Set up TDX boot code region
Posted by Sagi Shahar 1 month, 1 week ago
Add memory for TDX boot code in a separate memslot.

Use virt_map() to get identity map in this memory region to allow for
seamless transition from paging disabled to paging enabled code.

Copy the boot code into the memory region and set up the reset vectors
at this point. While it's possible to separate the memory allocation and
boot code initialization into separate functions, having all the
calculations for memory size and offsets in one place simplifies the
code and avoids duplications.

Handcode the reset vector as suggested by Sean Christopherson.

Suggested-by: Sean Christopherson <seanjc@google.com>
Co-developed-by: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Sagi Shahar <sagis@google.com>
---
 tools/testing/selftests/kvm/Makefile.kvm      |  1 +
 .../selftests/kvm/include/x86/tdx/tdx_util.h  |  2 +
 .../selftests/kvm/lib/x86/tdx/tdx_util.c      | 54 +++++++++++++++++++
 3 files changed, 57 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 03754ce2e983..c42b579fb7c5 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -31,6 +31,7 @@ LIBKVM_x86 += lib/x86/sev.c
 LIBKVM_x86 += lib/x86/svm.c
 LIBKVM_x86 += lib/x86/ucall.c
 LIBKVM_x86 += lib/x86/vmx.c
+LIBKVM_x86 += lib/x86/tdx/tdx_util.c
 LIBKVM_x86 += lib/x86/tdx/td_boot.S
 
 LIBKVM_arm64 += lib/arm64/gic.c
diff --git a/tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h b/tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h
index 286d5e3c24b1..ec05bcd59145 100644
--- a/tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h
+++ b/tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h
@@ -11,4 +11,6 @@ static inline bool is_tdx_vm(struct kvm_vm *vm)
 	return vm->type == KVM_X86_TDX_VM;
 }
 
+void vm_tdx_setup_boot_code_region(struct kvm_vm *vm);
+
 #endif // SELFTESTS_TDX_TDX_UTIL_H
diff --git a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
new file mode 100644
index 000000000000..15833b9eb5d5
--- /dev/null
+++ b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
@@ -0,0 +1,54 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+#include <stdint.h>
+
+#include "kvm_util.h"
+#include "processor.h"
+#include "tdx/td_boot.h"
+#include "tdx/tdx_util.h"
+
+/* Arbitrarily selected to avoid overlaps with anything else */
+#define TD_BOOT_CODE_SLOT	20
+
+#define X86_RESET_VECTOR	0xfffffff0ul
+#define X86_RESET_VECTOR_SIZE	16
+
+void vm_tdx_setup_boot_code_region(struct kvm_vm *vm)
+{
+	size_t total_code_size = TD_BOOT_CODE_SIZE + X86_RESET_VECTOR_SIZE;
+	vm_paddr_t boot_code_gpa = X86_RESET_VECTOR - TD_BOOT_CODE_SIZE;
+	vm_paddr_t alloc_gpa = round_down(boot_code_gpa, PAGE_SIZE);
+	size_t nr_pages = DIV_ROUND_UP(total_code_size, PAGE_SIZE);
+	vm_paddr_t gpa;
+	uint8_t *hva;
+
+	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS,
+				    alloc_gpa,
+				    TD_BOOT_CODE_SLOT, nr_pages,
+				    KVM_MEM_GUEST_MEMFD);
+
+	gpa = vm_phy_pages_alloc(vm, nr_pages, alloc_gpa, TD_BOOT_CODE_SLOT);
+	TEST_ASSERT(gpa == alloc_gpa, "Failed vm_phy_pages_alloc\n");
+
+	virt_map(vm, alloc_gpa, alloc_gpa, nr_pages);
+	hva = addr_gpa2hva(vm, boot_code_gpa);
+	memcpy(hva, td_boot, TD_BOOT_CODE_SIZE);
+
+	hva += TD_BOOT_CODE_SIZE;
+	TEST_ASSERT(hva == addr_gpa2hva(vm, X86_RESET_VECTOR),
+		    "Expected RESET vector at hva 0x%lx, got %lx",
+		    (unsigned long)addr_gpa2hva(vm, X86_RESET_VECTOR), (unsigned long)hva);
+
+	/*
+	 * Handcode "JMP rel8" at the RESET vector to jump back to the TD boot
+	 * code, as there are only 16 bytes at the RESET vector before RIP will
+	 * wrap back to zero.  Insert a trailing int3 so that the vCPU crashes
+	 * in case the JMP somehow falls through.  Note!  The target address is
+	 * relative to the end of the instruction!
+	 */
+	TEST_ASSERT(TD_BOOT_CODE_SIZE < 256,
+		    "TD boot code not addressable by 'JMP rel8'");
+	hva[0] = 0xeb;
+	hva[1] = 256 - 2 - TD_BOOT_CODE_SIZE;
+	hva[2] = 0xcc;
+}
-- 
2.51.0.rc1.193.gad69d77794-goog
Re: [PATCH v9 10/19] KVM: selftests: Set up TDX boot code region
Posted by Yan Zhao 1 month, 1 week ago
On Wed, Aug 20, 2025 at 09:29:03PM -0700, Sagi Shahar wrote:
> Add memory for TDX boot code in a separate memslot.
> 
> Use virt_map() to get identity map in this memory region to allow for
> seamless transition from paging disabled to paging enabled code.
> 
> Copy the boot code into the memory region and set up the reset vectors
> at this point. While it's possible to separate the memory allocation and
> boot code initialization into separate functions, having all the
> calculations for memory size and offsets in one place simplifies the
> code and avoids duplications.
> 
> Handcode the reset vector as suggested by Sean Christopherson.
> 
> Suggested-by: Sean Christopherson <seanjc@google.com>
> Co-developed-by: Erdem Aktas <erdemaktas@google.com>
> Signed-off-by: Erdem Aktas <erdemaktas@google.com>
> Signed-off-by: Sagi Shahar <sagis@google.com>
> ---
>  tools/testing/selftests/kvm/Makefile.kvm      |  1 +
>  .../selftests/kvm/include/x86/tdx/tdx_util.h  |  2 +
>  .../selftests/kvm/lib/x86/tdx/tdx_util.c      | 54 +++++++++++++++++++
>  3 files changed, 57 insertions(+)
>  create mode 100644 tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> 
> diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
> index 03754ce2e983..c42b579fb7c5 100644
> --- a/tools/testing/selftests/kvm/Makefile.kvm
> +++ b/tools/testing/selftests/kvm/Makefile.kvm
> @@ -31,6 +31,7 @@ LIBKVM_x86 += lib/x86/sev.c
>  LIBKVM_x86 += lib/x86/svm.c
>  LIBKVM_x86 += lib/x86/ucall.c
>  LIBKVM_x86 += lib/x86/vmx.c
> +LIBKVM_x86 += lib/x86/tdx/tdx_util.c
>  LIBKVM_x86 += lib/x86/tdx/td_boot.S
>  
>  LIBKVM_arm64 += lib/arm64/gic.c
> diff --git a/tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h b/tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h
> index 286d5e3c24b1..ec05bcd59145 100644
> --- a/tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h
> +++ b/tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h
> @@ -11,4 +11,6 @@ static inline bool is_tdx_vm(struct kvm_vm *vm)
>  	return vm->type == KVM_X86_TDX_VM;
>  }
>  
> +void vm_tdx_setup_boot_code_region(struct kvm_vm *vm);
> +
>  #endif // SELFTESTS_TDX_TDX_UTIL_H
> diff --git a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> new file mode 100644
> index 000000000000..15833b9eb5d5
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> @@ -0,0 +1,54 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +
> +#include <stdint.h>
> +
> +#include "kvm_util.h"
> +#include "processor.h"
> +#include "tdx/td_boot.h"
> +#include "tdx/tdx_util.h"
> +
> +/* Arbitrarily selected to avoid overlaps with anything else */
> +#define TD_BOOT_CODE_SLOT	20
> +
> +#define X86_RESET_VECTOR	0xfffffff0ul
> +#define X86_RESET_VECTOR_SIZE	16
> +
> +void vm_tdx_setup_boot_code_region(struct kvm_vm *vm)
> +{
> +	size_t total_code_size = TD_BOOT_CODE_SIZE + X86_RESET_VECTOR_SIZE;
> +	vm_paddr_t boot_code_gpa = X86_RESET_VECTOR - TD_BOOT_CODE_SIZE;
> +	vm_paddr_t alloc_gpa = round_down(boot_code_gpa, PAGE_SIZE);
> +	size_t nr_pages = DIV_ROUND_UP(total_code_size, PAGE_SIZE);
> +	vm_paddr_t gpa;
> +	uint8_t *hva;
> +
> +	vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS,
> +				    alloc_gpa,
> +				    TD_BOOT_CODE_SLOT, nr_pages,
> +				    KVM_MEM_GUEST_MEMFD);
> +
> +	gpa = vm_phy_pages_alloc(vm, nr_pages, alloc_gpa, TD_BOOT_CODE_SLOT);
> +	TEST_ASSERT(gpa == alloc_gpa, "Failed vm_phy_pages_alloc\n");
> +
> +	virt_map(vm, alloc_gpa, alloc_gpa, nr_pages);
> +	hva = addr_gpa2hva(vm, boot_code_gpa);
> +	memcpy(hva, td_boot, TD_BOOT_CODE_SIZE);
> +
> +	hva += TD_BOOT_CODE_SIZE;
> +	TEST_ASSERT(hva == addr_gpa2hva(vm, X86_RESET_VECTOR),
> +		    "Expected RESET vector at hva 0x%lx, got %lx",
> +		    (unsigned long)addr_gpa2hva(vm, X86_RESET_VECTOR), (unsigned long)hva);
> +
> +	/*
> +	 * Handcode "JMP rel8" at the RESET vector to jump back to the TD boot
> +	 * code, as there are only 16 bytes at the RESET vector before RIP will
> +	 * wrap back to zero.  Insert a trailing int3 so that the vCPU crashes
> +	 * in case the JMP somehow falls through.  Note!  The target address is
> +	 * relative to the end of the instruction!
> +	 */
> +	TEST_ASSERT(TD_BOOT_CODE_SIZE < 256,
Looks TD_BOOT_CODE_SIZE needs to be <= 126, as the jump range is limited to -128
to +127 for JMP rel8.

> +		    "TD boot code not addressable by 'JMP rel8'");
> +	hva[0] = 0xeb;
> +	hva[1] = 256 - 2 - TD_BOOT_CODE_SIZE;
> +	hva[2] = 0xcc;
> +}
> -- 
> 2.51.0.rc1.193.gad69d77794-goog
> 
>
Re: [PATCH v9 10/19] KVM: selftests: Set up TDX boot code region
Posted by Sean Christopherson 1 month, 1 week ago
On Mon, Aug 25, 2025, Yan Zhao wrote:
> > +	/*
> > +	 * Handcode "JMP rel8" at the RESET vector to jump back to the TD boot
> > +	 * code, as there are only 16 bytes at the RESET vector before RIP will
> > +	 * wrap back to zero.  Insert a trailing int3 so that the vCPU crashes
> > +	 * in case the JMP somehow falls through.  Note!  The target address is
> > +	 * relative to the end of the instruction!
> > +	 */
> > +	TEST_ASSERT(TD_BOOT_CODE_SIZE < 256,
> Looks TD_BOOT_CODE_SIZE needs to be <= 126, as the jump range is limited to -128
> to +127 for JMP rel8.

Gah, I managed to forget that relative targets obviously need to be signed values,
and I also forgot to account for the size of the JMP in the assert.  Go me.

Maybe express this as:

	TEST_ASSERT(TD_BOOT_CODE_SIZE + 2 < 128,
		    "TD boot code not addressable by 'JMP rel8'");
	
> > +		    "TD boot code not addressable by 'JMP rel8'");
> > +	hva[0] = 0xeb;
> > +	hva[1] = 256 - 2 - TD_BOOT_CODE_SIZE;

I think I lucked into getting this right though?

> > +	hva[2] = 0xcc;
> > +}
> > -- 
> > 2.51.0.rc1.193.gad69d77794-goog
> > 
> >
Re: [PATCH v9 10/19] KVM: selftests: Set up TDX boot code region
Posted by Yan Zhao 1 month, 1 week ago
On Tue, Aug 26, 2025 at 09:38:15AM -0700, Sean Christopherson wrote:
> On Mon, Aug 25, 2025, Yan Zhao wrote:
> > > +	/*
> > > +	 * Handcode "JMP rel8" at the RESET vector to jump back to the TD boot
> > > +	 * code, as there are only 16 bytes at the RESET vector before RIP will
> > > +	 * wrap back to zero.  Insert a trailing int3 so that the vCPU crashes
> > > +	 * in case the JMP somehow falls through.  Note!  The target address is
> > > +	 * relative to the end of the instruction!
> > > +	 */
> > > +	TEST_ASSERT(TD_BOOT_CODE_SIZE < 256,
> > Looks TD_BOOT_CODE_SIZE needs to be <= 126, as the jump range is limited to -128
> > to +127 for JMP rel8.
> 
> Gah, I managed to forget that relative targets obviously need to be signed values,
> and I also forgot to account for the size of the JMP in the assert.  Go me.
> 
> Maybe express this as:
> 
> 	TEST_ASSERT(TD_BOOT_CODE_SIZE + 2 < 128,
> 		    "TD boot code not addressable by 'JMP rel8'");
I like this version, which's is much clearer than asserting TD_BOOT_CODE_SIZE
alone.

nit: TD_BOOT_CODE_SIZE + 2 can be equal to 128, i.e.,

	TEST_ASSERT(TD_BOOT_CODE_SIZE + 2 <= 128,
		    "TD boot code not addressable by 'JMP rel8'");


> > > +		    "TD boot code not addressable by 'JMP rel8'");
> > > +	hva[0] = 0xeb;
> > > +	hva[1] = 256 - 2 - TD_BOOT_CODE_SIZE;
> 
> I think I lucked into getting this right though?
Yes, this one is correct :)


> > > +	hva[2] = 0xcc;
> > > +}
> > > -- 
> > > 2.51.0.rc1.193.gad69d77794-goog
> > > 
> > >