[PATCH v11 13/21] KVM: selftests: Add helpers to init TDX memory and finalize VM

Sagi Shahar posted 21 patches 4 months, 2 weeks ago
There is a newer version of this series
[PATCH v11 13/21] KVM: selftests: Add helpers to init TDX memory and finalize VM
Posted by Sagi Shahar 4 months, 2 weeks ago
From: Ackerley Tng <ackerleytng@google.com>

TDX protected memory needs to be measured and encrypted before it can be
used by the guest. Traverse the VM's memory regions and initialize all
the protected ranges by calling KVM_TDX_INIT_MEM_REGION.

Once all the memory is initialized, the VM can be finalized by calling
KVM_TDX_FINALIZE_VM.

Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Co-developed-by: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Erdem Aktas <erdemaktas@google.com>
Co-developed-by: Sagi Shahar <sagis@google.com>
Signed-off-by: Sagi Shahar <sagis@google.com>
---
 .../selftests/kvm/include/x86/tdx/tdx_util.h  |  2 +
 .../selftests/kvm/lib/x86/tdx/tdx_util.c      | 58 +++++++++++++++++++
 2 files changed, 60 insertions(+)

diff --git a/tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h b/tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h
index a2509959c7ce..2467b6c35557 100644
--- a/tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h
+++ b/tools/testing/selftests/kvm/include/x86/tdx/tdx_util.h
@@ -71,4 +71,6 @@ void vm_tdx_load_common_boot_parameters(struct kvm_vm *vm);
 void vm_tdx_load_vcpu_boot_parameters(struct kvm_vm *vm, struct kvm_vcpu *vcpu);
 void vm_tdx_set_vcpu_entry_point(struct kvm_vcpu *vcpu, void *guest_code);
 
+void vm_tdx_finalize(struct kvm_vm *vm);
+
 #endif // SELFTESTS_TDX_TDX_UTIL_H
diff --git a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
index 2551b3eac8f8..53cfadeff8de 100644
--- a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
+++ b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
@@ -270,3 +270,61 @@ void vm_tdx_init_vm(struct kvm_vm *vm, uint64_t attributes)
 
 	free(init_vm);
 }
+
+static void tdx_init_mem_region(struct kvm_vm *vm, void *source_pages,
+				uint64_t gpa, uint64_t size)
+{
+	uint32_t metadata = KVM_TDX_MEASURE_MEMORY_REGION;
+	struct kvm_tdx_init_mem_region mem_region = {
+		.source_addr = (uint64_t)source_pages,
+		.gpa = gpa,
+		.nr_pages = size / PAGE_SIZE,
+	};
+	struct kvm_vcpu *vcpu;
+
+	vcpu = list_first_entry_or_null(&vm->vcpus, struct kvm_vcpu, list);
+
+	TEST_ASSERT((mem_region.nr_pages > 0) &&
+		    ((mem_region.nr_pages * PAGE_SIZE) == size),
+		    "Cannot add partial pages to the guest memory.\n");
+	TEST_ASSERT(((uint64_t)source_pages & (PAGE_SIZE - 1)) == 0,
+		    "Source memory buffer is not page aligned\n");
+	vm_tdx_vcpu_ioctl(vcpu, KVM_TDX_INIT_MEM_REGION, metadata, &mem_region);
+}
+
+static void load_td_private_memory(struct kvm_vm *vm)
+{
+	struct userspace_mem_region *region;
+	int ctr;
+
+	hash_for_each(vm->regions.slot_hash, ctr, region, slot_node) {
+		const struct sparsebit *protected_pages = region->protected_phy_pages;
+		const vm_paddr_t gpa_base = region->region.guest_phys_addr;
+		const uint64_t hva_base = region->region.userspace_addr;
+		const sparsebit_idx_t lowest_page_in_region = gpa_base >> vm->page_shift;
+		sparsebit_idx_t i, j;
+
+		if (!sparsebit_any_set(protected_pages))
+			continue;
+
+		TEST_ASSERT(region->region.guest_memfd != -1,
+			    "TD private memory must be backed by guest_memfd");
+
+		sparsebit_for_each_set_range(protected_pages, i, j) {
+			const uint64_t size_to_load = (j - i + 1) * vm->page_size;
+			const uint64_t offset =
+				(i - lowest_page_in_region) * vm->page_size;
+			const uint64_t hva = hva_base + offset;
+			const uint64_t gpa = gpa_base + offset;
+
+			vm_mem_set_private(vm, gpa, size_to_load);
+			tdx_init_mem_region(vm, (void *)hva, gpa, size_to_load);
+		}
+	}
+}
+
+void vm_tdx_finalize(struct kvm_vm *vm)
+{
+	load_td_private_memory(vm);
+	vm_tdx_vm_ioctl(vm, KVM_TDX_FINALIZE_VM, 0, NULL);
+}
-- 
2.51.0.536.g15c5d4f767-goog
Re: [PATCH v11 13/21] KVM: selftests: Add helpers to init TDX memory and finalize VM
Posted by Ira Weiny 3 months, 3 weeks ago
Sagi Shahar wrote:
> From: Ackerley Tng <ackerleytng@google.com>
> 

[snip]

> diff --git a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> index 2551b3eac8f8..53cfadeff8de 100644
> --- a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> +++ b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> @@ -270,3 +270,61 @@ void vm_tdx_init_vm(struct kvm_vm *vm, uint64_t attributes)
>  
>  	free(init_vm);
>  }
> +

[snip]

> +
> +void vm_tdx_finalize(struct kvm_vm *vm)

Why is this not a new kvm_arch_vm_finalize_vcpu() call?

Ira

> +{
> +	load_td_private_memory(vm);
> +	vm_tdx_vm_ioctl(vm, KVM_TDX_FINALIZE_VM, 0, NULL);
> +}
> -- 
> 2.51.0.536.g15c5d4f767-goog
>
Re: [PATCH v11 13/21] KVM: selftests: Add helpers to init TDX memory and finalize VM
Posted by Sagi Shahar 3 months, 2 weeks ago
On Wed, Oct 15, 2025 at 11:25 AM Ira Weiny <ira.weiny@intel.com> wrote:
>
> Sagi Shahar wrote:
> > From: Ackerley Tng <ackerleytng@google.com>
> >
>
> [snip]
>
> > diff --git a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> > index 2551b3eac8f8..53cfadeff8de 100644
> > --- a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> > +++ b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> > @@ -270,3 +270,61 @@ void vm_tdx_init_vm(struct kvm_vm *vm, uint64_t attributes)
> >
> >       free(init_vm);
> >  }
> > +
>
> [snip]
>
> > +
> > +void vm_tdx_finalize(struct kvm_vm *vm)
>
> Why is this not a new kvm_arch_vm_finalize_vcpu() call?

What do you mean?
>
> Ira
>
> > +{
> > +     load_td_private_memory(vm);
> > +     vm_tdx_vm_ioctl(vm, KVM_TDX_FINALIZE_VM, 0, NULL);
> > +}
> > --
> > 2.51.0.536.g15c5d4f767-goog
> >
Re: [PATCH v11 13/21] KVM: selftests: Add helpers to init TDX memory and finalize VM
Posted by Sean Christopherson 3 months, 2 weeks ago
On Thu, Oct 23, 2025, Sagi Shahar wrote:
> On Wed, Oct 15, 2025 at 11:25 AM Ira Weiny <ira.weiny@intel.com> wrote:
> >
> > Sagi Shahar wrote:
> > > From: Ackerley Tng <ackerleytng@google.com>
> > >
> >
> > [snip]
> >
> > > diff --git a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> > > index 2551b3eac8f8..53cfadeff8de 100644
> > > --- a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> > > +++ b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> > > @@ -270,3 +270,61 @@ void vm_tdx_init_vm(struct kvm_vm *vm, uint64_t attributes)
> > >
> > >       free(init_vm);
> > >  }
> > > +
> >
> > [snip]
> >
> > > +
> > > +void vm_tdx_finalize(struct kvm_vm *vm)
> >
> > Why is this not a new kvm_arch_vm_finalize_vcpu() call?
> 
> What do you mean?

Ira is pointing out that upstream now has kvm_arch_vm_finalize_vcpus(), so you
can (and I agree, should) implement that for x86.c, and do vm_tdx_finalize() from
there (based on the VM shape) instead of requiring the caller to manually finalize
the TD.

Unlike SEV, where userspace can manipulate guest state prior to LAUNCH, TDX guest
state is unreachable from time zero, i.e. there is unlikely to be many (any?) use
cases where a selftest wants to do something between creating vCPUs and finalizing
the TD.
Re: [PATCH v11 13/21] KVM: selftests: Add helpers to init TDX memory and finalize VM
Posted by Sagi Shahar 3 months, 2 weeks ago
On Fri, Oct 24, 2025 at 11:02 AM Sean Christopherson <seanjc@google.com> wrote:
>
> On Thu, Oct 23, 2025, Sagi Shahar wrote:
> > On Wed, Oct 15, 2025 at 11:25 AM Ira Weiny <ira.weiny@intel.com> wrote:
> > >
> > > Sagi Shahar wrote:
> > > > From: Ackerley Tng <ackerleytng@google.com>
> > > >
> > >
> > > [snip]
> > >
> > > > diff --git a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> > > > index 2551b3eac8f8..53cfadeff8de 100644
> > > > --- a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> > > > +++ b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> > > > @@ -270,3 +270,61 @@ void vm_tdx_init_vm(struct kvm_vm *vm, uint64_t attributes)
> > > >
> > > >       free(init_vm);
> > > >  }
> > > > +
> > >
> > > [snip]
> > >
> > > > +
> > > > +void vm_tdx_finalize(struct kvm_vm *vm)
> > >
> > > Why is this not a new kvm_arch_vm_finalize_vcpu() call?
> >
> > What do you mean?
>
> Ira is pointing out that upstream now has kvm_arch_vm_finalize_vcpus(), so you
> can (and I agree, should) implement that for x86.c, and do vm_tdx_finalize() from
> there (based on the VM shape) instead of requiring the caller to manually finalize
> the TD.
>
> Unlike SEV, where userspace can manipulate guest state prior to LAUNCH, TDX guest
> state is unreachable from time zero, i.e. there is unlikely to be many (any?) use
> cases where a selftest wants to do something between creating vCPUs and finalizing
> the TD.

There are actually a few use cases for calling vm_tdx_finalize
seperately from the create phase. One such case is when a user wants
to add additional memslots for testing shared memory conversions.
Another one is installing interrupt handlers. But these are probably
rare enough that they can call __vm_create() and vm_vcpu_add()
manually instead of using the wrapper
Re: [PATCH v11 13/21] KVM: selftests: Add helpers to init TDX memory and finalize VM
Posted by Sean Christopherson 3 months, 2 weeks ago
On Fri, Oct 24, 2025, Sagi Shahar wrote:
> On Fri, Oct 24, 2025 at 11:02 AM Sean Christopherson <seanjc@google.com> wrote:
> >
> > On Thu, Oct 23, 2025, Sagi Shahar wrote:
> > > On Wed, Oct 15, 2025 at 11:25 AM Ira Weiny <ira.weiny@intel.com> wrote:
> > > >
> > > > Sagi Shahar wrote:
> > > > > From: Ackerley Tng <ackerleytng@google.com>
> > > > >
> > > >
> > > > [snip]
> > > >
> > > > > diff --git a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> > > > > index 2551b3eac8f8..53cfadeff8de 100644
> > > > > --- a/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> > > > > +++ b/tools/testing/selftests/kvm/lib/x86/tdx/tdx_util.c
> > > > > @@ -270,3 +270,61 @@ void vm_tdx_init_vm(struct kvm_vm *vm, uint64_t attributes)
> > > > >
> > > > >       free(init_vm);
> > > > >  }
> > > > > +
> > > >
> > > > [snip]
> > > >
> > > > > +
> > > > > +void vm_tdx_finalize(struct kvm_vm *vm)
> > > >
> > > > Why is this not a new kvm_arch_vm_finalize_vcpu() call?
> > >
> > > What do you mean?
> >
> > Ira is pointing out that upstream now has kvm_arch_vm_finalize_vcpus(), so you
> > can (and I agree, should) implement that for x86.c, and do vm_tdx_finalize() from
> > there (based on the VM shape) instead of requiring the caller to manually finalize
> > the TD.
> >
> > Unlike SEV, where userspace can manipulate guest state prior to LAUNCH, TDX guest
> > state is unreachable from time zero, i.e. there is unlikely to be many (any?) use
> > cases where a selftest wants to do something between creating vCPUs and finalizing
> > the TD.
> 
> There are actually a few use cases for calling vm_tdx_finalize
> seperately from the create phase. One such case is when a user wants
> to add additional memslots for testing shared memory conversions.

Why does creating memslots need to be done before finalizing the TD?  The extra
memslots shouldn't _need_ to be included in the initial image, and I don't recall
anything in the kernel that prevents adding a memslot after the TD is initialized.

> Another one is installing interrupt handlers. But these are probably
> rare enough that they can call __vm_create() and vm_vcpu_add()
> manually instead of using the wrapper

Hmm, for installing interrupt handlers, we could handle that by adding a guest
variant, e.g. so that the guest can install/remove exception handlers after
the initial image is finalized.  We probably want that anyways, otherwise tests
would be unable to manipulate handlers throughout a test.

In fact, it's tempting to kill off vm_install_exception_handler() in x86 entirely,
but that's probably not worth the churn (and some tests would be made more complex).