x86/hyper-v: Filter non-canonical addresses passed via HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST(_EX)

[PATCH] x86/hyper-v: Filter non-canonical addresses passed via HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST(_EX)

Posted by Manuel Andreas 3 months, 2 weeks ago

In KVM guests with Hyper-V hypercalls enabled, the hypercalls 
HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST and HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX 
allow a guest to request invalidation of portions of a virtual TLB.
For this, the hypercall parameter includes a list of GVAs that are supposed 
to be invalidated.

However, when non-canonical GVAs are passed, there is currently no 
filtering in place and they are eventually passed to checked invocations of 
INVVPID on Intel / INVLPGA on AMD.
While the AMD variant (INVLPGA) will silently ignore the non-canonical 
address and perform a no-op, the Intel variant (INVVPID) will fail and end 
up in invvpid_error, where a WARN_ONCE is triggered:

invvpid failed: ext=0x0 vpid=1 gva=0xaaaaaaaaaaaaa000
WARNING: CPU: 6 PID: 326 at arch/x86/kvm/vmx/vmx.c:482 
invvpid_error+0x91/0xa0 [kvm_intel]
Modules linked in: kvm_intel kvm 9pnet_virtio irqbypass fuse
CPU: 6 UID: 0 PID: 326 Comm: kvm-vm Not tainted 6.15.0 #14 PREEMPT(voluntary)
RIP: 0010:invvpid_error+0x91/0xa0 [kvm_intel]
Call Trace:
  <TASK>
  vmx_flush_tlb_gva+0x320/0x490 [kvm_intel]
  ? __pfx_vmx_flush_tlb_gva+0x10/0x10 [kvm_intel]
  ? kfifo_copy_out+0xcf/0x120
  kvm_hv_vcpu_flush_tlb+0x24f/0x4f0 [kvm]
  ? __pfx_kvm_hv_vcpu_flush_tlb+0x10/0x10 [kvm]
  ? kvm_pmu_is_valid_msr+0x6e/0x80 [kvm]
  ? kvm_get_msr_common+0x219/0x20f0 [kvm]
  kvm_arch_vcpu_ioctl_run+0x3013/0x5810 [kvm]
  /* ... */

Hyper-V documents that invalid GVAs (those that are beyond a partition's 
GVA space) are to be ignored. While not completely clear whether this 
ruling also applies to non-canonical GVAs, it is likely fine to make that 
assumption.

The following patch addresses the issue by skipping non-canonical GVAs 
before calling the architecture-specific invalidation primitive.
I've validated it against a PoC and the issue seems to be fixed.

Signed-off-by: Manuel Andreas <manuel.andreas@tum.de>
Suggested-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
  arch/x86/kvm/hyperv.c | 3 +++
  1 file changed, 3 insertions(+)

diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 24f0318c50d7..644a7aae6dab 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -1979,6 +1979,9 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu)
  		if (entries[i] == KVM_HV_TLB_FLUSHALL_ENTRY)
  			goto out_flush_all;

+                if (is_noncanonical_invlpg_address(entries[i], vcpu))
+                        continue;
+
  		/*
  		 * Lower 12 bits of 'address' encode the number of additional
  		 * pages to flush.
-- 
2.50.0

Re: [PATCH] x86/hyper-v: Filter non-canonical addresses passed via HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST(_EX)

Posted by Sean Christopherson 3 months, 2 weeks ago

On Wed, 25 Jun 2025 15:53:19 +0200, Manuel Andreas wrote:
> In KVM guests with Hyper-V hypercalls enabled, the hypercalls
> HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST and HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX
> allow a guest to request invalidation of portions of a virtual TLB.
> For this, the hypercall parameter includes a list of GVAs that are supposed
> to be invalidated.
> 
> However, when non-canonical GVAs are passed, there is currently no
> filtering in place and they are eventually passed to checked invocations of
> INVVPID on Intel / INVLPGA on AMD.
> While the AMD variant (INVLPGA) will silently ignore the non-canonical
> address and perform a no-op, the Intel variant (INVVPID) will fail and end
> up in invvpid_error, where a WARN_ONCE is triggered:
> 
> [...]

Applied to kvm-x86 fixes, with a massaged changelog, e.g. to call out that
"real" Hyper-V behaves this way.  Thanks!

[1/1] x86/hyper-v: Filter non-canonical addresses passed via HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST(_EX)
      https://github.com/kvm-x86/linux/commit/fa787ac07b3c

--
https://github.com/kvm-x86/kvm-unit-tests/tree/next

Re: [PATCH] x86/hyper-v: Filter non-canonical addresses passed via HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST(_EX)

Posted by Manuel Andreas 3 months, 2 weeks ago

On 6/26/25 12:25 AM, Sean Christopherson wrote:
> On Wed, 25 Jun 2025 15:53:19 +0200, Manuel Andreas wrote:
>> In KVM guests with Hyper-V hypercalls enabled, the hypercalls
>> HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST and HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX
>> allow a guest to request invalidation of portions of a virtual TLB.
>> For this, the hypercall parameter includes a list of GVAs that are supposed
>> to be invalidated.
>>
>> However, when non-canonical GVAs are passed, there is currently no
>> filtering in place and they are eventually passed to checked invocations of
>> INVVPID on Intel / INVLPGA on AMD.
>> While the AMD variant (INVLPGA) will silently ignore the non-canonical
>> address and perform a no-op, the Intel variant (INVVPID) will fail and end
>> up in invvpid_error, where a WARN_ONCE is triggered:
>>
>> [...]
> 
> Applied to kvm-x86 fixes, with a massaged changelog, e.g. to call out that
> "real" Hyper-V behaves this way.  Thanks!
> 
> [1/1] x86/hyper-v: Filter non-canonical addresses passed via HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST(_EX)
>        https://github.com/kvm-x86/linux/commit/fa787ac07b3c

Thanks for the quick approval and Vitaly for the Hyper-V testing!

Re: [PATCH] x86/hyper-v: Filter non-canonical addresses passed via HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST(_EX)

Posted by Vitaly Kuznetsov 3 months, 2 weeks ago

Manuel Andreas <manuel.andreas@tum.de> writes:

> In KVM guests with Hyper-V hypercalls enabled, the hypercalls 
> HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST and HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX 
> allow a guest to request invalidation of portions of a virtual TLB.
> For this, the hypercall parameter includes a list of GVAs that are supposed 
> to be invalidated.
>
> However, when non-canonical GVAs are passed, there is currently no 
> filtering in place and they are eventually passed to checked invocations of 
> INVVPID on Intel / INVLPGA on AMD.
> While the AMD variant (INVLPGA) will silently ignore the non-canonical 
> address and perform a no-op, the Intel variant (INVVPID) will fail and end 
> up in invvpid_error, where a WARN_ONCE is triggered:
>
> invvpid failed: ext=0x0 vpid=1 gva=0xaaaaaaaaaaaaa000
> WARNING: CPU: 6 PID: 326 at arch/x86/kvm/vmx/vmx.c:482 
> invvpid_error+0x91/0xa0 [kvm_intel]
> Modules linked in: kvm_intel kvm 9pnet_virtio irqbypass fuse
> CPU: 6 UID: 0 PID: 326 Comm: kvm-vm Not tainted 6.15.0 #14 PREEMPT(voluntary)
> RIP: 0010:invvpid_error+0x91/0xa0 [kvm_intel]
> Call Trace:
>   <TASK>
>   vmx_flush_tlb_gva+0x320/0x490 [kvm_intel]
>   ? __pfx_vmx_flush_tlb_gva+0x10/0x10 [kvm_intel]
>   ? kfifo_copy_out+0xcf/0x120
>   kvm_hv_vcpu_flush_tlb+0x24f/0x4f0 [kvm]
>   ? __pfx_kvm_hv_vcpu_flush_tlb+0x10/0x10 [kvm]
>   ? kvm_pmu_is_valid_msr+0x6e/0x80 [kvm]
>   ? kvm_get_msr_common+0x219/0x20f0 [kvm]
>   kvm_arch_vcpu_ioctl_run+0x3013/0x5810 [kvm]
>   /* ... */
>
> Hyper-V documents that invalid GVAs (those that are beyond a partition's 
> GVA space) are to be ignored. While not completely clear whether this 
> ruling also applies to non-canonical GVAs, it is likely fine to make that 
> assumption.

I wrote a simple test and ran it on Azure. Unless I screwed up, the
result confirms this assumption. The test was:

#include <linux/module.h>	/* Needed by all modules */
#include <linux/kernel.h>	/* Needed for KERN_INFO */

struct hv_tlb_flush {    /* HV_INPUT_FLUSH_VIRTUAL_ADDRESS_LIST */
        u64 address_space;
        u64 flags;
        u64 processor_mask;
        u64 gva_list[];
} __packed;

static inline u64 __hyperv_hypercall(u64 control, u64 input_address,
                                     u64 output_address)
{
	u64 res;

        asm volatile("mov %[output_address], %%r8\n\t"
                     "vmcall"
                     : "=a" (res),
                       "+c" (control), "+d" (input_address)
                     : [output_address] "r"(output_address)
                     : "cc", "memory", "r8", "r9", "r10", "r11");

	return res;
}

int init_module(void)
{
	u64 rcx = 0x0003UL | 1UL << 32; /* 1 rep */
	u64 status;
	struct hv_tlb_flush *flush = (struct hv_tlb_flush *)kzalloc(4096, GFP_KERNEL);
	void *out = kzalloc(4096, GFP_KERNEL);

	flush->flags = 0x3; /* all CPUS all address spaces */
	flush->processor_mask = 0x1;
	flush->gva_list[0] = 0xaaaaaaaaaaaaa000;

	status = __hyperv_hypercall(rcx, __pa(flush), __pa(out));

	printk("Flush result: %llx\n", status);

	kfree(flush);
	kfree(out);

	return -1;
}

void cleanup_module(void)
{
}

MODULE_LICENSE("GPL");

And the result of loading this module is:

[ 4228.888451] Flush result: 100000000

(1 in bit 32 means 1 rep completed, lower 16 bits == 0 indicate
HV_STATUS_SUCCESS).

>
> The following patch addresses the issue by skipping non-canonical GVAs 
> before calling the architecture-specific invalidation primitive.
> I've validated it against a PoC and the issue seems to be fixed.
>
> Signed-off-by: Manuel Andreas <manuel.andreas@tum.de>
> Suggested-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>   arch/x86/kvm/hyperv.c | 3 +++
>   1 file changed, 3 insertions(+)
>
> diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
> index 24f0318c50d7..644a7aae6dab 100644
> --- a/arch/x86/kvm/hyperv.c
> +++ b/arch/x86/kvm/hyperv.c
> @@ -1979,6 +1979,9 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu)
>   		if (entries[i] == KVM_HV_TLB_FLUSHALL_ENTRY)
>   			goto out_flush_all;
>
> +                if (is_noncanonical_invlpg_address(entries[i], vcpu))
> +                        continue;
> +
>   		/*
>   		 * Lower 12 bits of 'address' encode the number of additional
>   		 * pages to flush.

-- 
Vitaly