From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24669C433F5 for ; Fri, 11 Mar 2022 15:50:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349595AbiCKPvF (ORCPT ); Fri, 11 Mar 2022 10:51:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349528AbiCKPu7 (ORCPT ); Fri, 11 Mar 2022 10:50:59 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 065BB1C4689 for ; Fri, 11 Mar 2022 07:49:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013795; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7MOEJ5RjWZDPZbxRbiRpMy10rB9WaSfKp6mdphZp9/c=; b=FpkSo+eqGupcDeAKZeOQ5KoJpbqyPuSZLe0TufX6vobjXYz79enRxO111UHp/uwg6/x/qq 29lsl6fD/tgFPiXVKmVgR1X/u4H45Ag3xNyD5iJgz8CZytPtJXwJKxp2eDm+IiwD1AhUk5 jLpYucoYhN8ULvkmW/8/2PfIlwhZPH0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-557-Ee2F4p0fMmKOg74GF3BcHQ-1; Fri, 11 Mar 2022 10:49:52 -0500 X-MC-Unique: Ee2F4p0fMmKOg74GF3BcHQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7AFCE1091DA0; Fri, 11 Mar 2022 15:49:50 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id ED26E785FD; Fri, 11 Mar 2022 15:49:47 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 01/31] KVM: x86: hyper-v: Resurrect dedicated KVM_REQ_HV_TLB_FLUSH flag Date: Fri, 11 Mar 2022 16:49:13 +0100 Message-Id: <20220311154943.2299191-2-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In preparation to implementing fine-grained Hyper-V TLB flush and Direct TLB flush, resurrect dedicated KVM_REQ_HV_TLB_FLUSH request bit. As KVM_REQ_TLB_FLUSH_GUEST is a stronger operation, clear KVM_REQ_HV_TLB_FLUSH request in kvm_service_local_tlb_flush_requests() when KVM_REQ_TLB_FLUSH_GUEST was also requested. No functional change intended. Signed-off-by: Vitaly Kuznetsov --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/hyperv.c | 4 ++-- arch/x86/kvm/x86.c | 7 ++++++- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index da2f3a21e37b..11bcae77bba4 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -104,6 +104,8 @@ KVM_ARCH_REQ_FLAGS(30, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) #define KVM_REQ_MMU_FREE_OBSOLETE_ROOTS \ KVM_ARCH_REQ_FLAGS(31, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) +#define KVM_REQ_HV_TLB_FLUSH \ + KVM_ARCH_REQ_FLAGS(32, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) =20 #define CR0_RESERVED_BITS \ (~(unsigned long)(X86_CR0_PE | X86_CR0_MP | X86_CR0_EM | X86_CR0_TS \ diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index a32f54ab84a2..0cad1fed6007 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -1890,11 +1890,11 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, = struct kvm_hv_hcall *hc) * analyze it here, flush TLB regardless of the specified address space. */ if (all_cpus) { - kvm_make_all_cpus_request(kvm, KVM_REQ_TLB_FLUSH_GUEST); + kvm_make_all_cpus_request(kvm, KVM_REQ_HV_TLB_FLUSH); } else { sparse_set_to_vcpu_mask(kvm, sparse_banks, valid_bank_mask, vcpu_mask); =20 - kvm_make_vcpus_request_mask(kvm, KVM_REQ_TLB_FLUSH_GUEST, vcpu_mask); + kvm_make_vcpus_request_mask(kvm, KVM_REQ_HV_TLB_FLUSH, vcpu_mask); } =20 ret_success: diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index f79bf4552082..14f539fb9c02 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3339,7 +3339,12 @@ void kvm_service_local_tlb_flush_requests(struct kvm= _vcpu *vcpu) if (kvm_check_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu)) kvm_vcpu_flush_tlb_current(vcpu); =20 - if (kvm_check_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu)) + if (kvm_check_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu)) { + kvm_vcpu_flush_tlb_guest(vcpu); + kvm_clear_request(KVM_REQ_HV_TLB_FLUSH, vcpu); + } + + if (kvm_check_request(KVM_REQ_HV_TLB_FLUSH, vcpu)) kvm_vcpu_flush_tlb_guest(vcpu); } EXPORT_SYMBOL_GPL(kvm_service_local_tlb_flush_requests); --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 336ACC433F5 for ; Fri, 11 Mar 2022 15:50:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349733AbiCKPvR (ORCPT ); Fri, 11 Mar 2022 10:51:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349611AbiCKPvJ (ORCPT ); Fri, 11 Mar 2022 10:51:09 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 125321C9442 for ; Fri, 11 Mar 2022 07:49:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013798; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Wgg9qJl897kCOrcs6/XMfVs2Uts8Yt2p0e+XLxDlATU=; b=TO+/j0QIevZ6I2BQOVBF5bMl+SeTuS8Z9vIwjMXFUhE+9Am4DAZxIRV79XhZUa9Dh8jz38 Ze0oBLT0Cpv15q9gkjra49BILrjfCrk6lOgTPW5aFmjYpiSg3KE8cdauUMycrJ0mDSbHES NZcLB0KcTuW8J2hWfbY7d4fHk30kzlM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-663-1Tku3ur5PmCUGwxz73SwuA-1; Fri, 11 Mar 2022 10:49:55 -0500 X-MC-Unique: 1Tku3ur5PmCUGwxz73SwuA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 751BC824FA6; Fri, 11 Mar 2022 15:49:53 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id DAB6A785FD; Fri, 11 Mar 2022 15:49:50 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 02/31] KVM: x86: hyper-v: Introduce TLB flush ring Date: Fri, 11 Mar 2022 16:49:14 +0100 Message-Id: <20220311154943.2299191-3-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To allow flushing individual GVAs instead of always flushing the whole VPID a per-vCPU structure to pass the requests is needed. Introduce a simple ring write-locked structure to hold two types of entries: individual GVA (GFN + up to 4095 following GFNs in the lower 12 bits) and 'flush all'. The queuing rule is: if there's not enough space on the ring to put the request and leave at least 1 entry for 'flush all' - put 'flush all' entry. The size of the ring is arbitrary set to '16'. Note, kvm_hv_flush_tlb() only queues 'flush all' entries for now so there's very small functional change but the infrastructure is prepared to handle individual GVA flush requests. Signed-off-by: Vitaly Kuznetsov --- arch/x86/include/asm/kvm_host.h | 16 +++++++ arch/x86/kvm/hyperv.c | 74 +++++++++++++++++++++++++++++++++ arch/x86/kvm/hyperv.h | 13 ++++++ arch/x86/kvm/x86.c | 7 ++-- arch/x86/kvm/x86.h | 1 + 5 files changed, 108 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 11bcae77bba4..299f73d06680 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -579,6 +579,20 @@ struct kvm_vcpu_hv_synic { bool dont_zero_synic_pages; }; =20 +#define KVM_HV_TLB_FLUSH_RING_SIZE (16) + +struct kvm_vcpu_hv_tlbflush_entry { + u64 addr; + u64 flush_all:1; + u64 pad:63; +}; + +struct kvm_vcpu_hv_tlbflush_ring { + int read_idx, write_idx; + spinlock_t write_lock; + struct kvm_vcpu_hv_tlbflush_entry entries[KVM_HV_TLB_FLUSH_RING_SIZE]; +}; + /* Hyper-V per vcpu emulation context */ struct kvm_vcpu_hv { struct kvm_vcpu *vcpu; @@ -598,6 +612,8 @@ struct kvm_vcpu_hv { u32 enlightenments_ebx; /* HYPERV_CPUID_ENLIGHTMENT_INFO.EBX */ u32 syndbg_cap_eax; /* HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES.EAX */ } cpuid_cache; + + struct kvm_vcpu_hv_tlbflush_ring tlb_flush_ring; }; =20 /* Xen HVM per vcpu emulation context */ diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 0cad1fed6007..bc7e41fa5cd3 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -29,6 +29,7 @@ #include #include #include +#include #include =20 #include @@ -944,6 +945,8 @@ static int kvm_hv_vcpu_init(struct kvm_vcpu *vcpu) =20 hv_vcpu->vp_index =3D vcpu->vcpu_idx; =20 + spin_lock_init(&hv_vcpu->tlb_flush_ring.write_lock); + return 0; } =20 @@ -1803,6 +1806,65 @@ static u64 kvm_get_sparse_vp_set(struct kvm *kvm, st= ruct kvm_hv_hcall *hc, var_cnt * sizeof(*sparse_banks)); } =20 +static inline int hv_tlb_flush_ring_free(struct kvm_vcpu_hv *hv_vcpu, + int read_idx, int write_idx) +{ + if (write_idx >=3D read_idx) + return KVM_HV_TLB_FLUSH_RING_SIZE - (write_idx - read_idx) - 1; + + return read_idx - write_idx - 1; +} + +static void hv_tlb_flush_ring_enqueue(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_hv_tlbflush_ring *tlb_flush_ring; + struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); + int ring_free, write_idx, read_idx; + unsigned long flags; + + if (!hv_vcpu) + return; + + tlb_flush_ring =3D &hv_vcpu->tlb_flush_ring; + + spin_lock_irqsave(&tlb_flush_ring->write_lock, flags); + + read_idx =3D READ_ONCE(tlb_flush_ring->read_idx); + write_idx =3D READ_ONCE(tlb_flush_ring->write_idx); + + ring_free =3D hv_tlb_flush_ring_free(hv_vcpu, read_idx, write_idx); + /* Full ring always contains 'flush all' entry */ + if (!ring_free) + goto out_unlock; + + tlb_flush_ring->entries[write_idx].addr =3D 0; + tlb_flush_ring->entries[write_idx].flush_all =3D 1; + /* + * Advance write index only after filling in the entry to + * synchronize with lockless reader. + */ + smp_wmb(); + tlb_flush_ring->write_idx =3D (write_idx + 1) % KVM_HV_TLB_FLUSH_RING_SIZ= E; + +out_unlock: + spin_unlock_irqrestore(&tlb_flush_ring->write_lock, flags); +} + +void kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_hv_tlbflush_ring *tlb_flush_ring; + struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); + + kvm_vcpu_flush_tlb_guest(vcpu); + + if (!hv_vcpu) + return; + + tlb_flush_ring =3D &hv_vcpu->tlb_flush_ring; + + tlb_flush_ring->read_idx =3D tlb_flush_ring->write_idx; +} + static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) { struct kvm *kvm =3D vcpu->kvm; @@ -1811,6 +1873,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, st= ruct kvm_hv_hcall *hc) DECLARE_BITMAP(vcpu_mask, KVM_MAX_VCPUS); u64 valid_bank_mask; u64 sparse_banks[KVM_HV_MAX_SPARSE_VCPU_SET_BITS]; + struct kvm_vcpu *v; + unsigned long i; bool all_cpus; =20 /* @@ -1890,10 +1954,20 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, = struct kvm_hv_hcall *hc) * analyze it here, flush TLB regardless of the specified address space. */ if (all_cpus) { + kvm_for_each_vcpu(i, v, kvm) + hv_tlb_flush_ring_enqueue(v); + kvm_make_all_cpus_request(kvm, KVM_REQ_HV_TLB_FLUSH); } else { sparse_set_to_vcpu_mask(kvm, sparse_banks, valid_bank_mask, vcpu_mask); =20 + for_each_set_bit(i, vcpu_mask, KVM_MAX_VCPUS) { + v =3D kvm_get_vcpu(kvm, i); + if (!v) + continue; + hv_tlb_flush_ring_enqueue(v); + } + kvm_make_vcpus_request_mask(kvm, KVM_REQ_HV_TLB_FLUSH, vcpu_mask); } =20 diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h index e19c00ee9ab3..83960d1bdb1f 100644 --- a/arch/x86/kvm/hyperv.h +++ b/arch/x86/kvm/hyperv.h @@ -147,4 +147,17 @@ int kvm_vm_ioctl_hv_eventfd(struct kvm *kvm, struct kv= m_hyperv_eventfd *args); int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid, struct kvm_cpuid_entry2 __user *entries); =20 + +static inline void kvm_hv_vcpu_empty_flush_tlb(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); + + if (!hv_vcpu) + return; + + hv_vcpu->tlb_flush_ring.read_idx =3D hv_vcpu->tlb_flush_ring.write_idx; +} +void kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu); + + #endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 14f539fb9c02..0082c5691a05 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3303,7 +3303,7 @@ static void kvm_vcpu_flush_tlb_all(struct kvm_vcpu *v= cpu) static_call(kvm_x86_flush_tlb_all)(vcpu); } =20 -static void kvm_vcpu_flush_tlb_guest(struct kvm_vcpu *vcpu) +void kvm_vcpu_flush_tlb_guest(struct kvm_vcpu *vcpu) { ++vcpu->stat.tlb_flush; =20 @@ -3341,11 +3341,12 @@ void kvm_service_local_tlb_flush_requests(struct kv= m_vcpu *vcpu) =20 if (kvm_check_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu)) { kvm_vcpu_flush_tlb_guest(vcpu); - kvm_clear_request(KVM_REQ_HV_TLB_FLUSH, vcpu); + if (kvm_check_request(KVM_REQ_HV_TLB_FLUSH, vcpu)) + kvm_hv_vcpu_empty_flush_tlb(vcpu); } =20 if (kvm_check_request(KVM_REQ_HV_TLB_FLUSH, vcpu)) - kvm_vcpu_flush_tlb_guest(vcpu); + kvm_hv_vcpu_flush_tlb(vcpu); } EXPORT_SYMBOL_GPL(kvm_service_local_tlb_flush_requests); =20 diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index aa86abad914d..ed5c67b5d086 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -58,6 +58,7 @@ static inline unsigned int __shrink_ple_window(unsigned i= nt val, =20 #define MSR_IA32_CR_PAT_DEFAULT 0x0007040600070406ULL =20 +void kvm_vcpu_flush_tlb_guest(struct kvm_vcpu *vcpu); void kvm_service_local_tlb_flush_requests(struct kvm_vcpu *vcpu); int kvm_check_nested_events(struct kvm_vcpu *vcpu); =20 --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEBB6C433FE for ; Fri, 11 Mar 2022 15:50:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349636AbiCKPv2 (ORCPT ); Fri, 11 Mar 2022 10:51:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349675AbiCKPvP (ORCPT ); Fri, 11 Mar 2022 10:51:15 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0A9871CA5EA for ; Fri, 11 Mar 2022 07:50:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013803; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1jAroNsrhL3r9phAeQSUbmvXAtwcUuaQ+ZMwbX7MzuE=; b=JWD1r/NJsHLMAe7D4IYw7XZ9X1RqzSI0TY63ezjev3p2SOtewe3XHdtq6ID20Z0FCusRDn 5g5z0h2C07/ItbP/B9nEBaIGvwK0uHBqwdIZ9D8mGrVB0hYuTgEJpNNYcA237C3a5MSjSs d5f1IF1UJI/jtVxweWOrGnNFG5C0Plc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-168-8hqe017-Ok2_77qfV107kg-1; Fri, 11 Mar 2022 10:49:58 -0500 X-MC-Unique: 8hqe017-Ok2_77qfV107kg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6BC42801AFC; Fri, 11 Mar 2022 15:49:56 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id EAD61866D4; Fri, 11 Mar 2022 15:49:53 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 03/31] KVM: x86: hyper-v: Handle HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} calls gently Date: Fri, 11 Mar 2022 16:49:15 +0100 Message-Id: <20220311154943.2299191-4-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Currently, HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} calls are handled the exact same way as HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE{,EX}: by flushing the whole VPID and this is sub-optimal. Switch to handling these requests with 'flush_tlb_gva()' hooks instead. Use the newly introduced TLB flush ring to queue the requests. Signed-off-by: Vitaly Kuznetsov --- arch/x86/kvm/hyperv.c | 141 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 121 insertions(+), 20 deletions(-) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index bc7e41fa5cd3..4c8a37bce71e 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -1806,6 +1806,35 @@ static u64 kvm_get_sparse_vp_set(struct kvm *kvm, st= ruct kvm_hv_hcall *hc, var_cnt * sizeof(*sparse_banks)); } =20 +static int kvm_hv_get_tlbflush_entries(struct kvm *kvm, struct kvm_hv_hcal= l *hc, u64 entries[], + u32 data_offset, int consumed_xmm_halves) +{ + int i; + + if (hc->fast) { + /* + * Each XMM holds two entries, but do not count halves that + * have already been consumed. + */ + if (hc->rep_cnt > (2 * HV_HYPERCALL_MAX_XMM_REGISTERS - consumed_xmm_hal= ves)) + return -EINVAL; + + for (i =3D 0; i < hc->rep_cnt; i++) { + int j =3D i + consumed_xmm_halves; + + if (j % 2) + entries[i] =3D sse128_hi(hc->xmm[j / 2]); + else + entries[i] =3D sse128_lo(hc->xmm[j / 2]); + } + + return 0; + } + + return kvm_read_guest(kvm, hc->ingpa + data_offset, + entries, hc->rep_cnt * sizeof(entries[0])); +} + static inline int hv_tlb_flush_ring_free(struct kvm_vcpu_hv *hv_vcpu, int read_idx, int write_idx) { @@ -1815,12 +1844,14 @@ static inline int hv_tlb_flush_ring_free(struct kvm= _vcpu_hv *hv_vcpu, return read_idx - write_idx - 1; } =20 -static void hv_tlb_flush_ring_enqueue(struct kvm_vcpu *vcpu) +static void hv_tlb_flush_ring_enqueue(struct kvm_vcpu *vcpu, bool flush_al= l, + u64 *entries, int count) { struct kvm_vcpu_hv_tlbflush_ring *tlb_flush_ring; struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); int ring_free, write_idx, read_idx; unsigned long flags; + int i; =20 if (!hv_vcpu) return; @@ -1837,14 +1868,34 @@ static void hv_tlb_flush_ring_enqueue(struct kvm_vc= pu *vcpu) if (!ring_free) goto out_unlock; =20 - tlb_flush_ring->entries[write_idx].addr =3D 0; - tlb_flush_ring->entries[write_idx].flush_all =3D 1; /* - * Advance write index only after filling in the entry to - * synchronize with lockless reader. + * All entries should fit on the ring leaving one free for 'flush all' + * entry in case another request comes in. In case there's not enough + * space, just put 'flush all' entry there. + */ + if (!count || count >=3D ring_free - 1 || flush_all) { + tlb_flush_ring->entries[write_idx].addr =3D 0; + tlb_flush_ring->entries[write_idx].flush_all =3D 1; + /* + * Advance write index only after filling in the entry to + * synchronize with lockless reader. + */ + smp_wmb(); + tlb_flush_ring->write_idx =3D (write_idx + 1) % KVM_HV_TLB_FLUSH_RING_SI= ZE; + goto out_unlock; + } + + for (i =3D 0; i < count; i++) { + tlb_flush_ring->entries[write_idx].addr =3D entries[i]; + tlb_flush_ring->entries[write_idx].flush_all =3D 0; + write_idx =3D (write_idx + 1) % KVM_HV_TLB_FLUSH_RING_SIZE; + } + /* + * Advance write index only after filling in the entry to synchronize + * with lockless reader. */ smp_wmb(); - tlb_flush_ring->write_idx =3D (write_idx + 1) % KVM_HV_TLB_FLUSH_RING_SIZ= E; + tlb_flush_ring->write_idx =3D write_idx; =20 out_unlock: spin_unlock_irqrestore(&tlb_flush_ring->write_lock, flags); @@ -1854,15 +1905,47 @@ void kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) { struct kvm_vcpu_hv_tlbflush_ring *tlb_flush_ring; struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); - - kvm_vcpu_flush_tlb_guest(vcpu); - - if (!hv_vcpu) + struct kvm_vcpu_hv_tlbflush_entry *entry; + int read_idx, write_idx; + u64 address; + u32 count; + int i, j; + + if (!tdp_enabled || !hv_vcpu) { + kvm_vcpu_flush_tlb_guest(vcpu); return; + } =20 tlb_flush_ring =3D &hv_vcpu->tlb_flush_ring; + read_idx =3D READ_ONCE(tlb_flush_ring->read_idx); + write_idx =3D READ_ONCE(tlb_flush_ring->write_idx); + + /* Pairs with smp_wmb() in hv_tlb_flush_ring_enqueue() */ + smp_rmb(); =20 - tlb_flush_ring->read_idx =3D tlb_flush_ring->write_idx; + for (i =3D read_idx; i !=3D write_idx; i =3D (i + 1) % KVM_HV_TLB_FLUSH_R= ING_SIZE) { + entry =3D &tlb_flush_ring->entries[i]; + + if (entry->flush_all) + goto out_flush_all; + + /* + * Lower 12 bits of 'address' encode the number of additional + * pages to flush. + */ + address =3D entry->addr & PAGE_MASK; + count =3D (entry->addr & ~PAGE_MASK) + 1; + for (j =3D 0; j < count; j++) + static_call(kvm_x86_flush_tlb_gva)(vcpu, address + j * PAGE_SIZE); + } + ++vcpu->stat.tlb_flush; + goto out_empty_ring; + +out_flush_all: + kvm_vcpu_flush_tlb_guest(vcpu); + +out_empty_ring: + tlb_flush_ring->read_idx =3D write_idx; } =20 static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) @@ -1871,12 +1954,13 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, = struct kvm_hv_hcall *hc) struct hv_tlb_flush_ex flush_ex; struct hv_tlb_flush flush; DECLARE_BITMAP(vcpu_mask, KVM_MAX_VCPUS); + u64 entries[KVM_HV_TLB_FLUSH_RING_SIZE - 2]; u64 valid_bank_mask; u64 sparse_banks[KVM_HV_MAX_SPARSE_VCPU_SET_BITS]; struct kvm_vcpu *v; unsigned long i; - bool all_cpus; - + bool all_cpus, all_addr; + int data_offset =3D 0, consumed_xmm_halves =3D 0; /* * The Hyper-V TLFS doesn't allow more than 64 sparse banks, e.g. the * valid mask is a u64. Fail the build if KVM's max allowed number of @@ -1891,10 +1975,12 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, = struct kvm_hv_hcall *hc) flush.address_space =3D hc->ingpa; flush.flags =3D hc->outgpa; flush.processor_mask =3D sse128_lo(hc->xmm[0]); + consumed_xmm_halves =3D 1; } else { if (unlikely(kvm_read_guest(kvm, hc->ingpa, &flush, sizeof(flush)))) return HV_STATUS_INVALID_HYPERCALL_INPUT; + data_offset =3D sizeof(flush); } =20 trace_kvm_hv_flush_tlb(flush.processor_mask, @@ -1918,10 +2004,12 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, = struct kvm_hv_hcall *hc) flush_ex.flags =3D hc->outgpa; memcpy(&flush_ex.hv_vp_set, &hc->xmm[0], sizeof(hc->xmm[0])); + consumed_xmm_halves =3D 2; } else { if (unlikely(kvm_read_guest(kvm, hc->ingpa, &flush_ex, sizeof(flush_ex)))) return HV_STATUS_INVALID_HYPERCALL_INPUT; + data_offset =3D sizeof(flush_ex); } =20 trace_kvm_hv_flush_tlb_ex(flush_ex.hv_vp_set.valid_bank_mask, @@ -1937,25 +2025,38 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, = struct kvm_hv_hcall *hc) return HV_STATUS_INVALID_HYPERCALL_INPUT; =20 if (all_cpus) - goto do_flush; + goto read_flush_entries; =20 if (!hc->var_cnt) goto ret_success; =20 - if (kvm_get_sparse_vp_set(kvm, hc, 2, sparse_banks, - offsetof(struct hv_tlb_flush_ex, - hv_vp_set.bank_contents))) + if (kvm_get_sparse_vp_set(kvm, hc, consumed_xmm_halves, + sparse_banks, data_offset)) + return HV_STATUS_INVALID_HYPERCALL_INPUT; + data_offset +=3D hc->var_cnt * sizeof(sparse_banks[0]); + consumed_xmm_halves +=3D hc->var_cnt; + } + +read_flush_entries: + if (hc->code =3D=3D HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE || + hc->code =3D=3D HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX || + hc->rep_cnt > (KVM_HV_TLB_FLUSH_RING_SIZE - 2)) { + all_addr =3D true; + } else { + if (kvm_hv_get_tlbflush_entries(kvm, hc, entries, + data_offset, consumed_xmm_halves)) return HV_STATUS_INVALID_HYPERCALL_INPUT; + all_addr =3D false; } =20 -do_flush: + /* * vcpu->arch.cr3 may not be up-to-date for running vCPUs so we can't * analyze it here, flush TLB regardless of the specified address space. */ if (all_cpus) { kvm_for_each_vcpu(i, v, kvm) - hv_tlb_flush_ring_enqueue(v); + hv_tlb_flush_ring_enqueue(v, all_addr, entries, hc->rep_cnt); =20 kvm_make_all_cpus_request(kvm, KVM_REQ_HV_TLB_FLUSH); } else { @@ -1965,7 +2066,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, st= ruct kvm_hv_hcall *hc) v =3D kvm_get_vcpu(kvm, i); if (!v) continue; - hv_tlb_flush_ring_enqueue(v); + hv_tlb_flush_ring_enqueue(v, all_addr, entries, hc->rep_cnt); } =20 kvm_make_vcpus_request_mask(kvm, KVM_REQ_HV_TLB_FLUSH, vcpu_mask); --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 647F8C433FE for ; Fri, 11 Mar 2022 15:50:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245714AbiCKPvV (ORCPT ); Fri, 11 Mar 2022 10:51:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349531AbiCKPvN (ORCPT ); Fri, 11 Mar 2022 10:51:13 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id BDD391CA5C8 for ; Fri, 11 Mar 2022 07:50:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ekuNav+v5tacsqN4IGNEWWPQSDuBYlRTNy3FEXfhMiI=; b=bQm8FQuBZ+p6pcjhcMJdq7i3AMyVNzkVANJMPWO6vUtxA6wm0V3lBWBen1FWDcvJAr0VCM DvEJ6lTHcCZZLLoXMTe5VD3TXuCgZVvOOuA4HUn8IPDs5pDIkWdAAaatq0dMJYwGc9pqpq sA7e8jRbJHV84Iw7VTHh/Bkrons4YO0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-394-GbGzNc7_N-us7SW-j6AuTA-1; Fri, 11 Mar 2022 10:50:00 -0500 X-MC-Unique: GbGzNc7_N-us7SW-j6AuTA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 70A9C1006AA6; Fri, 11 Mar 2022 15:49:59 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id CB677785FD; Fri, 11 Mar 2022 15:49:56 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 04/31] KVM: x86: hyper-v: Expose support for extended gva ranges for flush hypercalls Date: Fri, 11 Mar 2022 16:49:16 +0100 Message-Id: <20220311154943.2299191-5-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Extended GVA ranges support bit seems to indicate whether lower 12 bits of GVA can be used to specify up to 4095 additional consequent GVAs to flush. This is somewhat described in TLFS. Previously, KVM was handling HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} requests by flushing the whole VPID so technically, extended GVA ranges were already supported. As such requests are handled more gently now, advertizing support for extended ranges starts making sense to reduce the size of TLB flush requests. Signed-off-by: Vitaly Kuznetsov --- arch/x86/include/asm/hyperv-tlfs.h | 2 ++ arch/x86/kvm/hyperv.c | 1 + 2 files changed, 3 insertions(+) diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hype= rv-tlfs.h index 0a9407dc0859..5225a85c08c3 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -61,6 +61,8 @@ #define HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE BIT(10) /* Support for debug MSRs available */ #define HV_FEATURE_DEBUG_MSRS_AVAILABLE BIT(11) +/* Support for extended gva ranges for flush hypercalls available */ +#define HV_FEATURE_EXT_GVA_RANGES_FLUSH BIT(14) /* * Support for returning hypercall output block via XMM * registers is available diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 4c8a37bce71e..f619acccbb6d 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2697,6 +2697,7 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kv= m_cpuid2 *cpuid, ent->ebx |=3D HV_DEBUGGING; ent->edx |=3D HV_X64_GUEST_DEBUGGING_AVAILABLE; ent->edx |=3D HV_FEATURE_DEBUG_MSRS_AVAILABLE; + ent->edx |=3D HV_FEATURE_EXT_GVA_RANGES_FLUSH; =20 /* * Direct Synthetic timers only make sense with in-kernel --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D7BCC433EF for ; Fri, 11 Mar 2022 15:50:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349800AbiCKPvb (ORCPT ); Fri, 11 Mar 2022 10:51:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349697AbiCKPvQ (ORCPT ); Fri, 11 Mar 2022 10:51:16 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 25B4E1CABD1 for ; Fri, 11 Mar 2022 07:50:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013804; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8gDwUidIwqdJttGlcdrWpuSeGlEE0r2bY8JHq9AKfxA=; b=GZAru+dyWHDavnOTtwhIpzGyfIQzgl19m/i5P9GVNgm0kC31FeBuARd40bOOTjKsan4bS8 OgVtAGhJnvyhzCmvIPGiO93wp/ZSqQ25zKh6kdsdGL92OFRuyDsIoYrRfHX7WXa/igL7eQ AVMEZg2RhGgjghFPypnJ3znySl6sg+M= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-497-Jc-1ZeiYOpilPD9DiF4GeQ-1; Fri, 11 Mar 2022 10:50:03 -0500 X-MC-Unique: Jc-1ZeiYOpilPD9DiF4GeQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2DE791091DA1; Fri, 11 Mar 2022 15:50:02 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id A9FD7785FD; Fri, 11 Mar 2022 15:49:59 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 05/31] KVM: x86: Prepare kvm_hv_flush_tlb() to handle L2's GPAs Date: Fri, 11 Mar 2022 16:49:17 +0100 Message-Id: <20220311154943.2299191-6-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To handle Direct TLB flush requests from L2, KVM needs to translate the specified L2 GPA to L1 GPA to read hypercall arguments from there. No fucntional change as KVM doesn't handle VMCALL/VMMCALL from L2 yet. Signed-off-by: Vitaly Kuznetsov --- arch/x86/kvm/hyperv.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index f619acccbb6d..53eb9540494b 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -23,6 +23,7 @@ #include "ioapic.h" #include "cpuid.h" #include "hyperv.h" +#include "mmu.h" #include "xen.h" =20 #include @@ -1969,6 +1970,12 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, s= truct kvm_hv_hcall *hc) */ BUILD_BUG_ON(KVM_HV_MAX_SPARSE_VCPU_SET_BITS > 64); =20 + if (!hc->fast && is_guest_mode(vcpu)) { + hc->ingpa =3D translate_nested_gpa(vcpu, hc->ingpa, 0, NULL); + if (unlikely(hc->ingpa =3D=3D UNMAPPED_GVA)) + return HV_STATUS_INVALID_HYPERCALL_INPUT; + } + if (hc->code =3D=3D HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST || hc->code =3D=3D HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE) { if (hc->fast) { --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 247ABC43217 for ; Fri, 11 Mar 2022 15:50:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238924AbiCKPvh (ORCPT ); Fri, 11 Mar 2022 10:51:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349611AbiCKPvS (ORCPT ); Fri, 11 Mar 2022 10:51:18 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 2F7141CBAB7 for ; Fri, 11 Mar 2022 07:50:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013811; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xc/ODvBs4ahi/YUBp9hveSuWTL5SxZ2u6EliH0Mv57o=; b=Tfjl+WaYqzAOXiYPAT8pmJ9Q/Wn0x+wr6E4H6dqQzGysyskPJ7RIt/3OtQ1K/P/HCmBZne ElMsUBYf4zxieYK/RsL0DDfqaLKGRn0izQgi9dDasMRzhArn9i5+GrlSAosEd+zeeMFJ8K ZNCSibBsjFRAJi+Dej9wKrgaS68byLY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-116-g3FEoLYsNqmT_L-58hCU1A-1; Fri, 11 Mar 2022 10:50:06 -0500 X-MC-Unique: g3FEoLYsNqmT_L-58hCU1A-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E5DF751DF; Fri, 11 Mar 2022 15:50:04 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 87BB4785FD; Fri, 11 Mar 2022 15:50:02 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 06/31] KVM: x86: hyper-v: Don't use sparse_set_to_vcpu_mask() in kvm_hv_send_ipi() Date: Fri, 11 Mar 2022 16:49:18 +0100 Message-Id: <20220311154943.2299191-7-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Get rid of on-stack allocation of vcpu_mask and optimize kvm_hv_send_ipi() for a smaller number of vCPUs in the request. When Hyper-V TLB flush is in use, HvSendSyntheticClusterIpi{,Ex} calls are not commonly used to send IPIs to a large number of vCPUs (and are rarely used in general). Introduce hv_is_vp_in_sparse_set() to directly check if the specified VP_ID is present in sparse vCPU set. Signed-off-by: Vitaly Kuznetsov --- arch/x86/kvm/hyperv.c | 35 ++++++++++++++++++++++++----------- 1 file changed, 24 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 53eb9540494b..7efa34fb15ef 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -1760,6 +1760,23 @@ static void sparse_set_to_vcpu_mask(struct kvm *kvm,= u64 *sparse_banks, } } =20 +static bool hv_is_vp_in_sparse_set(u32 vp_id, u64 valid_bank_mask, u64 spa= rse_banks[]) +{ + int bank, sbank =3D 0; + + if (!test_bit(vp_id / 64, (unsigned long *)&valid_bank_mask)) + return false; + + for_each_set_bit(bank, (unsigned long *)&valid_bank_mask, + KVM_HV_MAX_SPARSE_VCPU_SET_BITS) { + if (bank =3D=3D vp_id / 64) + break; + sbank++; + } + + return test_bit(vp_id % 64, (unsigned long *)&sparse_banks[sbank]); +} + struct kvm_hv_hcall { u64 param; u64 ingpa; @@ -2085,8 +2102,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, st= ruct kvm_hv_hcall *hc) ((u64)hc->rep_cnt << HV_HYPERCALL_REP_COMP_OFFSET); } =20 -static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector, - unsigned long *vcpu_bitmap) +static void kvm_hv_send_ipi_to_many(struct kvm *kvm, u32 vector, + u64 *sparse_banks, u64 valid_bank_mask) { struct kvm_lapic_irq irq =3D { .delivery_mode =3D APIC_DM_FIXED, @@ -2096,7 +2113,10 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u3= 2 vector, unsigned long i; =20 kvm_for_each_vcpu(i, vcpu, kvm) { - if (vcpu_bitmap && !test_bit(i, vcpu_bitmap)) + if (sparse_banks && + !hv_is_vp_in_sparse_set(kvm_hv_get_vpindex(vcpu), + valid_bank_mask, + sparse_banks)) continue; =20 /* We fail only when APIC is disabled */ @@ -2109,7 +2129,6 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, str= uct kvm_hv_hcall *hc) struct kvm *kvm =3D vcpu->kvm; struct hv_send_ipi_ex send_ipi_ex; struct hv_send_ipi send_ipi; - DECLARE_BITMAP(vcpu_mask, KVM_MAX_VCPUS); unsigned long valid_bank_mask; u64 sparse_banks[KVM_HV_MAX_SPARSE_VCPU_SET_BITS]; u32 vector; @@ -2171,13 +2190,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, st= ruct kvm_hv_hcall *hc) if ((vector < HV_IPI_LOW_VECTOR) || (vector > HV_IPI_HIGH_VECTOR)) return HV_STATUS_INVALID_HYPERCALL_INPUT; =20 - if (all_cpus) { - kvm_send_ipi_to_many(kvm, vector, NULL); - } else { - sparse_set_to_vcpu_mask(kvm, sparse_banks, valid_bank_mask, vcpu_mask); - - kvm_send_ipi_to_many(kvm, vector, vcpu_mask); - } + kvm_hv_send_ipi_to_many(kvm, vector, all_cpus ? NULL : sparse_banks, vali= d_bank_mask); =20 ret_success: return HV_STATUS_SUCCESS; --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD6ABC433F5 for ; Fri, 11 Mar 2022 15:50:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349893AbiCKPvw (ORCPT ); Fri, 11 Mar 2022 10:51:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349745AbiCKPvS (ORCPT ); Fri, 11 Mar 2022 10:51:18 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 9679E1CBA9C for ; Fri, 11 Mar 2022 07:50:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013810; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rlVYgUkdWRyYsMOIFcOnRDf5aAd17ft9dtYdjY2xq6w=; b=VjKH0w9KzLl/1KNVGkCmpX6rw6yTgSx/q4JPRmbDSlXU4V6J7ycM7NBhntOO9iIUfhqOoG Ne+sGOy01OAoLKI/7gFWUUzHzQqaw1UqR6BDopYvvhdhuGYThEVwdM2lNrU9ftI8MIjNFt QtqYpzImzi+qP8AOOA6IzGNcCL246mw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-261-kIFTeHXrNACzbZIpYcbKiQ-1; Fri, 11 Mar 2022 10:50:09 -0500 X-MC-Unique: kIFTeHXrNACzbZIpYcbKiQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C5EA51091DA0; Fri, 11 Mar 2022 15:50:07 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 51745785FD; Fri, 11 Mar 2022 15:50:05 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 07/31] KVM: x86: hyper-v: Create a separate ring for Direct TLB flush Date: Fri, 11 Mar 2022 16:49:19 +0100 Message-Id: <20220311154943.2299191-8-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To handle Direct TLB flush requests from L2 KVM needs to use a separate ring from regular Hyper-V TLB flush requests: e.g. when a request to flush something in L2 is made, the target vCPU can transition from L2 to L1, receive a request to flush a GVA for L1 and then try to enter L2 back. The first request needs to be processed then. Similarly, requests to flush GVAs in L1 must wait until L2 exits to L1. No functional change yet as KVM doesn't handle Direct TLB flush requests from L2 yet. Signed-off-by: Vitaly Kuznetsov --- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/hyperv.c | 7 ++++--- arch/x86/kvm/hyperv.h | 17 ++++++++++++++--- 3 files changed, 20 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 299f73d06680..750ac4055d0c 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -613,7 +613,8 @@ struct kvm_vcpu_hv { u32 syndbg_cap_eax; /* HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES.EAX */ } cpuid_cache; =20 - struct kvm_vcpu_hv_tlbflush_ring tlb_flush_ring; + /* Two rings for regular Hyper-V TLB flush and Direct TLB flush */ + struct kvm_vcpu_hv_tlbflush_ring tlb_flush_ring[2]; }; =20 /* Xen HVM per vcpu emulation context */ diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 7efa34fb15ef..9dfc122d5eca 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -946,7 +946,8 @@ static int kvm_hv_vcpu_init(struct kvm_vcpu *vcpu) =20 hv_vcpu->vp_index =3D vcpu->vcpu_idx; =20 - spin_lock_init(&hv_vcpu->tlb_flush_ring.write_lock); + spin_lock_init(&hv_vcpu->tlb_flush_ring[0].write_lock); + spin_lock_init(&hv_vcpu->tlb_flush_ring[1].write_lock); =20 return 0; } @@ -1874,7 +1875,7 @@ static void hv_tlb_flush_ring_enqueue(struct kvm_vcpu= *vcpu, bool flush_all, if (!hv_vcpu) return; =20 - tlb_flush_ring =3D &hv_vcpu->tlb_flush_ring; + tlb_flush_ring =3D &hv_vcpu->tlb_flush_ring[0]; =20 spin_lock_irqsave(&tlb_flush_ring->write_lock, flags); =20 @@ -1934,7 +1935,7 @@ void kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) return; } =20 - tlb_flush_ring =3D &hv_vcpu->tlb_flush_ring; + tlb_flush_ring =3D kvm_hv_get_tlb_flush_ring(vcpu); read_idx =3D READ_ONCE(tlb_flush_ring->read_idx); write_idx =3D READ_ONCE(tlb_flush_ring->write_idx); =20 diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h index 83960d1bdb1f..137f906eb8c3 100644 --- a/arch/x86/kvm/hyperv.h +++ b/arch/x86/kvm/hyperv.h @@ -22,6 +22,7 @@ #define __ARCH_X86_KVM_HYPERV_H__ =20 #include +#include "x86.h" =20 /* * The #defines related to the synthetic debugger are required by KDNet, b= ut @@ -147,15 +148,25 @@ int kvm_vm_ioctl_hv_eventfd(struct kvm *kvm, struct k= vm_hyperv_eventfd *args); int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid, struct kvm_cpuid_entry2 __user *entries); =20 +static inline struct kvm_vcpu_hv_tlbflush_ring *kvm_hv_get_tlb_flush_ring(= struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); + + if (!is_guest_mode(vcpu)) + return &hv_vcpu->tlb_flush_ring[0]; + + return &hv_vcpu->tlb_flush_ring[1]; +} =20 static inline void kvm_hv_vcpu_empty_flush_tlb(struct kvm_vcpu *vcpu) { - struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); + struct kvm_vcpu_hv_tlbflush_ring *tlb_flush_ring; =20 - if (!hv_vcpu) + if (!to_hv_vcpu(vcpu)) return; =20 - hv_vcpu->tlb_flush_ring.read_idx =3D hv_vcpu->tlb_flush_ring.write_idx; + tlb_flush_ring =3D kvm_hv_get_tlb_flush_ring(vcpu); + tlb_flush_ring->read_idx =3D tlb_flush_ring->write_idx; } void kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu); =20 --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88F3EC433EF for ; Fri, 11 Mar 2022 15:50:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349817AbiCKPvk (ORCPT ); Fri, 11 Mar 2022 10:51:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349705AbiCKPvV (ORCPT ); Fri, 11 Mar 2022 10:51:21 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 1E11A1C9B67 for ; Fri, 11 Mar 2022 07:50:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013815; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0tDVPG+cSC0wZSZXXfWO0Gv51YvV+L3gtZb8BmeIrIg=; b=UogXYkHF+xx9WOEmCuPirYtd9ua1BqCA4ieerjywhqYEg5+HTtaO8vM8WYHsSlqytl5UFh NAty+HTIxKXTYKMryEqJW9dm9oHHHpPYaLh/MTR5Kw0w91/GUzYN0WB58BU7WXEC57nzLu CRGeCCnLusPCjgkCksXIA/GlgbQBy4w= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-587-Ul6qfZXpOyKcmD6QQluqHw-1; Fri, 11 Mar 2022 10:50:12 -0500 X-MC-Unique: Ul6qfZXpOyKcmD6QQluqHw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A5733824FAB; Fri, 11 Mar 2022 15:50:10 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 27F9D785FD; Fri, 11 Mar 2022 15:50:07 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 08/31] KVM: x86: hyper-v: Use preallocated buffer in 'struct kvm_vcpu_hv' instead of on-stack 'sparse_banks' Date: Fri, 11 Mar 2022 16:49:20 +0100 Message-Id: <20220311154943.2299191-9-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To make kvm_hv_flush_tlb() ready to handle Direct TLB flush request KVM needs to allow for all 64 sparse vCPU banks regardless of KVM_MAX_VCPUs as L1 may use vCPU overcommit for L2. To avoid growing on-stack allocation, make 'sparse_banks' part of per-vCPU 'struct kvm_vcpu_hv' which is allocated dynamically. Note: sparse_set_to_vcpu_mask() keeps using on-stack allocation as it won't be used to handle Direct TLB flush requests. Signed-off-by: Vitaly Kuznetsov --- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/hyperv.c | 6 ++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 750ac4055d0c..1fce8232a2e9 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -615,6 +615,9 @@ struct kvm_vcpu_hv { =20 /* Two rings for regular Hyper-V TLB flush and Direct TLB flush */ struct kvm_vcpu_hv_tlbflush_ring tlb_flush_ring[2]; + + /* Preallocated buffer for handling hypercalls passing sparse vCPU set */ + u64 sparse_banks[64]; }; =20 /* Xen HVM per vcpu emulation context */ diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 9dfc122d5eca..b88e44a126b8 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -1969,13 +1969,14 @@ void kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) =20 static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) { + struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); + u64 *sparse_banks =3D hv_vcpu->sparse_banks; struct kvm *kvm =3D vcpu->kvm; struct hv_tlb_flush_ex flush_ex; struct hv_tlb_flush flush; DECLARE_BITMAP(vcpu_mask, KVM_MAX_VCPUS); u64 entries[KVM_HV_TLB_FLUSH_RING_SIZE - 2]; u64 valid_bank_mask; - u64 sparse_banks[KVM_HV_MAX_SPARSE_VCPU_SET_BITS]; struct kvm_vcpu *v; unsigned long i; bool all_cpus, all_addr; @@ -2127,11 +2128,12 @@ static void kvm_hv_send_ipi_to_many(struct kvm *kvm= , u32 vector, =20 static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) { + struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); + u64 *sparse_banks =3D hv_vcpu->sparse_banks; struct kvm *kvm =3D vcpu->kvm; struct hv_send_ipi_ex send_ipi_ex; struct hv_send_ipi send_ipi; unsigned long valid_bank_mask; - u64 sparse_banks[KVM_HV_MAX_SPARSE_VCPU_SET_BITS]; u32 vector; bool all_cpus; =20 --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19225C433FE for ; Fri, 11 Mar 2022 15:50:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349823AbiCKPvp (ORCPT ); Fri, 11 Mar 2022 10:51:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58344 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349787AbiCKPvW (ORCPT ); Fri, 11 Mar 2022 10:51:22 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 39B0A1CA5DD for ; Fri, 11 Mar 2022 07:50:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r0U93TwJKtCWSRp8OCIeSktwKEFUnEWgPi81ndGHy4o=; b=d9PByf+DC8bdWZX5OFSKaLR/W+lq/PYHePYgWJBQSS/ApWfCFiiS/QSdbldFM33N5QqRsx V5Y4F5p5iFW8vIJjKtjqnnV6GInjXmErZyisUYfoLxmiF9tE5CLOjpJTBrWTT6GEqXroEA NonODvKdKbc4Jzbluoh9anZdhA2lqpg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-641-YK2MpHZ8OsKdOJHHUKqrGw-1; Fri, 11 Mar 2022 10:50:15 -0500 X-MC-Unique: YK2MpHZ8OsKdOJHHUKqrGw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 84BA7824FA6; Fri, 11 Mar 2022 15:50:13 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 10AC9866C1; Fri, 11 Mar 2022 15:50:10 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 09/31] KVM: nVMX: Keep track of hv_vm_id/hv_vp_id when eVMCS is in use Date: Fri, 11 Mar 2022 16:49:21 +0100 Message-Id: <20220311154943.2299191-10-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To handle Direct TLB flush requests from L2, KVM needs to keep track of L2's VM_ID/VP_IDs which are set by L1 hypervisor. 'Partition assist page' address is also needed to handle post-flush exit to L1 upon request. Signed-off-by: Vitaly Kuznetsov --- arch/x86/include/asm/kvm_host.h | 6 ++++++ arch/x86/kvm/vmx/nested.c | 12 ++++++++++++ 2 files changed, 18 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 1fce8232a2e9..21b502aa9655 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -618,6 +618,12 @@ struct kvm_vcpu_hv { =20 /* Preallocated buffer for handling hypercalls passing sparse vCPU set */ u64 sparse_banks[64]; + + struct { + u64 pa_page_gpa; + u64 vm_id; + u32 vp_id; + } nested; }; =20 /* Xen HVM per vcpu emulation context */ diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index f18744f7ff82..0cd6d5475578 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -225,6 +225,7 @@ static void vmx_disable_shadow_vmcs(struct vcpu_vmx *vm= x) =20 static inline void nested_release_evmcs(struct kvm_vcpu *vcpu) { + struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); struct vcpu_vmx *vmx =3D to_vmx(vcpu); =20 if (evmptr_is_valid(vmx->nested.hv_evmcs_vmptr)) { @@ -233,6 +234,9 @@ static inline void nested_release_evmcs(struct kvm_vcpu= *vcpu) } =20 vmx->nested.hv_evmcs_vmptr =3D EVMPTR_INVALID; + hv_vcpu->nested.pa_page_gpa =3D INVALID_GPA; + hv_vcpu->nested.vm_id =3D 0; + hv_vcpu->nested.vp_id =3D 0; } =20 static void vmx_sync_vmcs_host_state(struct vcpu_vmx *vmx, @@ -1592,11 +1596,19 @@ static void copy_enlightened_to_vmcs12(struct vcpu_= vmx *vmx, u32 hv_clean_fields { struct vmcs12 *vmcs12 =3D vmx->nested.cached_vmcs12; struct hv_enlightened_vmcs *evmcs =3D vmx->nested.hv_evmcs; + struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(&vmx->vcpu); =20 /* HV_VMX_ENLIGHTENED_CLEAN_FIELD_NONE */ vmcs12->tpr_threshold =3D evmcs->tpr_threshold; vmcs12->guest_rip =3D evmcs->guest_rip; =20 + if (unlikely(!(hv_clean_fields & + HV_VMX_ENLIGHTENED_CLEAN_FIELD_ENLIGHTENMENTSCONTROL))) { + hv_vcpu->nested.pa_page_gpa =3D evmcs->partition_assist_page; + hv_vcpu->nested.vm_id =3D evmcs->hv_vm_id; + hv_vcpu->nested.vp_id =3D evmcs->hv_vp_id; + } + if (unlikely(!(hv_clean_fields & HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_BASIC))) { vmcs12->guest_rsp =3D evmcs->guest_rsp; --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96522C43217 for ; Fri, 11 Mar 2022 15:50:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349881AbiCKPvs (ORCPT ); Fri, 11 Mar 2022 10:51:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349801AbiCKPvb (ORCPT ); Fri, 11 Mar 2022 10:51:31 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 115F91CABF8 for ; Fri, 11 Mar 2022 07:50:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013819; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hURzgvRKcr/batMnM0awJ6HNtSiG3Y83/511GoAYwDE=; b=ErqhjCc58JRfjKkVNveGPWOSCw4xBAUvmr8bt3cLgMH0TviMxlQXb3cA1McS453XhKDDJB v5LhDHRMc07vWtigHZW9UpIR2D8PMniYGAjJTydhs4HtjiMkm9ydwxjX+F9NrFfSd2szWx a1W6fJFwmTQACIZjMSY/+1muMU4Pd8A= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-9-WIooUC8jOg66e011EQN3zQ-1; Fri, 11 Mar 2022 10:50:18 -0500 X-MC-Unique: WIooUC8jOg66e011EQN3zQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 649731006AA6; Fri, 11 Mar 2022 15:50:16 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id E4432866C1; Fri, 11 Mar 2022 15:50:13 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 10/31] KVM: nSVM: Keep track of Hyper-V hv_vm_id/hv_vp_id Date: Fri, 11 Mar 2022 16:49:22 +0100 Message-Id: <20220311154943.2299191-11-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Similar to nSVM, KVM needs to know L2's VM_ID/VP_ID and Partition assist page address to handle Direct TLB flush requests. Signed-off-by: Vitaly Kuznetsov --- arch/x86/kvm/svm/hyperv.h | 15 +++++++++++++++ arch/x86/kvm/svm/nested.c | 2 ++ 2 files changed, 17 insertions(+) diff --git a/arch/x86/kvm/svm/hyperv.h b/arch/x86/kvm/svm/hyperv.h index 7d6d97968fb9..27479469f672 100644 --- a/arch/x86/kvm/svm/hyperv.h +++ b/arch/x86/kvm/svm/hyperv.h @@ -32,4 +32,19 @@ struct hv_enlightenments { */ #define VMCB_HV_NESTED_ENLIGHTENMENTS VMCB_SW =20 +static inline void nested_svm_hv_update_vm_vp_ids(struct kvm_vcpu *vcpu) +{ + struct vcpu_svm *svm =3D to_svm(vcpu); + struct hv_enlightenments *hve =3D + (struct hv_enlightenments *)svm->nested.ctl.reserved_sw; + struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); + + if (!hv_vcpu) + return; + + hv_vcpu->nested.pa_page_gpa =3D hve->partition_assist_page; + hv_vcpu->nested.vm_id =3D hve->hv_vm_id; + hv_vcpu->nested.vp_id =3D hve->hv_vp_id; +} + #endif /* __ARCH_X86_KVM_SVM_HYPERV_H__ */ diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index 96bab464967f..f47570fa503e 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -752,6 +752,8 @@ int nested_svm_vmrun(struct kvm_vcpu *vcpu) =20 svm->nested.nested_run_pending =3D 1; =20 + nested_svm_hv_update_vm_vp_ids(vcpu); + if (enter_svm_guest_mode(vcpu, vmcb12_gpa, vmcb12, true)) goto out_exit_err; =20 --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BCE0C433F5 for ; Fri, 11 Mar 2022 15:50:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345132AbiCKPvz (ORCPT ); Fri, 11 Mar 2022 10:51:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58770 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349825AbiCKPvc (ORCPT ); Fri, 11 Mar 2022 10:51:32 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 331F01CC7E5 for ; Fri, 11 Mar 2022 07:50:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013824; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WUvZ8KBW+bseRhhnYRqi1Eg4UpUjAh2pCht8EKuMpbI=; b=Q3iOWWYHOB+ByzUtmxuOW97JwiOodGNB7aIcvRBZ1XGVJvxPlE0wK+CT/UjFrTp1nMtj7A kZg3hmKlu90wuqnTZqEdD/o3+DXVF8KII82eHHrWHpMpSrjRLCuvwC8XPf0y0E70OXLpp3 mOxpIwvkYbnLJTV7tCycI3BfxSW0x+s= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-175-OZnWHRliPk-r6TWcYQ58Fg-1; Fri, 11 Mar 2022 10:50:21 -0500 X-MC-Unique: OZnWHRliPk-r6TWcYQ58Fg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 75B1F1006AA5; Fri, 11 Mar 2022 15:50:19 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id E7ADA785FD; Fri, 11 Mar 2022 15:50:16 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 11/31] KVM: x86: Introduce .post_hv_direct_flush() nested hook Date: Fri, 11 Mar 2022 16:49:23 +0100 Message-Id: <20220311154943.2299191-12-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Hyper-V supports injecting synthetic L2->L1 exit after performing Direct TLB flush operation but the procedure is vendor specific. Introduce .post_hv_direct_flush() nested hook for it. Signed-off-by: Vitaly Kuznetsov --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/Makefile | 3 ++- arch/x86/kvm/svm/hyperv.c | 11 +++++++++++ arch/x86/kvm/svm/hyperv.h | 2 ++ arch/x86/kvm/svm/nested.c | 1 + arch/x86/kvm/vmx/evmcs.c | 4 ++++ arch/x86/kvm/vmx/evmcs.h | 1 + arch/x86/kvm/vmx/nested.c | 1 + 8 files changed, 23 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/svm/hyperv.c diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 21b502aa9655..45ee05f4f1a0 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1538,6 +1538,7 @@ struct kvm_x86_nested_ops { int (*enable_evmcs)(struct kvm_vcpu *vcpu, uint16_t *vmcs_version); uint16_t (*get_evmcs_version)(struct kvm_vcpu *vcpu); + void (*post_hv_direct_flush)(struct kvm_vcpu *vcpu); }; =20 struct kvm_x86_init_ops { diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index 30f244b64523..b6d53b045692 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -25,7 +25,8 @@ kvm-intel-y +=3D vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o= vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o kvm-intel-$(CONFIG_X86_SGX_KVM) +=3D vmx/sgx.o =20 -kvm-amd-y +=3D svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o = svm/sev.o +kvm-amd-y +=3D svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \ + svm/sev.o svm/hyperv.o =20 ifdef CONFIG_HYPERV kvm-amd-y +=3D svm/svm_onhyperv.o diff --git a/arch/x86/kvm/svm/hyperv.c b/arch/x86/kvm/svm/hyperv.c new file mode 100644 index 000000000000..0142fde34738 --- /dev/null +++ b/arch/x86/kvm/svm/hyperv.c @@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * AMD SVM specific code for Hyper-V on KVM. + * + * Copyright 2022 Red Hat, Inc. and/or its affiliates. + */ +#include "hyperv.h" + +void svm_post_hv_direct_flush(struct kvm_vcpu *vcpu) +{ +} diff --git a/arch/x86/kvm/svm/hyperv.h b/arch/x86/kvm/svm/hyperv.h index 27479469f672..478db6f61d69 100644 --- a/arch/x86/kvm/svm/hyperv.h +++ b/arch/x86/kvm/svm/hyperv.h @@ -47,4 +47,6 @@ static inline void nested_svm_hv_update_vm_vp_ids(struct = kvm_vcpu *vcpu) hv_vcpu->nested.vp_id =3D hve->hv_vp_id; } =20 +void svm_post_hv_direct_flush(struct kvm_vcpu *vcpu); + #endif /* __ARCH_X86_KVM_SVM_HYPERV_H__ */ diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index f47570fa503e..46ea234363cf 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -1573,4 +1573,5 @@ struct kvm_x86_nested_ops svm_nested_ops =3D { .get_nested_state_pages =3D svm_get_nested_state_pages, .get_state =3D svm_get_nested_state, .set_state =3D svm_set_nested_state, + .post_hv_direct_flush =3D svm_post_hv_direct_flush, }; diff --git a/arch/x86/kvm/vmx/evmcs.c b/arch/x86/kvm/vmx/evmcs.c index 87e3dc10edf4..1705c4973636 100644 --- a/arch/x86/kvm/vmx/evmcs.c +++ b/arch/x86/kvm/vmx/evmcs.c @@ -437,3 +437,7 @@ int nested_enable_evmcs(struct kvm_vcpu *vcpu, =20 return 0; } + +void vmx_post_hv_direct_flush(struct kvm_vcpu *vcpu) +{ +} diff --git a/arch/x86/kvm/vmx/evmcs.h b/arch/x86/kvm/vmx/evmcs.h index 8d70f9aea94b..8862692a4c5d 100644 --- a/arch/x86/kvm/vmx/evmcs.h +++ b/arch/x86/kvm/vmx/evmcs.h @@ -244,5 +244,6 @@ int nested_enable_evmcs(struct kvm_vcpu *vcpu, uint16_t *vmcs_version); void nested_evmcs_filter_control_msr(u32 msr_index, u64 *pdata); int nested_evmcs_check_controls(struct vmcs12 *vmcs12); +void vmx_post_hv_direct_flush(struct kvm_vcpu *vcpu); =20 #endif /* __KVM_X86_VMX_EVMCS_H */ diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 0cd6d5475578..3121b761b1ee 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -6824,4 +6824,5 @@ struct kvm_x86_nested_ops vmx_nested_ops =3D { .write_log_dirty =3D nested_vmx_write_pml_buffer, .enable_evmcs =3D nested_enable_evmcs, .get_evmcs_version =3D nested_get_evmcs_version, + .post_hv_direct_flush =3D vmx_post_hv_direct_flush, }; --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C4BAC4332F for ; Fri, 11 Mar 2022 15:50:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349744AbiCKPv7 (ORCPT ); Fri, 11 Mar 2022 10:51:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349845AbiCKPvd (ORCPT ); Fri, 11 Mar 2022 10:51:33 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 00BF71CCB01 for ; Fri, 11 Mar 2022 07:50:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013827; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SBWkFgpgeF9UVavT67BPAf0/9XPzgwWECjGG8ZN+XCY=; b=eBwl1XKCHXOfyKLkbwU9yubuHLAg0NILKM5rq8khZINW5yDlYX/ej9KVoru39ZWSWLQgCJ zQjYd/448iFxqmUQFASUciJlYXptAPgy2VdrA2cDcb5A3M1As1HIXOi9ZL4+q0hShx8qet Vk4bmyL5gYyO0mLbmpiHDKNU5FzDp28= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-7-3HCHAwmfOK-1DtB8s4GZGQ-1; Fri, 11 Mar 2022 10:50:23 -0500 X-MC-Unique: 3HCHAwmfOK-1DtB8s4GZGQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4FE8B824FA9; Fri, 11 Mar 2022 15:50:22 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id D1A55785FD; Fri, 11 Mar 2022 15:50:19 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 12/31] KVM: x86: hyper-v: Introduce kvm_hv_is_tlb_flush_hcall() Date: Fri, 11 Mar 2022 16:49:24 +0100 Message-Id: <20220311154943.2299191-13-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The newly introduced helper checks whether vCPU is performing a Hyper-V TLB flush hypercall. This is required to filter out Direct TLB flush hypercalls from L2 for processing. Signed-off-by: Vitaly Kuznetsov --- arch/x86/kvm/hyperv.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h index 137f906eb8c3..0c0877a6aa74 100644 --- a/arch/x86/kvm/hyperv.h +++ b/arch/x86/kvm/hyperv.h @@ -168,6 +168,30 @@ static inline void kvm_hv_vcpu_empty_flush_tlb(struct = kvm_vcpu *vcpu) tlb_flush_ring =3D kvm_hv_get_tlb_flush_ring(vcpu); tlb_flush_ring->read_idx =3D tlb_flush_ring->write_idx; } + +static inline bool kvm_hv_is_tlb_flush_hcall(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); + u16 code; + + if (!hv_vcpu) + return false; + +#ifdef CONFIG_X86_64 + if (is_64_bit_hypercall(vcpu)) { + code =3D kvm_rcx_read(vcpu) & 0xffff; + } else +#endif + { + code =3D kvm_rax_read(vcpu) & 0xffff; + } + + return (code =3D=3D HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE || + code =3D=3D HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST || + code =3D=3D HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX || + code =3D=3D HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX); +} + void kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu); =20 =20 --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E889EC433EF for ; Fri, 11 Mar 2022 15:51:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349926AbiCKPwC (ORCPT ); Fri, 11 Mar 2022 10:52:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349649AbiCKPvg (ORCPT ); Fri, 11 Mar 2022 10:51:36 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 19DD01C8D96 for ; Fri, 11 Mar 2022 07:50:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013830; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gkr32gLCpegSELJcT7qdogv1QJC+6j2y6oSy6+rCKt0=; b=Nacp8k1U/cPsXw51o2CPdnRPf2CGnehRkX1gJ3sLA14hbNDlf6yTVf3/SyEteXRZNWU4ki 0Lhwi8Kh05/Rc9c9XZTpzolUaubGFF5vbZoXYibnOUoawRTGWx1/6sj6FOo7Rl5OCtfjwR Z3q7yAsQLQqJJnVEH8q1l4Dh9LAIhUE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-215-vxE5ESU-M2WTlZ8Iy9B28Q-1; Fri, 11 Mar 2022 10:50:27 -0500 X-MC-Unique: vxE5ESU-M2WTlZ8Iy9B28Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 989D351DF; Fri, 11 Mar 2022 15:50:25 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id B40AF785FD; Fri, 11 Mar 2022 15:50:22 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 13/31] KVM: x86: hyper-v: Direct TLB flush Date: Fri, 11 Mar 2022 16:49:25 +0100 Message-Id: <20220311154943.2299191-14-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Handle Direct TLB flush requests from L2 by going through all vCPUs and checking whether there are vCPUs running the same VM_ID with a VP_ID specified in the requests. Perform synthetic exit to L2 upon finish. Note, while checking VM_ID/VP_ID of running vCPUs seem to be a bit racy, we count on the fact that KVM flushes the whole L2 VPID upon transition. Also, KVM_REQ_HV_TLB_FLUSH request needs to be done upon transition between L1 and L2 to make sure all pending requests are always processed. Note, while nVMX/nSVM code does not handle VMCALL/VMMCALL from L2 yet. Signed-off-by: Vitaly Kuznetsov --- arch/x86/kvm/hyperv.c | 65 ++++++++++++++++++++++++++++++++++++------- arch/x86/kvm/trace.h | 21 ++++++++------ 2 files changed, 68 insertions(+), 18 deletions(-) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index b88e44a126b8..865b4d9a26fe 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -34,6 +34,7 @@ #include =20 #include +#include #include =20 #include "trace.h" @@ -1863,8 +1864,8 @@ static inline int hv_tlb_flush_ring_free(struct kvm_v= cpu_hv *hv_vcpu, return read_idx - write_idx - 1; } =20 -static void hv_tlb_flush_ring_enqueue(struct kvm_vcpu *vcpu, bool flush_al= l, - u64 *entries, int count) +static void hv_tlb_flush_ring_enqueue(struct kvm_vcpu *vcpu, bool direct, + bool flush_all, u64 *entries, int count) { struct kvm_vcpu_hv_tlbflush_ring *tlb_flush_ring; struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); @@ -1875,7 +1876,7 @@ static void hv_tlb_flush_ring_enqueue(struct kvm_vcpu= *vcpu, bool flush_all, if (!hv_vcpu) return; =20 - tlb_flush_ring =3D &hv_vcpu->tlb_flush_ring[0]; + tlb_flush_ring =3D direct ? &hv_vcpu->tlb_flush_ring[1] : &hv_vcpu->tlb_f= lush_ring[0]; =20 spin_lock_irqsave(&tlb_flush_ring->write_lock, flags); =20 @@ -2010,7 +2011,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, st= ruct kvm_hv_hcall *hc) } =20 trace_kvm_hv_flush_tlb(flush.processor_mask, - flush.address_space, flush.flags); + flush.address_space, flush.flags, + is_guest_mode(vcpu)); =20 valid_bank_mask =3D BIT_ULL(0); sparse_banks[0] =3D flush.processor_mask; @@ -2041,7 +2043,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, st= ruct kvm_hv_hcall *hc) trace_kvm_hv_flush_tlb_ex(flush_ex.hv_vp_set.valid_bank_mask, flush_ex.hv_vp_set.format, flush_ex.address_space, - flush_ex.flags); + flush_ex.flags, is_guest_mode(vcpu)); =20 valid_bank_mask =3D flush_ex.hv_vp_set.valid_bank_mask; all_cpus =3D flush_ex.hv_vp_set.format !=3D @@ -2080,19 +2082,45 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, = struct kvm_hv_hcall *hc) * vcpu->arch.cr3 may not be up-to-date for running vCPUs so we can't * analyze it here, flush TLB regardless of the specified address space. */ - if (all_cpus) { + if (all_cpus && !is_guest_mode(vcpu)) { kvm_for_each_vcpu(i, v, kvm) - hv_tlb_flush_ring_enqueue(v, all_addr, entries, hc->rep_cnt); + hv_tlb_flush_ring_enqueue(v, false, all_addr, entries, hc->rep_cnt); =20 kvm_make_all_cpus_request(kvm, KVM_REQ_HV_TLB_FLUSH); - } else { + } else if (!is_guest_mode(vcpu)) { sparse_set_to_vcpu_mask(kvm, sparse_banks, valid_bank_mask, vcpu_mask); =20 for_each_set_bit(i, vcpu_mask, KVM_MAX_VCPUS) { v =3D kvm_get_vcpu(kvm, i); if (!v) continue; - hv_tlb_flush_ring_enqueue(v, all_addr, entries, hc->rep_cnt); + hv_tlb_flush_ring_enqueue(v, false, all_addr, entries, hc->rep_cnt); + } + + kvm_make_vcpus_request_mask(kvm, KVM_REQ_HV_TLB_FLUSH, vcpu_mask); + } else { + struct kvm_vcpu_hv *hv_v; + + bitmap_zero(vcpu_mask, KVM_MAX_VCPUS); + + kvm_for_each_vcpu(i, v, kvm) { + hv_v =3D to_hv_vcpu(v); + + /* + * TLB is fully flushed on L2 VM change: either by KVM + * (on a eVMPTR switch) or by L1 hypervisor (in case it + * re-purposes the active eVMCS for a different VM/VP). + */ + if (!hv_v || hv_v->nested.vm_id !=3D hv_vcpu->nested.vm_id) + continue; + + if (!all_cpus && + !hv_is_vp_in_sparse_set(hv_v->nested.vp_id, valid_bank_mask, + sparse_banks)) + continue; + + __set_bit(i, vcpu_mask); + hv_tlb_flush_ring_enqueue(v, true, all_addr, entries, hc->rep_cnt); } =20 kvm_make_vcpus_request_mask(kvm, KVM_REQ_HV_TLB_FLUSH, vcpu_mask); @@ -2280,10 +2308,27 @@ static void kvm_hv_hypercall_set_result(struct kvm_= vcpu *vcpu, u64 result) =20 static int kvm_hv_hypercall_complete(struct kvm_vcpu *vcpu, u64 result) { + int ret; + trace_kvm_hv_hypercall_done(result); kvm_hv_hypercall_set_result(vcpu, result); ++vcpu->stat.hypercalls; - return kvm_skip_emulated_instruction(vcpu); + ret =3D kvm_skip_emulated_instruction(vcpu); + + if (unlikely(hv_result_success(result) && is_guest_mode(vcpu) + && kvm_hv_is_tlb_flush_hcall(vcpu))) { + struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); + u32 tlb_lock_count; + + if (unlikely(kvm_read_guest(vcpu->kvm, hv_vcpu->nested.pa_page_gpa, + &tlb_lock_count, sizeof(tlb_lock_count)))) + kvm_inject_gp(vcpu, 0); + + if (tlb_lock_count) + kvm_x86_ops.nested_ops->post_hv_direct_flush(vcpu); + } + + return ret; } =20 static int kvm_hv_hypercall_complete_userspace(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h index 193f5ba930d1..82e1646b9433 100644 --- a/arch/x86/kvm/trace.h +++ b/arch/x86/kvm/trace.h @@ -1477,38 +1477,41 @@ TRACE_EVENT(kvm_hv_timer_state, * Tracepoint for kvm_hv_flush_tlb. */ TRACE_EVENT(kvm_hv_flush_tlb, - TP_PROTO(u64 processor_mask, u64 address_space, u64 flags), - TP_ARGS(processor_mask, address_space, flags), + TP_PROTO(u64 processor_mask, u64 address_space, u64 flags, bool direct), + TP_ARGS(processor_mask, address_space, flags, direct), =20 TP_STRUCT__entry( __field(u64, processor_mask) __field(u64, address_space) __field(u64, flags) + __field(bool, direct) ), =20 TP_fast_assign( __entry->processor_mask =3D processor_mask; __entry->address_space =3D address_space; __entry->flags =3D flags; + __entry->direct =3D direct; ), =20 - TP_printk("processor_mask 0x%llx address_space 0x%llx flags 0x%llx", + TP_printk("processor_mask 0x%llx address_space 0x%llx flags 0x%llx %s", __entry->processor_mask, __entry->address_space, - __entry->flags) + __entry->flags, __entry->direct ? "(direct)" : "") ); =20 /* * Tracepoint for kvm_hv_flush_tlb_ex. */ TRACE_EVENT(kvm_hv_flush_tlb_ex, - TP_PROTO(u64 valid_bank_mask, u64 format, u64 address_space, u64 flags), - TP_ARGS(valid_bank_mask, format, address_space, flags), + TP_PROTO(u64 valid_bank_mask, u64 format, u64 address_space, u64 flags, b= ool direct), + TP_ARGS(valid_bank_mask, format, address_space, flags, direct), =20 TP_STRUCT__entry( __field(u64, valid_bank_mask) __field(u64, format) __field(u64, address_space) __field(u64, flags) + __field(bool, direct) ), =20 TP_fast_assign( @@ -1516,12 +1519,14 @@ TRACE_EVENT(kvm_hv_flush_tlb_ex, __entry->format =3D format; __entry->address_space =3D address_space; __entry->flags =3D flags; + __entry->direct =3D direct; ), =20 TP_printk("valid_bank_mask 0x%llx format 0x%llx " - "address_space 0x%llx flags 0x%llx", + "address_space 0x%llx flags 0x%llx %s", __entry->valid_bank_mask, __entry->format, - __entry->address_space, __entry->flags) + __entry->address_space, __entry->flags, + __entry->direct ? "(direct)" : "") ); =20 /* --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D675C433F5 for ; Fri, 11 Mar 2022 15:51:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349879AbiCKPwL (ORCPT ); Fri, 11 Mar 2022 10:52:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349813AbiCKPvj (ORCPT ); Fri, 11 Mar 2022 10:51:39 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 5AD3C1CC7CE for ; Fri, 11 Mar 2022 07:50:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013833; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AJxhfbbUpp3TCfbP2M1jKurKEmQJOIdOoRd+GiZKb4c=; b=UTXKxhqX1ul4e91AROFgHJaO39wsjbjoBj/xEeCmSC23q7Trd/OE+kMRLW/vpN37itmN+d g734AKVZHcc85I9t9WXAez1kiAQQfFGM1GfLzvFQQXfc68ppj+mJG7+tnvly7C+lTMmL31 4Ns7uNOQJ8vTNLwDibSh9s9RqTwCBmQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-376-QmWc_Fs6OAiW6ic17gOgfg-1; Fri, 11 Mar 2022 10:50:30 -0500 X-MC-Unique: QmWc_Fs6OAiW6ic17gOgfg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C42FB51DF; Fri, 11 Mar 2022 15:50:28 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id EF21E785FD; Fri, 11 Mar 2022 15:50:25 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 14/31] KVM: x86: hyper-v: Introduce fast kvm_hv_direct_tlb_flush_exposed() check Date: Fri, 11 Mar 2022 16:49:26 +0100 Message-Id: <20220311154943.2299191-15-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Introduce a helper to quickly check if KVM needs to handle VMCALL/VMMCALL from L2 in L0 to process Direct TLB flush requests. Signed-off-by: Vitaly Kuznetsov --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/hyperv.c | 6 ++++++ arch/x86/kvm/hyperv.h | 7 +++++++ 3 files changed, 14 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 45ee05f4f1a0..bcaf05951628 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -611,6 +611,7 @@ struct kvm_vcpu_hv { u32 enlightenments_eax; /* HYPERV_CPUID_ENLIGHTMENT_INFO.EAX */ u32 enlightenments_ebx; /* HYPERV_CPUID_ENLIGHTMENT_INFO.EBX */ u32 syndbg_cap_eax; /* HYPERV_CPUID_SYNDBG_PLATFORM_CAPABILITIES.EAX */ + u32 nested_features_eax; /* HYPERV_CPUID_NESTED_FEATURES.EAX */ } cpuid_cache; =20 /* Two rings for regular Hyper-V TLB flush and Direct TLB flush */ diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 865b4d9a26fe..31acc08a6b23 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2270,6 +2270,12 @@ void kvm_hv_set_cpuid(struct kvm_vcpu *vcpu) hv_vcpu->cpuid_cache.syndbg_cap_eax =3D entry->eax; else hv_vcpu->cpuid_cache.syndbg_cap_eax =3D 0; + + entry =3D kvm_find_cpuid_entry(vcpu, HYPERV_CPUID_NESTED_FEATURES, 0); + if (entry) + hv_vcpu->cpuid_cache.nested_features_eax =3D entry->eax; + else + hv_vcpu->cpuid_cache.nested_features_eax =3D 0; } =20 int kvm_hv_set_enforce_cpuid(struct kvm_vcpu *vcpu, bool enforce) diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h index 0c0877a6aa74..0c9062aed182 100644 --- a/arch/x86/kvm/hyperv.h +++ b/arch/x86/kvm/hyperv.h @@ -169,6 +169,13 @@ static inline void kvm_hv_vcpu_empty_flush_tlb(struct = kvm_vcpu *vcpu) tlb_flush_ring->read_idx =3D tlb_flush_ring->write_idx; } =20 +static inline bool kvm_hv_direct_tlb_flush_exposed(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); + + return hv_vcpu && (hv_vcpu->cpuid_cache.nested_features_eax & HV_X64_NEST= ED_DIRECT_FLUSH); +} + static inline bool kvm_hv_is_tlb_flush_hcall(struct kvm_vcpu *vcpu) { struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E84DC4332F for ; Fri, 11 Mar 2022 15:51:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235213AbiCKPwG (ORCPT ); Fri, 11 Mar 2022 10:52:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237320AbiCKPvl (ORCPT ); Fri, 11 Mar 2022 10:51:41 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 5E3D41CABD6 for ; Fri, 11 Mar 2022 07:50:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013835; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ymMl8hA8K/esTw1tg/apm+DYb9LQJWsl4/Hevby8goQ=; b=YqTUgiM4UH24HnY5350ZY4Rq+JlmkjtnfXt/4FgLAm32QBzWguNBXJTo/OBEwdlmezdY11 GC65K0Pe3lbgzpFhAAP72XmgVesxc2NjhvJXvlVNV3XfZzgC0sM4TFx5yi48HWbTFzTdK4 DZJn+CVQPyX3Qi4ETnUps061ZrDvBp4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-674-EwBvS5ZsM22Wz1fMY-uaWA-1; Fri, 11 Mar 2022 10:50:33 -0500 X-MC-Unique: EwBvS5ZsM22Wz1fMY-uaWA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D4D83801AFC; Fri, 11 Mar 2022 15:50:31 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2DB93785FD; Fri, 11 Mar 2022 15:50:28 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 15/31] x86/hyperv: Fix 'struct hv_enlightened_vmcs' definition Date: Fri, 11 Mar 2022 16:49:27 +0100 Message-Id: <20220311154943.2299191-16-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Section 1.9 of TLFS v6.0b says: "All structures are padded in such a way that fields are aligned naturally (that is, an 8-byte field is aligned to an offset of 8 bytes and so on)". 'struct enlightened_vmcs' has a glitch: ... struct { u32 nested_flush_hypercall:1; /* 836: 0 4= */ u32 msr_bitmap:1; /* 836: 1 4 */ u32 reserved:30; /* 836: 2 4 */ } hv_enlightenments_control; /* 836 4 */ u32 hv_vp_id; /* 840 4 */ u64 hv_vm_id; /* 844 8 */ u64 partition_assist_page; /* 852 8 */ ... And the observed values in 'partition_assist_page' make no sense at all. Fix the layout by padding the structure properly. Fixes: 68d1eb72ee99 ("x86/hyper-v: define struct hv_enlightened_vmcs and cl= ean field bits") Reviewed-by: Michael Kelley Signed-off-by: Vitaly Kuznetsov --- arch/x86/include/asm/hyperv-tlfs.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hype= rv-tlfs.h index 5225a85c08c3..e7ddae8e02c6 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -548,7 +548,7 @@ struct hv_enlightened_vmcs { u64 guest_rip; =20 u32 hv_clean_fields; - u32 hv_padding_32; + u32 padding32_1; u32 hv_synthetic_controls; struct { u32 nested_flush_hypercall:1; @@ -556,7 +556,7 @@ struct hv_enlightened_vmcs { u32 reserved:30; } __packed hv_enlightenments_control; u32 hv_vp_id; - + u32 padding32_2; u64 hv_vm_id; u64 partition_assist_page; u64 padding64_4[4]; --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A404C433F5 for ; Fri, 11 Mar 2022 15:51:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349882AbiCKPwO (ORCPT ); Fri, 11 Mar 2022 10:52:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58226 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349837AbiCKPvr (ORCPT ); Fri, 11 Mar 2022 10:51:47 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 30BAD1CC7D3 for ; Fri, 11 Mar 2022 07:50:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013839; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6r/JqSRUySk6LAWAAPPTCCWXEHb1+Sqs9cztwTx0ODo=; b=Sfk7ep0v9cmrwWEY+xLCzBzpum717uH9J9DKrXC9l8tbY1NQP8ehZQGJai1zfpQW8YmjhH bu/CUmBsrH1YnjsuwqTunQlFo4a45XbrmLoLvy1TXHIEpAVYo0VS7gh2NNR04U0+OtW1xj U8ZUf4cbgv6uoQ6B6Pv3WQUJaqEuH/Q= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-262-9akMTh3jODir0AtxJpsvyA-1; Fri, 11 Mar 2022 10:50:36 -0500 X-MC-Unique: 9akMTh3jODir0AtxJpsvyA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B981F824FA6; Fri, 11 Mar 2022 15:50:34 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 40DB6866D1; Fri, 11 Mar 2022 15:50:32 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 16/31] KVM: nVMX: hyper-v: Direct TLB flush Date: Fri, 11 Mar 2022 16:49:28 +0100 Message-Id: <20220311154943.2299191-17-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Enable Direct TLB flush feature on nVMX when: - Enlightened VMCS is in use. - Direct TLB flush flag is enabled in eVMCS. - Direct TLB flush is enabled in partition assist page. Perform synthetic vmexit to L1 after processing TLB flush call upon request (HV_VMX_SYNTHETIC_EXIT_REASON_TRAP_AFTER_FLUSH). Signed-off-by: Vitaly Kuznetsov --- arch/x86/kvm/vmx/evmcs.c | 20 ++++++++++++++++++++ arch/x86/kvm/vmx/evmcs.h | 3 +++ arch/x86/kvm/vmx/nested.c | 16 ++++++++++++++++ 3 files changed, 39 insertions(+) diff --git a/arch/x86/kvm/vmx/evmcs.c b/arch/x86/kvm/vmx/evmcs.c index 1705c4973636..cdf7ec5cb64c 100644 --- a/arch/x86/kvm/vmx/evmcs.c +++ b/arch/x86/kvm/vmx/evmcs.c @@ -6,6 +6,7 @@ #include "../hyperv.h" #include "../cpuid.h" #include "evmcs.h" +#include "nested.h" #include "vmcs.h" #include "vmx.h" #include "trace.h" @@ -438,6 +439,25 @@ int nested_enable_evmcs(struct kvm_vcpu *vcpu, return 0; } =20 +bool nested_evmcs_direct_flush_enabled(struct kvm_vcpu *vcpu) +{ + struct vcpu_vmx *vmx =3D to_vmx(vcpu); + struct hv_enlightened_vmcs *evmcs =3D vmx->nested.hv_evmcs; + struct hv_vp_assist_page assist_page; + + if (!evmcs) + return false; + + if (!evmcs->hv_enlightenments_control.nested_flush_hypercall) + return false; + + if (unlikely(!kvm_hv_get_assist_page(vcpu, &assist_page))) + return false; + + return assist_page.nested_control.features.directhypercall; +} + void vmx_post_hv_direct_flush(struct kvm_vcpu *vcpu) { + nested_vmx_vmexit(vcpu, HV_VMX_SYNTHETIC_EXIT_REASON_TRAP_AFTER_FLUSH, 0,= 0); } diff --git a/arch/x86/kvm/vmx/evmcs.h b/arch/x86/kvm/vmx/evmcs.h index 8862692a4c5d..ab0949c22d2d 100644 --- a/arch/x86/kvm/vmx/evmcs.h +++ b/arch/x86/kvm/vmx/evmcs.h @@ -65,6 +65,8 @@ DECLARE_STATIC_KEY_FALSE(enable_evmcs); #define EVMCS1_UNSUPPORTED_VMENTRY_CTRL (VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CT= RL) #define EVMCS1_UNSUPPORTED_VMFUNC (VMX_VMFUNC_EPTP_SWITCHING) =20 +#define HV_VMX_SYNTHETIC_EXIT_REASON_TRAP_AFTER_FLUSH 0x10000031 + struct evmcs_field { u16 offset; u16 clean_field; @@ -244,6 +246,7 @@ int nested_enable_evmcs(struct kvm_vcpu *vcpu, uint16_t *vmcs_version); void nested_evmcs_filter_control_msr(u32 msr_index, u64 *pdata); int nested_evmcs_check_controls(struct vmcs12 *vmcs12); +bool nested_evmcs_direct_flush_enabled(struct kvm_vcpu *vcpu); void vmx_post_hv_direct_flush(struct kvm_vcpu *vcpu); =20 #endif /* __KVM_X86_VMX_EVMCS_H */ diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 3121b761b1ee..efed9a199c84 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -1168,6 +1168,17 @@ static void nested_vmx_transition_tlb_flush(struct k= vm_vcpu *vcpu, { struct vcpu_vmx *vmx =3D to_vmx(vcpu); =20 + /* + * KVM_REQ_HV_TLB_FLUSH flushes entries from either L1's VPID or + * L2's VPID upon request from the guest. Make sure we check for + * pending entries for the case when the request got misplaced (e.g. + * a transition from L2->L1 happened while processing Direct TLB flush + * request or vice versa). kvm_hv_vcpu_flush_tlb() will not flush + * anything if there are no requests in the corresponding buffer. + */ + if (to_hv_vcpu(vcpu)) + kvm_make_request(KVM_REQ_HV_TLB_FLUSH, vcpu); + /* * If vmcs12 doesn't use VPID, L1 expects linear and combined mappings * for *all* contexts to be flushed on VM-Enter/VM-Exit, i.e. it's a @@ -5972,6 +5983,11 @@ static bool nested_vmx_l0_wants_exit(struct kvm_vcpu= *vcpu, * Handle L2's bus locks in L0 directly. */ return true; + case EXIT_REASON_VMCALL: + /* Hyper-V Direct TLB flush hypercall is handled by L0 */ + return kvm_hv_direct_tlb_flush_exposed(vcpu) && + nested_evmcs_direct_flush_enabled(vcpu) && + kvm_hv_is_tlb_flush_hcall(vcpu); default: break; } --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CE2DC433F5 for ; Fri, 11 Mar 2022 15:51:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242710AbiCKPwS (ORCPT ); Fri, 11 Mar 2022 10:52:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349836AbiCKPvs (ORCPT ); Fri, 11 Mar 2022 10:51:48 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id B432F1CC7FA for ; Fri, 11 Mar 2022 07:50:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013842; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1BUkCceWwPF6IV25ZH1+8yGdQLQ11zSPD8IwItivN4M=; b=E8eKdDwIevr3dyi1RMBFqOaOenn5vTScKxgOcvQYLmaUKyqSwNK2/GZENiypB8Uc0QgdMJ 6JFQVbmDt/mHEyAOogS8TNiZ2UN+4mONwpiWl7LkBhB82XNjeL1r4sbZy493rbKKGOVP9m iRy08AbZGyn7gTrfGR8QjcYUhYELI/k= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-675-NX2EUs27Pp2GRaWfd_b8MA-1; Fri, 11 Mar 2022 10:50:39 -0500 X-MC-Unique: NX2EUs27Pp2GRaWfd_b8MA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C737951E0; Fri, 11 Mar 2022 15:50:37 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 17EA1785FD; Fri, 11 Mar 2022 15:50:34 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 17/31] KVM: x86: KVM_REQ_TLB_FLUSH_CURRENT is a superset of KVM_REQ_HV_TLB_FLUSH too Date: Fri, 11 Mar 2022 16:49:29 +0100 Message-Id: <20220311154943.2299191-18-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" KVM_REQ_TLB_FLUSH_CURRENT is an even stronger operation than KVM_REQ_TLB_FLUSH_GUEST so KVM_REQ_HV_TLB_FLUSH needs not to be processed after it. Signed-off-by: Vitaly Kuznetsov --- arch/x86/kvm/x86.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 0082c5691a05..81f1039c9aff 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3336,8 +3336,11 @@ static inline void kvm_vcpu_flush_tlb_current(struct= kvm_vcpu *vcpu) */ void kvm_service_local_tlb_flush_requests(struct kvm_vcpu *vcpu) { - if (kvm_check_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu)) + if (kvm_check_request(KVM_REQ_TLB_FLUSH_CURRENT, vcpu)) { kvm_vcpu_flush_tlb_current(vcpu); + if (kvm_check_request(KVM_REQ_HV_TLB_FLUSH, vcpu)) + kvm_hv_vcpu_empty_flush_tlb(vcpu); + } =20 if (kvm_check_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu)) { kvm_vcpu_flush_tlb_guest(vcpu); --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BD17C433EF for ; Fri, 11 Mar 2022 15:51:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349967AbiCKPwU (ORCPT ); Fri, 11 Mar 2022 10:52:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349891AbiCKPvw (ORCPT ); Fri, 11 Mar 2022 10:51:52 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 35C981CC7FD for ; Fri, 11 Mar 2022 07:50:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013845; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u3c9w/aK7c+8N73IFmes1GpB1imyjMjXrb2rGtlAH/I=; b=E/+Ai7sNTBvadvs2zUJqydlwYeETbnAAY1ZbuZmbsgzsaZtK5gIVvlxK9MMMV9iIB6zrkk skPkq3F/bq1LLJD1cOXsIsN5axkWfEvBx6Lcrt4h315lN3DOzbxA0IrFD5Ac6Us+4kyLRC hLweI0NYBhEHn8XC4cj/hH1nRtqfUDI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-568-tE6HNNPOMb-yeDUjJ2MbxQ-1; Fri, 11 Mar 2022 10:50:42 -0500 X-MC-Unique: tE6HNNPOMb-yeDUjJ2MbxQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A5D1051DC; Fri, 11 Mar 2022 15:50:40 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2A1F0785FD; Fri, 11 Mar 2022 15:50:38 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 18/31] KVM: nSVM: hyper-v: Direct TLB flush Date: Fri, 11 Mar 2022 16:49:30 +0100 Message-Id: <20220311154943.2299191-19-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Implement Hyper-V Direct TLB flush for nSVM feature. The feature needs to be enabled both in extended 'nested controls' in VMCB and partition assist page. According to TLFS, synthetic vmexit to L1 is performed with - HV_SVM_EXITCODE_ENL exit_code. - HV_SVM_ENL_EXITCODE_TRAP_AFTER_FLUSH exit_info_1. Signed-off-by: Vitaly Kuznetsov --- arch/x86/kvm/svm/hyperv.c | 7 +++++++ arch/x86/kvm/svm/hyperv.h | 20 ++++++++++++++++++++ arch/x86/kvm/svm/nested.c | 22 +++++++++++++++++++++- 3 files changed, 48 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/svm/hyperv.c b/arch/x86/kvm/svm/hyperv.c index 0142fde34738..f3298c70053e 100644 --- a/arch/x86/kvm/svm/hyperv.c +++ b/arch/x86/kvm/svm/hyperv.c @@ -8,4 +8,11 @@ =20 void svm_post_hv_direct_flush(struct kvm_vcpu *vcpu) { + struct vcpu_svm *svm =3D to_svm(vcpu); + + svm->vmcb->control.exit_code =3D HV_SVM_EXITCODE_ENL; + svm->vmcb->control.exit_code_hi =3D 0; + svm->vmcb->control.exit_info_1 =3D HV_SVM_ENL_EXITCODE_TRAP_AFTER_FLUSH; + svm->vmcb->control.exit_info_2 =3D 0; + nested_svm_vmexit(svm); } diff --git a/arch/x86/kvm/svm/hyperv.h b/arch/x86/kvm/svm/hyperv.h index 478db6f61d69..80d12e075b4f 100644 --- a/arch/x86/kvm/svm/hyperv.h +++ b/arch/x86/kvm/svm/hyperv.h @@ -9,6 +9,7 @@ #include =20 #include "../hyperv.h" +#include "svm.h" =20 /* * Hyper-V uses the software reserved 32 bytes in VMCB @@ -32,6 +33,9 @@ struct hv_enlightenments { */ #define VMCB_HV_NESTED_ENLIGHTENMENTS VMCB_SW =20 +#define HV_SVM_EXITCODE_ENL 0xF0000000 +#define HV_SVM_ENL_EXITCODE_TRAP_AFTER_FLUSH (1) + static inline void nested_svm_hv_update_vm_vp_ids(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm =3D to_svm(vcpu); @@ -47,6 +51,22 @@ static inline void nested_svm_hv_update_vm_vp_ids(struct= kvm_vcpu *vcpu) hv_vcpu->nested.vp_id =3D hve->hv_vp_id; } =20 +static inline bool nested_svm_direct_flush_enabled(struct kvm_vcpu *vcpu) +{ + struct vcpu_svm *svm =3D to_svm(vcpu); + struct hv_enlightenments *hve =3D + (struct hv_enlightenments *)svm->nested.ctl.reserved_sw; + struct hv_vp_assist_page assist_page; + + if (unlikely(!kvm_hv_get_assist_page(vcpu, &assist_page))) + return false; + + if (!hve->hv_enlightenments_control.nested_flush_hypercall) + return false; + + return assist_page.nested_control.features.directhypercall; +} + void svm_post_hv_direct_flush(struct kvm_vcpu *vcpu); =20 #endif /* __ARCH_X86_KVM_SVM_HYPERV_H__ */ diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index 46ea234363cf..3f6c2f088e26 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -153,7 +153,8 @@ void recalc_intercepts(struct vcpu_svm *svm) } =20 /* We don't want to see VMMCALLs from a nested guest */ - vmcb_clr_intercept(c, INTERCEPT_VMMCALL); + if (!nested_svm_direct_flush_enabled(&svm->vcpu)) + vmcb_clr_intercept(c, INTERCEPT_VMMCALL); =20 for (i =3D 0; i < MAX_INTERCEPT; i++) c->intercepts[i] |=3D g->intercepts[i]; @@ -461,6 +462,17 @@ static inline bool nested_npt_enabled(struct vcpu_svm = *svm) =20 static void nested_svm_transition_tlb_flush(struct kvm_vcpu *vcpu) { + /* + * KVM_REQ_HV_TLB_FLUSH flushes entries from either L1's VPID or + * L2's VPID upon request from the guest. Make sure we check for + * pending entries for the case when the request got misplaced (e.g. + * a transition from L2->L1 happened while processing Direct TLB flush + * request or vice versa). kvm_hv_vcpu_flush_tlb() will not flush + * anything if there are no requests in the corresponding buffer. + */ + if (to_hv_vcpu(vcpu)) + kvm_make_request(KVM_REQ_HV_TLB_FLUSH, vcpu); + /* * TODO: optimize unconditional TLB flush/MMU sync. A partial list of * things to fix before this can be conditional: @@ -1266,6 +1278,7 @@ static int svm_check_nested_events(struct kvm_vcpu *v= cpu) int nested_svm_exit_special(struct vcpu_svm *svm) { u32 exit_code =3D svm->vmcb->control.exit_code; + struct kvm_vcpu *vcpu =3D &svm->vcpu; =20 switch (exit_code) { case SVM_EXIT_INTR: @@ -1284,6 +1297,13 @@ int nested_svm_exit_special(struct vcpu_svm *svm) return NESTED_EXIT_HOST; break; } + case SVM_EXIT_VMMCALL: + /* Hyper-V Direct TLB flush hypercall is handled by L0 */ + if (kvm_hv_direct_tlb_flush_exposed(vcpu) && + nested_svm_direct_flush_enabled(vcpu) && + kvm_hv_is_tlb_flush_hcall(vcpu)) + return NESTED_EXIT_HOST; + break; default: break; } --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FCA5C433FE for ; Fri, 11 Mar 2022 15:51:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350015AbiCKPwk (ORCPT ); Fri, 11 Mar 2022 10:52:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59184 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349929AbiCKPwC (ORCPT ); Fri, 11 Mar 2022 10:52:02 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 7094A1C9B4F for ; Fri, 11 Mar 2022 07:50:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3smDROMP8QMSLDecB2DTW8XgUBKKv8Df2PL4vzKcmzw=; b=Hs9XBGaoOevbhLMD6l8rh3rxdXFv75Ao6Y8Zmjq24CZBBWLQR8cvWjCfBHRR3+1YIQZUxk H0FbpaYQISOBkiHqiZsAwCyY7ouY5/UTb9iRG0brqoRakHZvM7N6g4EJMw/BGmooE/GIrU IkNnBa1bAh8LL8XByrfmUk6ZaNPb/Zk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-57-cGvnSmZ9OxeG8_tWAp336g-1; Fri, 11 Mar 2022 10:50:45 -0500 X-MC-Unique: cGvnSmZ9OxeG8_tWAp336g-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9EC711006AA5; Fri, 11 Mar 2022 15:50:43 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 23966785FD; Fri, 11 Mar 2022 15:50:40 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 19/31] KVM: x86: Expose Hyper-V Direct TLB flush feature Date: Fri, 11 Mar 2022 16:49:31 +0100 Message-Id: <20220311154943.2299191-20-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" With both nSVM and nVMX implementations in place, KVM can export Hyper-V Direct TLB flush feature to userspace. Signed-off-by: Vitaly Kuznetsov --- arch/x86/kvm/hyperv.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 31acc08a6b23..d52de033ce36 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2815,6 +2815,7 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kv= m_cpuid2 *cpuid, =20 case HYPERV_CPUID_NESTED_FEATURES: ent->eax =3D evmcs_ver; + ent->eax |=3D HV_X64_NESTED_DIRECT_FLUSH; ent->eax |=3D HV_X64_NESTED_MSR_BITMAP; =20 break; --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F730C433F5 for ; Fri, 11 Mar 2022 15:51:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350037AbiCKPwn (ORCPT ); Fri, 11 Mar 2022 10:52:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349886AbiCKPwP (ORCPT ); Fri, 11 Mar 2022 10:52:15 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 5F71B1CD7D8 for ; Fri, 11 Mar 2022 07:50:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013853; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sN6DWASh/AKFZGnLsQPjnzaSKJmnnbjQwij/4RRQmag=; b=bXv65HyXfkB9vpAb+eQz9bKxc+13mvnHJyjnEQKrggIFahdep3l3r6HHo+3aZENMEy0ZM8 k2ChT43oNqYS+K1TKrriRuBfmr6kTaK7P4yrOHaVaCjGGfRrBXOOlxQROTskXHYRgphvUA 6gZ70LUEra0KUKfeYRdKLW+zHfg73Xs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-218-AikEhRbCN32ehAOwjbc4rA-1; Fri, 11 Mar 2022 10:50:48 -0500 X-MC-Unique: AikEhRbCN32ehAOwjbc4rA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9E48D801AFC; Fri, 11 Mar 2022 15:50:46 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 09960785FD; Fri, 11 Mar 2022 15:50:43 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 20/31] KVM: selftests: Add hyperv_svm_test to .gitignore Date: Fri, 11 Mar 2022 16:49:32 +0100 Message-Id: <20220311154943.2299191-21-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add hyperv_svm_test to .gitignore. Fixes: e67bd7df28a0 ("KVM: selftests: nSVM: Add enlightened MSR-Bitmap self= test") Signed-off-by: Vitaly Kuznetsov --- tools/testing/selftests/kvm/.gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftes= ts/kvm/.gitignore index 052ddfe4b23a..e42620dbaac7 100644 --- a/tools/testing/selftests/kvm/.gitignore +++ b/tools/testing/selftests/kvm/.gitignore @@ -21,6 +21,7 @@ /x86_64/hyperv_clock /x86_64/hyperv_cpuid /x86_64/hyperv_features +/x86_64/hyperv_svm_test /x86_64/mmio_warning_test /x86_64/mmu_role_test /x86_64/platform_info_test --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5656C433F5 for ; Fri, 11 Mar 2022 15:53:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345367AbiCKPye (ORCPT ); Fri, 11 Mar 2022 10:54:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349909AbiCKPwc (ORCPT ); Fri, 11 Mar 2022 10:52:32 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 1A2391CD9C5 for ; Fri, 11 Mar 2022 07:50:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013855; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5n7gy9H/BMqsJ4RsjxsgJmI1zewtaeQWOy9VL2DQsf4=; b=VR3CdX0VLeMdB14MOmUiaMs7iiFS+INo6FTjiJEx6lVWwyJ7YSdpaG6TEYWjqHfXa9aFd7 gaPBB+wPNTQ/3s07oXSTCCJJvtmDyFfziEK4tcRWQpE4am1eXyjhhe6ZFgM1q8Y7G7cUY7 GkUGUugRZuMrJWIbLJZ6CLzCe160mYA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-75-0FebaiqVNFydDrWw5CHSAg-1; Fri, 11 Mar 2022 10:50:50 -0500 X-MC-Unique: 0FebaiqVNFydDrWw5CHSAg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2B7731854E27; Fri, 11 Mar 2022 15:50:49 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id C5C22866D4; Fri, 11 Mar 2022 15:50:46 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 21/31] KVM: selftests: Better XMM read/write helpers Date: Fri, 11 Mar 2022 16:49:33 +0100 Message-Id: <20220311154943.2299191-22-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" set_xmm()/get_xmm() helpers are fairly useless as they only read 64 bits from 128-bit registers. Moreover, these helpers are not used. Borrow _kvm_read_sse_reg()/_kvm_write_sse_reg() from KVM limiting them to XMM0-XMM8 for now. Signed-off-by: Vitaly Kuznetsov --- .../selftests/kvm/include/x86_64/processor.h | 70 ++++++++++--------- 1 file changed, 36 insertions(+), 34 deletions(-) diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools= /testing/selftests/kvm/include/x86_64/processor.h index 8a470da7b71a..4962cd0ceacd 100644 --- a/tools/testing/selftests/kvm/include/x86_64/processor.h +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h @@ -296,71 +296,73 @@ static inline void cpuid(uint32_t *eax, uint32_t *ebx, : "memory"); } =20 -#define SET_XMM(__var, __xmm) \ - asm volatile("movq %0, %%"#__xmm : : "r"(__var) : #__xmm) +typedef u32 __attribute__((vector_size(16))) sse128_t; +#define __sse128_u union { sse128_t vec; u64 as_u64[2]; u32 as_u32[4]; } +#define sse128_lo(x) ({ __sse128_u t; t.vec =3D x; t.as_u64[0]; }) +#define sse128_hi(x) ({ __sse128_u t; t.vec =3D x; t.as_u64[1]; }) =20 -static inline void set_xmm(int n, unsigned long val) +static inline void read_sse_reg(int reg, sse128_t *data) { - switch (n) { + switch (reg) { case 0: - SET_XMM(val, xmm0); + asm("movdqa %%xmm0, %0" : "=3Dm"(*data)); break; case 1: - SET_XMM(val, xmm1); + asm("movdqa %%xmm1, %0" : "=3Dm"(*data)); break; case 2: - SET_XMM(val, xmm2); + asm("movdqa %%xmm2, %0" : "=3Dm"(*data)); break; case 3: - SET_XMM(val, xmm3); + asm("movdqa %%xmm3, %0" : "=3Dm"(*data)); break; case 4: - SET_XMM(val, xmm4); + asm("movdqa %%xmm4, %0" : "=3Dm"(*data)); break; case 5: - SET_XMM(val, xmm5); + asm("movdqa %%xmm5, %0" : "=3Dm"(*data)); break; case 6: - SET_XMM(val, xmm6); + asm("movdqa %%xmm6, %0" : "=3Dm"(*data)); break; case 7: - SET_XMM(val, xmm7); + asm("movdqa %%xmm7, %0" : "=3Dm"(*data)); break; + default: + BUG(); } } =20 -#define GET_XMM(__xmm) \ -({ \ - unsigned long __val; \ - asm volatile("movq %%"#__xmm", %0" : "=3Dr"(__val)); \ - __val; \ -}) - -static inline unsigned long get_xmm(int n) +static inline void write_sse_reg(int reg, const sse128_t *data) { - assert(n >=3D 0 && n <=3D 7); - - switch (n) { + switch (reg) { case 0: - return GET_XMM(xmm0); + asm("movdqa %0, %%xmm0" : : "m"(*data)); + break; case 1: - return GET_XMM(xmm1); + asm("movdqa %0, %%xmm1" : : "m"(*data)); + break; case 2: - return GET_XMM(xmm2); + asm("movdqa %0, %%xmm2" : : "m"(*data)); + break; case 3: - return GET_XMM(xmm3); + asm("movdqa %0, %%xmm3" : : "m"(*data)); + break; case 4: - return GET_XMM(xmm4); + asm("movdqa %0, %%xmm4" : : "m"(*data)); + break; case 5: - return GET_XMM(xmm5); + asm("movdqa %0, %%xmm5" : : "m"(*data)); + break; case 6: - return GET_XMM(xmm6); + asm("movdqa %0, %%xmm6" : : "m"(*data)); + break; case 7: - return GET_XMM(xmm7); + asm("movdqa %0, %%xmm7" : : "m"(*data)); + break; + default: + BUG(); } - - /* never reached */ - return 0; } =20 bool is_intel_cpu(void); --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC18BC433FE for ; Fri, 11 Mar 2022 15:53:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350094AbiCKPw5 (ORCPT ); Fri, 11 Mar 2022 10:52:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58224 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349988AbiCKPwd (ORCPT ); Fri, 11 Mar 2022 10:52:33 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id CB13F1CD9D8 for ; Fri, 11 Mar 2022 07:50:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013856; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xUuo315Ep47JqKWpYA6vZ5HF93dGtuZOeMzrFoF56JQ=; b=DkbRFpdWjiX3baiKvJb7xgUh5Vs60W+ANsa1/BTx5V4kEimvv0xPfKKMmSY+IY41jh3fpv hjRI6z9PFmUlb8NGFaFs/6o9p8YMTyRVc3IdQ82ON0b+AXrq9Z2OucTMCGn6lbQT+S2H8w pnfgnXDeKn7P8agl+wYkG0HMrNyC6Og= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-401-tjk-ZLzJMzalk9oPO9O3Og-1; Fri, 11 Mar 2022 10:50:53 -0500 X-MC-Unique: tjk-ZLzJMzalk9oPO9O3Og-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 027451854E26; Fri, 11 Mar 2022 15:50:52 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8AFFF866D4; Fri, 11 Mar 2022 15:50:49 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 22/31] KVM: selftests: Hyper-V PV IPI selftest Date: Fri, 11 Mar 2022 16:49:34 +0100 Message-Id: <20220311154943.2299191-23-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Introduce a selftest for Hyper-V PV IPI hypercalls (HvCallSendSyntheticClusterIpi, HvCallSendSyntheticClusterIpiEx). The test creates one 'sender' vCPU and two 'receiver' vCPU and then issues various combinations of send IPI hypercalls in both 'normal' and 'fast' (with XMM input where necessary) mode. Later, the test checks whether IPIs were delivered to the expected destination vCPU[s]. Signed-off-by: Vitaly Kuznetsov --- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/x86_64/hyperv.h | 3 + .../selftests/kvm/x86_64/hyperv_features.c | 5 +- .../testing/selftests/kvm/x86_64/hyperv_ipi.c | 362 ++++++++++++++++++ 5 files changed, 369 insertions(+), 3 deletions(-) create mode 100644 tools/testing/selftests/kvm/x86_64/hyperv_ipi.c diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftes= ts/kvm/.gitignore index e42620dbaac7..8ef4b4571b23 100644 --- a/tools/testing/selftests/kvm/.gitignore +++ b/tools/testing/selftests/kvm/.gitignore @@ -21,6 +21,7 @@ /x86_64/hyperv_clock /x86_64/hyperv_cpuid /x86_64/hyperv_features +/x86_64/hyperv_ipi /x86_64/hyperv_svm_test /x86_64/mmio_warning_test /x86_64/mmu_role_test diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests= /kvm/Makefile index f7fa5655e535..69bec991fb12 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -51,6 +51,7 @@ TEST_GEN_PROGS_x86_64 +=3D x86_64/emulator_error_test TEST_GEN_PROGS_x86_64 +=3D x86_64/hyperv_clock TEST_GEN_PROGS_x86_64 +=3D x86_64/hyperv_cpuid TEST_GEN_PROGS_x86_64 +=3D x86_64/hyperv_features +TEST_GEN_PROGS_x86_64 +=3D x86_64/hyperv_ipi TEST_GEN_PROGS_x86_64 +=3D x86_64/hyperv_svm_test TEST_GEN_PROGS_x86_64 +=3D x86_64/kvm_clock_test TEST_GEN_PROGS_x86_64 +=3D x86_64/kvm_pv_test diff --git a/tools/testing/selftests/kvm/include/x86_64/hyperv.h b/tools/te= sting/selftests/kvm/include/x86_64/hyperv.h index b66910702c0a..f51d6fab8e93 100644 --- a/tools/testing/selftests/kvm/include/x86_64/hyperv.h +++ b/tools/testing/selftests/kvm/include/x86_64/hyperv.h @@ -184,5 +184,8 @@ =20 /* hypercall options */ #define HV_HYPERCALL_FAST_BIT BIT(16) +#define HV_HYPERCALL_VARHEAD_OFFSET 17 + +#define HYPERV_LINUX_OS_ID ((u64)0x8100 << 48) =20 #endif /* !SELFTEST_KVM_HYPERV_H */ diff --git a/tools/testing/selftests/kvm/x86_64/hyperv_features.c b/tools/t= esting/selftests/kvm/x86_64/hyperv_features.c index 672915ce73d8..98c020356925 100644 --- a/tools/testing/selftests/kvm/x86_64/hyperv_features.c +++ b/tools/testing/selftests/kvm/x86_64/hyperv_features.c @@ -14,7 +14,6 @@ #include "hyperv.h" =20 #define VCPU_ID 0 -#define LINUX_OS_ID ((u64)0x8100 << 48) =20 extern unsigned char rdmsr_start; extern unsigned char rdmsr_end; @@ -127,7 +126,7 @@ static void guest_hcall(vm_vaddr_t pgs_gpa, struct hcal= l_data *hcall) int i =3D 0; u64 res, input, output; =20 - wrmsr(HV_X64_MSR_GUEST_OS_ID, LINUX_OS_ID); + wrmsr(HV_X64_MSR_GUEST_OS_ID, HYPERV_LINUX_OS_ID); wrmsr(HV_X64_MSR_HYPERCALL, pgs_gpa); =20 while (hcall->control) { @@ -230,7 +229,7 @@ static void guest_test_msrs_access(void) */ msr->idx =3D HV_X64_MSR_GUEST_OS_ID; msr->write =3D 1; - msr->write_val =3D LINUX_OS_ID; + msr->write_val =3D HYPERV_LINUX_OS_ID; msr->available =3D 1; break; case 3: diff --git a/tools/testing/selftests/kvm/x86_64/hyperv_ipi.c b/tools/testin= g/selftests/kvm/x86_64/hyperv_ipi.c new file mode 100644 index 000000000000..6c697fe7eca4 --- /dev/null +++ b/tools/testing/selftests/kvm/x86_64/hyperv_ipi.c @@ -0,0 +1,362 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Hyper-V HvCallSendSyntheticClusterIpi{,Ex} tests + * + * Copyright (C) 2022, Red Hat, Inc. + * + */ + +#define _GNU_SOURCE /* for program_invocation_short_name */ +#include +#include + +#include "kvm_util.h" +#include "hyperv.h" +#include "processor.h" +#include "test_util.h" +#include "vmx.h" + +#define SENDER_VCPU_ID 1 +#define RECEIVER_VCPU_ID_1 2 +#define RECEIVER_VCPU_ID_2 65 + +#define IPI_VECTOR 0xfe + +static volatile uint64_t ipis_rcvd[RECEIVER_VCPU_ID_2 + 1]; + +struct thread_params { + struct kvm_vm *vm; + uint32_t vcpu_id; +}; + +struct hv_vpset { + u64 format; + u64 valid_bank_mask; + u64 bank_contents[2]; +}; + +enum HV_GENERIC_SET_FORMAT { + HV_GENERIC_SET_SPARSE_4K, + HV_GENERIC_SET_ALL, +}; + +/* HvCallSendSyntheticClusterIpi hypercall */ +struct hv_send_ipi { + u32 vector; + u32 reserved; + u64 cpu_mask; +}; + +/* HvCallSendSyntheticClusterIpiEx hypercall */ +struct hv_send_ipi_ex { + u32 vector; + u32 reserved; + struct hv_vpset vp_set; +}; + +static inline void hv_init(vm_vaddr_t pgs_gpa) +{ + wrmsr(HV_X64_MSR_GUEST_OS_ID, HYPERV_LINUX_OS_ID); + wrmsr(HV_X64_MSR_HYPERCALL, pgs_gpa); +} + +static void receiver_code(void *hcall_page, vm_vaddr_t pgs_gpa) +{ + x2apic_enable(); + hv_init(pgs_gpa); + + for (;;) + asm volatile("sti; hlt; cli"); +} + +static void guest_ipi_handler(struct ex_regs *regs) +{ + u32 vcpu_id =3D rdmsr(HV_X64_MSR_VP_INDEX); + + ipis_rcvd[vcpu_id]++; + wrmsr(HV_X64_MSR_EOI, 1); +} + +static inline u64 hypercall(u64 control, vm_vaddr_t arg1, vm_vaddr_t arg2) +{ + u64 hv_status; + + asm volatile("mov %3, %%r8\n" + "vmcall" + : "=3Da" (hv_status), + "+c" (control), "+d" (arg1) + : "r" (arg2) + : "cc", "memory", "r8", "r9", "r10", "r11"); + + return hv_status; +} + +static inline void nop_loop(void) +{ + int i; + + for (i =3D 0; i < 100000000; i++) + asm volatile("nop"); +} + +static inline void sync_to_xmm(void *data) +{ + int i; + + for (i =3D 0; i < 8; i++) + write_sse_reg(i, (sse128_t *)(data + sizeof(sse128_t) * i)); +} + +static void sender_guest_code(void *hcall_page, vm_vaddr_t pgs_gpa) +{ + struct hv_send_ipi *ipi =3D (struct hv_send_ipi *)hcall_page; + struct hv_send_ipi_ex *ipi_ex =3D (struct hv_send_ipi_ex *)hcall_page; + int stage =3D 1, ipis_expected[2] =3D {0}; + u64 res; + + hv_init(pgs_gpa); + GUEST_SYNC(stage++); + + /* 'Slow' HvCallSendSyntheticClusterIpi to RECEIVER_VCPU_ID_1 */ + ipi->vector =3D IPI_VECTOR; + ipi->cpu_mask =3D 1 << RECEIVER_VCPU_ID_1; + res =3D hypercall(HVCALL_SEND_IPI, pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + nop_loop(); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] =3D=3D ++ipis_expected[0]); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] =3D=3D ipis_expected[1]); + GUEST_SYNC(stage++); + /* 'Fast' HvCallSendSyntheticClusterIpi to RECEIVER_VCPU_ID_1 */ + res =3D hypercall(HVCALL_SEND_IPI | HV_HYPERCALL_FAST_BIT, + IPI_VECTOR, 1 << RECEIVER_VCPU_ID_1); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + nop_loop(); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] =3D=3D ++ipis_expected[0]); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] =3D=3D ipis_expected[1]); + GUEST_SYNC(stage++); + + /* 'Slow' HvCallSendSyntheticClusterIpiEx to RECEIVER_VCPU_ID_1 */ + memset(hcall_page, 0, 4096); + ipi_ex->vector =3D IPI_VECTOR; + ipi_ex->vp_set.format =3D HV_GENERIC_SET_SPARSE_4K; + ipi_ex->vp_set.valid_bank_mask =3D 1 << 0; + ipi_ex->vp_set.bank_contents[0] =3D BIT(RECEIVER_VCPU_ID_1); + res =3D hypercall(HVCALL_SEND_IPI_EX | (1 << HV_HYPERCALL_VARHEAD_OFFSET), + pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + nop_loop(); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] =3D=3D ++ipis_expected[0]); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] =3D=3D ipis_expected[1]); + GUEST_SYNC(stage++); + /* 'XMM Fast' HvCallSendSyntheticClusterIpiEx to RECEIVER_VCPU_ID_1 */ + sync_to_xmm(&ipi_ex->vp_set.valid_bank_mask); + res =3D hypercall(HVCALL_SEND_IPI_EX | HV_HYPERCALL_FAST_BIT | + (1 << HV_HYPERCALL_VARHEAD_OFFSET), + IPI_VECTOR, HV_GENERIC_SET_SPARSE_4K); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + nop_loop(); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] =3D=3D ++ipis_expected[0]); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] =3D=3D ipis_expected[1]); + GUEST_SYNC(stage++); + + /* 'Slow' HvCallSendSyntheticClusterIpiEx to RECEIVER_VCPU_ID_2 */ + memset(hcall_page, 0, 4096); + ipi_ex->vector =3D IPI_VECTOR; + ipi_ex->vp_set.format =3D HV_GENERIC_SET_SPARSE_4K; + ipi_ex->vp_set.valid_bank_mask =3D 1 << 1; + ipi_ex->vp_set.bank_contents[0] =3D BIT(RECEIVER_VCPU_ID_2 - 64); + res =3D hypercall(HVCALL_SEND_IPI_EX | (1 << HV_HYPERCALL_VARHEAD_OFFSET), + pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + nop_loop(); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] =3D=3D ipis_expected[0]); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] =3D=3D ++ipis_expected[1]); + GUEST_SYNC(stage++); + /* 'XMM Fast' HvCallSendSyntheticClusterIpiEx to RECEIVER_VCPU_ID_2 */ + sync_to_xmm(&ipi_ex->vp_set.valid_bank_mask); + res =3D hypercall(HVCALL_SEND_IPI_EX | HV_HYPERCALL_FAST_BIT | + (1 << HV_HYPERCALL_VARHEAD_OFFSET), + IPI_VECTOR, HV_GENERIC_SET_SPARSE_4K); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + nop_loop(); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] =3D=3D ipis_expected[0]); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] =3D=3D ++ipis_expected[1]); + GUEST_SYNC(stage++); + + /* 'Slow' HvCallSendSyntheticClusterIpiEx to both RECEIVER_VCPU_ID_{1,2} = */ + memset(hcall_page, 0, 4096); + ipi_ex->vector =3D IPI_VECTOR; + ipi_ex->vp_set.format =3D HV_GENERIC_SET_SPARSE_4K; + ipi_ex->vp_set.valid_bank_mask =3D 1 << 1 | 1; + ipi_ex->vp_set.bank_contents[0] =3D BIT(RECEIVER_VCPU_ID_1); + ipi_ex->vp_set.bank_contents[1] =3D BIT(RECEIVER_VCPU_ID_2 - 64); + res =3D hypercall(HVCALL_SEND_IPI_EX | (2 << HV_HYPERCALL_VARHEAD_OFFSET), + pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + nop_loop(); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] =3D=3D ++ipis_expected[0]); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] =3D=3D ++ipis_expected[1]); + GUEST_SYNC(stage++); + /* 'XMM Fast' HvCallSendSyntheticClusterIpiEx to both RECEIVER_VCPU_ID_{1= , 2} */ + sync_to_xmm(&ipi_ex->vp_set.valid_bank_mask); + res =3D hypercall(HVCALL_SEND_IPI_EX | HV_HYPERCALL_FAST_BIT | + (2 << HV_HYPERCALL_VARHEAD_OFFSET), + IPI_VECTOR, HV_GENERIC_SET_SPARSE_4K); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + nop_loop(); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] =3D=3D ++ipis_expected[0]); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] =3D=3D ++ipis_expected[1]); + GUEST_SYNC(stage++); + + /* 'Slow' HvCallSendSyntheticClusterIpiEx to HV_GENERIC_SET_ALL */ + memset(hcall_page, 0, 4096); + ipi_ex->vector =3D IPI_VECTOR; + ipi_ex->vp_set.format =3D HV_GENERIC_SET_ALL; + res =3D hypercall(HVCALL_SEND_IPI_EX, + pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + nop_loop(); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] =3D=3D ++ipis_expected[0]); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] =3D=3D ++ipis_expected[1]); + GUEST_SYNC(stage++); + /* 'XMM Fast' HvCallSendSyntheticClusterIpiEx to HV_GENERIC_SET_ALL */ + sync_to_xmm(&ipi_ex->vp_set.valid_bank_mask); + res =3D hypercall(HVCALL_SEND_IPI_EX | HV_HYPERCALL_FAST_BIT, + IPI_VECTOR, HV_GENERIC_SET_ALL); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + nop_loop(); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] =3D=3D ++ipis_expected[0]); + GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] =3D=3D ++ipis_expected[1]); + GUEST_SYNC(stage++); + + GUEST_DONE(); +} + +static void *vcpu_thread(void *arg) +{ + struct thread_params *params =3D (struct thread_params *)arg; + struct ucall uc; + int old; + int r; + unsigned int exit_reason; + + r =3D pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, &old); + TEST_ASSERT(r =3D=3D 0, + "pthread_setcanceltype failed on vcpu_id=3D%u with errno=3D%d", + params->vcpu_id, r); + + vcpu_run(params->vm, params->vcpu_id); + exit_reason =3D vcpu_state(params->vm, params->vcpu_id)->exit_reason; + + TEST_ASSERT(exit_reason =3D=3D KVM_EXIT_IO, + "vCPU %u exited with unexpected exit reason %u-%s, expected KVM_EXIT= _IO", + params->vcpu_id, exit_reason, exit_reason_str(exit_reason)); + + if (get_ucall(params->vm, params->vcpu_id, &uc) =3D=3D UCALL_ABORT) { + TEST_ASSERT(false, + "vCPU %u exited with error: %s.\n", + params->vcpu_id, (const char *)uc.args[0]); + } + + return NULL; +} + +static void cancel_join_vcpu_thread(pthread_t thread, uint32_t vcpu_id) +{ + void *retval; + int r; + + r =3D pthread_cancel(thread); + TEST_ASSERT(r =3D=3D 0, + "pthread_cancel on vcpu_id=3D%d failed with errno=3D%d", + vcpu_id, r); + + r =3D pthread_join(thread, &retval); + TEST_ASSERT(r =3D=3D 0, + "pthread_join on vcpu_id=3D%d failed with errno=3D%d", + vcpu_id, r); + TEST_ASSERT(retval =3D=3D PTHREAD_CANCELED, + "expected retval=3D%p, got %p", PTHREAD_CANCELED, + retval); +} + +int main(int argc, char *argv[]) +{ + int r; + pthread_t threads[2]; + struct thread_params params[2]; + struct kvm_vm *vm; + struct kvm_run *run; + vm_vaddr_t hcall_page; + struct ucall uc; + int stage =3D 1; + + vm =3D vm_create_default(SENDER_VCPU_ID, 0, sender_guest_code); + params[0].vm =3D vm; + params[1].vm =3D vm; + + /* Hypercall input/output */ + hcall_page =3D vm_vaddr_alloc_pages(vm, 2); + memset(addr_gva2hva(vm, hcall_page), 0x0, 2 * getpagesize()); + + vm_init_descriptor_tables(vm); + + vm_vcpu_add_default(vm, RECEIVER_VCPU_ID_1, receiver_code); + vcpu_init_descriptor_tables(vm, RECEIVER_VCPU_ID_1); + vcpu_args_set(vm, RECEIVER_VCPU_ID_1, 2, hcall_page, addr_gva2gpa(vm, hca= ll_page)); + vcpu_set_msr(vm, RECEIVER_VCPU_ID_1, HV_X64_MSR_VP_INDEX, RECEIVER_VCPU_I= D_1); + vcpu_set_hv_cpuid(vm, RECEIVER_VCPU_ID_1); + + vm_vcpu_add_default(vm, RECEIVER_VCPU_ID_2, receiver_code); + vcpu_init_descriptor_tables(vm, RECEIVER_VCPU_ID_2); + vcpu_args_set(vm, RECEIVER_VCPU_ID_2, 2, hcall_page, addr_gva2gpa(vm, hca= ll_page)); + vcpu_set_msr(vm, RECEIVER_VCPU_ID_2, HV_X64_MSR_VP_INDEX, RECEIVER_VCPU_I= D_2); + vcpu_set_hv_cpuid(vm, RECEIVER_VCPU_ID_2); + + vm_install_exception_handler(vm, IPI_VECTOR, guest_ipi_handler); + + vcpu_args_set(vm, SENDER_VCPU_ID, 2, hcall_page, addr_gva2gpa(vm, hcall_p= age)); + vcpu_set_hv_cpuid(vm, SENDER_VCPU_ID); + + params[0].vcpu_id =3D RECEIVER_VCPU_ID_1; + r =3D pthread_create(&threads[0], NULL, vcpu_thread, ¶ms[0]); + TEST_ASSERT(r =3D=3D 0, + "pthread_create halter failed errno=3D%d", errno); + + params[1].vcpu_id =3D RECEIVER_VCPU_ID_2; + r =3D pthread_create(&threads[1], NULL, vcpu_thread, ¶ms[1]); + TEST_ASSERT(r =3D=3D 0, + "pthread_create halter failed errno=3D%d", errno); + + run =3D vcpu_state(vm, SENDER_VCPU_ID); + + while (true) { + r =3D _vcpu_run(vm, SENDER_VCPU_ID); + TEST_ASSERT(!r, "vcpu_run failed: %d\n", r); + TEST_ASSERT(run->exit_reason =3D=3D KVM_EXIT_IO, + "unexpected exit reason: %u (%s)", + run->exit_reason, exit_reason_str(run->exit_reason)); + + switch (get_ucall(vm, SENDER_VCPU_ID, &uc)) { + case UCALL_SYNC: + TEST_ASSERT(uc.args[1] =3D=3D stage, + "Unexpected stage: %ld (%d expected)\n", + uc.args[1], stage); + break; + case UCALL_ABORT: + TEST_FAIL("%s at %s:%ld", (const char *)uc.args[0], + __FILE__, uc.args[1]); + return 1; + case UCALL_DONE: + return 0; + } + + stage++; + } + + cancel_join_vcpu_thread(threads[0], RECEIVER_VCPU_ID_1); + cancel_join_vcpu_thread(threads[1], RECEIVER_VCPU_ID_2); + kvm_vm_free(vm); + + return 0; +} --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 954B2C433FE for ; Fri, 11 Mar 2022 15:51:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349917AbiCKPwr (ORCPT ); Fri, 11 Mar 2022 10:52:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58178 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349954AbiCKPwi (ORCPT ); Fri, 11 Mar 2022 10:52:38 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 92F9D1CDDCE for ; Fri, 11 Mar 2022 07:51:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013861; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LgKxZ+60OBmBTv0uCZcgzTAGPMwScB8wIZWfQnzZcjo=; b=KcwjS+8MJDrYoQsNbi+ppVkAAejXRZZ6h8CXWchUXhGTwofjSubvfLEV2XOVfTGEHZ8vgl RsjTSslWqojv2eqMaU/aPP1LJ70Zx7BfH7DZ5lzwJtOMwImPMgdoIS0XpM6v63R/Mq4f0z QZ80WLeoNMU7dHNP/Vn2ohoaz7M2GJw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-492-2ueYG3m-ODO_eMOnBPB-TA-1; Fri, 11 Mar 2022 10:50:56 -0500 X-MC-Unique: 2ueYG3m-ODO_eMOnBPB-TA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C51ED1006AA6; Fri, 11 Mar 2022 15:50:54 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 53F1A866D4; Fri, 11 Mar 2022 15:50:52 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 23/31] KVM: selftests: Make it possible to replace PTEs with __virt_pg_map() Date: Fri, 11 Mar 2022 16:49:35 +0100 Message-Id: <20220311154943.2299191-24-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" __virt_pg_map() makes an assumption that leaf PTE is not present. This is not suitable if the test wants to replace an already present PTE. Hyper-V PV TLB flush test is going to need that. Signed-off-by: Vitaly Kuznetsov --- tools/testing/selftests/kvm/include/x86_64/processor.h | 2 +- tools/testing/selftests/kvm/lib/x86_64/processor.c | 6 +++--- tools/testing/selftests/kvm/x86_64/mmu_role_test.c | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools= /testing/selftests/kvm/include/x86_64/processor.h index 4962cd0ceacd..1a3550c99eb1 100644 --- a/tools/testing/selftests/kvm/include/x86_64/processor.h +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h @@ -468,7 +468,7 @@ enum x86_page_size { X86_PAGE_SIZE_1G, }; void __virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr, - enum x86_page_size page_size); + enum x86_page_size page_size, bool replace); =20 /* * Basic CPU control in CR0 diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/tes= ting/selftests/kvm/lib/x86_64/processor.c index 9f000dfb5594..20df3e84d777 100644 --- a/tools/testing/selftests/kvm/lib/x86_64/processor.c +++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c @@ -229,7 +229,7 @@ static struct pageUpperEntry *virt_create_upper_pte(str= uct kvm_vm *vm, } =20 void __virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr, - enum x86_page_size page_size) + enum x86_page_size page_size, bool replace) { const uint64_t pg_size =3D 1ull << ((page_size * 9) + 12); struct pageUpperEntry *pml4e, *pdpe, *pde; @@ -270,7 +270,7 @@ void __virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, u= int64_t paddr, =20 /* Fill in page table entry. */ pte =3D virt_get_pte(vm, pde->pfn, vaddr, 0); - TEST_ASSERT(!pte->present, + TEST_ASSERT(replace || !pte->present, "PTE already present for 4k page at vaddr: 0x%lx\n", vaddr); pte->pfn =3D paddr >> vm->page_shift; pte->writable =3D true; @@ -279,7 +279,7 @@ void __virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, u= int64_t paddr, =20 void virt_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr) { - __virt_pg_map(vm, vaddr, paddr, X86_PAGE_SIZE_4K); + __virt_pg_map(vm, vaddr, paddr, X86_PAGE_SIZE_4K, false); } =20 static struct pageTableEntry *_vm_get_page_table_entry(struct kvm_vm *vm, = int vcpuid, diff --git a/tools/testing/selftests/kvm/x86_64/mmu_role_test.c b/tools/tes= ting/selftests/kvm/x86_64/mmu_role_test.c index da2325fcad87..e3fdf320b9f4 100644 --- a/tools/testing/selftests/kvm/x86_64/mmu_role_test.c +++ b/tools/testing/selftests/kvm/x86_64/mmu_role_test.c @@ -35,7 +35,7 @@ static void mmu_role_test(u32 *cpuid_reg, u32 evil_cpuid_= val) run =3D vcpu_state(vm, VCPU_ID); =20 /* Map 1gb page without a backing memlot. */ - __virt_pg_map(vm, MMIO_GPA, MMIO_GPA, X86_PAGE_SIZE_1G); + __virt_pg_map(vm, MMIO_GPA, MMIO_GPA, X86_PAGE_SIZE_1G, false); =20 r =3D _vcpu_run(vm, VCPU_ID); =20 --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E44B9C433F5 for ; Fri, 11 Mar 2022 15:53:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349981AbiCKPyn (ORCPT ); Fri, 11 Mar 2022 10:54:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349929AbiCKPwl (ORCPT ); Fri, 11 Mar 2022 10:52:41 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 0A5851CF09A for ; Fri, 11 Mar 2022 07:51:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013862; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DIHS3kRygsNVnv/+QoF/sy3dTTlLzRbYXkv0d81jBmc=; b=btNzy+cbA5FQMBRsirz94UT2FoCQaS6vpzgDTFN/SPef15uignyPvDVLY++3ZVHZl9Fzgj DB4tcjuGh92nnQTGQBiUrDlzKGkBkme4lc9RPVpqZ/yPu3bWggfG3jYJp+4mb6WkoTEnyy /CJbZUYocQ7t0l/FKYKT/WcQO3wblp8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-611-z0LY-ub6Pv6NB3PF8-szfA-1; Fri, 11 Mar 2022 10:50:59 -0500 X-MC-Unique: z0LY-ub6Pv6NB3PF8-szfA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 15C0551DC; Fri, 11 Mar 2022 15:50:58 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 30509866D4; Fri, 11 Mar 2022 15:50:55 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 24/31] KVM: selftests: Hyper-V PV TLB flush selftest Date: Fri, 11 Mar 2022 16:49:36 +0100 Message-Id: <20220311154943.2299191-25-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Introduce a selftest for Hyper-V PV TLB flush hypercalls (HvFlushVirtualAddressSpace/HvFlushVirtualAddressSpaceEx, HvFlushVirtualAddressList/HvFlushVirtualAddressListEx). The test creates one 'sender' vCPU and two 'worker' vCPU which do busy loop reading from a certain GVA checking the observed value. Sender vCPU drops to the host to swap the data page with another page filled with a different value. The expectation for workers is also altered. Without TLB flush on worker vCPUs, they may continue to observe old value. To guard against accidental TLB flushes for worker vCPUs the test is repeated 100 times. Hyper-V TLB flush hypercalls are tested in both 'normal' and 'XMM fast' modes. Signed-off-by: Vitaly Kuznetsov --- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/x86_64/hyperv.h | 1 + .../selftests/kvm/x86_64/hyperv_tlb_flush.c | 647 ++++++++++++++++++ 4 files changed, 650 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftes= ts/kvm/.gitignore index 8ef4b4571b23..f0f7fba0acd1 100644 --- a/tools/testing/selftests/kvm/.gitignore +++ b/tools/testing/selftests/kvm/.gitignore @@ -23,6 +23,7 @@ /x86_64/hyperv_features /x86_64/hyperv_ipi /x86_64/hyperv_svm_test +/x86_64/hyperv_tlb_flush /x86_64/mmio_warning_test /x86_64/mmu_role_test /x86_64/platform_info_test diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests= /kvm/Makefile index 69bec991fb12..bd878effa20d 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -53,6 +53,7 @@ TEST_GEN_PROGS_x86_64 +=3D x86_64/hyperv_cpuid TEST_GEN_PROGS_x86_64 +=3D x86_64/hyperv_features TEST_GEN_PROGS_x86_64 +=3D x86_64/hyperv_ipi TEST_GEN_PROGS_x86_64 +=3D x86_64/hyperv_svm_test +TEST_GEN_PROGS_x86_64 +=3D x86_64/hyperv_tlb_flush TEST_GEN_PROGS_x86_64 +=3D x86_64/kvm_clock_test TEST_GEN_PROGS_x86_64 +=3D x86_64/kvm_pv_test TEST_GEN_PROGS_x86_64 +=3D x86_64/mmio_warning_test diff --git a/tools/testing/selftests/kvm/include/x86_64/hyperv.h b/tools/te= sting/selftests/kvm/include/x86_64/hyperv.h index f51d6fab8e93..1e34dd7c5075 100644 --- a/tools/testing/selftests/kvm/include/x86_64/hyperv.h +++ b/tools/testing/selftests/kvm/include/x86_64/hyperv.h @@ -185,6 +185,7 @@ /* hypercall options */ #define HV_HYPERCALL_FAST_BIT BIT(16) #define HV_HYPERCALL_VARHEAD_OFFSET 17 +#define HV_HYPERCALL_REP_COMP_OFFSET 32 =20 #define HYPERV_LINUX_OS_ID ((u64)0x8100 << 48) =20 diff --git a/tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c b/tools/= testing/selftests/kvm/x86_64/hyperv_tlb_flush.c new file mode 100644 index 000000000000..00bcae45ddd2 --- /dev/null +++ b/tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c @@ -0,0 +1,647 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Hyper-V HvFlushVirtualAddress{List,Space}{,Ex} tests + * + * Copyright (C) 2022, Red Hat, Inc. + * + */ + +#define _GNU_SOURCE /* for program_invocation_short_name */ +#include +#include + +#include "kvm_util.h" +#include "hyperv.h" +#include "processor.h" +#include "test_util.h" +#include "vmx.h" + +#define SENDER_VCPU_ID 1 +#define WORKER_VCPU_ID_1 2 +#define WORKER_VCPU_ID_2 65 + +#define NTRY 100 + +struct thread_params { + struct kvm_vm *vm; + uint32_t vcpu_id; +}; + +struct hv_vpset { + u64 format; + u64 valid_bank_mask; + u64 bank_contents[]; +}; + +enum HV_GENERIC_SET_FORMAT { + HV_GENERIC_SET_SPARSE_4K, + HV_GENERIC_SET_ALL, +}; + +#define HV_FLUSH_ALL_PROCESSORS BIT(0) +#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES BIT(1) +#define HV_FLUSH_NON_GLOBAL_MAPPINGS_ONLY BIT(2) +#define HV_FLUSH_USE_EXTENDED_RANGE_FORMAT BIT(3) + +/* HvFlushVirtualAddressSpace, HvFlushVirtualAddressList hypercalls */ +struct hv_tlb_flush { + u64 address_space; + u64 flags; + u64 processor_mask; + u64 gva_list[]; +} __packed; + +/* HvFlushVirtualAddressSpaceEx, HvFlushVirtualAddressListEx hypercalls */ +struct hv_tlb_flush_ex { + u64 address_space; + u64 flags; + struct hv_vpset hv_vp_set; + u64 gva_list[]; +} __packed; + +static inline void hv_init(vm_vaddr_t pgs_gpa) +{ + wrmsr(HV_X64_MSR_GUEST_OS_ID, HYPERV_LINUX_OS_ID); + wrmsr(HV_X64_MSR_HYPERCALL, pgs_gpa); +} + +static void worker_code(void *test_pages, vm_vaddr_t pgs_gpa) +{ + u32 vcpu_id =3D rdmsr(HV_X64_MSR_VP_INDEX); + unsigned char chr; + + x2apic_enable(); + hv_init(pgs_gpa); + + for (;;) { + chr =3D READ_ONCE(*(unsigned char *)(test_pages + 4096 * 2 + vcpu_id)); + if (chr) + GUEST_ASSERT(*(unsigned char *)test_pages =3D=3D chr); + asm volatile("nop"); + } +} + +static inline u64 hypercall(u64 control, vm_vaddr_t arg1, vm_vaddr_t arg2) +{ + u64 hv_status; + + asm volatile("mov %3, %%r8\n" + "vmcall" + : "=3Da" (hv_status), + "+c" (control), "+d" (arg1) + : "r" (arg2) + : "cc", "memory", "r8", "r9", "r10", "r11"); + + return hv_status; +} + +static inline void nop_loop(void) +{ + int i; + + for (i =3D 0; i < 10000000; i++) + asm volatile("nop"); +} + +static inline void sync_to_xmm(void *data) +{ + int i; + + for (i =3D 0; i < 8; i++) + write_sse_reg(i, (sse128_t *)(data + sizeof(sse128_t) * i)); +} + +static void set_expected_char(void *addr, unsigned char chr, int vcpu_id) +{ + asm volatile("mfence"); + *(unsigned char *)(addr + 2 * 4096 + vcpu_id) =3D chr; +} + +static void sender_guest_code(void *hcall_page, void *test_pages, vm_vaddr= _t pgs_gpa) +{ + struct hv_tlb_flush *flush =3D (struct hv_tlb_flush *)hcall_page; + struct hv_tlb_flush_ex *flush_ex =3D (struct hv_tlb_flush_ex *)hcall_page; + int stage =3D 1, i; + u64 res; + + hv_init(pgs_gpa); + + /* "Slow" hypercalls */ + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE for WORKER_VCPU_ID_1 */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES; + flush->processor_mask =3D BIT(WORKER_VCPU_ID_1); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE, pgs_gpa, pgs_gpa += 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST for WORKER_VCPU_ID_1 */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES; + flush->processor_mask =3D BIT(WORKER_VCPU_ID_1); + flush->gva_list[0] =3D (u64)test_pages; + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST | + (1UL << HV_HYPERCALL_REP_COMP_OFFSET), + pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE for HV_FLUSH_ALL_PROCESSORS */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROC= ESSORS; + flush->processor_mask =3D 0; + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE, pgs_gpa, pgs_gpa += 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST for HV_FLUSH_ALL_PROCESSORS */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROC= ESSORS; + flush->gva_list[0] =3D (u64)test_pages; + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST | + (1UL << HV_HYPERCALL_REP_COMP_OFFSET), + pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX for WORKER_VCPU_ID_2 */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush_ex->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES; + flush_ex->hv_vp_set.format =3D HV_GENERIC_SET_SPARSE_4K; + flush_ex->hv_vp_set.valid_bank_mask =3D BIT_ULL(WORKER_VCPU_ID_2 / 64); + flush_ex->hv_vp_set.bank_contents[0] =3D BIT_ULL(WORKER_VCPU_ID_2 % 64); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX | + (1 << HV_HYPERCALL_VARHEAD_OFFSET), + pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX for WORKER_VCPU_ID_2 */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush_ex->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES; + flush_ex->hv_vp_set.format =3D HV_GENERIC_SET_SPARSE_4K; + flush_ex->hv_vp_set.valid_bank_mask =3D BIT_ULL(WORKER_VCPU_ID_2 / 64); + flush_ex->hv_vp_set.bank_contents[0] =3D BIT_ULL(WORKER_VCPU_ID_2 % 64); + /* bank_contents and gva_list occupy the same space, thus [1] */ + flush_ex->gva_list[1] =3D (u64)test_pages; + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX | + (1 << HV_HYPERCALL_VARHEAD_OFFSET) | + (1UL << HV_HYPERCALL_REP_COMP_OFFSET), + pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX for both vCPUs */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush_ex->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES; + flush_ex->hv_vp_set.format =3D HV_GENERIC_SET_SPARSE_4K; + flush_ex->hv_vp_set.valid_bank_mask =3D BIT_ULL(WORKER_VCPU_ID_2 / 64) | + BIT_ULL(WORKER_VCPU_ID_1 / 64); + flush_ex->hv_vp_set.bank_contents[0] =3D BIT_ULL(WORKER_VCPU_ID_1 % 64); + flush_ex->hv_vp_set.bank_contents[1] =3D BIT_ULL(WORKER_VCPU_ID_2 % 64); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX | + (2 << HV_HYPERCALL_VARHEAD_OFFSET), + pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX for both vCPUs */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush_ex->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES; + flush_ex->hv_vp_set.format =3D HV_GENERIC_SET_SPARSE_4K; + flush_ex->hv_vp_set.valid_bank_mask =3D BIT_ULL(WORKER_VCPU_ID_1 / 64) | + BIT_ULL(WORKER_VCPU_ID_2 / 64); + flush_ex->hv_vp_set.bank_contents[0] =3D BIT_ULL(WORKER_VCPU_ID_1 % 64); + flush_ex->hv_vp_set.bank_contents[1] =3D BIT_ULL(WORKER_VCPU_ID_2 % 64); + /* bank_contents and gva_list occupy the same space, thus [2] */ + flush_ex->gva_list[2] =3D (u64)test_pages; + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX | + (2 << HV_HYPERCALL_VARHEAD_OFFSET) | + (1UL << HV_HYPERCALL_REP_COMP_OFFSET), + pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX for HV_GENERIC_SET_ALL */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush_ex->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES; + flush_ex->hv_vp_set.format =3D HV_GENERIC_SET_ALL; + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX, + pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX for HV_GENERIC_SET_ALL */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush_ex->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES; + flush_ex->hv_vp_set.format =3D HV_GENERIC_SET_ALL; + flush_ex->gva_list[0] =3D (u64)test_pages; + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX | + (1UL << HV_HYPERCALL_REP_COMP_OFFSET), + pgs_gpa, pgs_gpa + 4096); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* "Fast" hypercalls */ + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE for WORKER_VCPU_ID_1 */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush->processor_mask =3D BIT(WORKER_VCPU_ID_1); + sync_to_xmm(&flush->processor_mask); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | + HV_HYPERCALL_FAST_BIT, 0x0, HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST for WORKER_VCPU_ID_1 */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush->processor_mask =3D BIT(WORKER_VCPU_ID_1); + flush->gva_list[0] =3D (u64)test_pages; + sync_to_xmm(&flush->processor_mask); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST | HV_HYPERCALL_FAST_= BIT | + (1UL << HV_HYPERCALL_REP_COMP_OFFSET), + 0x0, HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE for HV_FLUSH_ALL_PROCESSORS */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + sync_to_xmm(&flush->processor_mask); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | HV_HYPERCALL_FAST= _BIT, 0x0, + HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROCESSORS); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST for HV_FLUSH_ALL_PROCESSORS */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush->gva_list[0] =3D (u64)test_pages; + sync_to_xmm(&flush->processor_mask); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST | HV_HYPERCALL_FAST_= BIT | + (1UL << HV_HYPERCALL_REP_COMP_OFFSET), 0x0, + HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROCESSORS); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX for WORKER_VCPU_ID_2 */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush_ex->hv_vp_set.format =3D HV_GENERIC_SET_SPARSE_4K; + flush_ex->hv_vp_set.valid_bank_mask =3D BIT_ULL(WORKER_VCPU_ID_2 / 64); + flush_ex->hv_vp_set.bank_contents[0] =3D BIT_ULL(WORKER_VCPU_ID_2 % 64); + sync_to_xmm(&flush_ex->hv_vp_set); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX | HV_HYPERCALL_F= AST_BIT | + (1 << HV_HYPERCALL_VARHEAD_OFFSET), + 0x0, HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX for WORKER_VCPU_ID_2 */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush_ex->hv_vp_set.format =3D HV_GENERIC_SET_SPARSE_4K; + flush_ex->hv_vp_set.valid_bank_mask =3D BIT_ULL(WORKER_VCPU_ID_2 / 64); + flush_ex->hv_vp_set.bank_contents[0] =3D BIT_ULL(WORKER_VCPU_ID_2 % 64); + /* bank_contents and gva_list occupy the same space, thus [1] */ + flush_ex->gva_list[1] =3D (u64)test_pages; + sync_to_xmm(&flush_ex->hv_vp_set); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX | HV_HYPERCALL_FA= ST_BIT | + (1 << HV_HYPERCALL_VARHEAD_OFFSET) | + (1UL << HV_HYPERCALL_REP_COMP_OFFSET), + 0x0, HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX for both vCPUs */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush_ex->hv_vp_set.format =3D HV_GENERIC_SET_SPARSE_4K; + flush_ex->hv_vp_set.valid_bank_mask =3D BIT_ULL(WORKER_VCPU_ID_2 / 64) | + BIT_ULL(WORKER_VCPU_ID_1 / 64); + flush_ex->hv_vp_set.bank_contents[0] =3D BIT_ULL(WORKER_VCPU_ID_1 % 64); + flush_ex->hv_vp_set.bank_contents[1] =3D BIT_ULL(WORKER_VCPU_ID_2 % 64); + sync_to_xmm(&flush_ex->hv_vp_set); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX | HV_HYPERCALL_F= AST_BIT | + (2 << HV_HYPERCALL_VARHEAD_OFFSET), + 0x0, HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX for both vCPUs */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush_ex->hv_vp_set.format =3D HV_GENERIC_SET_SPARSE_4K; + flush_ex->hv_vp_set.valid_bank_mask =3D BIT_ULL(WORKER_VCPU_ID_1 / 64) | + BIT_ULL(WORKER_VCPU_ID_2 / 64); + flush_ex->hv_vp_set.bank_contents[0] =3D BIT_ULL(WORKER_VCPU_ID_1 % 64); + flush_ex->hv_vp_set.bank_contents[1] =3D BIT_ULL(WORKER_VCPU_ID_2 % 64); + /* bank_contents and gva_list occupy the same space, thus [2] */ + flush_ex->gva_list[2] =3D (u64)test_pages; + sync_to_xmm(&flush_ex->hv_vp_set); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX | HV_HYPERCALL_FA= ST_BIT | + (2 << HV_HYPERCALL_VARHEAD_OFFSET) | + (1UL << HV_HYPERCALL_REP_COMP_OFFSET), + 0x0, HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX for HV_GENERIC_SET_ALL */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush_ex->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES; + flush_ex->hv_vp_set.format =3D HV_GENERIC_SET_ALL; + sync_to_xmm(&flush_ex->hv_vp_set); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX | HV_HYPERCALL_F= AST_BIT, + 0x0, HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + /* HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX for HV_GENERIC_SET_ALL */ + for (i =3D 0; i < NTRY; i++) { + memset(hcall_page, 0, 4096); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_1); + set_expected_char(test_pages, 0x0, WORKER_VCPU_ID_2); + GUEST_SYNC(stage++); + flush_ex->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES; + flush_ex->hv_vp_set.format =3D HV_GENERIC_SET_ALL; + flush_ex->gva_list[0] =3D (u64)test_pages; + sync_to_xmm(&flush_ex->hv_vp_set); + res =3D hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX | HV_HYPERCALL_FA= ST_BIT | + (1UL << HV_HYPERCALL_REP_COMP_OFFSET), + 0x0, HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES); + GUEST_ASSERT((res & 0xffff) =3D=3D 0); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_1); + set_expected_char(test_pages, i % 2 ? 0x1 : 0x2, WORKER_VCPU_ID_2); + nop_loop(); + } + + GUEST_DONE(); +} + +static void *vcpu_thread(void *arg) +{ + struct thread_params *params =3D (struct thread_params *)arg; + struct ucall uc; + int old; + int r; + unsigned int exit_reason; + + r =3D pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, &old); + TEST_ASSERT(r =3D=3D 0, + "pthread_setcanceltype failed on vcpu_id=3D%u with errno=3D%d", + params->vcpu_id, r); + + vcpu_run(params->vm, params->vcpu_id); + exit_reason =3D vcpu_state(params->vm, params->vcpu_id)->exit_reason; + + TEST_ASSERT(exit_reason =3D=3D KVM_EXIT_IO, + "vCPU %u exited with unexpected exit reason %u-%s, expected KVM_EXIT= _IO", + params->vcpu_id, exit_reason, exit_reason_str(exit_reason)); + + if (get_ucall(params->vm, params->vcpu_id, &uc) =3D=3D UCALL_ABORT) { + TEST_ASSERT(false, + "vCPU %u exited with error: %s.\n", + params->vcpu_id, (const char *)uc.args[0]); + } + + return NULL; +} + +static void cancel_join_vcpu_thread(pthread_t thread, uint32_t vcpu_id) +{ + void *retval; + int r; + + r =3D pthread_cancel(thread); + TEST_ASSERT(r =3D=3D 0, + "pthread_cancel on vcpu_id=3D%d failed with errno=3D%d", + vcpu_id, r); + + r =3D pthread_join(thread, &retval); + TEST_ASSERT(r =3D=3D 0, + "pthread_join on vcpu_id=3D%d failed with errno=3D%d", + vcpu_id, r); + TEST_ASSERT(retval =3D=3D PTHREAD_CANCELED, + "expected retval=3D%p, got %p", PTHREAD_CANCELED, + retval); +} + +int main(int argc, char *argv[]) +{ + int r; + pthread_t threads[2]; + struct thread_params params[2]; + struct kvm_vm *vm; + struct kvm_run *run; + vm_vaddr_t hcall_page, test_pages; + struct ucall uc; + int stage =3D 1; + + vm =3D vm_create_default(SENDER_VCPU_ID, 0, sender_guest_code); + params[0].vm =3D vm; + params[1].vm =3D vm; + + /* Hypercall input/output */ + hcall_page =3D vm_vaddr_alloc_pages(vm, 2); + memset(addr_gva2hva(vm, hcall_page), 0x0, 2 * getpagesize()); + + /* + * Test pages: the first one is filled with '0x1's, the second with '0x2's + * and the test will swap their mappings. The third page keeps the indica= tion + * about the current state of mappings. + */ + test_pages =3D vm_vaddr_alloc_pages(vm, 3); + memset(addr_gva2hva(vm, test_pages), 0x1, 4096); + memset(addr_gva2hva(vm, test_pages) + 4096, 0x2, 4096); + set_expected_char(addr_gva2hva(vm, test_pages), 0x0, WORKER_VCPU_ID_1); + set_expected_char(addr_gva2hva(vm, test_pages), 0x0, WORKER_VCPU_ID_2); + + vm_vcpu_add_default(vm, WORKER_VCPU_ID_1, worker_code); + vcpu_args_set(vm, WORKER_VCPU_ID_1, 2, test_pages, addr_gva2gpa(vm, hcall= _page)); + vcpu_set_msr(vm, WORKER_VCPU_ID_1, HV_X64_MSR_VP_INDEX, WORKER_VCPU_ID_1); + vcpu_set_hv_cpuid(vm, WORKER_VCPU_ID_1); + + vm_vcpu_add_default(vm, WORKER_VCPU_ID_2, worker_code); + vcpu_args_set(vm, WORKER_VCPU_ID_2, 2, test_pages, addr_gva2gpa(vm, hcall= _page)); + vcpu_set_msr(vm, WORKER_VCPU_ID_2, HV_X64_MSR_VP_INDEX, WORKER_VCPU_ID_2); + vcpu_set_hv_cpuid(vm, WORKER_VCPU_ID_2); + + vcpu_args_set(vm, SENDER_VCPU_ID, 3, hcall_page, test_pages, + addr_gva2gpa(vm, hcall_page)); + vcpu_set_hv_cpuid(vm, SENDER_VCPU_ID); + + params[0].vcpu_id =3D WORKER_VCPU_ID_1; + r =3D pthread_create(&threads[0], NULL, vcpu_thread, ¶ms[0]); + TEST_ASSERT(r =3D=3D 0, + "pthread_create halter failed errno=3D%d", errno); + + params[1].vcpu_id =3D WORKER_VCPU_ID_2; + r =3D pthread_create(&threads[1], NULL, vcpu_thread, ¶ms[1]); + TEST_ASSERT(r =3D=3D 0, + "pthread_create halter failed errno=3D%d", errno); + + run =3D vcpu_state(vm, SENDER_VCPU_ID); + + while (true) { + r =3D _vcpu_run(vm, SENDER_VCPU_ID); + TEST_ASSERT(!r, "vcpu_run failed: %d\n", r); + TEST_ASSERT(run->exit_reason =3D=3D KVM_EXIT_IO, + "unexpected exit reason: %u (%s)", + run->exit_reason, exit_reason_str(run->exit_reason)); + + switch (get_ucall(vm, SENDER_VCPU_ID, &uc)) { + case UCALL_SYNC: + TEST_ASSERT(uc.args[1] =3D=3D stage, + "Unexpected stage: %ld (%d expected)\n", + uc.args[1], stage); + break; + case UCALL_ABORT: + TEST_FAIL("%s at %s:%ld", (const char *)uc.args[0], + __FILE__, uc.args[1]); + return 1; + case UCALL_DONE: + return 0; + } + + /* Swap test pages */ + if (stage % 2) { + __virt_pg_map(vm, test_pages, addr_gva2gpa(vm, test_pages) + 4096, + X86_PAGE_SIZE_4K, true); + __virt_pg_map(vm, test_pages + 4096, addr_gva2gpa(vm, test_pages) - 409= 6, + X86_PAGE_SIZE_4K, true); + } else { + __virt_pg_map(vm, test_pages, addr_gva2gpa(vm, test_pages) - 4096, + X86_PAGE_SIZE_4K, true); + __virt_pg_map(vm, test_pages + 4096, addr_gva2gpa(vm, test_pages) + 409= 6, + X86_PAGE_SIZE_4K, true); + } + + stage++; + } + + cancel_join_vcpu_thread(threads[0], WORKER_VCPU_ID_1); + cancel_join_vcpu_thread(threads[1], WORKER_VCPU_ID_2); + kvm_vm_free(vm); + + return 0; +} --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90CDAC433EF for ; Fri, 11 Mar 2022 15:54:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349706AbiCKPzm (ORCPT ); Fri, 11 Mar 2022 10:55:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58226 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350291AbiCKPxx (ORCPT ); Fri, 11 Mar 2022 10:53:53 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 61E281CC7EF for ; Fri, 11 Mar 2022 07:51:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013890; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I+R/eNqjb1wpUKu1EY5DiMRn7IAObJ3SaayfFkvzkX4=; b=Y8GV/hUSKVjFbQzfHmn+7j8uvBwvIwOqDJbcwNtn5NeBya2F0yF/ZBm/hhwm/tnXSQiyHJ 2IaFMKm6jOxtqsuIhS+6da/cqm2W0SzvusUVWDxFyLZiSnLfcPBQj4nuoHzgo6PGKgkTYG BgGY3uXVdChWz22IqTQpCJLhR9jd8G0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-373-nzN-DciFNh-KEfBTSZlOow-1; Fri, 11 Mar 2022 10:51:27 -0500 X-MC-Unique: nzN-DciFNh-KEfBTSZlOow-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4F39E1800D50; Fri, 11 Mar 2022 15:51:25 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6FAC5866DB; Fri, 11 Mar 2022 15:50:58 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 25/31] KVM: selftests: Sync 'struct hv_enlightened_vmcs' definition with hyperv-tlfs.h Date: Fri, 11 Mar 2022 16:49:37 +0100 Message-Id: <20220311154943.2299191-26-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" 'struct hv_enlightened_vmcs' definition in selftests is not '__packed' and so we rely on the compiler doing the right padding. This is not obvious so it seems beneficial to use the same definition as in kernel. Signed-off-by: Vitaly Kuznetsov --- tools/testing/selftests/kvm/include/x86_64/evmcs.h | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/kvm/include/x86_64/evmcs.h b/tools/tes= ting/selftests/kvm/include/x86_64/evmcs.h index cc5d14a45702..b6067b555110 100644 --- a/tools/testing/selftests/kvm/include/x86_64/evmcs.h +++ b/tools/testing/selftests/kvm/include/x86_64/evmcs.h @@ -41,6 +41,8 @@ struct hv_enlightened_vmcs { u16 host_gs_selector; u16 host_tr_selector; =20 + u16 padding16_1; + u64 host_ia32_pat; u64 host_ia32_efer; =20 @@ -159,7 +161,7 @@ struct hv_enlightened_vmcs { u64 ept_pointer; =20 u16 virtual_processor_id; - u16 padding16[3]; + u16 padding16_2[3]; =20 u64 padding64_2[5]; u64 guest_physical_address; @@ -195,15 +197,15 @@ struct hv_enlightened_vmcs { u64 guest_rip; =20 u32 hv_clean_fields; - u32 hv_padding_32; + u32 padding32_1; u32 hv_synthetic_controls; struct { u32 nested_flush_hypercall:1; u32 msr_bitmap:1; u32 reserved:30; - } hv_enlightenments_control; + } __packed hv_enlightenments_control; u32 hv_vp_id; - + u32 padding32_2; u64 hv_vm_id; u64 partition_assist_page; u64 padding64_4[4]; @@ -211,7 +213,7 @@ struct hv_enlightened_vmcs { u64 padding64_5[7]; u64 xss_exit_bitmap; u64 padding64_6[7]; -}; +} __packed; =20 #define HV_VMX_ENLIGHTENED_CLEAN_FIELD_NONE 0 #define HV_VMX_ENLIGHTENED_CLEAN_FIELD_IO_BITMAP BIT(0) --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07B66C433F5 for ; Fri, 11 Mar 2022 15:54:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345668AbiCKPzB (ORCPT ); Fri, 11 Mar 2022 10:55:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350304AbiCKPxx (ORCPT ); Fri, 11 Mar 2022 10:53:53 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 686B61CD9E1 for ; Fri, 11 Mar 2022 07:51:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013896; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bBLhChg87FRrs5MpYt/tg2Gm57CzbrV4UINPPthvRVg=; b=hLjj1fXeqicKolTyNu1Z0MkEkeUlm2oOHHuRGbnQafzMtz+8e7jXXmW/0zGBAm7o7XwDSj sYp2S5EhNu9PwVWPgWY7eWQqNDtQf0MkK42hk5qU22sDp/bv+AFHKMsma2/2+uOrDlHYZB gLiujjdvC55zwIVHj3Qo3DmFX8YzOJQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-411-mjRGmV_kOQmlD8jynA-wxQ-1; Fri, 11 Mar 2022 10:51:33 -0500 X-MC-Unique: mjRGmV_kOQmlD8jynA-wxQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 59A5C801AFC; Fri, 11 Mar 2022 15:51:31 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id AE1F3866D4; Fri, 11 Mar 2022 15:51:25 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 26/31] KVM: selftests: nVMX: Allocate Hyper-V partition assist page Date: Fri, 11 Mar 2022 16:49:38 +0100 Message-Id: <20220311154943.2299191-27-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In preparation to testing Hyper-V Direct TLB flush hypercalls, allocate so-called Partition assist page and link it to 'struct vmx_pages'. Signed-off-by: Vitaly Kuznetsov --- tools/testing/selftests/kvm/include/x86_64/vmx.h | 4 ++++ tools/testing/selftests/kvm/lib/x86_64/vmx.c | 7 +++++++ 2 files changed, 11 insertions(+) diff --git a/tools/testing/selftests/kvm/include/x86_64/vmx.h b/tools/testi= ng/selftests/kvm/include/x86_64/vmx.h index 583ceb0d1457..f99922ca8259 100644 --- a/tools/testing/selftests/kvm/include/x86_64/vmx.h +++ b/tools/testing/selftests/kvm/include/x86_64/vmx.h @@ -567,6 +567,10 @@ struct vmx_pages { uint64_t enlightened_vmcs_gpa; void *enlightened_vmcs; =20 + void *partition_assist_hva; + uint64_t partition_assist_gpa; + void *partition_assist; + void *eptp_hva; uint64_t eptp_gpa; void *eptp; diff --git a/tools/testing/selftests/kvm/lib/x86_64/vmx.c b/tools/testing/s= elftests/kvm/lib/x86_64/vmx.c index d089d8b850b5..3db21e0e1a8f 100644 --- a/tools/testing/selftests/kvm/lib/x86_64/vmx.c +++ b/tools/testing/selftests/kvm/lib/x86_64/vmx.c @@ -124,6 +124,13 @@ vcpu_alloc_vmx(struct kvm_vm *vm, vm_vaddr_t *p_vmx_gv= a) vmx->enlightened_vmcs_gpa =3D addr_gva2gpa(vm, (uintptr_t)vmx->enlightened_vmcs); =20 + /* Setup of a region of guest memory for the partition assist page. */ + vmx->partition_assist =3D (void *)vm_vaddr_alloc_page(vm); + vmx->partition_assist_hva =3D + addr_gva2hva(vm, (uintptr_t)vmx->partition_assist); + vmx->partition_assist_gpa =3D + addr_gva2gpa(vm, (uintptr_t)vmx->partition_assist); + *p_vmx_gva =3D vmx_gva; return vmx; } --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F058BC433EF for ; Fri, 11 Mar 2022 15:54:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349986AbiCKPzN (ORCPT ); Fri, 11 Mar 2022 10:55:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350333AbiCKPxz (ORCPT ); Fri, 11 Mar 2022 10:53:55 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 25E5B1CDDC5 for ; Fri, 11 Mar 2022 07:51:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013898; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=126nTNRXhHFrXixY2xtOW/3ENn+8ieWmJUoTJhv6R28=; b=QT2J5mj5wRoD1GbwJ1cZKilFI5hEtQhPSvZxRKZjqQvlXPVNiP3oKEsRFtCU7ONRT0KBia SZWGFpSmFHb9n/wk/+896/IiK1su5HqEaU2guHBw1+pxdrs6sL26jXEqtIS7v/EIb2ebpA cGqcHcJC1xclBIVJZ+XMlXson824+X0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-209-XdTkIyCrMIeIFwJYcyFaUA-1; Fri, 11 Mar 2022 10:51:35 -0500 X-MC-Unique: XdTkIyCrMIeIFwJYcyFaUA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6ADE6801AFE; Fri, 11 Mar 2022 15:51:34 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id BA657785FD; Fri, 11 Mar 2022 15:51:31 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 27/31] KVM: selftests: nSVM: Allocate Hyper-V partition assist and VP assist pages Date: Fri, 11 Mar 2022 16:49:39 +0100 Message-Id: <20220311154943.2299191-28-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In preparation to testing Hyper-V Direct TLB flush hypercalls, allocate VP assist and Partition assist pages and link them to 'struct svm_test_data'. Signed-off-by: Vitaly Kuznetsov --- tools/testing/selftests/kvm/include/x86_64/svm_util.h | 10 ++++++++++ tools/testing/selftests/kvm/lib/x86_64/svm.c | 10 ++++++++++ 2 files changed, 20 insertions(+) diff --git a/tools/testing/selftests/kvm/include/x86_64/svm_util.h b/tools/= testing/selftests/kvm/include/x86_64/svm_util.h index a25aabd8f5e7..640859b58fd6 100644 --- a/tools/testing/selftests/kvm/include/x86_64/svm_util.h +++ b/tools/testing/selftests/kvm/include/x86_64/svm_util.h @@ -34,6 +34,16 @@ struct svm_test_data { void *msr; /* gva */ void *msr_hva; uint64_t msr_gpa; + + /* Hyper-V VP assist page */ + void *vp_assist; /* gva */ + void *vp_assist_hva; + uint64_t vp_assist_gpa; + + /* Hyper-V Partition assist page */ + void *partition_assist; /* gva */ + void *partition_assist_hva; + uint64_t partition_assist_gpa; }; =20 struct svm_test_data *vcpu_alloc_svm(struct kvm_vm *vm, vm_vaddr_t *p_svm_= gva); diff --git a/tools/testing/selftests/kvm/lib/x86_64/svm.c b/tools/testing/s= elftests/kvm/lib/x86_64/svm.c index 736ee4a23df6..c284e8f87f5c 100644 --- a/tools/testing/selftests/kvm/lib/x86_64/svm.c +++ b/tools/testing/selftests/kvm/lib/x86_64/svm.c @@ -48,6 +48,16 @@ vcpu_alloc_svm(struct kvm_vm *vm, vm_vaddr_t *p_svm_gva) svm->msr_gpa =3D addr_gva2gpa(vm, (uintptr_t)svm->msr); memset(svm->msr_hva, 0, getpagesize()); =20 + svm->vp_assist =3D (void *)vm_vaddr_alloc_page(vm); + svm->vp_assist_hva =3D addr_gva2hva(vm, (uintptr_t)svm->vp_assist); + svm->vp_assist_gpa =3D addr_gva2gpa(vm, (uintptr_t)svm->vp_assist); + memset(svm->vp_assist_hva, 0, getpagesize()); + + svm->partition_assist =3D (void *)vm_vaddr_alloc_page(vm); + svm->partition_assist_hva =3D addr_gva2hva(vm, (uintptr_t)svm->partition_= assist); + svm->partition_assist_gpa =3D addr_gva2gpa(vm, (uintptr_t)svm->partition_= assist); + memset(svm->partition_assist_hva, 0, getpagesize()); + *p_svm_gva =3D svm_gva; return svm; } --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 325A9C433F5 for ; Fri, 11 Mar 2022 15:54:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235817AbiCKPzK (ORCPT ); Fri, 11 Mar 2022 10:55:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350321AbiCKPxy (ORCPT ); Fri, 11 Mar 2022 10:53:54 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id CCB761CD9F2 for ; Fri, 11 Mar 2022 07:51:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013902; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KPIyVXRJ+3a82MhUAQ3Qk34MlgnHhoRbfQUWHnDgoBY=; b=RGmu0NFADovwEIBH8BoJdisyvwt7vEEv1OF7CifO0Aoi+XbgZRTGN5DMbofylZxi+jZfir D60Y3B/QU7Y3e97+Jt939jWke6fUCP9njEmhEcEyodICTNQGPisR1PLc1xdHLcmHKc7sN1 WNCEsaxiNWN43mwz27fNSeV6izteS2c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-259-Y6aRIYVNNYuGQu1T37oWzA-1; Fri, 11 Mar 2022 10:51:38 -0500 X-MC-Unique: Y6aRIYVNNYuGQu1T37oWzA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 65157180A08E; Fri, 11 Mar 2022 15:51:37 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id C2678785FD; Fri, 11 Mar 2022 15:51:34 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 28/31] KVM: selftests: Sync 'struct hv_vp_assist_page' definition with hyperv-tlfs.h Date: Fri, 11 Mar 2022 16:49:40 +0100 Message-Id: <20220311154943.2299191-29-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" 'struct hv_vp_assist_page' definition doesn't match TLFS. Also, define 'struct hv_nested_enlightenments_control' and use it instead of opaque '__u64'. Signed-off-by: Vitaly Kuznetsov --- .../selftests/kvm/include/x86_64/evmcs.h | 22 ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/kvm/include/x86_64/evmcs.h b/tools/tes= ting/selftests/kvm/include/x86_64/evmcs.h index b6067b555110..9c965ba73dec 100644 --- a/tools/testing/selftests/kvm/include/x86_64/evmcs.h +++ b/tools/testing/selftests/kvm/include/x86_64/evmcs.h @@ -20,14 +20,26 @@ =20 extern bool enable_evmcs; =20 +struct hv_nested_enlightenments_control { + struct { + __u32 directhypercall:1; + __u32 reserved:31; + } features; + struct { + __u32 reserved; + } hypercallControls; +} __packed; + +/* Define virtual processor assist page structure. */ struct hv_vp_assist_page { __u32 apic_assist; - __u32 reserved; - __u64 vtl_control[2]; - __u64 nested_enlightenments_control[2]; - __u32 enlighten_vmentry; + __u32 reserved1; + __u64 vtl_control[3]; + struct hv_nested_enlightenments_control nested_control; + __u8 enlighten_vmentry; + __u8 reserved2[7]; __u64 current_nested_vmcs; -}; +} __packed; =20 struct hv_enlightened_vmcs { u32 revision_id; --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51C01C433EF for ; Fri, 11 Mar 2022 15:54:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349949AbiCKPzS (ORCPT ); Fri, 11 Mar 2022 10:55:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350346AbiCKPx5 (ORCPT ); Fri, 11 Mar 2022 10:53:57 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 14F591CC7F8 for ; Fri, 11 Mar 2022 07:51:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013905; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w+T44QIV/PbnYC1lhRap5XuNB8m7fz3Fw6iTjBuFInM=; b=WSAM7ZdYtV6Q5o6GZ1NCA2BLcy95XfShnWk6i6mQSo1qUBZr9WX2oszhXUK5x9AOlZW5xU 56hDnxMAfsdbFfrdM+6YTHFyiinniecrqUoTSxWPfvPnGuPR+AE36S3/ncp002vIoPfYef hET6MdMkFHHiG38YUtPE6QAXNk29864= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-231-Pxq48A1jPa2vt_WMMLYSuQ-1; Fri, 11 Mar 2022 10:51:42 -0500 X-MC-Unique: Pxq48A1jPa2vt_WMMLYSuQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8FABB1006AA6; Fri, 11 Mar 2022 15:51:40 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id C3728785FD; Fri, 11 Mar 2022 15:51:37 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 29/31] KVM: selftests: evmcs_test: Direct TLB flush test Date: Fri, 11 Mar 2022 16:49:41 +0100 Message-Id: <20220311154943.2299191-30-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Enable Hyper-V Direct TLB flush and check that Hyper-V TLB flush hypercalls from L2 don't exit to L1 unless 'TlbLockCount' is set in the Partition assist page. Signed-off-by: Vitaly Kuznetsov --- .../selftests/kvm/include/x86_64/evmcs.h | 2 + .../testing/selftests/kvm/x86_64/evmcs_test.c | 52 ++++++++++++++++++- 2 files changed, 52 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/include/x86_64/evmcs.h b/tools/tes= ting/selftests/kvm/include/x86_64/evmcs.h index 9c965ba73dec..36c0a67d8602 100644 --- a/tools/testing/selftests/kvm/include/x86_64/evmcs.h +++ b/tools/testing/selftests/kvm/include/x86_64/evmcs.h @@ -252,6 +252,8 @@ struct hv_enlightened_vmcs { #define HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_MASK \ (~((1ull << HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_SHIFT) - 1)) =20 +#define HV_VMX_SYNTHETIC_EXIT_REASON_TRAP_AFTER_FLUSH 0x10000031 + extern struct hv_enlightened_vmcs *current_evmcs; extern struct hv_vp_assist_page *current_vp_assist; =20 diff --git a/tools/testing/selftests/kvm/x86_64/evmcs_test.c b/tools/testin= g/selftests/kvm/x86_64/evmcs_test.c index d12e043aa2ee..411c4dbeac09 100644 --- a/tools/testing/selftests/kvm/x86_64/evmcs_test.c +++ b/tools/testing/selftests/kvm/x86_64/evmcs_test.c @@ -16,6 +16,7 @@ =20 #include "kvm_util.h" =20 +#include "hyperv.h" #include "vmx.h" =20 #define VCPU_ID 5 @@ -49,6 +50,16 @@ static inline void rdmsr_gs_base(void) "r13", "r14", "r15"); } =20 +static inline void hypercall(u64 control, vm_vaddr_t arg1, vm_vaddr_t arg2) +{ + asm volatile("mov %3, %%r8\n" + "vmcall" + : "+c" (control), "+d" (arg1) + : "r" (arg2) + : "cc", "memory", "rax", "rbx", "r8", "r9", "r10", + "r11", "r12", "r13", "r14", "r15"); +} + void l2_guest_code(void) { GUEST_SYNC(7); @@ -67,15 +78,27 @@ void l2_guest_code(void) vmcall(); rdmsr_gs_base(); /* intercepted */ =20 + /* Direct TLB flush tests */ + hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | HV_HYPERCALL_FAST_BIT, 0x0, + HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROCESSORS); + rdmsr_fs_base(); + hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | HV_HYPERCALL_FAST_BIT, 0x0, + HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROCESSORS); + /* Make sure we're no issuing Hyper-V TLB flush call again */ + __asm__ __volatile__ ("mov $0xdeadbeef, %rcx"); + /* Done, exit to L1 and never come back. */ vmcall(); } =20 -void guest_code(struct vmx_pages *vmx_pages) +void guest_code(struct vmx_pages *vmx_pages, vm_vaddr_t pgs_gpa) { #define L2_GUEST_STACK_SIZE 64 unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE]; =20 + wrmsr(HV_X64_MSR_GUEST_OS_ID, HYPERV_LINUX_OS_ID); + wrmsr(HV_X64_MSR_HYPERCALL, pgs_gpa); + x2apic_enable(); =20 GUEST_SYNC(1); @@ -105,6 +128,14 @@ void guest_code(struct vmx_pages *vmx_pages) vmwrite(PIN_BASED_VM_EXEC_CONTROL, vmreadz(PIN_BASED_VM_EXEC_CONTROL) | PIN_BASED_NMI_EXITING); =20 + /* Direct TLB flush setup */ + current_evmcs->partition_assist_page =3D vmx_pages->partition_assist_gpa; + current_evmcs->hv_enlightenments_control.nested_flush_hypercall =3D 1; + current_evmcs->hv_vm_id =3D 1; + current_evmcs->hv_vp_id =3D 1; + current_vp_assist->nested_control.features.directhypercall =3D 1; + *(u32 *)(vmx_pages->partition_assist) =3D 0; + GUEST_ASSERT(!vmlaunch()); GUEST_ASSERT(vmptrstz() =3D=3D vmx_pages->enlightened_vmcs_gpa); =20 @@ -149,6 +180,18 @@ void guest_code(struct vmx_pages *vmx_pages) GUEST_ASSERT(vmreadz(VM_EXIT_REASON) =3D=3D EXIT_REASON_MSR_READ); current_evmcs->guest_rip +=3D 2; /* rdmsr */ =20 + /* + * Direct TLB flush test. First VMCALL should be handled directly by L0, + * no VMCALL exit expested. + */ + GUEST_ASSERT(!vmresume()); + GUEST_ASSERT(vmreadz(VM_EXIT_REASON) =3D=3D EXIT_REASON_MSR_READ); + current_evmcs->guest_rip +=3D 2; /* rdmsr */ + /* Enable synthetic vmexit */ + *(u32 *)(vmx_pages->partition_assist) =3D 1; + GUEST_ASSERT(!vmresume()); + GUEST_ASSERT(vmreadz(VM_EXIT_REASON) =3D=3D HV_VMX_SYNTHETIC_EXIT_REASON_= TRAP_AFTER_FLUSH); + GUEST_ASSERT(!vmresume()); GUEST_ASSERT(vmreadz(VM_EXIT_REASON) =3D=3D EXIT_REASON_VMCALL); GUEST_SYNC(11); @@ -201,6 +244,7 @@ static void save_restore_vm(struct kvm_vm *vm) int main(int argc, char *argv[]) { vm_vaddr_t vmx_pages_gva =3D 0; + vm_vaddr_t hcall_page; =20 struct kvm_vm *vm; struct kvm_run *run; @@ -217,11 +261,15 @@ int main(int argc, char *argv[]) exit(KSFT_SKIP); } =20 + hcall_page =3D vm_vaddr_alloc_pages(vm, 1); + memset(addr_gva2hva(vm, hcall_page), 0x0, getpagesize()); + vcpu_set_hv_cpuid(vm, VCPU_ID); vcpu_enable_evmcs(vm, VCPU_ID); =20 vcpu_alloc_vmx(vm, &vmx_pages_gva); - vcpu_args_set(vm, VCPU_ID, 1, vmx_pages_gva); + vcpu_args_set(vm, VCPU_ID, 2, vmx_pages_gva, addr_gva2gpa(vm, hcall_page)= ); + vcpu_set_msr(vm, VCPU_ID, HV_X64_MSR_VP_INDEX, VCPU_ID); =20 vm_init_descriptor_tables(vm); vcpu_init_descriptor_tables(vm, VCPU_ID); --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B82A1C433FE for ; Fri, 11 Mar 2022 15:54:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235632AbiCKPzV (ORCPT ); Fri, 11 Mar 2022 10:55:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350367AbiCKPx5 (ORCPT ); Fri, 11 Mar 2022 10:53:57 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id E314C1CE986 for ; Fri, 11 Mar 2022 07:52:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013908; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=58iM70DiT8r7nT41Z6sKc8b4vIC4qiaKkLBgA80Zknk=; b=LHjQjw1+OypsQXkyRm36a/Ad57X16ejFKoaosSuQ/w5CiQOc8liSM68b+5Z2Vrlsb931xn d4VfXU/s6VDW+bkxupRBc9Op4R4DRyCS79kRK9SZxCYMKjxh+nLmfVjlhP3m4LxW5qhT5c +YXeCH8Gw0thTGAdLeqUjl9LUIfC7aw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-564-zd7Hh4TiNKef7fkSrCbIhg-1; Fri, 11 Mar 2022 10:51:45 -0500 X-MC-Unique: zd7Hh4TiNKef7fkSrCbIhg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8677451DF; Fri, 11 Mar 2022 15:51:43 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id ECA9F785FD; Fri, 11 Mar 2022 15:51:40 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 30/31] KVM: selftests: Move Hyper-V VP assist page enablement out of evmcs.h Date: Fri, 11 Mar 2022 16:49:42 +0100 Message-Id: <20220311154943.2299191-31-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Hyper-V VP assist page is not eVMCS specific, it is also used for enlightened nSVM. Move the code to vendor neutral place. Signed-off-by: Vitaly Kuznetsov --- tools/testing/selftests/kvm/Makefile | 2 +- .../selftests/kvm/include/x86_64/evmcs.h | 40 +------------------ .../selftests/kvm/include/x86_64/hyperv.h | 31 ++++++++++++++ .../testing/selftests/kvm/lib/x86_64/hyperv.c | 21 ++++++++++ .../testing/selftests/kvm/x86_64/evmcs_test.c | 1 + 5 files changed, 56 insertions(+), 39 deletions(-) create mode 100644 tools/testing/selftests/kvm/lib/x86_64/hyperv.c diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests= /kvm/Makefile index bd878effa20d..e615a4646e7b 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -38,7 +38,7 @@ ifeq ($(ARCH),riscv) endif =20 LIBKVM =3D lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/rbtree.c lib= /sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c -LIBKVM_x86_64 =3D lib/x86_64/apic.c lib/x86_64/processor.c lib/x86_64/vmx.= c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S +LIBKVM_x86_64 =3D lib/x86_64/apic.c lib/x86_64/hyperv.c lib/x86_64/process= or.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handle= rs.S LIBKVM_aarch64 =3D lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64= /handlers.S lib/aarch64/spinlock.c lib/aarch64/gic.c lib/aarch64/gic_v3.c l= ib/aarch64/vgic.c LIBKVM_s390x =3D lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318= _test_handler.c LIBKVM_riscv =3D lib/riscv/processor.c lib/riscv/ucall.c diff --git a/tools/testing/selftests/kvm/include/x86_64/evmcs.h b/tools/tes= ting/selftests/kvm/include/x86_64/evmcs.h index 36c0a67d8602..026586b53013 100644 --- a/tools/testing/selftests/kvm/include/x86_64/evmcs.h +++ b/tools/testing/selftests/kvm/include/x86_64/evmcs.h @@ -10,6 +10,7 @@ #define SELFTEST_KVM_EVMCS_H =20 #include +#include "hyperv.h" #include "vmx.h" =20 #define u16 uint16_t @@ -20,27 +21,6 @@ =20 extern bool enable_evmcs; =20 -struct hv_nested_enlightenments_control { - struct { - __u32 directhypercall:1; - __u32 reserved:31; - } features; - struct { - __u32 reserved; - } hypercallControls; -} __packed; - -/* Define virtual processor assist page structure. */ -struct hv_vp_assist_page { - __u32 apic_assist; - __u32 reserved1; - __u64 vtl_control[3]; - struct hv_nested_enlightenments_control nested_control; - __u8 enlighten_vmentry; - __u8 reserved2[7]; - __u64 current_nested_vmcs; -} __packed; - struct hv_enlightened_vmcs { u32 revision_id; u32 abort; @@ -246,31 +226,15 @@ struct hv_enlightened_vmcs { #define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ENLIGHTENMENTSCONTROL BIT(15) #define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL 0xFFFF =20 -#define HV_X64_MSR_VP_ASSIST_PAGE 0x40000073 -#define HV_X64_MSR_VP_ASSIST_PAGE_ENABLE 0x00000001 -#define HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_SHIFT 12 -#define HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_MASK \ - (~((1ull << HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_SHIFT) - 1)) - #define HV_VMX_SYNTHETIC_EXIT_REASON_TRAP_AFTER_FLUSH 0x10000031 =20 extern struct hv_enlightened_vmcs *current_evmcs; -extern struct hv_vp_assist_page *current_vp_assist; =20 int vcpu_enable_evmcs(struct kvm_vm *vm, int vcpu_id); =20 -static inline int enable_vp_assist(uint64_t vp_assist_pa, void *vp_assist) +static inline void evmcs_enable(void) { - u64 val =3D (vp_assist_pa & HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_MASK) | - HV_X64_MSR_VP_ASSIST_PAGE_ENABLE; - - wrmsr(HV_X64_MSR_VP_ASSIST_PAGE, val); - - current_vp_assist =3D vp_assist; - enable_evmcs =3D true; - - return 0; } =20 static inline int evmcs_vmptrld(uint64_t vmcs_pa, void *vmcs) diff --git a/tools/testing/selftests/kvm/include/x86_64/hyperv.h b/tools/te= sting/selftests/kvm/include/x86_64/hyperv.h index 1e34dd7c5075..095c15fc5381 100644 --- a/tools/testing/selftests/kvm/include/x86_64/hyperv.h +++ b/tools/testing/selftests/kvm/include/x86_64/hyperv.h @@ -189,4 +189,35 @@ =20 #define HYPERV_LINUX_OS_ID ((u64)0x8100 << 48) =20 +#define HV_X64_MSR_VP_ASSIST_PAGE 0x40000073 +#define HV_X64_MSR_VP_ASSIST_PAGE_ENABLE 0x00000001 +#define HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_SHIFT 12 +#define HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_MASK \ + (~((1ull << HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_SHIFT) - 1)) + +struct hv_nested_enlightenments_control { + struct { + __u32 directhypercall:1; + __u32 reserved:31; + } features; + struct { + __u32 reserved; + } hypercallControls; +} __packed; + +/* Define virtual processor assist page structure. */ +struct hv_vp_assist_page { + __u32 apic_assist; + __u32 reserved1; + __u64 vtl_control[3]; + struct hv_nested_enlightenments_control nested_control; + __u8 enlighten_vmentry; + __u8 reserved2[7]; + __u64 current_nested_vmcs; +} __packed; + +extern struct hv_vp_assist_page *current_vp_assist; + +int enable_vp_assist(uint64_t vp_assist_pa, void *vp_assist); + #endif /* !SELFTEST_KVM_HYPERV_H */ diff --git a/tools/testing/selftests/kvm/lib/x86_64/hyperv.c b/tools/testin= g/selftests/kvm/lib/x86_64/hyperv.c new file mode 100644 index 000000000000..a8c6b156f92d --- /dev/null +++ b/tools/testing/selftests/kvm/lib/x86_64/hyperv.c @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * tools/testing/selftests/kvm/lib/x86_64/hyperv.c + * + * Copyright (C) 2021, Red Hat Inc. + */ +#include +#include "processor.h" +#include "hyperv.h" + +int enable_vp_assist(uint64_t vp_assist_pa, void *vp_assist) +{ + uint64_t val =3D (vp_assist_pa & HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_MASK) | + HV_X64_MSR_VP_ASSIST_PAGE_ENABLE; + + wrmsr(HV_X64_MSR_VP_ASSIST_PAGE, val); + + current_vp_assist =3D vp_assist; + + return 0; +} diff --git a/tools/testing/selftests/kvm/x86_64/evmcs_test.c b/tools/testin= g/selftests/kvm/x86_64/evmcs_test.c index 411c4dbeac09..32ec43a8b101 100644 --- a/tools/testing/selftests/kvm/x86_64/evmcs_test.c +++ b/tools/testing/selftests/kvm/x86_64/evmcs_test.c @@ -105,6 +105,7 @@ void guest_code(struct vmx_pages *vmx_pages, vm_vaddr_t= pgs_gpa) GUEST_SYNC(2); =20 enable_vp_assist(vmx_pages->vp_assist_gpa, vmx_pages->vp_assist); + evmcs_enable(); =20 GUEST_ASSERT(vmx_pages->vmcs_gpa); GUEST_ASSERT(prepare_for_vmx_operation(vmx_pages)); --=20 2.35.1 From nobody Tue Jun 23 03:13:17 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC889C433F5 for ; Fri, 11 Mar 2022 15:54:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350062AbiCKPzZ (ORCPT ); Fri, 11 Mar 2022 10:55:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350385AbiCKPx6 (ORCPT ); Fri, 11 Mar 2022 10:53:58 -0500 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 197191CF09D for ; Fri, 11 Mar 2022 07:52:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1647013913; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zlNKffXUsYA/dr1sDc5Q+AFd+03QHFEyPT6fGMhOla0=; b=hfMsoWjpHPWMULWZOUvPhkDe/onnIo9rEfj6ATntKVLzQlMD6jXbWyyTVatZ0G3C/ujt8v bhktNWrR2oYw9VqWFlrD0yb1IHvVn+huF/Y8VtlQamJTSNIOGZmqSfQ2eIAYEWWrvM6vtU KMAoQJXdO2qtsIFjDjSqvmAU6CrjlTY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-661-UgCb4dLPNYeHKjIf77Ugvg-1; Fri, 11 Mar 2022 10:51:47 -0500 X-MC-Unique: UgCb4dLPNYeHKjIf77Ugvg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5BC25180A08F; Fri, 11 Mar 2022 15:51:46 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.40.194.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id E5B9B785FD; Fri, 11 Mar 2022 15:51:43 +0000 (UTC) From: Vitaly Kuznetsov To: kvm@vger.kernel.org, Paolo Bonzini Cc: Sean Christopherson , Wanpeng Li , Jim Mattson , Michael Kelley , linux-hyperv@vger.kernel.org, Siddharth Chandrasekaran , linux-kernel@vger.kernel.org Subject: [PATCH 31/31] KVM: selftests: hyperv_svm_test: Add Direct TLB flush test Date: Fri, 11 Mar 2022 16:49:43 +0100 Message-Id: <20220311154943.2299191-32-vkuznets@redhat.com> In-Reply-To: <20220311154943.2299191-1-vkuznets@redhat.com> References: <20220311154943.2299191-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Enable Hyper-V Direct TLB flush and check that Hyper-V TLB flush hypercalls from L2 don't exit to L1 unless 'TlbLockCount' is set in the Partition assist page. Signed-off-by: Vitaly Kuznetsov --- .../selftests/kvm/x86_64/hyperv_svm_test.c | 60 +++++++++++++++++-- 1 file changed, 56 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c b/tools/t= esting/selftests/kvm/x86_64/hyperv_svm_test.c index 21f5ca9197da..e2849e58582f 100644 --- a/tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c +++ b/tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c @@ -42,11 +42,24 @@ struct hv_enlightenments { */ #define VMCB_HV_NESTED_ENLIGHTENMENTS (1U << 31) =20 +#define HV_SVM_EXITCODE_ENL 0xF0000000 +#define HV_SVM_ENL_EXITCODE_TRAP_AFTER_FLUSH (1) + static inline void vmmcall(void) { __asm__ __volatile__("vmmcall"); } =20 +static inline void hypercall(u64 control, vm_vaddr_t arg1, vm_vaddr_t arg2) +{ + asm volatile("mov %3, %%r8\n" + "vmmcall" + : "+c" (control), "+d" (arg1) + : "r" (arg2) + : "cc", "memory", "rax", "rbx", "r8", "r9", "r10", + "r11", "r12", "r13", "r14", "r15"); +} + void l2_guest_code(void) { GUEST_SYNC(3); @@ -62,11 +75,21 @@ void l2_guest_code(void) =20 GUEST_SYNC(5); =20 + /* Direct TLB flush tests */ + hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | HV_HYPERCALL_FAST_BIT, 0x0, + HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROCESSORS); + rdmsr(MSR_FS_BASE); + hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | HV_HYPERCALL_FAST_BIT, 0x0, + HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROCESSORS); + /* Make sure we're not issuing Hyper-V TLB flush call again */ + __asm__ __volatile__ ("mov $0xdeadbeef, %rcx"); + /* Done, exit to L1 and never come back. */ vmmcall(); } =20 -static void __attribute__((__flatten__)) guest_code(struct svm_test_data *= svm) +static void __attribute__((__flatten__)) guest_code(struct svm_test_data *= svm, + vm_vaddr_t pgs_gpa) { unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE]; struct vmcb *vmcb =3D svm->vmcb; @@ -75,13 +98,23 @@ static void __attribute__((__flatten__)) guest_code(str= uct svm_test_data *svm) =20 GUEST_SYNC(1); =20 - wrmsr(HV_X64_MSR_GUEST_OS_ID, (u64)0x8100 << 48); + wrmsr(HV_X64_MSR_GUEST_OS_ID, HYPERV_LINUX_OS_ID); + wrmsr(HV_X64_MSR_HYPERCALL, pgs_gpa); + enable_vp_assist(svm->vp_assist_gpa, svm->vp_assist); =20 GUEST_ASSERT(svm->vmcb_gpa); /* Prepare for L2 execution. */ generic_svm_setup(svm, l2_guest_code, &l2_guest_stack[L2_GUEST_STACK_SIZE]); =20 + /* Direct TLB flush setup */ + hve->partition_assist_page =3D svm->partition_assist_gpa; + hve->hv_enlightenments_control.nested_flush_hypercall =3D 1; + hve->hv_vm_id =3D 1; + hve->hv_vp_id =3D 1; + current_vp_assist->nested_control.features.directhypercall =3D 1; + *(u32 *)(svm->partition_assist) =3D 0; + GUEST_SYNC(2); run_guest(vmcb, svm->vmcb_gpa); GUEST_ASSERT(vmcb->control.exit_code =3D=3D SVM_EXIT_VMMCALL); @@ -116,6 +149,20 @@ static void __attribute__((__flatten__)) guest_code(st= ruct svm_test_data *svm) GUEST_ASSERT(vmcb->control.exit_code =3D=3D SVM_EXIT_MSR); vmcb->save.rip +=3D 2; /* rdmsr */ =20 + + /* + * Direct TLB flush test. First VMCALL should be handled directly by L0, + * no VMCALL exit expested. + */ + run_guest(vmcb, svm->vmcb_gpa); + GUEST_ASSERT(vmcb->control.exit_code =3D=3D SVM_EXIT_MSR); + vmcb->save.rip +=3D 2; /* rdmsr */ + /* Enable synthetic vmexit */ + *(u32 *)(svm->partition_assist) =3D 1; + run_guest(vmcb, svm->vmcb_gpa); + GUEST_ASSERT(vmcb->control.exit_code =3D=3D HV_SVM_EXITCODE_ENL); + GUEST_ASSERT(vmcb->control.exit_info_1 =3D=3D HV_SVM_ENL_EXITCODE_TRAP_AF= TER_FLUSH); + run_guest(vmcb, svm->vmcb_gpa); GUEST_ASSERT(vmcb->control.exit_code =3D=3D SVM_EXIT_VMMCALL); GUEST_SYNC(6); @@ -126,7 +173,7 @@ static void __attribute__((__flatten__)) guest_code(str= uct svm_test_data *svm) int main(int argc, char *argv[]) { vm_vaddr_t nested_gva =3D 0; - + vm_vaddr_t hcall_page; struct kvm_vm *vm; struct kvm_run *run; struct ucall uc; @@ -141,7 +188,12 @@ int main(int argc, char *argv[]) vcpu_set_hv_cpuid(vm, VCPU_ID); run =3D vcpu_state(vm, VCPU_ID); vcpu_alloc_svm(vm, &nested_gva); - vcpu_args_set(vm, VCPU_ID, 1, nested_gva); + + hcall_page =3D vm_vaddr_alloc_pages(vm, 1); + memset(addr_gva2hva(vm, hcall_page), 0x0, getpagesize()); + + vcpu_args_set(vm, VCPU_ID, 2, nested_gva, addr_gva2gpa(vm, hcall_page)); + vcpu_set_msr(vm, VCPU_ID, HV_X64_MSR_VP_INDEX, VCPU_ID); =20 for (stage =3D 1;; stage++) { _vcpu_run(vm, VCPU_ID); --=20 2.35.1