From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 613813D5673 for ; Fri, 18 Sep 2026 08:15:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719355; cv=none; b=D0d9TInaXynjfJClR3SCbfdxJOc69GkNDp8Z0Kl8KM2u0d230uEWJL0cFcQdGcs0HDwyIU8v7U4qCQPivCpbIoiSTXqP6fYifBJ37Stuq28KLV4AEaYOE81wh+o1C9UF5rtUJ2ZaBi/gxq8bOWrWJGUAE2wp10afIe0Eze/yeoE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719355; c=relaxed/simple; bh=LecEQKu/SDwbIJ98w082y35hYPM4bZ2YqckHZDStxmM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=t8PXmz6e86zDhSI2+dWVndgbkfGu36r7mJepti7kzNo7+A4q2VH4LwVqF8bq9TTl/OoA9mPYmhaFKK4ZZ53B7yv14ymHLdqayPOSRd4eBWUwgS/vbvwClwqK3bVyQ+oEJtymFP9RWy8Y9YUFvEU3dp5QNZISgOe3/s/ox3B5dxU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fkp7rGHf; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fkp7rGHf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719351; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p+RBgkHWDBWjcz9ZqG24LQx/Q7KFkaJuhhsDYniJ5qc=; b=fkp7rGHf9s/dxvG5DcDVQL35OCYMfDAeJZIe5uTv+u8X1LB6PEqUOvjuPMnSoHpkYvUAMi 7fkUOFRDsEce9AcxW2RWQzVLNua0mT54EWO4HrJsYzE/e8JqUoQRv+mqQwoOUM67gwR09x rGr3sRljIoebzWhM+XHtt8SoS0HSnY8= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-111-3aSHCHnJPLag7WyhqsRq-A-1; Fri, 18 Sep 2026 04:15:46 -0400 X-MC-Unique: 3aSHCHnJPLag7WyhqsRq-A-1 X-Mimecast-MFC-AGG-ID: 3aSHCHnJPLag7WyhqsRq-A_1789719345 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4FD3A194511A; Fri, 18 Sep 2026 08:15:45 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A607F1800348; Fri, 18 Sep 2026 08:15:44 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 01/28] KVM: selftests: Take into account mixed memory fault flags Date: Fri, 18 Sep 2026 04:15:16 -0400 Message-ID: <20260918081543.139871-2-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne Update x86_64/private_mem_kvm_exits_test to take into account memory fault flags might contain multiple bits set while remaining valid---for example KVM might return KVM_MEMORY_EXIT_FLAG_READ in addition to KVM_MEMORY_EXIT_FLAG_PRIVATE. Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- .../testing/selftests/kvm/x86/private_mem_kvm_exits_test.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c b= /tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c index e6d16e9d6120..33fd73465432 100644 --- a/tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c +++ b/tools/testing/selftests/kvm/x86/private_mem_kvm_exits_test.c @@ -74,7 +74,8 @@ static void test_private_access_memslot_deleted(void) exit_reason =3D (u32)(u64)thread_return; =20 TEST_ASSERT_EQ(exit_reason, KVM_EXIT_MEMORY_FAULT); - TEST_ASSERT_EQ(vcpu->run->memory_fault.flags, KVM_MEMORY_EXIT_FLAG_PRIVAT= E); + TEST_ASSERT(vcpu->run->memory_fault.flags & KVM_MEMORY_EXIT_FLAG_PRIVATE, + "Memory fault didn't occur on a private memory access"); TEST_ASSERT_EQ(vcpu->run->memory_fault.gpa, EXITS_TEST_GPA); TEST_ASSERT_EQ(vcpu->run->memory_fault.size, EXITS_TEST_SIZE); =20 @@ -103,7 +104,8 @@ static void test_private_access_memslot_not_private(voi= d) exit_reason =3D run_vcpu_get_exit_reason(vcpu); =20 TEST_ASSERT_EQ(exit_reason, KVM_EXIT_MEMORY_FAULT); - TEST_ASSERT_EQ(vcpu->run->memory_fault.flags, KVM_MEMORY_EXIT_FLAG_PRIVAT= E); + TEST_ASSERT(vcpu->run->memory_fault.flags & KVM_MEMORY_EXIT_FLAG_PRIVATE, + "Memory fault didn't occur on a private memory access"); TEST_ASSERT_EQ(vcpu->run->memory_fault.gpa, EXITS_TEST_GPA); TEST_ASSERT_EQ(vcpu->run->memory_fault.size, EXITS_TEST_SIZE); =20 --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDC2E2FABFB for ; Fri, 18 Sep 2026 08:15:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719360; cv=none; b=apKUsj7bVUjteatPu8sXfQPwtrHEdWqYO4ZVC3t5q/c07hiRIzgJAhRDhrPLFZ6dYayA/rJee6Uikdd4IFFrSzpZ51RH5rErf8BeSLZQoDE/ODvedQvR/VEDgnPXbCcTeHmGM5tKRASfsgjbNFh1qLwGv4jqNSNgi1C+0k7OPKQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719360; c=relaxed/simple; bh=zzEHyCrP6YSfLJgk6zrPkxbRErf6W7r56cYPcCj1LHc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ZUDGfkY+HVjtAXWbs/pqdiXvY+olaBgPpTF9Duh78KJi6YtvPxRBS10CPIPs37jsK7v8xHZocbqNn11T5t5iO5h1vbvijChE2vF3Rx2wWECwTqMUZ5PIDpJA9qSDtgYQyMp44+mVF6r9gyjZi5Vs3EuswMsPQHotTI2XXL1AnUM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=O/OhazVF; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="O/OhazVF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719353; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w60wt/dkt6/s0NeTy3kqOW1+30h7wm/BdjoJ3Ym0CcI=; b=O/OhazVFLkUfy4KTcrF7fKRXz0v1qGrUK3lfjZdNo5K+Lab9AC6ZHAOUp346wCzGAYS2uJ Ggv2TWyz2Ezd0J1TQULhffkqLHTkQoM2Ln80vVC2i3uIWzjN8aZhKRSI42+MzyK9/c/Si9 aMmswpSSCz8nU199vcwowMgnjNEVXo4= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-596-U2b55Cj7NYyeaLgOS0wqhA-1; Fri, 18 Sep 2026 04:15:47 -0400 X-MC-Unique: U2b55Cj7NYyeaLgOS0wqhA-1 X-Mimecast-MFC-AGG-ID: U2b55Cj7NYyeaLgOS0wqhA_1789719346 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 49567180AECA; Fri, 18 Sep 2026 08:15:46 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 771B11800348; Fri, 18 Sep 2026 08:15:45 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com, Anish Moorthy , Sean Christopherson Subject: [PATCH v2 02/28] KVM: Define and communicate KVM_EXIT_MEMORY_FAULT RWX flags to userspace Date: Fri, 18 Sep 2026 04:15:17 -0400 Message-ID: <20260918081543.139871-3-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" From: Anish Moorthy kvm_prepare_memory_fault_exit() already takes parameters describing the RWX-ness of the relevant access but doesn't actually do anything with them. Define and use the flags necessary to pass this information on to userspace. Suggested-by: Sean Christopherson Link: https://lore.kernel.org/kvm/ZR4N8cwzTMDanPUY@google.com/ Signed-off-by: Anish Moorthy Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- Documentation/virt/kvm/api.rst | 5 +++++ include/linux/kvm_host.h | 9 ++++++++- include/uapi/linux/kvm.h | 3 +++ 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 4eb7e75a7473..37028ce019e3 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -7279,6 +7279,9 @@ spec refer, https://github.com/riscv/riscv-sbi-doc. =20 /* KVM_EXIT_MEMORY_FAULT */ struct { + #define KVM_MEMORY_EXIT_FLAG_READ (1ULL << 0) + #define KVM_MEMORY_EXIT_FLAG_WRITE (1ULL << 1) + #define KVM_MEMORY_EXIT_FLAG_EXEC (1ULL << 2) #define KVM_MEMORY_EXIT_FLAG_PRIVATE (1ULL << 3) __u64 flags; __u64 gpa; @@ -7290,6 +7293,8 @@ could not be resolved by KVM. The 'gpa' and 'size' (= in bytes) describe the guest physical address range [gpa, gpa + size) of the fault. The 'flags' = field describes properties of the faulting access that are likely pertinent: =20 + - KVM_MEMORY_EXIT_FLAG_READ/WRITE/EXEC - When set, indicates that the mem= ory + fault occurred on a read/write/exec access respectively. - KVM_MEMORY_EXIT_FLAG_PRIVATE - When set, indicates the memory fault occ= urred on a private memory access. When clear, indicates the fault occurred o= n a shared access. diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 03bfc92864b6..ce5757e85ae3 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2546,8 +2546,15 @@ static inline void kvm_prepare_memory_fault_exit(str= uct kvm_vcpu *vcpu, vcpu->run->memory_fault.gpa =3D gpa; vcpu->run->memory_fault.size =3D size; =20 - /* RWX flags are not (yet) defined or communicated to userspace. */ vcpu->run->memory_fault.flags =3D 0; + + if (is_write) + vcpu->run->memory_fault.flags |=3D KVM_MEMORY_EXIT_FLAG_WRITE; + else if (is_exec) + vcpu->run->memory_fault.flags |=3D KVM_MEMORY_EXIT_FLAG_EXEC; + else + vcpu->run->memory_fault.flags |=3D KVM_MEMORY_EXIT_FLAG_READ; + if (is_private) vcpu->run->memory_fault.flags |=3D KVM_MEMORY_EXIT_FLAG_PRIVATE; } diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index ac2d77d14963..15a3090f067e 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -457,6 +457,9 @@ struct kvm_run { } notify; /* KVM_EXIT_MEMORY_FAULT */ struct { +#define KVM_MEMORY_EXIT_FLAG_READ (1ULL << 0) +#define KVM_MEMORY_EXIT_FLAG_WRITE (1ULL << 1) +#define KVM_MEMORY_EXIT_FLAG_EXEC (1ULL << 2) #define KVM_MEMORY_EXIT_FLAG_PRIVATE (1ULL << 3) __u64 flags; __u64 gpa; --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA39F3A59B1 for ; Fri, 18 Sep 2026 08:15:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719352; cv=none; b=OR+im5CSUZoV8QQjzspJZBzlb9wLWvy+KwCCz4QHzmw4+oQgHvY23SJKB+v3OPTNbs8iTkRUBfWHs4vqaTJsublMav9sluITR3K1PJL6H3ZoQIiHpdTtrhcLNCHOiDfvZKSWKNeQYwSL593SFlnUBqY6h3n8/jabZLvhSaL5bQ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719352; c=relaxed/simple; bh=pbuEFP9JBUPeu4FwbDZIxGLj/5C/IgErLvJLmGdTKJU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WWlzr+zabVS8ZkG1VNqGQPc/a76De22Y0ZUTg8CUMz7hNeqZ3VA4lFbKIgKly/JS/eqfeiq8v+Ktbork1/XObobmjLVOW3sFgO/IJwNqfDaTRVRiaD2xy17EgSW//CgTll3dn6gcGxzOPAYVE/24gzmYSTbwg6ES8QEvU/b6Hg8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=K6kukmVj; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="K6kukmVj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719349; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=C+igli3ADcJ/ELfbVLsspCnGXZetx5NgGwJwZm27a1g=; b=K6kukmVjlrjwc6n8WDu5Rp1eldsbiwRe45KTYot4fDxA/6GwQcMvDtoALTviBV8gjv6UVw q//lArfjCfPXioOFD3Uq/TgDxlSdBcXg9MwgkoaDWNd67tnJ4NhGKqFnSgafGq869WsbxU uKIgBX3DT0DQFyyDLCphiUmr3BBFq9k= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-195-7d-m2j5MPBebFoIWJBVbcg-1; Fri, 18 Sep 2026 04:15:48 -0400 X-MC-Unique: 7d-m2j5MPBebFoIWJBVbcg-1 X-Mimecast-MFC-AGG-ID: 7d-m2j5MPBebFoIWJBVbcg_1789719347 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1CD8D195C242; Fri, 18 Sep 2026 08:15:47 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 713F318004D4; Fri, 18 Sep 2026 08:15:46 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 03/28] KVM: selftests: Test address translation for Hyper-V direct L2 hypercalls Date: Fri, 18 Sep 2026 04:15:18 -0400 Message-ID: <20260918081543.139871-4-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" Test that the TLB flush hypercalls, when executed by L2, translate their input addresses through the guest's EPT or NPT. Signed-off-by: Paolo Bonzini --- tools/testing/selftests/kvm/x86/hyperv_evmcs.c | 13 ++++++++++++- tools/testing/selftests/kvm/x86/hyperv_svm_test.c | 13 ++++++++++++- 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/x86/hyperv_evmcs.c b/tools/testing= /selftests/kvm/x86/hyperv_evmcs.c index 63ea1533e4ea..fa80bc78aef7 100644 --- a/tools/testing/selftests/kvm/x86/hyperv_evmcs.c +++ b/tools/testing/selftests/kvm/x86/hyperv_evmcs.c @@ -18,6 +18,8 @@ #include "hyperv.h" #include "vmx.h" =20 +#define L2_HCALL_INPUT_GPA 0x100000000ull + static int ud_count; =20 static void guest_ud_handler(struct ex_regs *regs) @@ -60,6 +62,8 @@ void l2_guest_code(void) rdmsr_from_l2(MSR_GS_BASE); /* intercepted */ =20 /* L2 TLB flush tests */ + hyperv_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE, + L2_HCALL_INPUT_GPA, 0); hyperv_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | HV_HYPERCALL_FAST_B= IT, 0x0, HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROCESSORS); rdmsr_from_l2(MSR_FS_BASE); @@ -212,6 +216,7 @@ static struct kvm_vcpu *save_restore_vm(struct kvm_vm *= vm, =20 /* Restore state in a new VM. */ vcpu =3D vm_recreate_with_one_vcpu(vm); + vm_enable_ept(vm); vcpu_set_hv_cpuid(vcpu); vcpu_enable_evmcs(vcpu); vcpu_load_state(vcpu, state); @@ -228,7 +233,7 @@ static struct kvm_vcpu *save_restore_vm(struct kvm_vm *= vm, int main(int argc, char *argv[]) { gva_t vmx_pages_gva =3D 0, hv_pages_gva =3D 0; - gva_t hcall_page; + gva_t hcall_input_page, hcall_page; =20 struct kvm_vcpu *vcpu; struct kvm_vm *vm; @@ -239,17 +244,23 @@ int main(int argc, char *argv[]) TEST_REQUIRE(kvm_has_cap(KVM_CAP_NESTED_STATE)); TEST_REQUIRE(kvm_has_cap(KVM_CAP_HYPERV_ENLIGHTENED_VMCS)); TEST_REQUIRE(kvm_hv_cpu_has(HV_X64_NESTED_DIRECT_FLUSH)); + TEST_REQUIRE(kvm_cpu_has_ept()); =20 vm =3D vm_create_with_one_vcpu(&vcpu, guest_code); + vm_enable_ept(vm); =20 hcall_page =3D vm_alloc_pages(vm, 1); memset(addr_gva2hva(vm, hcall_page), 0x0, getpagesize()); + hcall_input_page =3D vm_alloc_page(vm); + memset(addr_gva2hva(vm, hcall_input_page), 0, getpagesize()); =20 vcpu_set_hv_cpuid(vcpu); vcpu_enable_evmcs(vcpu); =20 vcpu_alloc_vmx(vm, &vmx_pages_gva); vcpu_alloc_hyperv_test_pages(vm, &hv_pages_gva); + tdp_identity_map_default_memslots(vm); + tdp_map(vm, L2_HCALL_INPUT_GPA, addr_gva2gpa(vm, hcall_input_page), PAGE_= SIZE); vcpu_args_set(vcpu, 3, vmx_pages_gva, hv_pages_gva, addr_gva2gpa(vm, hcal= l_page)); vcpu_set_msr(vcpu, HV_X64_MSR_VP_INDEX, vcpu->id); =20 diff --git a/tools/testing/selftests/kvm/x86/hyperv_svm_test.c b/tools/test= ing/selftests/kvm/x86/hyperv_svm_test.c index 1f74b0fa9b83..18f0ad6debd8 100644 --- a/tools/testing/selftests/kvm/x86/hyperv_svm_test.c +++ b/tools/testing/selftests/kvm/x86/hyperv_svm_test.c @@ -18,6 +18,8 @@ #include "svm_util.h" #include "hyperv.h" =20 +#define L2_HCALL_INPUT_GPA 0x100000000ull + /* Exit to L1 from L2 with RDMSR instruction */ static inline void rdmsr_from_l2(u32 msr) { @@ -45,6 +47,8 @@ void l2_guest_code(void) GUEST_SYNC(5); =20 /* L2 TLB flush tests */ + hyperv_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE, + L2_HCALL_INPUT_GPA, 0); hyperv_hypercall(HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE | HV_HYPERCALL_FAST_BIT, 0x0, HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | @@ -146,7 +150,7 @@ static void __attribute__((__flatten__)) guest_code(str= uct svm_test_data *svm, int main(int argc, char *argv[]) { gva_t nested_gva =3D 0, hv_pages_gva =3D 0; - gva_t hcall_page; + gva_t hcall_input_page, hcall_page; struct kvm_vcpu *vcpu; struct kvm_vm *vm; struct ucall uc; @@ -154,15 +158,22 @@ int main(int argc, char *argv[]) =20 TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_SVM)); TEST_REQUIRE(kvm_hv_cpu_has(HV_X64_NESTED_DIRECT_FLUSH)); + TEST_REQUIRE(kvm_cpu_has_npt()); =20 /* Create VM */ vm =3D vm_create_with_one_vcpu(&vcpu, guest_code); + vm_enable_npt(vm); vcpu_set_hv_cpuid(vcpu); vcpu_alloc_svm(vm, &nested_gva); vcpu_alloc_hyperv_test_pages(vm, &hv_pages_gva); =20 hcall_page =3D vm_alloc_pages(vm, 1); memset(addr_gva2hva(vm, hcall_page), 0x0, getpagesize()); + hcall_input_page =3D vm_alloc_page(vm); + memset(addr_gva2hva(vm, hcall_input_page), 0, getpagesize()); + + tdp_identity_map_default_memslots(vm); + tdp_map(vm, L2_HCALL_INPUT_GPA, addr_gva2gpa(vm, hcall_input_page), PAGE_= SIZE); =20 vcpu_args_set(vcpu, 3, nested_gva, hv_pages_gva, addr_gva2gpa(vm, hcall_p= age)); vcpu_set_msr(vcpu, HV_X64_MSR_VP_INDEX, vcpu->id); --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A26B3308F07 for ; Fri, 18 Sep 2026 08:15:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719359; cv=none; b=YEXDTalGqzkMCfRkdqDJjsOPKyPIzefIznk0mapsytCY5cN8O/2/mqI87tw4p3ahb79t3euBg+sBdg0h40/GJCO+hLtKAsNKnXLRpUQFfn4e6pRaxWbluhmAg/U0zek8Ne4lF3YRGv/OrbfQuO/TvxhQCT3EWzlx8siBMvemdwo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719359; c=relaxed/simple; bh=zcNwGv45CExz7j/TVlzPBkHjSt0mpoZ14T1ighPkpqY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=D2NnosGt47qCF+bHjIYo0GuLyMyK4Voy2U8tIccL3S249c6AcM3GzzTkE9OUn1eWjm7/MRvsQx7DApadApK5hk/Z3g5Xbg+Q35FkGosiXZgfG7UzLY998OxRiUR8QxjpqkLJhG9zyQK1H+akjFZTt+Bz/XIwTgFJBqXCmmhxiSE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Q/ILPn8Y; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Q/ILPn8Y" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719352; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f/mcFx7CpqXrw9dX0AsthsvY4YVJ7zqnvie71XVwgx4=; b=Q/ILPn8YidCtGcqkec0yrFbNOr9gmNs3PiH5wPVbhObpPrMNxReq98ilomCSBvrV3A6ppS VO67rTHyOJzrvOTSQpQ/y5KgqJ8qqM9jstmNNhZGLx/sX6Za6WrWRi0ywTGJnqPeUBBgIU 7SIqBLyrZh9cB2R4o1D7LHYx3b4ZkJM= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-648-6kAO5bzdPCGg4RcQq4J4ew-1; Fri, 18 Sep 2026 04:15:49 -0400 X-MC-Unique: 6kAO5bzdPCGg4RcQq4J4ew-1 X-Mimecast-MFC-AGG-ID: 6kAO5bzdPCGg4RcQq4J4ew_1789719348 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EE864195396F; Fri, 18 Sep 2026 08:15:47 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4842D1800348; Fri, 18 Sep 2026 08:15:47 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 04/28] KVM: apply nGPA->GPA translation to KVM_HC_CLOCK_PAIRING Date: Fri, 18 Sep 2026 04:15:19 -0400 Message-ID: <20260918081543.139871-5-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" KVM_HC_CLOCK_PAIRING writes to a guest physical addresses. Treat it as an L2 address when running nested, which is consistent with the Hyper-V TLB flush hypercalls for example. This is technically backwards incompatible, but it should be extremely rare and thus we can treat it as a bugfix, I think. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/x86.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 4b3681796c75..2fbaa070f741 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7217,6 +7217,11 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcp= u, gpa_t paddr, clock_pairing.flags =3D 0; memset(&clock_pairing.pad, 0, sizeof(clock_pairing.pad)); =20 + paddr =3D kvm_translate_gpa(vcpu, &vcpu->arch.gva_walk, paddr, + PFERR_WRITE_MASK | PFERR_GUEST_FINAL_MASK, NULL, 0); + if (paddr =3D=3D INVALID_GPA) + return -KVM_EFAULT; + ret =3D 0; if (kvm_write_guest(vcpu->kvm, paddr, &clock_pairing, sizeof(struct kvm_clock_pairing))) --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BB934ABBCB for ; Fri, 18 Sep 2026 08:15:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719363; cv=none; b=e1dVK7eyjPKCU/tUb1G76M4/WP/9Q3umcowOGg8+KrGzMmPddPbUXDYRIGdp4CiKxj4MTWt9pqxbycxo5mo2KnyWWTXIM1UGUxFeOxw0OuwJ6sw7vRzryg8d68YcQD2WsP+rNsX/CO1o+knkL/5/tB70CiceFOk8RhAfbPrYFfY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719363; c=relaxed/simple; bh=v5wR7U5cYsBH/nkva/SeFYy3gKlLaPJXWaoMcHltJQs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=areGzGNnqXGShrVb/zS9LiZyeBSa15DYHvGDV1OX0SxDI6xBoxRfpj33/rSZaDeN1+GQt603pbEWo19D6fhAlbTIe4IA8s8CtIYJZRbwsJ7mXO20G3zk/+krA1kW3Is5b3G8sixNleFydxhb+q/qmPsNW06ce1tBirOWJ+Bfg9I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fMfGcMqA; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fMfGcMqA" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719354; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+hVsZbuskpwJXUXR/rDnVL9ommRvM8B/74gs7hn/GOA=; b=fMfGcMqAyQbeYYMsifFY2mbzMIANzicVKo1Zsaga6ILOd8OqgqjacbIrMqmNpgQ3Bwu4WG LNf6ANC0pjW2pfzGaxcf7qjqagK5eepW0NDH9NO374Vy6IGZlfNq53Tscpv9YOfJJnP/hk LqyLH3uA6cwP4DteDt1saBrtMrZ40jM= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-662-3QvOjbg8NxWxPre2Gi0oUQ-1; Fri, 18 Sep 2026 04:15:50 -0400 X-MC-Unique: 3QvOjbg8NxWxPre2Gi0oUQ-1 X-Mimecast-MFC-AGG-ID: 3QvOjbg8NxWxPre2Gi0oUQ_1789719349 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BD86A19772E3; Fri, 18 Sep 2026 08:15:48 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 20E87180056E; Fri, 18 Sep 2026 08:15:48 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 05/28] KVM: x86: Introduce memory fault on invalid hypercalls reads/writes Date: Fri, 18 Sep 2026 04:15:20 -0400 Message-ID: <20260918081543.139871-6-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" Allow userspace to perform arbitrary actions when a hypercall refers to an invalid address, by exiting with KVM_EXIT_MEMORY_FAULT. This will for example allow userspace to perform a VTL call. Co-developed-by: Nicolas Saenz Julienne Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- Documentation/virt/kvm/api.rst | 19 ++++ arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/hyperv.c | 157 +++++++++++++++++++++++++------- arch/x86/kvm/x86.c | 38 ++++++-- include/uapi/linux/kvm.h | 1 + 5 files changed, 174 insertions(+), 42 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 37028ce019e3..abc2ff1f8c84 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -8984,6 +8984,25 @@ enabled, cmma can't be enabled anymore and pfmfi and= the storage key interpretation are disabled. If cmma has already been enabled or the hpage_2g module parameter is not set to 1, -EINVAL is returned. =20 +7.48 KVM_CAP_HCALL_FAULT_EXIT +------------------------------------ + +:Architectures: x86 +:Parameters: args[0] is 0 to disable, 1 to enable + +When enabled, KVM checks the memory that is read or written by +hypercalls (including slow Hyper-V hypercalls and KVM_HC_CLOCK_PAIRING). +An inaccessible input page causes a KVM_EXIT_MEMORY_FAULT with +KVM_MEMORY_EXIT_FLAG_READ. An inaccessible or read-only output page causes +a KVM_EXIT_MEMORY_FAULT with KVM_MEMORY_EXIT_FLAG_WRITE. The reported +range identifies the page containing the parameter GPA. + +Hypercall parameters that are unused by the selected hypercall are not che= cked. + +For Hyper-V, unknown hypercalls are passed to userspace without checking t= heir +parameter pages. The fault GPA is in the physical address space of the +VM managed by userspace, after nested GPA translation. + 8. Other capabilities. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 683bb8bf43a9..c08781bb0327 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1169,6 +1169,7 @@ struct kvm_arch { bool has_protected_eoi; bool has_protected_pmu; bool pre_fault_allowed; + bool hcall_fault_exit; struct hlist_head *mmu_page_hash; struct list_head active_mmu_pages; struct kvm_possible_nx_huge_pages possible_nx_huge_pages[KVM_NR_MMU_TYPES= ]; diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 604651cb2739..9f5adcd26cba 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2033,7 +2033,68 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) return -ENOSPC; } =20 -static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) +static int kvm_hv_hypercall_check_gpa(struct kvm_vcpu *vcpu, gpa_t gpa, + bool write) +{ + bool writable =3D true; + gfn_t gfn =3D gpa_to_gfn(gpa); + unsigned long addr; + + addr =3D kvm_vcpu_gfn_to_hva_prot(vcpu, gfn, &writable); + if (!kvm_is_error_hva(addr) && (!write || writable)) + return 0; + + kvm_prepare_memory_fault_exit(vcpu, gfn_to_gpa(gfn), PAGE_SIZE, + write, false, false); + return -EFAULT; +} + +static unsigned int kvm_hv_hypercall_mem_access(u16 code) +{ + switch (code) { + case HVCALL_SIGNAL_EVENT: + case HVCALL_POST_MESSAGE: + case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST: + case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX: + case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE: + case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX: + case HVCALL_SEND_IPI: + case HVCALL_SEND_IPI_EX: + return KVM_MEMORY_EXIT_FLAG_READ; + case HVCALL_POST_DEBUG_DATA: + case HVCALL_RETRIEVE_DEBUG_DATA: + return KVM_MEMORY_EXIT_FLAG_READ | KVM_MEMORY_EXIT_FLAG_WRITE; + case HVCALL_RESET_DEBUG_SESSION: + case HV_EXT_CALL_QUERY_CAPABILITIES: + return KVM_MEMORY_EXIT_FLAG_WRITE; + } + + return 0; +} + +static int kvm_hv_hypercall_check_params(struct kvm_vcpu *vcpu, + struct kvm_hv_hcall *hc) +{ + unsigned access; + int r; + + if (hc->fast || !vcpu->kvm->arch.hcall_fault_exit) + return 0; + + access =3D kvm_hv_hypercall_mem_access(hc->code); + if (access & KVM_MEMORY_EXIT_FLAG_READ) { + r =3D kvm_hv_hypercall_check_gpa(vcpu, hc->ingpa, false); + if (r) + return r; + } + + if (access & KVM_MEMORY_EXIT_FLAG_WRITE) + return kvm_hv_hypercall_check_gpa(vcpu, hc->outgpa, true); + + return 0; +} + +static s64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) { struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); unsigned long *vcpu_mask =3D hv_vcpu->vcpu_mask; @@ -2053,6 +2114,19 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, s= truct kvm_hv_hcall *hc) struct kvm_vcpu *v; unsigned long i; bool all_cpus; + int r; + + /* Slow direct hypercalls from L2 provide a nested GPA. */ + if (!hc->fast) { + hc->ingpa =3D kvm_translate_gpa(vcpu, &vcpu->arch.gva_walk, hc->ingpa, + PFERR_GUEST_FINAL_MASK, NULL, 0); + if (unlikely(hc->ingpa =3D=3D INVALID_GPA)) + return HV_STATUS_INVALID_HYPERCALL_INPUT; + } + + r =3D kvm_hv_hypercall_check_params(vcpu, hc); + if (r) + return r; =20 /* * The Hyper-V TLFS doesn't allow more than HV_MAX_SPARSE_VCPU_BANKS @@ -2061,20 +2135,6 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, s= truct kvm_hv_hcall *hc) */ BUILD_BUG_ON(KVM_HV_MAX_SPARSE_VCPU_SET_BITS > HV_MAX_SPARSE_VCPU_BANKS); =20 - /* - * 'Slow' hypercall's first parameter is the address in guest's memory - * where hypercall parameters are placed. This is either a GPA or a - * nested GPA when KVM is handling the call from L2 ('direct' TLB - * flush). Translate the address here so the memory can be uniformly - * read with kvm_read_guest(). - */ - if (!hc->fast) { - hc->ingpa =3D kvm_translate_gpa(vcpu, &vcpu->arch.gva_walk, hc->ingpa, - PFERR_GUEST_FINAL_MASK, NULL, 0); - if (unlikely(hc->ingpa =3D=3D INVALID_GPA)) - return HV_STATUS_INVALID_HYPERCALL_INPUT; - } - if (hc->code =3D=3D HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST || hc->code =3D=3D HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE) { if (hc->fast) { @@ -2242,7 +2302,7 @@ static void kvm_hv_send_ipi_to_many(struct kvm *kvm, = u32 vector, } } =20 -static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) +static int kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) { struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); u64 *sparse_banks =3D hv_vcpu->sparse_banks; @@ -2252,6 +2312,11 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, st= ruct kvm_hv_hcall *hc) u64 valid_bank_mask; u32 vector; bool all_cpus; + int r; + + r =3D kvm_hv_hypercall_check_params(vcpu, hc); + if (r) + return r; =20 if (!lapic_in_kernel(vcpu)) return HV_STATUS_INVALID_HYPERCALL_INPUT; @@ -2432,10 +2497,15 @@ static int kvm_hv_hypercall_complete_userspace(stru= ct kvm_vcpu *vcpu) return kvm_hv_hypercall_complete(vcpu, vcpu->run->hyperv.u.hcall.result); } =20 -static u16 kvm_hvcall_signal_event(struct kvm_vcpu *vcpu, struct kvm_hv_hc= all *hc) +static int kvm_hvcall_signal_event(struct kvm_vcpu *vcpu, struct kvm_hv_hc= all *hc) { struct kvm_hv *hv =3D to_kvm_hv(vcpu->kvm); struct eventfd_ctx *eventfd; + int ret; + + ret =3D kvm_hv_hypercall_check_params(vcpu, hc); + if (ret) + return ret; =20 if (unlikely(!hc->fast)) { int ret; @@ -2547,11 +2617,30 @@ static bool hv_check_hypercall_access(struct kvm_vc= pu_hv *hv_vcpu, u16 code) return true; } =20 +static int kvm_hv_hypercall_userspace_exit(struct kvm_vcpu *vcpu, + struct kvm_hv_hcall *hc) +{ + int r; + + r =3D kvm_hv_hypercall_check_params(vcpu, hc); + if (r) + return r; + + vcpu->run->exit_reason =3D KVM_EXIT_HYPERV; + vcpu->run->hyperv.type =3D KVM_EXIT_HYPERV_HCALL; + vcpu->run->hyperv.u.hcall.input =3D hc->param; + vcpu->run->hyperv.u.hcall.params[0] =3D hc->ingpa; + vcpu->run->hyperv.u.hcall.params[1] =3D hc->outgpa; + vcpu->arch.complete_userspace_io =3D kvm_hv_hypercall_complete_userspace; + return 0; +} + int kvm_hv_hypercall(struct kvm_vcpu *vcpu) { struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); struct kvm_hv_hcall hc; u64 ret =3D HV_STATUS_SUCCESS; + s64 r; =20 /* * hypercall generates UD from non zero cpl and real mode @@ -2619,7 +2708,10 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) ret =3D HV_STATUS_INVALID_HYPERCALL_INPUT; break; } - ret =3D kvm_hvcall_signal_event(vcpu, &hc); + r =3D kvm_hvcall_signal_event(vcpu, &hc); + if (r < 0) + return r; + ret =3D r; if (ret !=3D HV_STATUS_INVALID_PORT_ID) break; fallthrough; /* maybe userspace knows this conn_id */ @@ -2629,7 +2721,7 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) ret =3D HV_STATUS_INVALID_HYPERCALL_INPUT; break; } - goto hypercall_userspace_exit; + return kvm_hv_hypercall_userspace_exit(vcpu, &hc); case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST: if (unlikely(hc.var_cnt)) { ret =3D HV_STATUS_INVALID_HYPERCALL_INPUT; @@ -2641,7 +2733,10 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) ret =3D HV_STATUS_INVALID_HYPERCALL_INPUT; break; } - ret =3D kvm_hv_flush_tlb(vcpu, &hc); + r =3D kvm_hv_flush_tlb(vcpu, &hc); + if (r < 0) + return r; + ret =3D r; break; case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE: if (unlikely(hc.var_cnt)) { @@ -2654,7 +2749,10 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) ret =3D HV_STATUS_INVALID_HYPERCALL_INPUT; break; } - ret =3D kvm_hv_flush_tlb(vcpu, &hc); + r =3D kvm_hv_flush_tlb(vcpu, &hc); + if (r < 0) + return r; + ret =3D r; break; case HVCALL_SEND_IPI: if (unlikely(hc.var_cnt)) { @@ -2667,7 +2765,10 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) ret =3D HV_STATUS_INVALID_HYPERCALL_INPUT; break; } - ret =3D kvm_hv_send_ipi(vcpu, &hc); + r =3D kvm_hv_send_ipi(vcpu, &hc); + if (r < 0) + return r; + ret =3D r; break; case HVCALL_POST_DEBUG_DATA: case HVCALL_RETRIEVE_DEBUG_DATA: @@ -2688,14 +2789,14 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) ret =3D HV_STATUS_OPERATION_DENIED; break; } - goto hypercall_userspace_exit; + return kvm_hv_hypercall_userspace_exit(vcpu, &hc); } case HV_EXT_CALL_QUERY_CAPABILITIES ... HV_EXT_CALL_MAX: if (unlikely(hc.fast)) { ret =3D HV_STATUS_INVALID_PARAMETER; break; } - goto hypercall_userspace_exit; + return kvm_hv_hypercall_userspace_exit(vcpu, &hc); default: ret =3D HV_STATUS_INVALID_HYPERCALL_CODE; break; @@ -2704,14 +2805,6 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) hypercall_complete: return kvm_hv_hypercall_complete(vcpu, ret); =20 -hypercall_userspace_exit: - vcpu->run->exit_reason =3D KVM_EXIT_HYPERV; - vcpu->run->hyperv.type =3D KVM_EXIT_HYPERV_HCALL; - vcpu->run->hyperv.u.hcall.input =3D hc.param; - vcpu->run->hyperv.u.hcall.params[0] =3D hc.ingpa; - vcpu->run->hyperv.u.hcall.params[1] =3D hc.outgpa; - vcpu->arch.complete_userspace_io =3D kvm_hv_hypercall_complete_userspace; - return 0; } =20 void kvm_hv_init_vm(struct kvm *kvm) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 2fbaa070f741..c10b3fd48b65 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2242,6 +2242,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, lon= g ext) case KVM_CAP_HYPERV_ENFORCE_CPUID: case KVM_CAP_SYS_HYPERV_CPUID: #endif + case KVM_CAP_HCALL_FAULT_EXIT: case KVM_CAP_PCI_SEGMENT: case KVM_CAP_DEBUGREGS: case KVM_CAP_X86_ROBUST_SINGLESTEP: @@ -4210,6 +4211,10 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, mutex_unlock(&kvm->lock); break; } + case KVM_CAP_HCALL_FAULT_EXIT: + kvm->arch.hcall_fault_exit =3D cap->args[0]; + r =3D 0; + break; default: r =3D -EINVAL; break; @@ -7191,25 +7196,27 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_x86_vendor_exit); =20 #ifdef CONFIG_X86_64 static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, - unsigned long clock_type) + unsigned long clock_type, unsigned long *ret) { struct kvm_clock_pairing clock_pairing; struct timespec64 ts; + bool writable; + unsigned long hva; u64 cycle; - int ret; =20 + *ret =3D -KVM_EOPNOTSUPP; if (clock_type !=3D KVM_CLOCK_PAIRING_WALLCLOCK) - return -KVM_EOPNOTSUPP; + return 0; =20 /* * When tsc is in permanent catchup mode guests won't be able to use * pvclock_read_retry loop to get consistent view of pvclock */ if (vcpu->arch.tsc_always_catchup) - return -KVM_EOPNOTSUPP; + return 0; =20 if (!kvm_get_walltime_and_clockread(&ts, &cycle)) - return -KVM_EOPNOTSUPP; + return 0; =20 clock_pairing.sec =3D ts.tv_sec; clock_pairing.nsec =3D ts.tv_nsec; @@ -7217,17 +7224,27 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vc= pu, gpa_t paddr, clock_pairing.flags =3D 0; memset(&clock_pairing.pad, 0, sizeof(clock_pairing.pad)); =20 + *ret =3D -KVM_EFAULT; paddr =3D kvm_translate_gpa(vcpu, &vcpu->arch.gva_walk, paddr, PFERR_WRITE_MASK | PFERR_GUEST_FINAL_MASK, NULL, 0); if (paddr =3D=3D INVALID_GPA) - return -KVM_EFAULT; + return 0; + + if (vcpu->kvm->arch.hcall_fault_exit) { + hva =3D kvm_vcpu_gfn_to_hva_prot(vcpu, gpa_to_gfn(paddr), &writable); + if (kvm_is_error_hva(hva) || !writable) { + kvm_prepare_memory_fault_exit(vcpu, paddr & PAGE_MASK, PAGE_SIZE, + true, false, false); + return -EFAULT; + } + } =20 - ret =3D 0; if (kvm_write_guest(vcpu->kvm, paddr, &clock_pairing, sizeof(struct kvm_clock_pairing))) - ret =3D -KVM_EFAULT; + return 0; =20 - return ret; + *ret =3D 0; + return 0; } #endif =20 @@ -7385,7 +7402,8 @@ int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, = int cpl, break; #ifdef CONFIG_X86_64 case KVM_HC_CLOCK_PAIRING: - ret =3D kvm_pv_clock_pairing(vcpu, a0, a1); + if (kvm_pv_clock_pairing(vcpu, a0, a1, &ret)) + return -EFAULT; break; #endif case KVM_HC_SEND_IPI: diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 15a3090f067e..c23f1ea62eaf 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1002,6 +1002,7 @@ struct kvm_enable_cap { #define KVM_CAP_S390_HPAGE_2G 249 #define KVM_CAP_PPC_COMPAT_CAPS 250 #define KVM_CAP_ARM_PMU_V3_STRICT 251 +#define KVM_CAP_HCALL_FAULT_EXIT 252 =20 struct kvm_irq_routing_irqchip { __u32 irqchip; --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3DE74071CA for ; Fri, 18 Sep 2026 08:15:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719355; cv=none; b=Hn/0KclawesbAw6u1yLhQDpihgoFtnulRtbCTFIBdN0IBeJIIoSl3/0d3bopyb+Up7omby4lVk90Ks371ZlWsJ4wsTVYhBrCgBiBe9lqv0Gp3YkCWUjIgck8EmhocpmZAVjRVfbSXnALunfViZLSk7nvZOpMpg/wVPLDJ7fv8I0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719355; c=relaxed/simple; bh=j718blU6RN5DJGFUvrjtjMbPldS9KGIw0B1zGTjX5Lg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=l4kJnhUPCNFUWIc/8E2nZb1jOPpGGGnN+v8hx2kyLnoDeV+pzGlxZdnGGaDn3p9g/3bJHykTypMTM6XFEffZLbThKMkidxt+hzoFffDtZzhmsZTcLJF2KFubhgtMlk4BXsG3YLnFpUe8EONvebyOu8kOUDoTdrAZFyOGpytK3Jk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BADMIL/I; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BADMIL/I" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719351; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CoQsP7oEaShRQSDplU9Ab5jzKYzLus15zhUEw3A9VY8=; b=BADMIL/ItBSsHFOWWKMUo3wrndtpbvrDitUy04oSpL+Z0pyufCy10VPLrpc1ndVg0PZI79 TwzVC2nZ6yJQ9X+TAQsrAXJcgk8FyReJAxMKSOKk6LU0zzW7pf/6a812SfXelaVTsQw6dy kIYBepyaHeksJYLyx3rtP58CRn0TJso= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-663-z4N7MFywMyalXd-j9JsOxw-1; Fri, 18 Sep 2026 04:15:50 -0400 X-MC-Unique: z4N7MFywMyalXd-j9JsOxw-1 X-Mimecast-MFC-AGG-ID: z4N7MFywMyalXd-j9JsOxw_1789719349 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8CECD194511A; Fri, 18 Sep 2026 08:15:49 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E47B81800370; Fri, 18 Sep 2026 08:15:48 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 06/28] KVM: selftests: test hypercall memory fault exits Date: Fri, 18 Sep 2026 04:15:21 -0400 Message-ID: <20260918081543.139871-7-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" Cover various scenarios where hypercalls are invoked with invalid or read-only GPAs, and check that they exit to userspace. Signed-off-by: Paolo Bonzini --- tools/testing/selftests/kvm/Makefile.kvm | 1 + tools/testing/selftests/kvm/x86/hcall_fault.c | 236 ++++++++++++++++++ .../testing/selftests/kvm/x86/hyperv_evmcs.c | 4 + .../selftests/kvm/x86/hyperv_svm_test.c | 2 + 4 files changed, 243 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86/hcall_fault.c diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selft= ests/kvm/Makefile.kvm index 96bab7002d39..6e2bbf743eaa 100644 --- a/tools/testing/selftests/kvm/Makefile.kvm +++ b/tools/testing/selftests/kvm/Makefile.kvm @@ -79,6 +79,7 @@ TEST_GEN_PROGS_x86 +=3D x86/evmcs_smm_controls_test TEST_GEN_PROGS_x86 +=3D x86/exit_on_emulation_failure_test TEST_GEN_PROGS_x86 +=3D x86/fastops_test TEST_GEN_PROGS_x86 +=3D x86/fix_hypercall_test +TEST_GEN_PROGS_x86 +=3D x86/hcall_fault TEST_GEN_PROGS_x86 +=3D x86/hwcr_msr_test TEST_GEN_PROGS_x86 +=3D x86/hyperv_clock TEST_GEN_PROGS_x86 +=3D x86/hyperv_cpuid diff --git a/tools/testing/selftests/kvm/x86/hcall_fault.c b/tools/testing/= selftests/kvm/x86/hcall_fault.c new file mode 100644 index 000000000000..63e38c87616f --- /dev/null +++ b/tools/testing/selftests/kvm/x86/hcall_fault.c @@ -0,0 +1,236 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include + +#include "kvm_util.h" +#include "processor.h" +#include "hyperv.h" + +#define TEST_MEM_GPA 0x100000000ull +#define READONLY_GPA (TEST_MEM_GPA + PAGE_SIZE) +#define UNMAPPED_IN_GPA (TEST_MEM_GPA + 2 * PAGE_SIZE) +#define UNMAPPED_OUT_GPA (TEST_MEM_GPA + 3 * PAGE_SIZE) + +struct hcall_test { + /* TEST_MEM_GPA in the guest */ + u64 control; + u64 ingpa; + u64 outgpa; + u64 expected_status; + u8 unused[PAGE_SIZE - 32]; + + /* READONLY_GPA in the guest */ + u8 readonly_data[]; +}; + +#define HV_FLUSH_ALL_PROCESSORS BIT(0) +#define HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES BIT(1) + +/* HvFlushVirtualAddressSpace, HvFlushVirtualAddressList hypercalls */ +struct hv_tlb_flush { + u64 address_space; + u64 flags; + u64 processor_mask; + u64 gva_list[]; +} __packed; + +static void guest_code(gpa_t hcall_page, struct hcall_test *test) +{ + u64 result; + u8 vector; + + wrmsr(HV_X64_MSR_GUEST_OS_ID, HYPERV_LINUX_OS_ID); + wrmsr(HV_X64_MSR_HYPERCALL, hcall_page); + + vector =3D __hyperv_hypercall(test->control, test->ingpa, test->outgpa, &= result); + GUEST_ASSERT(!vector); + GUEST_ASSERT_EQ(result & 0xffff, test->expected_status); + GUEST_DONE(); +} + +static void clock_pairing_guest(gpa_t gpa) +{ + kvm_hypercall(KVM_HC_CLOCK_PAIRING, gpa, KVM_CLOCK_PAIRING_WALLCLOCK, 0, = 0); + GUEST_DONE(); +} + +static struct kvm_vm *create_vm(struct kvm_vcpu **vcpu, struct hcall_test = **test, + gpa_t *hcall_page_gpa) +{ + gva_t hcall_page, test_gva; + struct kvm_vm *vm; + + vm =3D vm_create_with_one_vcpu(vcpu, guest_code); + vcpu_set_hv_cpuid(*vcpu); + vm_enable_cap(vm, KVM_CAP_HCALL_FAULT_EXIT, 1); + + hcall_page =3D vm_alloc_page(vm); + memset(addr_gva2hva(vm, hcall_page), 0, PAGE_SIZE); + *hcall_page_gpa =3D addr_gva2gpa(vm, hcall_page); + + vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, TEST_MEM_GPA, + 10, 1, 0); + memset(addr_gpa2hva(vm, TEST_MEM_GPA), 0, PAGE_SIZE); + + vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, READONLY_GPA, + 11, 1, KVM_MEM_READONLY); + memset(addr_gpa2hva(vm, READONLY_GPA), 0, PAGE_SIZE); + + test_gva =3D vm_unused_gva_gap(vm, PAGE_SIZE * 2, 1 << 24); + for (int i =3D 0; i < 2; i++) + virt_pg_map(vm, test_gva + PAGE_SIZE * i, TEST_MEM_GPA + PAGE_SIZE * i); + *test =3D addr_gva2hva(vm, test_gva); + + vcpu_args_set(*vcpu, 2, *hcall_page_gpa, test_gva); + return vm; +} + +static void run_vm(struct kvm_vcpu *vcpu) +{ + struct ucall uc; + + vcpu_run(vcpu); + TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_IO); + + switch (get_ucall(vcpu, &uc)) { + case UCALL_ABORT: + REPORT_GUEST_ASSERT(uc); + break; + case UCALL_DONE: + break; + default: + TEST_FAIL("Unexpected ucall: %lu", uc.cmd); + } +} + +static void test_unused_gpas(void) +{ + struct hcall_test *test; + struct kvm_vcpu *vcpu; + struct kvm_vm *vm; + gpa_t hcall_page; + + vm =3D create_vm(&vcpu, &test, &hcall_page); + test->control =3D HVCALL_NOTIFY_LONG_SPIN_WAIT; + test->ingpa =3D UNMAPPED_IN_GPA; + test->outgpa =3D UNMAPPED_OUT_GPA; + test->expected_status =3D HV_STATUS_SUCCESS; + run_vm(vcpu); + kvm_vm_free(vm); + + vm =3D create_vm(&vcpu, &test, &hcall_page); + test->control =3D 0xbeef; + test->ingpa =3D UNMAPPED_IN_GPA; + test->outgpa =3D UNMAPPED_OUT_GPA; + test->expected_status =3D HV_STATUS_INVALID_HYPERCALL_CODE; + run_vm(vcpu); + kvm_vm_free(vm); +} + +static void assert_memory_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 flag= s) +{ + int r; + + r =3D _vcpu_run(vcpu); + TEST_ASSERT(r =3D=3D -1 && errno =3D=3D EFAULT, KVM_IOCTL_ERROR(KVM_RUN, = r)); + TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_MEMORY_FAULT); + TEST_ASSERT_EQ(vcpu->run->memory_fault.flags, flags); + TEST_ASSERT_EQ(vcpu->run->memory_fault.gpa, gpa & PAGE_MASK); + TEST_ASSERT_EQ(vcpu->run->memory_fault.size, PAGE_SIZE); +} + +/* Read-only hypercalls, read-only inputs, valid/invalid argument */ +static void test_readonly_input(void) +{ + struct hv_tlb_flush *flush; + struct hcall_test *test; + struct kvm_vcpu *vcpu; + struct kvm_vm *vm; + gpa_t hcall_page; + + vm =3D create_vm(&vcpu, &test, &hcall_page); + flush =3D (struct hv_tlb_flush *)&test->readonly_data; + flush->flags =3D HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES | HV_FLUSH_ALL_PROCE= SSORS; + flush->processor_mask =3D 0; + test->control =3D HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE; + test->ingpa =3D READONLY_GPA; + test->outgpa =3D UNMAPPED_OUT_GPA; + test->expected_status =3D HV_STATUS_SUCCESS; + run_vm(vcpu); + kvm_vm_free(vm); + + /* fails immediately because it requires SynIC */ + vm =3D create_vm(&vcpu, &test, &hcall_page); + test->control =3D HVCALL_POST_MESSAGE; + test->ingpa =3D UNMAPPED_IN_GPA; + test->outgpa =3D UNMAPPED_OUT_GPA; + test->expected_status =3D HV_STATUS_INVALID_HYPERCALL_INPUT; + run_vm(vcpu); + kvm_vm_free(vm); +} + +/* Read-only hypercall, unmapped input */ +static void test_input_fault(void) +{ + struct hcall_test *test; + struct kvm_vcpu *vcpu; + struct kvm_vm *vm; + gpa_t hcall_page; + + vm =3D create_vm(&vcpu, &test, &hcall_page); + test->control =3D HVCALL_SIGNAL_EVENT; + test->ingpa =3D UNMAPPED_IN_GPA; + test->outgpa =3D UNMAPPED_OUT_GPA; + assert_memory_fault(vcpu, test->ingpa, KVM_MEMORY_EXIT_FLAG_READ); + kvm_vm_free(vm); +} + +/* Read-write hypercall, unmapped or readonly input and output */ +static void test_output_fault(bool readonly_in, bool readonly_out) +{ + struct hcall_test *test; + struct kvm_vcpu *vcpu; + struct kvm_vm *vm; + gpa_t hcall_page; + + vm =3D create_vm(&vcpu, &test, &hcall_page); + test->control =3D HV_EXT_CALL_QUERY_CAPABILITIES; + test->ingpa =3D readonly_in ? READONLY_GPA : UNMAPPED_IN_GPA; + test->outgpa =3D readonly_out ? READONLY_GPA : UNMAPPED_OUT_GPA; + + assert_memory_fault(vcpu, test->outgpa, KVM_MEMORY_EXIT_FLAG_WRITE); + kvm_vm_free(vm); +} + +static void test_clock_pairing_fault(bool readonly) +{ + struct kvm_vcpu *vcpu; + struct kvm_vm *vm; + gpa_t gpa; + + /* Unlike create_vm do not enable Hyper-V hypercalls. */ + vm =3D vm_create_with_one_vcpu(&vcpu, clock_pairing_guest); + vm_enable_cap(vm, KVM_CAP_HCALL_FAULT_EXIT, 1); + gpa =3D readonly ? READONLY_GPA : UNMAPPED_OUT_GPA; + vcpu_args_set(vcpu, 1, gpa); + + assert_memory_fault(vcpu, gpa, KVM_MEMORY_EXIT_FLAG_WRITE); + kvm_vm_free(vm); +} + +int main(void) +{ + TEST_REQUIRE(kvm_has_cap(KVM_CAP_HYPERV_CPUID)); + TEST_REQUIRE(kvm_has_cap(KVM_CAP_HCALL_FAULT_EXIT)); + TEST_REQUIRE(kvm_cpuid_has(kvm_get_supported_hv_cpuid(), + HV_ENABLE_EXTENDED_HYPERCALLS)); + + test_unused_gpas(); + test_readonly_input(); + test_input_fault(); + test_output_fault(false, false); + test_output_fault(true, false); + test_output_fault(false, true); + test_clock_pairing_fault(false); + test_clock_pairing_fault(true); + return 0; +} diff --git a/tools/testing/selftests/kvm/x86/hyperv_evmcs.c b/tools/testing= /selftests/kvm/x86/hyperv_evmcs.c index fa80bc78aef7..cd8d8acb2419 100644 --- a/tools/testing/selftests/kvm/x86/hyperv_evmcs.c +++ b/tools/testing/selftests/kvm/x86/hyperv_evmcs.c @@ -216,6 +216,8 @@ static struct kvm_vcpu *save_restore_vm(struct kvm_vm *= vm, =20 /* Restore state in a new VM. */ vcpu =3D vm_recreate_with_one_vcpu(vm); + if (kvm_has_cap(KVM_CAP_HCALL_FAULT_EXIT)) + vm_enable_cap(vm, KVM_CAP_HCALL_FAULT_EXIT, 1); vm_enable_ept(vm); vcpu_set_hv_cpuid(vcpu); vcpu_enable_evmcs(vcpu); @@ -247,6 +249,8 @@ int main(int argc, char *argv[]) TEST_REQUIRE(kvm_cpu_has_ept()); =20 vm =3D vm_create_with_one_vcpu(&vcpu, guest_code); + if (kvm_has_cap(KVM_CAP_HCALL_FAULT_EXIT)) + vm_enable_cap(vm, KVM_CAP_HCALL_FAULT_EXIT, 1); vm_enable_ept(vm); =20 hcall_page =3D vm_alloc_pages(vm, 1); diff --git a/tools/testing/selftests/kvm/x86/hyperv_svm_test.c b/tools/test= ing/selftests/kvm/x86/hyperv_svm_test.c index 18f0ad6debd8..ecad242e6623 100644 --- a/tools/testing/selftests/kvm/x86/hyperv_svm_test.c +++ b/tools/testing/selftests/kvm/x86/hyperv_svm_test.c @@ -162,6 +162,8 @@ int main(int argc, char *argv[]) =20 /* Create VM */ vm =3D vm_create_with_one_vcpu(&vcpu, guest_code); + if (kvm_has_cap(KVM_CAP_HCALL_FAULT_EXIT)) + vm_enable_cap(vm, KVM_CAP_HCALL_FAULT_EXIT, 1); vm_enable_npt(vm); vcpu_set_hv_cpuid(vcpu); vcpu_alloc_svm(vm, &nested_gva); --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 394EA3D9043 for ; Fri, 18 Sep 2026 08:15:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719365; cv=none; b=HLd71AHLJYI8ROyy/po2Rwlqb2zHuhTqiDB/MWUH29XF5y/2kwDi91yt1nKphxPyUNJZmcpJM1aPfIQ4ufdIj/obc2ytxPwUJHg4nXVR+Lq1hVbLlFqA95vDXkpQfNmoCcl41mHECJrjbl78mcn6Bvi+Jxecw2oHMEzAMCVWT2Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719365; c=relaxed/simple; bh=rgk9M4pgE1QviE0CqliLyO+0jWnMkZ/1sJnfXb4SX+s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=AXRBHgMFctGdZjU34bUczFG9AVS+8IXrMA22+985lVqJmFVUSWLtZe8z5I2gv4OWvQQ9nx4QzlJEHBC5XX0Xj5XRBU2jjIlqzHyj9D1kHZeebo7Ypl5tviY8xTN+/bwmP9UyA0WEmH8gkmOImq9Ogzrdm8Mn+/IIjDkRMWA50Ig= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=R7yC1WLR; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="R7yC1WLR" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719355; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ujSswSFjYRCTMYkmmhaCZWF4ecoRPMD1qhDcHmUI9W0=; b=R7yC1WLRRS3jQHFssz1UHCG4bV5AHKlBg8k/6R3OpMIuaAxhDB4WpYL5HgbVRijAeYlDYv jM4n0WYddfT0PnM1+Lw3hU8Bw0Zmf6OppdORLIQybLKIkrQ1Y7EyuUzHHpEUUHqeq9ewNo K0K1ke534XZbJnvB5gT/7iPrSAgFzIE= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-262-BgJo-VkcP6CXQjxsZPrajw-1; Fri, 18 Sep 2026 04:15:51 -0400 X-MC-Unique: BgJo-VkcP6CXQjxsZPrajw-1 X-Mimecast-MFC-AGG-ID: BgJo-VkcP6CXQjxsZPrajw_1789719350 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7062918000A3; Fri, 18 Sep 2026 08:15:50 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B38291800370; Fri, 18 Sep 2026 08:15:49 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 07/28] KVM: x86/mmu: intersect writability from __kvm_faultin_pfn with fault->map_writable Date: Fri, 18 Sep 2026 04:15:22 -0400 Message-ID: <20260918081543.139871-8-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" fault->map_writable is currently a pure output of __kvm_faultin_pfn(), which is the only thing that restricts it. This will no longer hold once memory protections derived from memory attributes are applied: those compute their own access permissions that combine with those from __kvm_faultin_pfn(). Applying them *before* faulting in the pfn lets a fault that violates the attributes exit to userspace without the cost of gup and/or an async #PF; but it means that permissions will then be restricted in two independent steps, first by memory attributes and then by __kvm_faultin_pfn= (). Switch fault->map_writable to that model by letting kvm_mmu_faultin_pfn() only clear bits rather than assign them. No functional change intended: nothing writes fault->map_writable between the initializer and __kvm_mmu_faultin_pfn() yet, so the AND is equivalent to the assignment it replaces. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 064ecc33b926..5996468b7120 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4634,7 +4634,7 @@ static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *= vcpu, return r; } =20 - fault->map_writable =3D !(fault->slot->flags & KVM_MEM_READONLY); + fault->map_writable &=3D !(fault->slot->flags & KVM_MEM_READONLY); fault->max_level =3D kvm_max_level_for_order(max_order); =20 return RET_PF_CONTINUE; @@ -4644,13 +4644,14 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *v= cpu, struct kvm_page_fault *fault) { unsigned int foll =3D fault->write ? FOLL_WRITE : 0; + bool writable; =20 if (fault->is_private || kvm_memslot_is_gmem_only(fault->slot)) return kvm_mmu_faultin_pfn_gmem(vcpu, fault); =20 foll |=3D FOLL_NOWAIT; fault->pfn =3D __kvm_faultin_pfn(fault->slot, fault->gfn, foll, - &fault->map_writable, &fault->refcounted_page); + &writable, &fault->refcounted_page); =20 /* * If resolving the page failed because I/O is needed to fault-in the @@ -4659,7 +4660,7 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcp= u, * other failures are terminal, i.e. retrying won't help. */ if (fault->pfn !=3D KVM_PFN_ERR_NEEDS_IO) - return RET_PF_CONTINUE; + goto out_pf_continue; =20 if (!fault->prefetch && kvm_can_do_async_pf(vcpu)) { trace_kvm_try_async_get_page(fault->addr, fault->gfn); @@ -4680,8 +4681,10 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vc= pu, foll |=3D FOLL_INTERRUPTIBLE; foll &=3D ~FOLL_NOWAIT; fault->pfn =3D __kvm_faultin_pfn(fault->slot, fault->gfn, foll, - &fault->map_writable, &fault->refcounted_page); + &writable, &fault->refcounted_page); =20 +out_pf_continue: + fault->map_writable &=3D writable; return RET_PF_CONTINUE; } =20 @@ -5000,6 +5003,7 @@ static int kvm_mmu_do_page_fault(struct kvm_vcpu *vcp= u, gpa_t cr2_or_gpa, .is_private =3D err & PFERR_PRIVATE_ACCESS, =20 .pfn =3D KVM_PFN_ERR_FAULT, + .map_writable =3D true, }; int r; =20 --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3DC47499F0C for ; Fri, 18 Sep 2026 08:15:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719365; cv=none; b=BWibebw4aeZLGve2ArFM9FEOmLP9QIgRstdcMEcTcVbZrvAz0xSeL3+W5xvs3nhosnNkp5MZvGwMG0KroyezQZT4VJbwb2Re3avQw21fEVs1z9ZePBqhFokbqlcPq0/r65EGimghUXxAr7RTRih/SkjqYHYdf3ed/1ozvkWpc9M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719365; c=relaxed/simple; bh=lcJN8vLjIwK4WDPsjbRdO6X3eIhgV3xUE56pBLYgXUk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=sHF5QMCXj1JeQSzpBbSKc5x/CJKkqBtQHkn1v4ydEfV/6UIusDEBD98PnoDJFiVTLLopX5dq8/8TXwflHIrvpfitj/I9I7Nj2eJfvTdQk2AEAJPhmYnsIg2Y/hrrzVQZHxBK43/8U1yWZI19KfTwUSzAT6PTDBUVGk7NpNkSuqg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=c8XT6R4h; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="c8XT6R4h" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719356; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=34U/H3bFnFyk6+Y+T6dy301GKxXjP7tSD65G/7DySy4=; b=c8XT6R4hEKzL24zcLfXMwfusPFPhVAoHYjb3ZUzKXC7XARKIL1Mpkt5ds+umMmfGbA6q4/ /LXf5E/s4Lez/5rmYm/7fDk5G0CKNDHYCHQWU4n5/UeBHlavUu7UZrtivI/uFbq/0xnNTT ZtGmUXoIgmf5MFkW2+00Cq7JbRegqFI= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-665-Tj-Gm__fPu2b0mP5B-cC3w-1; Fri, 18 Sep 2026 04:15:52 -0400 X-MC-Unique: Tj-Gm__fPu2b0mP5B-cC3w-1 X-Mimecast-MFC-AGG-ID: Tj-Gm__fPu2b0mP5B-cC3w_1789719351 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 29F8418011E9; Fri, 18 Sep 2026 08:15:51 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 82D091800370; Fri, 18 Sep 2026 08:15:50 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 08/28] KVM: x86/mmu: Extend map_writable to a full ACC_* mask Date: Fri, 18 Sep 2026 04:15:23 -0400 Message-ID: <20260918081543.139871-9-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" Support for memory protection attributes opens the door to installing non-executable mappings. Instead of introducing yet another member in struct kvm_page_fault and another argument to make_spte(), make the existing member map_writable a mask of ACC_* bits. This also avoids the need for make_spte() to map a single bool to either the NX bit or the XS/XU bits together. Unlike for mappings that are not writable because the fault did not request write premission, it is not not necessary to track executability for these SPTEs; the gfn is always available and it will be possible to access the attributes directly in FNAME(sync_spte). Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 21 ++++++++++++--------- arch/x86/kvm/mmu/mmu_internal.h | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 8 +++++--- arch/x86/kvm/mmu/spte.c | 12 ++++++------ arch/x86/kvm/mmu/spte.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 2 +- 6 files changed, 26 insertions(+), 21 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5996468b7120..b72ccbee0d86 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3105,7 +3105,7 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct= kvm_memory_slot *slot, u64 spte; =20 /* Prefetching always gets a writable pfn. */ - bool host_writable =3D !fault || fault->map_writable; + unsigned host_access =3D fault ? fault->host_access : ACC_ALL; bool prefetch =3D !fault || fault->prefetch; bool write_fault =3D fault && fault->write; =20 @@ -3142,7 +3142,7 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct= kvm_memory_slot *slot, } =20 wrprot =3D make_spte(vcpu, sp, slot, pte_access, gfn, pfn, *sptep, prefet= ch, - false, host_writable, &spte); + false, host_access, &spte); =20 if (*sptep =3D=3D spte) { ret =3D RET_PF_SPURIOUS; @@ -3589,7 +3589,7 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *v= cpu, =20 fault->slot =3D NULL; fault->pfn =3D KVM_PFN_NOSLOT; - fault->map_writable =3D false; + fault->host_access =3D 0; =20 /* * If MMIO caching is disabled, emulate immediately without @@ -4614,7 +4614,8 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu= *vcpu, struct kvm_page_fault *fault, int r) { kvm_release_faultin_page(vcpu->kvm, fault->refcounted_page, - r =3D=3D RET_PF_RETRY, fault->map_writable); + r =3D=3D RET_PF_RETRY, + !!(fault->host_access & ACC_WRITE_MASK)); } =20 static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu *vcpu, @@ -4634,9 +4635,10 @@ static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu = *vcpu, return r; } =20 - fault->map_writable &=3D !(fault->slot->flags & KVM_MEM_READONLY); - fault->max_level =3D kvm_max_level_for_order(max_order); + if (fault->slot->flags & KVM_MEM_READONLY) + fault->host_access &=3D ~ACC_WRITE_MASK; =20 + fault->max_level =3D kvm_max_level_for_order(max_order); return RET_PF_CONTINUE; } =20 @@ -4684,7 +4686,8 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcp= u, &writable, &fault->refcounted_page); =20 out_pf_continue: - fault->map_writable &=3D writable; + if (!writable) + fault->host_access &=3D ~ACC_WRITE_MASK; return RET_PF_CONTINUE; } =20 @@ -5003,7 +5006,7 @@ static int kvm_mmu_do_page_fault(struct kvm_vcpu *vcp= u, gpa_t cr2_or_gpa, .is_private =3D err & PFERR_PRIVATE_ACCESS, =20 .pfn =3D KVM_PFN_ERR_FAULT, - .map_writable =3D true, + .host_access =3D ACC_ALL, }; int r; =20 @@ -5197,7 +5200,7 @@ int kvm_tdp_mmu_map_private_pfn(struct kvm_vcpu *vcpu= , gfn_t gfn, kvm_pfn_t pfn) .gfn =3D gfn, .slot =3D kvm_vcpu_gfn_to_memslot(vcpu, gfn), .pfn =3D pfn, - .map_writable =3D true, + .host_access =3D ACC_ALL, }; struct kvm *kvm =3D vcpu->kvm; int r; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index c29002c60126..00215b9f309f 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -280,7 +280,7 @@ struct kvm_page_fault { unsigned long mmu_seq; kvm_pfn_t pfn; struct page *refcounted_page; - bool map_writable; + u8 host_access; =20 /* * Indicates the guest is trying to write a gfn that contains one or diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 27427e7f22fa..e6ec14165f40 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -935,7 +935,7 @@ static gpa_t FNAME(gva_to_gpa)(struct kvm_vcpu *vcpu, s= truct kvm_pagewalk *w, */ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp= , int i) { - bool host_writable; + u8 host_access; gpa_t first_pte_gpa; u64 *sptep, spte; struct kvm_memory_slot *slot; @@ -992,11 +992,13 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, st= ruct kvm_mmu_page *sp, int =20 sptep =3D &sp->spt[i]; spte =3D *sptep; - host_writable =3D spte & shadow_host_writable_mask; + host_access =3D ACC_ALL; + if (!(spte & shadow_host_writable_mask)) + host_access &=3D ~ACC_WRITE_MASK; slot =3D kvm_vcpu_gfn_to_memslot(vcpu, gfn); make_spte(vcpu, sp, slot, pte_access, gfn, spte_to_pfn(spte), spte, true, true, - host_writable, &spte); + host_access, &spte); =20 /* * There is no need to mark the pfn dirty, as the new protections must diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 5fc27e9733b3..1434164fa372 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -189,7 +189,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_pa= ge *sp, const struct kvm_memory_slot *slot, unsigned int pte_access, gfn_t gfn, kvm_pfn_t pfn, u64 old_spte, bool prefetch, bool synchronizing, - bool host_writable, u64 *new_spte) + unsigned int host_access, u64 *new_spte) { int level =3D sp->role.level; u64 spte =3D SPTE_MMU_PRESENT_MASK; @@ -207,6 +207,11 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_p= age *sp, if (!prefetch || synchronizing) spte |=3D shadow_accessed_mask; =20 + if (host_access & ACC_WRITE_MASK) + spte |=3D shadow_host_writable_mask; + + pte_access &=3D host_access; + /* * For simplicity, enforce the NX huge page mitigation even if not * strictly necessary. KVM could ignore the mitigation if paging is @@ -246,11 +251,6 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_p= age *sp, if (kvm_x86_ops.get_mt_mask) spte |=3D kvm_x86_call(get_mt_mask)(vcpu, gfn, kvm_is_mmio_pfn(pfn, &is_host_mmio)); - if (host_writable) - spte |=3D shadow_host_writable_mask; - else - pte_access &=3D ~ACC_WRITE_MASK; - if (shadow_me_value && !kvm_is_mmio_pfn(pfn, &is_host_mmio)) spte |=3D shadow_me_value; =20 diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index e730717824b3..589f3954633e 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -563,7 +563,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_pa= ge *sp, const struct kvm_memory_slot *slot, unsigned int pte_access, gfn_t gfn, kvm_pfn_t pfn, u64 old_spte, bool prefetch, bool synchronizing, - bool host_writable, u64 *new_spte); + unsigned int host_access, u64 *new_spte); u64 make_small_spte(struct kvm *kvm, u64 huge_spte, union kvm_mmu_page_role role, int index); u64 make_huge_spte(struct kvm *kvm, u64 small_spte, int level); diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 44dad106fad1..dce44b9ce73a 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1143,7 +1143,7 @@ static int tdp_mmu_map_handle_target_level(struct kvm= _vcpu *vcpu, else wrprot =3D make_spte(vcpu, sp, fault->slot, sp->role.access, iter->gfn, fault->pfn, iter->old_spte, fault->prefetch, - false, fault->map_writable, &new_spte); + false, fault->host_access, &new_spte); =20 if (new_spte =3D=3D iter->old_spte) ret =3D RET_PF_SPURIOUS; --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34AEA38330E for ; Fri, 18 Sep 2026 08:15:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719363; cv=none; b=dUXCEuQkRIxigkpRj+VtQDEUxtLxowGM3J8g63svVkHkugMCVc9rLjA5qlFfC5IMotkhGa00AH3OyDjDfRP9MGdjoEp+edNZgYNSJmFzK/KA15RGREX1sfDUT+3K7gl5m5YASM0a2stsEGzpJr24aeyz6hBZ0J+rHk8xuAq+fms= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719363; c=relaxed/simple; bh=XPMcHgIeRQB+eRAm7FCUaCYMyDw7uk/BDzk27PKxyjk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Yo+xe24Tf0A+ZpGs6ix0bnzPCRVtvwmGtAfOQI45uZ0da4bTbIvY00r9fBMw584+O3J4Zn2IICo67Al+kBexe93Hf5dyAYuz2zz31O9xkv9jcUYp13l+FhwDOWIsKi5NRqJV4VbzDR0Wqqivob+cgz50OQ8brL96e2H7lDdXWQE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Chj7F5MF; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Chj7F5MF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719355; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mdubStH0LPL3cLRWju/ttbHM7W6/Ya5BMaguJDp2QnM=; b=Chj7F5MFwQJ3gqt6ctoAnAkGT3soLQm0DoI0nCejrmjfO7LD1+vengz2Bu9QvZlPraDSQb DikMmYMbmB47osHYd9bPw5M0rx5ETDj073AHldeWhenSd74mYxrX8Qepr3hNxPEOYvRM16 95CVQy5xJUx3AK8gdhfFBN6LYPegZPo= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-330-mO_kOgZpPWie48f_gBZDjg-1; Fri, 18 Sep 2026 04:15:53 -0400 X-MC-Unique: mO_kOgZpPWie48f_gBZDjg-1 X-Mimecast-MFC-AGG-ID: mO_kOgZpPWie48f_gBZDjg_1789719352 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 48A831955E75; Fri, 18 Sep 2026 08:15:52 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 522391800370; Fri, 18 Sep 2026 08:15:51 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 09/28] KVM: x86/mmu: Init memslot hugepage information for non-private_mem VMs too Date: Fri, 18 Sep 2026 04:15:24 -0400 Message-ID: <20260918081543.139871-10-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne The list of supported memory attributes is about to grow, and they will be available on any VM type (as opposed to only ones targeted at confidential computing). As such, update the check in kvm_mmu_init_memslot_memory_attributes() to initialize huge page information if any kind of memory attributes is available for the VM. Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 2 +- include/linux/kvm_host.h | 5 +++++ virt/kvm/kvm_main.c | 2 +- 3 files changed, 7 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b72ccbee0d86..ba462c671f7d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -8275,7 +8275,7 @@ void kvm_mmu_init_memslot_memory_attributes(struct kv= m *kvm, { int level; =20 - if (!kvm_arch_has_private_mem(kvm)) + if (!kvm_supported_mem_attributes(kvm)) return; =20 for (level =3D PG_LEVEL_2M; level <=3D KVM_MAX_HUGEPAGE_LEVEL; level++) { diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ce5757e85ae3..acffb2f79b3e 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2579,12 +2579,17 @@ bool kvm_arch_pre_set_memory_attributes(struct kvm = *kvm, struct kvm_gfn_range *range); bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range); +u64 kvm_supported_mem_attributes(struct kvm *kvm); =20 static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) { return kvm_get_memory_attributes(kvm, gfn) & KVM_MEMORY_ATTRIBUTE_PRIVATE; } #else +static inline u64 kvm_supported_mem_attributes(struct kvm *kvm) +{ + return 0; +} static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) { return false; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 65eb26a0520d..3d574e16de45 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2420,7 +2420,7 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm *k= vm, #endif /* CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT */ =20 #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES -static u64 kvm_supported_mem_attributes(struct kvm *kvm) +u64 kvm_supported_mem_attributes(struct kvm *kvm) { if (!kvm || kvm_arch_has_private_mem(kvm)) return KVM_MEMORY_ATTRIBUTE_PRIVATE; --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F2DC49D583 for ; Fri, 18 Sep 2026 08:15:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719365; cv=none; b=SUkkgLCihi/Qo2o1pTXX2lvxHrJWO18EEBKy6InfAKwKswsVNKf5Z7p1j+lWN0eGRcaRA3pguGhldYMd/P7MQq6YAZnERq8qg75BY0R5NpQXSdn1oRD467X04XiEeoTF4E005ZrZ+C2s5PbVvM05jAwDbcvHydlgCpsehPLmgWs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719365; c=relaxed/simple; bh=Tcp4fTVI0amjVPArGkLYPcjTd7iV6Y46W81dENOVLnk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=RY/XuW6+x9b8ohPIPOUbPTjM0m2sY0I01PlIWuqVY2VKVvNCGAVcBAtNzladPt0tr6TkB90Peu94Ii0UHr8YW/2HMRbuXf+DO2z5TouQMZK6c3wOg52m+3PiGoBpLV3xR7YaBstQx5U6Dkk2DAdBYHAZJuVkG6NuwIXUrdlHadk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Ca4C+mKR; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ca4C+mKR" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719358; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iVXB86C7ZXFdBtEEXIU4ekHGfWImSWkvjDcOJJv1cy4=; b=Ca4C+mKRioIvGNpNgzMG46r/Aw26pjSp3fV3QblMGPyxJ2DrphqT38Cz5UweEd1sjy48y4 TiJHiqGfndCT6DnpaEBp8650AdChJVEIntuw1a13HTxdYnA2aWc+xM1aV1rc1OqfgUKta2 IkME5paY+ewYG092otnZKPDK3gnAnLQ= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-447-47zKpceCNfWyDOkVs6hpjg-1; Fri, 18 Sep 2026 04:15:54 -0400 X-MC-Unique: 47zKpceCNfWyDOkVs6hpjg-1 X-Mimecast-MFC-AGG-ID: 47zKpceCNfWyDOkVs6hpjg_1789719353 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 92F1F1954236; Fri, 18 Sep 2026 08:15:53 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E0EDE30001A2; Fri, 18 Sep 2026 08:15:52 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 10/28] KVM: pass kvm == NULL case to kvm_arch_has_private_mem Date: Fri, 18 Sep 2026 04:15:25 -0400 Message-ID: <20260918081543.139871-11-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Allow the architecture-specific code to enable CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES without exposing KVM_MEMORY_ATTRIBUTE_PRIVATE. This is mostly for consistency after introducing memory protection attributes; non-x86 architectures might want to add support for protection attributes without having KVM_MEMORY_ATTRIBUTE_PRIVATE show up in KVM_CHECK_EXTENSION. Signed-off-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index c08781bb0327..c619552740a8 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1853,7 +1853,7 @@ enum kvm_intr_type { (!!in_nmi() =3D=3D ((vcpu)->arch.handling_intr_from_guest =3D=3D KVM_HAN= DLING_NMI))) =20 #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES -#define kvm_arch_has_private_mem(kvm) ((kvm)->arch.has_private_mem) +#define kvm_arch_has_private_mem(kvm) (!(kvm) || (kvm)->arch.has_private_m= em) #endif =20 #define kvm_arch_has_readonly_mem(kvm) (!(kvm)->arch.has_protected_state) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 3d574e16de45..9d1836382d19 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2422,7 +2422,7 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm *k= vm, #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES u64 kvm_supported_mem_attributes(struct kvm *kvm) { - if (!kvm || kvm_arch_has_private_mem(kvm)) + if (kvm_arch_has_private_mem(kvm)) return KVM_MEMORY_ATTRIBUTE_PRIVATE; =20 return 0; --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A30A64A64E6 for ; Fri, 18 Sep 2026 08:16:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719367; cv=none; b=K36iPvgsMz5zBaFQM1Q9AiWnUqToYZgpgd/kuSAs4SpnO32voAyY8G3AoBo/tuzRjRdyrK7N/ZZFHuhEHcYibILlmzt8ptxvLdPZidyHNbUxqz9KJiqs33flT44hoHVt4oUPKSOUnMe6lV91bb8OKtvNF8O3H+TilwzXGzqj0cI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719367; c=relaxed/simple; bh=KFbaMqbiLFRP87KR2IT5l978ib4/M2BclxnGHkZsbZ8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=KczLQ7Q8exvaIXJoYp+Jl7I83hmBDjVc6FXx8wx+xEr9UKD14gVm9M5zzT0142CHwO0HYs4i+U9z9cvLBwYtFmong4oA3ewteYC+wbhtC0q/xR34+qW92MqEJVr2+2RW9RpdBc7BirKRYWpJDRxir5lW7wqNue6JB3Y0asnRmjc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=KNRydRIN; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KNRydRIN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719360; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KZw+s9+hNkGrvJFNAo5F/Jik+U9iZGIw5s6mYsPaxqg=; b=KNRydRINo1GmPgnsmL2/8VLwuOczNZ9sZwiVdP8eFzAyXdr0YWT1Yk4/eCNLlQdgcjeQcy vlJwW9AXySTRyc99x8/VkC9tjnyiv9SwsR8IpDiWL1qSckkgMinwgK5sd19/KhWn4IdoUR c31IHBrSoE9g0iEtEp1LcKOnGwQWQzM= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-449-kIxNs8okMZKEfFLXOTofQA-1; Fri, 18 Sep 2026 04:15:55 -0400 X-MC-Unique: kIxNs8okMZKEfFLXOTofQA-1 X-Mimecast-MFC-AGG-ID: kIxNs8okMZKEfFLXOTofQA_1789719354 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 649A519560B3; Fri, 18 Sep 2026 08:15:54 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BA0B730001A2; Fri, 18 Sep 2026 08:15:53 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 11/28] KVM: adjust for presence of more than one attribute Date: Fri, 18 Sep 2026 04:15:26 -0400 Message-ID: <20260918081543.139871-12-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Remove or adjust code that assumed that the only valid attribute was KVM_MEMORY_ATTRIBUTE_PRIVATE, in preparation for the introduction of RWX memory attributes. In particular, the optimization in kvm_range_has_memory_attributes simply is not valid if attrs =3D=3D 0 but some of the attributes are being ignored. The WARNs in arch/x86/kvm/mmu/mmu.c are just a duplicate of checking kvm_supported_mem_attributes() during the ioctl, so drop it. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 9 +-------- virt/kvm/kvm_main.c | 6 ++++-- 2 files changed, 5 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index ba462c671f7d..77d9c62d2a07 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -8140,9 +8140,6 @@ bool kvm_arch_pre_set_memory_attributes(struct kvm *k= vm, * Zapping SPTEs in this case ensures KVM will reassess whether or not * a hugepage can be used for affected ranges. */ - if (WARN_ON_ONCE(!kvm_arch_has_private_mem(kvm))) - return false; - if (WARN_ON_ONCE(range->end <=3D range->start)) return false; =20 @@ -8218,11 +8215,7 @@ bool kvm_arch_post_set_memory_attributes(struct kvm = *kvm, * can't map memory PRIVATE. KVM mustn't create a SHARED hugepage over * a range that has PRIVATE GFNs, and conversely converting a range to * SHARED may now allow hugepages. - */ - if (WARN_ON_ONCE(!kvm_arch_has_private_mem(kvm))) - return false; - - /* + * * The sequence matters here: upper levels consume the result of lower * level's scanning. */ diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 9d1836382d19..85fdd33c51ab 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2436,10 +2436,12 @@ bool kvm_range_has_memory_attributes(struct kvm *kv= m, gfn_t start, gfn_t end, unsigned long mask, unsigned long attrs) { XA_STATE(xas, &kvm->mem_attr_array, start); + unsigned long supported_attrs; unsigned long index; void *entry; =20 - mask &=3D kvm_supported_mem_attributes(kvm); + supported_attrs =3D kvm_supported_mem_attributes(kvm); + mask &=3D supported_attrs; if (attrs & ~mask) return false; =20 @@ -2447,7 +2449,7 @@ bool kvm_range_has_memory_attributes(struct kvm *kvm,= gfn_t start, gfn_t end, return (kvm_get_memory_attributes(kvm, start) & mask) =3D=3D attrs; =20 guard(rcu)(); - if (!attrs) + if (!attrs && mask =3D=3D supported_attrs) return !xas_find(&xas, end - 1); =20 for (index =3D start; index < end; index++) { --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0933485CF7 for ; Fri, 18 Sep 2026 08:16:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719368; cv=none; b=J5jQ4OqsPqj9DDOIgVCMwSD8cu9JK/DmCe7yFc4F7g8oBAGn0PeI0vAh2n1kVcEYaj9MCG23wqDHYtYvZcDy/dgQ63/C+P9QFeJ6G7zDb47rKaylp3UjYPryMoeh3u0tz8JMhcIAehvRbc0p7Usd4qosG1Ic4L3DTAafKeorYNY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719368; c=relaxed/simple; bh=YU1RSB3tZW4H9GuENgsxEPTL++6T6u9XlO8HNcE2XWM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=duQ8F+8/1J/ADTwXhYXYFB8KTe7uutdpJfrNoUkLCh8Q3vvnLcoyE8cI/mr29tHHFiEkHvPl/GdjaudASbn8aT2zm6rCwWUNGU7PNRl8rHt4bnaAxjKhJJbEKzpGylcUK0+0ks43aWHrHzy1fdvA1pn+Vw+RD/XihJv0LmPYGf0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Xig2iW/h; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Xig2iW/h" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719359; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sR53KYSP3YZe5CDWYlXVBqA1sDrK0TUryY4DDJSrDG4=; b=Xig2iW/hbibXzCwRnb2+VBU2nUf9s78IqaWgt0nTYIbvQAg6hGFKmCk5ohz3fUdGVJVHPB 48khfbwktncUDOtCFXlGr9SjtQBGvx3BrRyiC/x7TVPB30mSg6tQrwluU+cKeVK4L+Ox3/ R1ppjOrCUnPpwTdj6Tv3HehX17GKhI8= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-683-JNQ_VpuqNQmDIQplP0LB3w-1; Fri, 18 Sep 2026 04:15:56 -0400 X-MC-Unique: JNQ_VpuqNQmDIQplP0LB3w-1 X-Mimecast-MFC-AGG-ID: JNQ_VpuqNQmDIQplP0LB3w_1789719355 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3E96A19560BB; Fri, 18 Sep 2026 08:15:55 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8B73430001A2; Fri, 18 Sep 2026 08:15:54 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 12/28] KVM: Introduce NR/NW/NX memory attributes Date: Fri, 18 Sep 2026 04:15:27 -0400 Message-ID: <20260918081543.139871-13-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne Introduce memory attributes to map guest physical memory regions as non-readable, non-writable, and/or non-executable. Only a subset of flag combinations is supported. Notably write and exec permissions require read permission, and memory protection attributes are incompatible with private memory. As mentioned in 5a475554db1e ("KVM: Introduce per-page memory attributes", 2023-11-13), bits 0-2 of the memory attributes were reserved for RWX protection; they are negated to support current memory attribute users which use 0 to indicate no special treatment. Since 0 is not available, a non-negated version of the flags would need an extra bit to express no-access (R=3D0/W=3D0/X=3D0) mappings. Unfortunately this precaution did not age too well; KVM now supports MBEC/GMET and adding mode-based memory protections will require a non-contiguous bit. But that's something left for later. Different architectures may have different limitations on the set of valid protections, for example execution-only and XU=3D0 mappings are supported by Intel but not AMD processors[1]. So, add an architecture-spec= ific callback and add a basic implementation for x86. [1] When adding support for MBEC/GMET, since NX would remain to mean no execution at all, it is possible to use either an NXS bit or two separate NXS/NXU bits in addition to NX. The former would only support permissions that are available with either MBEC or GMET. Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- Documentation/virt/kvm/api.rst | 14 +++++++--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/Kconfig | 4 +-- arch/x86/kvm/mmu/mmu.c | 47 +++++++++++++++++++++++++-------- arch/x86/kvm/x86.c | 2 -- include/linux/kvm_host.h | 28 +++++++++++++++++++- include/uapi/linux/kvm.h | 3 +++ virt/kvm/kvm_main.c | 32 +++++++++++++++++++--- 8 files changed, 107 insertions(+), 24 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index abc2ff1f8c84..81126ac11bf8 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6414,15 +6414,23 @@ of guest physical memory. __u64 flags; }; =20 + #define KVM_MEMORY_ATTRIBUTE_NR (1ULL << 0) + #define KVM_MEMORY_ATTRIBUTE_NW (1ULL << 1) + #define KVM_MEMORY_ATTRIBUTE_NX (1ULL << 2) #define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3) =20 The address and size must be page aligned. The supported attributes can be retrieved via ioctl(KVM_CHECK_EXTENSION) on KVM_CAP_MEMORY_ATTRIBUTES. If executed on a VM, KVM_CAP_MEMORY_ATTRIBUTES precisely returns the attribut= es supported by that VM. If executed at system scope, KVM_CAP_MEMORY_ATTRIBU= TES -returns all attributes supported by KVM. The only attribute defined at th= is -time is KVM_MEMORY_ATTRIBUTE_PRIVATE, which marks the associated gfn as be= ing -guest private memory. +returns all attributes supported by KVM. The attribute defined at this +time are: + + - KVM_MEMORY_ATTRIBUTE_NR/NW/NX - Respectively marks the memory region as + non-read, non-write and/or non-exec. Note that write-only, exec-only a= nd + write-exec mappings are not supported. + - KVM_MEMORY_ATTRIBUTE_PRIVATE - Which marks the associated gfn as being = guest + private memory. =20 Note, there is no "get" API. Userspace is responsible for explicitly trac= king the state of a gfn/page as needed. diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index c619552740a8..52c432f9bc66 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1853,6 +1853,7 @@ enum kvm_intr_type { (!!in_nmi() =3D=3D ((vcpu)->arch.handling_intr_from_guest =3D=3D KVM_HAN= DLING_NMI))) =20 #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#define kvm_arch_has_memory_protection_attributes(kvm) (!(kvm) || !(kvm)->= arch.has_private_mem) #define kvm_arch_has_private_mem(kvm) (!(kvm) || (kvm)->arch.has_private_m= em) #endif =20 diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index 538ed1e80332..4c630fd64fd2 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -48,6 +48,7 @@ config KVM_X86 select KVM_GENERIC_PRE_FAULT_MEMORY select KVM_WERROR if WERROR select KVM_GUEST_MEMFD if X86_64 + select KVM_GENERIC_MEMORY_ATTRIBUTES =20 config KVM tristate "Kernel-based Virtual Machine (KVM) support" @@ -84,7 +85,6 @@ config KVM_SW_PROTECTED_VM bool "Enable support for KVM software-protected VMs" depends on EXPERT depends on KVM_X86 && X86_64 - select KVM_GENERIC_MEMORY_ATTRIBUTES help Enable support for KVM software-protected VMs. Currently, software- protected VMs are purely a development and testing vehicle for @@ -135,7 +135,6 @@ config KVM_INTEL_TDX bool "Intel Trust Domain Extensions (TDX) support" default y depends on INTEL_TDX_HOST - select KVM_GENERIC_MEMORY_ATTRIBUTES select HAVE_KVM_ARCH_GMEM_POPULATE help Provides support for launching Intel Trust Domain Extensions (TDX) @@ -159,7 +158,6 @@ config KVM_AMD_SEV depends on KVM_AMD && X86_64 depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=3Dy && CRYPTO_DEV_CCP_DD=3Dm) select ARCH_HAS_CC_PLATFORM - select KVM_GENERIC_MEMORY_ATTRIBUTES select HAVE_KVM_ARCH_GMEM_CONVERT select HAVE_KVM_ARCH_GMEM_RECLAIM select HAVE_KVM_ARCH_GMEM_INVALIDATE diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 77d9c62d2a07..797b18201d98 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -8104,7 +8104,6 @@ void kvm_mmu_pre_destroy_vm(struct kvm *kvm) vhost_task_stop(kvm->arch.nx_huge_page_recovery_thread); } =20 -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES static bool hugepage_test_mixed(struct kvm_memory_slot *slot, gfn_t gfn, int level) { @@ -8130,16 +8129,23 @@ bool kvm_arch_pre_set_memory_attributes(struct kvm = *kvm, int level; =20 /* - * Zap SPTEs even if the slot can't be mapped PRIVATE. KVM x86 only - * supports KVM_MEMORY_ATTRIBUTE_PRIVATE, and so it *seems* like KVM - * can simply ignore such slots. But if userspace is making memory - * PRIVATE, then KVM must prevent the guest from accessing the memory - * as shared. And if userspace is making memory SHARED and this point - * is reached, then at least one page within the range was previously - * PRIVATE, i.e. the slot's possible hugepage ranges are changing. - * Zapping SPTEs in this case ensures KVM will reassess whether or not - * a hugepage can be used for affected ranges. + * For KVM_MEMORY_ATTRIBUTE_PRIVATE: + * Zap SPTEs even if the slot can't be mapped PRIVATE. KVM x86 only + * supports KVM_MEMORY_ATTRIBUTE_PRIVATE, and so it *seems* like KVM + * can simply ignore such slots. But if userspace is making memory + * PRIVATE, then KVM must prevent the guest from accessing the memory + * as shared. And if userspace is making memory SHARED and this point + * is reached, then at least one page within the range was previously + * PRIVATE, i.e. the slot's possible hugepage ranges are changing. + * Zapping SPTEs in this case ensures KVM will reassess whether or not + * a hugepage can be used for affected ranges. + * + * For KVM_MEMORY_ATTRIBUTE_NR/NW/NX: + * Zap even when loosening restrictions R=3D>RW, which is nost strictly + * necessary, but will allow KVM to reasses whether a hugepage can be + * used for the affected pages. */ + if (WARN_ON_ONCE(range->end <=3D range->start)) return false; =20 @@ -8299,4 +8305,23 @@ void kvm_mmu_init_memslot_memory_attributes(struct k= vm *kvm, } } } -#endif + +/* The bits are flipped but remain in the same position. */ +#define KVM_PROT_READ KVM_MEMORY_ATTRIBUTE_NR +#define KVM_PROT_WRITE KVM_MEMORY_ATTRIBUTE_NW +#define KVM_PROT_EXEC KVM_MEMORY_ATTRIBUTE_NX + +bool kvm_arch_mem_attributes_supported_prot(struct kvm *kvm, unsigned long= attrs) +{ + unsigned long prot =3D (attrs & KVM_MEMORY_ATTRIBUTE_PROT) ^ KVM_MEMORY_A= TTRIBUTE_PROT; + + /* Private memory and access permissions are incompatible */ + if (attrs & KVM_MEMORY_ATTRIBUTE_PRIVATE) + return false; + + /* For now do now support exec-only, even though EPT can handle it. */ + if (prot && !(prot & KVM_PROT_READ)) + return false; + + return true; +} diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c10b3fd48b65..416b2ebf1c70 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10163,9 +10163,7 @@ static int kvm_alloc_memslot_metadata(struct kvm *k= vm, } } =20 -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES kvm_mmu_init_memslot_memory_attributes(kvm, slot); -#endif =20 if (kvm_page_track_create_memslot(kvm, slot, npages)) goto out_free; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index acffb2f79b3e..c215e3bdf043 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -722,7 +722,9 @@ static inline int kvm_arch_vcpu_memslots_id(struct kvm_= vcpu *vcpu) } #endif =20 -#ifndef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +bool kvm_arch_mem_attributes_supported_prot(struct kvm *kvm, unsigned long= attrs); +#else static inline bool kvm_arch_has_private_mem(struct kvm *kvm) { return false; @@ -2567,7 +2569,25 @@ static inline bool kvm_memslot_is_gmem_only(const st= ruct kvm_memory_slot *slot) return slot->flags & KVM_MEMSLOT_GMEM_ONLY; } =20 +static inline bool kvm_mem_attributes_may_read(u64 attrs) +{ + return !(attrs & KVM_MEMORY_ATTRIBUTE_NR); +} + +static inline bool kvm_mem_attributes_may_write(u64 attrs) +{ + return !(attrs & KVM_MEMORY_ATTRIBUTE_NW); +} + +static inline bool kvm_mem_attributes_may_exec(u64 attrs) +{ + return !(attrs & KVM_MEMORY_ATTRIBUTE_NX); +} + #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +#define KVM_MEMORY_ATTRIBUTE_PROT \ + (KVM_MEMORY_ATTRIBUTE_NR | KVM_MEMORY_ATTRIBUTE_NW | KVM_MEMORY_ATTRIBUTE= _NX) + static inline unsigned long kvm_get_memory_attributes(struct kvm *kvm, gfn= _t gfn) { return xa_to_value(xa_load(&kvm->mem_attr_array, gfn)); @@ -2579,6 +2599,7 @@ bool kvm_arch_pre_set_memory_attributes(struct kvm *k= vm, struct kvm_gfn_range *range); bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range); +bool kvm_mem_attributes_valid(struct kvm *kvm, unsigned long attrs); u64 kvm_supported_mem_attributes(struct kvm *kvm); =20 static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) @@ -2586,6 +2607,11 @@ static inline bool kvm_mem_is_private(struct kvm *kv= m, gfn_t gfn) return kvm_get_memory_attributes(kvm, gfn) & KVM_MEMORY_ATTRIBUTE_PRIVATE; } #else +static inline bool kvm_mem_attributes_valid(struct kvm *kvm, + unsigned long attrs) +{ + return false; +} static inline u64 kvm_supported_mem_attributes(struct kvm *kvm) { return 0; diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index c23f1ea62eaf..ffbc68a6e2c9 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1657,6 +1657,9 @@ struct kvm_memory_attributes { __u64 flags; }; =20 +#define KVM_MEMORY_ATTRIBUTE_NR (1ULL << 0) +#define KVM_MEMORY_ATTRIBUTE_NW (1ULL << 1) +#define KVM_MEMORY_ATTRIBUTE_NX (1ULL << 2) #define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3) =20 #define KVM_CREATE_GUEST_MEMFD _IOWR(KVMIO, 0xd4, struct kvm_create_guest= _memfd) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 85fdd33c51ab..f98104169862 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2422,10 +2422,15 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm = *kvm, #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES u64 kvm_supported_mem_attributes(struct kvm *kvm) { - if (kvm_arch_has_private_mem(kvm)) - return KVM_MEMORY_ATTRIBUTE_PRIVATE; + u64 supported_attrs =3D 0; =20 - return 0; + if (kvm_arch_has_memory_protection_attributes(kvm)) + supported_attrs |=3D KVM_MEMORY_ATTRIBUTE_PROT; + + if (kvm_arch_has_private_mem(kvm)) + supported_attrs |=3D KVM_MEMORY_ATTRIBUTE_PRIVATE; + + return supported_attrs; } =20 /* @@ -2599,6 +2604,25 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm= , gfn_t start, gfn_t end, =20 return r; } + +bool __weak kvm_arch_mem_attributes_supported_prot(struct kvm *kvm, unsign= ed long attrs) +{ + WARN_ON_ONCE("KVM_MEMORY_ATTRIBUTE_PROT requires kvm_arch_mem_attributes_= supported_prot()"); + return false; +} + +bool kvm_mem_attributes_valid(struct kvm *kvm, unsigned long attrs) +{ + if (attrs & ~kvm_supported_mem_attributes(kvm)) + return false; + + if ((attrs & KVM_MEMORY_ATTRIBUTE_PROT) && + !kvm_arch_mem_attributes_supported_prot(kvm, attrs)) + return false; + + return true; +} + static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, struct kvm_memory_attributes *attrs) { @@ -2607,7 +2631,7 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm= *kvm, /* flags is currently not used. */ if (attrs->flags) return -EINVAL; - if (attrs->attributes & ~kvm_supported_mem_attributes(kvm)) + if (!kvm_mem_attributes_valid(kvm, attrs->attributes)) return -EINVAL; if (attrs->size =3D=3D 0 || attrs->address + attrs->size < attrs->address) return -EINVAL; --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42DEB35C69F for ; Fri, 18 Sep 2026 08:16:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719369; cv=none; b=aRaRvAppF4RXdmdGaJXV7cOfZmiQ5Rw20zowerDUEr2kCc82QlWG3725p5gcgGK++G93r4XrVa++fND81jtoDgAPLY2jWgntbmFUX4wq6IUoJRCnwC2W0pDty/iOJ+EtW2v41YYggPE7Z0A5Jtmv6k2Y51rk8Wx+oXAbfUtBl20= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719369; c=relaxed/simple; bh=0KSm60RmbVxBGC+ZwxwAp4zJAlyv12vvrsCfKg1n46o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=iCqcampJ2d/0w9gU/faWPAxEByjUSpHFJyCpjx5QtVLpW8vrL7FH+Ctbm8N7kcW1Pq+hSR1CQ0VKtjE+w/OdpKl+dEKkjoVy+j9/BG7uFXV0eJHQYEoGpvWIAVPXzRFsvMPSCndlvHevfVI9NnpEHWFrZZeka88ZG4hX9ySeiUU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=az+fRn0O; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="az+fRn0O" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719360; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ds2Lc+aMh2WymrMTxljwuJtfkBBHdr6kIvvoHFilRIs=; b=az+fRn0OmzUGN9kB3/jfDZgsol3ANDBiIikxl4dtzJOvzhGee8jHM4sOEN4kyuS4wF5b/3 5/pMamq9zYeHCgxFV4/VDiJI0uTUNVI7IzdUfc5s6lOp/w7G28q+ImN1K4ZMTedhmIxGZ7 5G63q0m9tRrh1mAxpqr4nJEOgqUgduQ= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-446-QK---mVANnWbuPIW32toZw-1; Fri, 18 Sep 2026 04:15:57 -0400 X-MC-Unique: QK---mVANnWbuPIW32toZw-1 X-Mimecast-MFC-AGG-ID: QK---mVANnWbuPIW32toZw_1789719356 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0F89318512C6; Fri, 18 Sep 2026 08:15:56 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 64DAD30001A2; Fri, 18 Sep 2026 08:15:55 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 13/28] KVM: Include memory protections in result of gfn->hva conversion Date: Fri, 18 Sep 2026 04:15:28 -0400 Message-ID: <20260918081543.139871-14-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne All paths that lead to guest memory accesses now need to check whether memory attributes allow that access. Users of gfn_to_hva, kvm_vcpu_gfn_to_hva and their *_prot variant can get it more or less for free via erroneous return values, so do this first. Note however that this is not true of the variants that take a cached kvm_memslots pointer. These include caches (gfn-to-hva and gfn-to-pfn) and page faults, both of which will need specific changes; but the more optimized functions in virt/kvm/kvm_main.c such as kvm_read_guest() and kvm_write_guest() also retrieve the memslot high in the call chain, and therefore they will need changes in __kvm_read/write_guest_page(). Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- include/linux/kvm_host.h | 18 +++++++++++++ virt/kvm/kvm_main.c | 55 +++++++++++++++++++++++++++++++++++++--- 2 files changed, 69 insertions(+), 4 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index c215e3bdf043..01223301f0e5 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2620,8 +2620,26 @@ static inline bool kvm_mem_is_private(struct kvm *kv= m, gfn_t gfn) { return false; } +static inline unsigned long kvm_get_memory_attributes(struct kvm *kvm, gfn= _t gfn) +{ + return 0; +} #endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */ =20 +static inline int kvm_mem_attributes_may_read_gfn(struct kvm *kvm, gfn_t g= fn) +{ + unsigned long attrs =3D kvm_get_memory_attributes(kvm, gfn); + + return kvm_mem_attributes_may_read(attrs); +} + +static inline int kvm_mem_attributes_may_write_gfn(struct kvm *kvm, gfn_t = gfn) +{ + unsigned long attrs =3D kvm_get_memory_attributes(kvm, gfn); + + return kvm_mem_attributes_may_write(attrs); +} + #ifdef CONFIG_KVM_GUEST_MEMFD int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, kvm_pfn_t *pfn, struct page **page, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index f98104169862..b1cc570be8df 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2767,13 +2767,38 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(gfn_to_hva_memslot); =20 unsigned long gfn_to_hva(struct kvm *kvm, gfn_t gfn) { - return gfn_to_hva_many(gfn_to_memslot(kvm, gfn), gfn, NULL); + unsigned long addr; + + addr =3D gfn_to_hva_many(gfn_to_memslot(kvm, gfn), gfn, NULL); + if (kvm_is_error_hva(addr)) + return addr; + + if (!kvm_mem_attributes_may_read_gfn(kvm, gfn)) + return KVM_HVA_ERR_BAD; + + if (!kvm_mem_attributes_may_write_gfn(kvm, gfn)) + return KVM_HVA_ERR_RO_BAD; + + return addr; } EXPORT_SYMBOL_FOR_KVM_INTERNAL(gfn_to_hva); =20 unsigned long kvm_vcpu_gfn_to_hva(struct kvm_vcpu *vcpu, gfn_t gfn) { - return gfn_to_hva_many(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn, NULL); + struct kvm *kvm =3D vcpu->kvm; + unsigned long addr; + + addr =3D gfn_to_hva_many(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn, NULL); + if (kvm_is_error_hva(addr)) + return addr; + + if (!kvm_mem_attributes_may_read_gfn(kvm, gfn)) + return KVM_HVA_ERR_BAD; + + if (!kvm_mem_attributes_may_write_gfn(kvm, gfn)) + return KVM_HVA_ERR_RO_BAD; + + return addr; } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_vcpu_gfn_to_hva); =20 @@ -2799,15 +2824,37 @@ unsigned long gfn_to_hva_memslot_prot(struct kvm_me= mory_slot *slot, unsigned long gfn_to_hva_prot(struct kvm *kvm, gfn_t gfn, bool *writable) { struct kvm_memory_slot *slot =3D gfn_to_memslot(kvm, gfn); + unsigned long addr; =20 - return gfn_to_hva_memslot_prot(slot, gfn, writable); + addr =3D gfn_to_hva_memslot_prot(slot, gfn, writable); + if (kvm_is_error_hva(addr)) + return addr; + + if (!kvm_mem_attributes_may_read_gfn(kvm, gfn)) + return KVM_HVA_ERR_BAD; + + if (writable && !kvm_mem_attributes_may_write_gfn(kvm, gfn)) + *writable =3D false; + + return addr; } =20 unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, b= ool *writable) { struct kvm_memory_slot *slot =3D kvm_vcpu_gfn_to_memslot(vcpu, gfn); + unsigned long addr; =20 - return gfn_to_hva_memslot_prot(slot, gfn, writable); + addr =3D gfn_to_hva_memslot_prot(slot, gfn, writable); + if (kvm_is_error_hva(addr)) + return addr; + + if (!kvm_mem_attributes_may_read_gfn(vcpu->kvm, gfn)) + return KVM_HVA_ERR_BAD; + + if (writable && !kvm_mem_attributes_may_write_gfn(vcpu->kvm, gfn)) + *writable =3D false; + + return addr; } =20 static bool kvm_is_ad_tracked_page(struct page *page) --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8A9F4ACC86 for ; Fri, 18 Sep 2026 08:16:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719367; cv=none; b=tOjXK+8CYngUmaOrhLrAVr+GF977OroC9y/0VXJ+dqKjjslootsKJiAl47kFIS5jgPl9Jut83Ydw/wf1cna0mIDgnQoJ6dOetNAEyAAY0TcmhpzHC7jNxJ9nh8T1Js7LEbBD1X+/wLGiUECuuY3KQlC+VQWvL59ZQGk0EUFrheM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719367; c=relaxed/simple; bh=QOO98tAN/8/3LDBC/DGy5oKr+hlvwrBnFKaB/8Inwn4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Tq99wIvYIUoKtncO7M9XtKKkXwbiMuWmbPWgCTES9wHH3J41DMkF3cSrrQpwx5vmMisWQJ8U4iJP3Dwb3Po5MZtFMMTfImD+sYfxJ0NCK+aPXMj1FYsm2M3TSurPtVPIW8h0m330mZ6eSx4IDAeIGYWLy1T1t2jfOOuOs9Tx5IM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=MpgA5Tc1; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MpgA5Tc1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719361; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uKhaNYp4wxpZzu7WulZV4wey1GkqmjaDkMFDmXM/CIA=; b=MpgA5Tc1Di9c7IqKFqHSWwROEId1BO4gxCQwvOydYB0rV0T19OAVbTdrJxCDP/5Uh7+Rty DPquaJSR+w64gHITwBWXwL4HHjjCIbC/Mk06TUWU5Md7pWcz6GRiikMl3xdZ6lMCZCudXd dRd5ODwnvwTDckPkjYnZyzODD+On1RI= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-498-SnqQHGLJO7yb6UKzsQ7MDg-1; Fri, 18 Sep 2026 04:15:58 -0400 X-MC-Unique: SnqQHGLJO7yb6UKzsQ7MDg-1 X-Mimecast-MFC-AGG-ID: SnqQHGLJO7yb6UKzsQ7MDg_1789719356 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D4249195419D; Fri, 18 Sep 2026 08:15:56 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 3598030001A2; Fri, 18 Sep 2026 08:15:56 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 14/28] KVM: Introduce kvm_fetch_guest_page() and use it for x86 Date: Fri, 18 Sep 2026 04:15:29 -0400 Message-ID: <20260918081543.139871-15-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Introduce a separate function so that the emulator will be able to take into account execution memory attributes rather than read. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/x86.c | 4 ++-- include/linux/kvm_host.h | 2 ++ virt/kvm/kvm_main.c | 9 +++++++++ 3 files changed, 13 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 416b2ebf1c70..6c0c95b92b5a 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4872,8 +4872,8 @@ static int kvm_fetch_guest_virt(struct x86_emulate_ct= xt *ctxt, offset =3D addr & (PAGE_SIZE-1); if (WARN_ON(offset + bytes > PAGE_SIZE)) bytes =3D (unsigned)PAGE_SIZE - offset; - ret =3D kvm_vcpu_read_guest_page(vcpu, gpa >> PAGE_SHIFT, val, - offset, bytes); + ret =3D kvm_vcpu_fetch_guest_page(vcpu, gpa >> PAGE_SHIFT, val, + offset, bytes); if (unlikely(ret < 0)) return X86EMUL_IO_NEEDED; =20 diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 01223301f0e5..cddf0b943757 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1444,6 +1444,8 @@ DEFINE_VCPU_MAP_CLASS(_readonly); =20 unsigned long kvm_vcpu_gfn_to_hva(struct kvm_vcpu *vcpu, gfn_t gfn); unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, b= ool *writable); +int kvm_vcpu_fetch_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, + int offset, int len); int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data,= int offset, int len); int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu, gpa_t gpa, void *dat= a, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b1cc570be8df..7cb7ff202fc0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3291,6 +3291,15 @@ int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, = gfn_t gfn, void *data, } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_vcpu_read_guest_page); =20 +int kvm_vcpu_fetch_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, + int offset, int len) +{ + struct kvm_memory_slot *slot =3D kvm_vcpu_gfn_to_memslot(vcpu, gfn); + + return __kvm_read_guest_page(vcpu->kvm, slot, gfn, data, offset, len); +} +EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_vcpu_fetch_guest_page); + int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long l= en) { gfn_t gfn =3D gpa >> PAGE_SHIFT; --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1CA8B4137A2 for ; Fri, 18 Sep 2026 08:16:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719369; cv=none; b=oSnaKDdOyN8ecDIfePW6S90nvGTlMaQ/tAX4TK9l7ILwiBR9aHrhBLSTAOsZToTWDETLIlmkYebBJLIgTzjiOxDRrMBggrumjJfkZQPd+QiZkzoBWh/9a3sm45oU19EsgJVWEauGM+umMGwj8CQS5nGMiQXgYAUwvfkvXCnm0/g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719369; c=relaxed/simple; bh=kRAjpQl3F1/aE2QGbMJs9cExZ1mJktOpK1EDeCPZgWg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UCjrD1Xe3aXogUFd2vzkz9ackvyws77Z4kEZZuW6Bvo/wrzB+koQn0j9y5Ta1Xg/cxqvVEv07KnfUPSePaOKTdXCSfdZYzoFi+GK7oKV3WEGMk/VxeOC53Efd3ruGwy0rML4RYAfMWS8PzXZ77NrQkueN7edj+RA7OGjU8n/1Nk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=aKPMciux; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="aKPMciux" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719362; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+jGMqMQItfg7KHf0X3JCN7/XeOQQfBgPY7FxDE1WR7U=; b=aKPMciuxoWu0bvCtJ1P3kUSa8pf4OLNMKwraIbW13Vfpqzh6DZPbqd3ov3Q17GejlzNwk1 hxCAF49NQRfH/wXFMlAKhtt2JyR954OykDUMiF90l3RWwhCVwyhxz1yuKxAlmc3qG4C0fE RuLjhjdm059A+Yg+UCzPcC4oZLskHqM= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-81-jqHocLqfPXiKRY6MimbbTw-1; Fri, 18 Sep 2026 04:15:58 -0400 X-MC-Unique: jqHocLqfPXiKRY6MimbbTw-1 X-Mimecast-MFC-AGG-ID: jqHocLqfPXiKRY6MimbbTw_1789719357 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BD6CC195394A; Fri, 18 Sep 2026 08:15:57 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 070B83000218; Fri, 18 Sep 2026 08:15:56 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 15/28] KVM: Take memory protections into account for memory read/write/fetch Date: Fri, 18 Sep 2026 04:15:30 -0400 Message-ID: <20260918081543.139871-16-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne Take into account memory attributes when accessing guest memory through the kvm_{read,write,fetch}*() family of functions. All of these pass a struct kvm_memory_slot pointer to the actual workhorse functions, in order to share code between the VM-wide and vCPU-specific version of the functions (the latter of which handles the multi-address-space case). For this reason they need specific changes and do not work even though the gfn_to_hva() path has been taught already about memory protection attributes. Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- virt/kvm/kvm_main.c | 34 ++++++++++++++++++++++++++-------- 1 file changed, 26 insertions(+), 8 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 7cb7ff202fc0..3932e526870a 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3255,11 +3255,13 @@ static int next_segment(unsigned long len, int offs= et) } =20 /* Copy @len bytes from guest memory at '(@gfn * PAGE_SIZE) + @offset' to = @data */ -static int __kvm_read_guest_page(struct kvm_memory_slot *slot, gfn_t gfn, - void *data, int offset, int len) +static int __kvm_read_guest_page(struct kvm *kvm, struct kvm_memory_slot *= slot, + gfn_t gfn, void *data, int offset, int len, + unsigned long attr) { int r; unsigned long addr; + unsigned long attrs; =20 if (WARN_ON_ONCE(offset + len > PAGE_SIZE)) return -EFAULT; @@ -3267,6 +3269,11 @@ static int __kvm_read_guest_page(struct kvm_memory_s= lot *slot, gfn_t gfn, addr =3D gfn_to_hva_memslot_prot(slot, gfn, NULL); if (kvm_is_error_hva(addr)) return -EFAULT; + + attrs =3D kvm_get_memory_attributes(kvm, gfn); + if (attrs & attr) + return -EFAULT; + r =3D __copy_from_user(data, (void __user *)addr + offset, len); if (r) return -EFAULT; @@ -3278,7 +3285,8 @@ int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, v= oid *data, int offset, { struct kvm_memory_slot *slot =3D gfn_to_memslot(kvm, gfn); =20 - return __kvm_read_guest_page(slot, gfn, data, offset, len); + return __kvm_read_guest_page(kvm, slot, gfn, data, offset, len, + KVM_MEMORY_ATTRIBUTE_NR); } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_read_guest_page); =20 @@ -3287,7 +3295,8 @@ int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, g= fn_t gfn, void *data, { struct kvm_memory_slot *slot =3D kvm_vcpu_gfn_to_memslot(vcpu, gfn); =20 - return __kvm_read_guest_page(slot, gfn, data, offset, len); + return __kvm_read_guest_page(vcpu->kvm, slot, gfn, data, offset, len, + KVM_MEMORY_ATTRIBUTE_NR); } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_vcpu_read_guest_page); =20 @@ -3296,7 +3305,8 @@ int kvm_vcpu_fetch_guest_page(struct kvm_vcpu *vcpu, = gfn_t gfn, void *data, { struct kvm_memory_slot *slot =3D kvm_vcpu_gfn_to_memslot(vcpu, gfn); =20 - return __kvm_read_guest_page(vcpu->kvm, slot, gfn, data, offset, len); + return __kvm_read_guest_page(vcpu->kvm, slot, gfn, data, offset, len, + KVM_MEMORY_ATTRIBUTE_NX); } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_vcpu_fetch_guest_page); =20 @@ -3340,8 +3350,9 @@ int kvm_vcpu_read_guest(struct kvm_vcpu *vcpu, gpa_t = gpa, void *data, unsigned l } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_vcpu_read_guest); =20 -static int __kvm_read_guest_atomic(struct kvm_memory_slot *slot, gfn_t gfn, - void *data, int offset, unsigned long len) +static int __kvm_read_guest_atomic(struct kvm *kvm, + struct kvm_memory_slot *slot, gfn_t gfn, + void *data, int offset, unsigned long len) { int r; unsigned long addr; @@ -3349,6 +3360,9 @@ static int __kvm_read_guest_atomic(struct kvm_memory_= slot *slot, gfn_t gfn, if (WARN_ON_ONCE(offset + len > PAGE_SIZE)) return -EFAULT; =20 + if (!kvm_mem_attributes_may_read_gfn(kvm, gfn)) + return -EFAULT; + addr =3D gfn_to_hva_memslot_prot(slot, gfn, NULL); if (kvm_is_error_hva(addr)) return -EFAULT; @@ -3367,7 +3381,7 @@ int kvm_vcpu_read_guest_atomic(struct kvm_vcpu *vcpu,= gpa_t gpa, struct kvm_memory_slot *slot =3D kvm_vcpu_gfn_to_memslot(vcpu, gfn); int offset =3D offset_in_page(gpa); =20 - return __kvm_read_guest_atomic(slot, gfn, data, offset, len); + return __kvm_read_guest_atomic(vcpu->kvm, slot, gfn, data, offset, len); } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_vcpu_read_guest_atomic); =20 @@ -3385,6 +3399,10 @@ static int __kvm_write_guest_page(struct kvm *kvm, addr =3D gfn_to_hva_memslot(memslot, gfn); if (kvm_is_error_hva(addr)) return -EFAULT; + + if (!kvm_mem_attributes_may_write_gfn(kvm, gfn)) + return -EFAULT; + r =3D __copy_to_user((void __user *)addr + offset, data, len); if (r) return -EFAULT; --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F28B44B0E3B for ; Fri, 18 Sep 2026 08:16:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719372; cv=none; b=nCntbqS/++IPaFFiLvj3DESqJFeJuaFuuxdbrvn8LQ3rMLq12sqaM2j5k1Rnxgtcor+xxhPwoCNYtWhgJe7CfdtFMJ1oIaq3igyL1Z8WDFfyFTAeAIxLbcTOw3P6V7xCEzcOCCrOKFUm/Mn22MUKxb6QSDEQp6G/Cy39CWKbsIU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719372; c=relaxed/simple; bh=obeBnTSJ02v2RMdTMxsu9Fevk06HaAr7bV1BDU65XPI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nCTlTVIvFBMV+qd8+jqUjs4kOgsYu+uApYAtF95UfIrr4dFgWaLpeKqOh3NfBoDwxoq07rAoS75ye6cWind3Dsxoqwz0D3riwjkIrhzOD8TJaFh3CYrvuN+4qzx+h2JtQXAj7r+85ZmvG42ggiutfoMttUno3eNrZQf4rSCJkDM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=NeKT8Uzt; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NeKT8Uzt" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719365; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gWfubM76ShOdscVreN61oLBluSMWPZIXXO7BhgwWri4=; b=NeKT8UztUHK5hdP+6cHo3UYk1gMZoiBmd51GguuxyLs4e9Dg30EFjXt20yfY2kTvFcv79R 59I0faUCX2IJ9cFzmgP12UkF/CP05RjaV52WV1K7zgUvfz1+q9+zhx+9uK4EmQCpKnGwil f0tGdf3kbg6SrqIrQuTXm3htF8+eBAY= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-102-TJRXpLzMPAqpsjV62D8SNw-1; Fri, 18 Sep 2026 04:15:59 -0400 X-MC-Unique: TJRXpLzMPAqpsjV62D8SNw-1 X-Mimecast-MFC-AGG-ID: TJRXpLzMPAqpsjV62D8SNw_1789719358 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 76D351944F03; Fri, 18 Sep 2026 08:15:58 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CCAE230001A2; Fri, 18 Sep 2026 08:15:57 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 16/28] KVM: Encapsulate memattrs array into anonymous struct Date: Fri, 18 Sep 2026 04:15:31 -0400 Message-ID: <20260918081543.139871-17-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne The metadata surrounding memory attributes is about to grow, so encapsulate the memory attributes array within an anonymous struct to provide namespacing. Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- include/linux/kvm_host.h | 8 +++++--- virt/kvm/kvm_main.c | 10 +++++----- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index cddf0b943757..24d04322d89e 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -875,8 +875,10 @@ struct kvm { struct notifier_block pm_notifier; #endif #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES - /* Protected by slots_lock (for writes) and RCU (for reads) */ - struct xarray mem_attr_array; + struct { + /* Protected by slots_lock (for writes) and RCU (for reads) */ + struct xarray array; + } mem_attrs; #endif char stats_id[KVM_STATS_NAME_SIZE]; }; @@ -2592,7 +2594,7 @@ static inline bool kvm_mem_attributes_may_exec(u64 at= trs) =20 static inline unsigned long kvm_get_memory_attributes(struct kvm *kvm, gfn= _t gfn) { - return xa_to_value(xa_load(&kvm->mem_attr_array, gfn)); + return xa_to_value(xa_load(&kvm->mem_attrs.array, gfn)); } =20 bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t e= nd, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 3932e526870a..994a33a09448 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1117,7 +1117,7 @@ static struct kvm *kvm_create_vm(unsigned long type, = const char *fdname) rcuwait_init(&kvm->mn_memslots_update_rcuwait); xa_init(&kvm->vcpu_array); #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES - xa_init(&kvm->mem_attr_array); + xa_init(&kvm->mem_attrs.array); #endif =20 INIT_LIST_HEAD(&kvm->gpc_list); @@ -1302,7 +1302,7 @@ static void kvm_destroy_vm(struct kvm *kvm) srcu_barrier(&kvm->srcu); cleanup_srcu_struct(&kvm->srcu); #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES - xa_destroy(&kvm->mem_attr_array); + xa_destroy(&kvm->mem_attrs.array); #endif kvm_arch_free_vm(kvm); preempt_notifier_dec(); @@ -2440,7 +2440,7 @@ u64 kvm_supported_mem_attributes(struct kvm *kvm) bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t e= nd, unsigned long mask, unsigned long attrs) { - XA_STATE(xas, &kvm->mem_attr_array, start); + XA_STATE(xas, &kvm->mem_attrs.array, start); unsigned long supported_attrs; unsigned long index; void *entry; @@ -2581,7 +2581,7 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm,= gfn_t start, gfn_t end, * partway through setting the new attributes. */ for (i =3D start; i < end; i++) { - r =3D xa_reserve(&kvm->mem_attr_array, i, GFP_KERNEL_ACCOUNT); + r =3D xa_reserve(&kvm->mem_attrs.array, i, GFP_KERNEL_ACCOUNT); if (r) goto out_unlock; =20 @@ -2591,7 +2591,7 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm,= gfn_t start, gfn_t end, kvm_handle_gfn_range(kvm, &pre_set_range); =20 for (i =3D start; i < end; i++) { - r =3D xa_err(xa_store(&kvm->mem_attr_array, i, entry, + r =3D xa_err(xa_store(&kvm->mem_attrs.array, i, entry, GFP_KERNEL_ACCOUNT)); KVM_BUG_ON(r, kvm); cond_resched(); --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 213BC4349B4 for ; Fri, 18 Sep 2026 08:16:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719370; cv=none; b=FKEIDYVjrPJy7VBaq0RgLDEI8cUQTebLjRorO2T5EcXsdD8+sgT/FzPBQHc8nU2l0bF3WOxJfFz6STkoubNG3/ba7tTk5vaZg+gRtTRCQ18RTAZqJ+Swgbe1lE0oZWaOL10VeBbWeU4g/w/sVaw6KEE1u7TcKyqOE6mT37KtM2I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719370; c=relaxed/simple; bh=3W9xldtaGe7Ff2NkNCQJOcGzl15r04ouTawyPDxtIbs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=CaQgMaEAb5lYZJYQs4Mo4yxi1mNvzRVqAqrIn5Cg3jrXQclQLmcdtF6Yszh5FOc/YOlSOqUQ3bFHivAIpeziOYQZJuN0u3iEVYliKsR4izZIrOsLnh/5O6MmgxRpxOoTOMMJEOvSesLS3jSdgsvi36vo172LtGz5n5GnqVJ30j0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ht4K3JGO; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ht4K3JGO" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qTS8CM36DMSHYjVobhMBvWQts9TN4lc2TtJAx1vS/HI=; b=ht4K3JGOJ2BIDPD+qWWVTHmgi+hnRj6oE5HP0trZTX8Bktdaxx8LxhHbpmqmjaWr9KxCVJ pD4fYFUYDvVosQTxFoQ/wCI05GDRcUDC9eRH/zmusXgcx19AlELZhrvaCuUPSpvmT/GPVt k5y9KXQxTTCdgWbd5h82gCMBEnlQkpc= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-37-nM4JldxnPD6IlP0LxSbQbw-1; Fri, 18 Sep 2026 04:16:00 -0400 X-MC-Unique: nM4JldxnPD6IlP0LxSbQbw-1 X-Mimecast-MFC-AGG-ID: nM4JldxnPD6IlP0LxSbQbw_1789719359 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 48E89195604E; Fri, 18 Sep 2026 08:15:59 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9DE5E30001A2; Fri, 18 Sep 2026 08:15:58 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 17/28] KVM: Introduce kvm_check_gen()/kvm_memslots_check_gen() Date: Fri, 18 Sep 2026 04:15:32 -0400 Message-ID: <20260918081543.139871-18-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" In many cases, retrieving kvm_memslots is followed by a check on the generation of the slots. Introduce a helper function that either does the check alone, or compounds it with returning the struct kvm_memslots* to the caller. Signed-off-by: Paolo Bonzini --- arch/x86/kvm/x86.c | 10 ++-------- include/linux/kvm_host.h | 13 +++++++++++++ virt/kvm/kvm_main.c | 8 ++++---- virt/kvm/pfncache.c | 10 ++++------ 4 files changed, 23 insertions(+), 18 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6c0c95b92b5a..5b7ff5d9af75 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2049,7 +2049,6 @@ static void record_steal_time(struct kvm_vcpu *vcpu) { struct gfn_to_hva_cache *ghc =3D &vcpu->arch.st.cache; struct kvm_steal_time __user *st; - struct kvm_memslots *slots; gpa_t gpa =3D vcpu->arch.st.msr_val & KVM_STEAL_VALID_BITS; u64 steal; u32 version; @@ -2065,9 +2064,7 @@ static void record_steal_time(struct kvm_vcpu *vcpu) if (WARN_ON_ONCE(current->mm !=3D vcpu->kvm->mm)) return; =20 - slots =3D kvm_memslots(vcpu->kvm); - - if (unlikely(slots->generation !=3D ghc->generation || + if (unlikely(!kvm_check_gen(vcpu->kvm, ghc->generation) || gpa !=3D ghc->gpa || kvm_is_error_hva(ghc->hva) || !ghc->memslot)) { /* We rely on the fact that it fits in a single page. */ @@ -2617,7 +2614,6 @@ static void kvm_steal_time_set_preempted(struct kvm_v= cpu *vcpu) { struct gfn_to_hva_cache *ghc =3D &vcpu->arch.st.cache; struct kvm_steal_time __user *st; - struct kvm_memslots *slots; static const u8 preempted =3D KVM_VCPU_PREEMPTED; gpa_t gpa =3D vcpu->arch.st.msr_val & KVM_STEAL_VALID_BITS; =20 @@ -2644,9 +2640,7 @@ static void kvm_steal_time_set_preempted(struct kvm_v= cpu *vcpu) if (unlikely(current->mm !=3D vcpu->kvm->mm)) return; =20 - slots =3D kvm_memslots(vcpu->kvm); - - if (unlikely(slots->generation !=3D ghc->generation || + if (unlikely(!kvm_check_gen(vcpu->kvm, ghc->generation) || gpa !=3D ghc->gpa || kvm_is_error_hva(ghc->hva) || !ghc->memslot)) return; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 24d04322d89e..7599ab7ad7ef 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2644,6 +2644,19 @@ static inline int kvm_mem_attributes_may_write_gfn(s= truct kvm *kvm, gfn_t gfn) return kvm_mem_attributes_may_write(attrs); } =20 +static inline bool kvm_memslots_check_gen(struct kvm *kvm, u64 slots_gener= ation, struct kvm_memslots **p_slots) +{ + struct kvm_memslots *slots =3D *p_slots =3D kvm_memslots(kvm); + return slots->generation =3D=3D slots_generation; +} + +static inline bool kvm_check_gen(struct kvm *kvm, u64 slots_generation) +{ + struct kvm_memslots *slots; + return kvm_memslots_check_gen(kvm, slots_generation, &slots); +} + + #ifdef CONFIG_KVM_GUEST_MEMFD int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, kvm_pfn_t *pfn, struct page **page, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 994a33a09448..89e79d3f27f0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3523,14 +3523,14 @@ int kvm_write_guest_offset_cached(struct kvm *kvm, = struct gfn_to_hva_cache *ghc, void *data, unsigned int offset, unsigned long len) { - struct kvm_memslots *slots =3D kvm_memslots(kvm); + struct kvm_memslots *slots; int r; gpa_t gpa =3D ghc->gpa + offset; =20 if (WARN_ON_ONCE(len + offset > ghc->len)) return -EINVAL; =20 - if (slots->generation !=3D ghc->generation) { + if (unlikely(!kvm_memslots_check_gen(kvm, ghc->generation, &slots))) { if (__kvm_gfn_to_hva_cache_init(slots, ghc, ghc->gpa, ghc->len)) return -EFAULT; } @@ -3561,14 +3561,14 @@ int kvm_read_guest_offset_cached(struct kvm *kvm, s= truct gfn_to_hva_cache *ghc, void *data, unsigned int offset, unsigned long len) { - struct kvm_memslots *slots =3D kvm_memslots(kvm); + struct kvm_memslots *slots; int r; gpa_t gpa =3D ghc->gpa + offset; =20 if (WARN_ON_ONCE(len + offset > ghc->len)) return -EINVAL; =20 - if (slots->generation !=3D ghc->generation) { + if (unlikely(!kvm_memslots_check_gen(kvm, ghc->generation, &slots))) { if (__kvm_gfn_to_hva_cache_init(slots, ghc, ghc->gpa, ghc->len)) return -EFAULT; } diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 728d2c1b488a..e09703d249bb 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -72,8 +72,6 @@ static bool kvm_gpc_is_valid_len(gpa_t gpa, unsigned long= uhva, =20 bool kvm_gpc_check(struct gfn_to_pfn_cache *gpc, unsigned long len) { - struct kvm_memslots *slots =3D kvm_memslots(gpc->kvm); - if (!gpc->active) return false; =20 @@ -81,7 +79,7 @@ bool kvm_gpc_check(struct gfn_to_pfn_cache *gpc, unsigned= long len) * If the page was cached from a memslot, make sure the memslots have * not been re-configured. */ - if (!kvm_is_error_gpa(gpc->gpa) && gpc->generation !=3D slots->generation) + if (!kvm_is_error_gpa(gpc->gpa) && !kvm_check_gen(gpc->kvm, gpc->generati= on)) return false; =20 if (kvm_is_error_hva(gpc->uhva)) @@ -290,12 +288,12 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache = *gpc, gpa_t gpa, unsigned l if (gpc->uhva !=3D old_uhva) hva_change =3D true; } else { - struct kvm_memslots *slots =3D kvm_memslots(gpc->kvm); + struct kvm_memslots *slots; =20 page_offset =3D offset_in_page(gpa); =20 - if (gpc->gpa !=3D gpa || gpc->generation !=3D slots->generation || - kvm_is_error_hva(gpc->uhva)) { + if (!kvm_memslots_check_gen(gpc->kvm, gpc->generation, &slots) || + gpc->gpa !=3D gpa || kvm_is_error_hva(gpc->uhva)) { gfn_t gfn =3D gpa_to_gfn(gpa); =20 gpc->gpa =3D gpa; --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A76254B4034 for ; Fri, 18 Sep 2026 08:16:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719376; cv=none; b=Zs0u5jIXmEW/g/bfSg1aYmnWqCksZzbl3WAuHAHiFXD+MRLZwjTEMIufMUrS5N/BeI8hRYxyzC3ZD9WpKy4XlWA8M9rViL3KloooFGsuiCqbo2etN1TAoC1jKqUM0zSq10D4ackQAvNsMsQOHaLv1w65C11rm+wkOO7R8zJbz4c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719376; c=relaxed/simple; bh=Rqw7WdxDkTOzti5oYxvNHbTMDeYstGC73Rgtwviq6Ko=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=RAHIW1+rrbQVy+oFBpgtLXCKK7cxAV4x3yyOxnhAjkgFRHka7zwHrUQFaZm8sGc2A3FImrjM0VTtT00486RU6ZEvvgnQJw08syTGqmmuNhuHUY+yuw8K78+ls/EKN0xAdnfeq1aoNlFxeTzWtLQkgo2qPdRcro+3Br1ES/nkql0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=NKUBo8fg; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NKUBo8fg" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719368; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7HKcfuP1Jy6EksSrMLXvjTNw6qdZL15IchFVEPuv5vY=; b=NKUBo8fgWJPWSCBiFjkEo+ucFF9Vk19td9iMdA0gnLfqUFxcIPDU/rhvB58ceEqdtY0tdN dkRTg1xOyRv+wfhWtNansDCZNBxpKAk6UtCKilBfoOMPosR8AxSqn3K4ltXN8ZlnI0+KlL pFMVqLgeUxdgxogImfL1T+WJHbfRz9Q= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-90-F1hJrHhKOWCqGXQc3unFwQ-1; Fri, 18 Sep 2026 04:16:01 -0400 X-MC-Unique: F1hJrHhKOWCqGXQc3unFwQ-1 X-Mimecast-MFC-AGG-ID: F1hJrHhKOWCqGXQc3unFwQ_1789719360 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 28BE51955D55; Fri, 18 Sep 2026 08:16:00 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6F4FC30001A2; Fri, 18 Sep 2026 08:15:59 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 18/28] KVM: Introduce a generation number for memory attributes Date: Fri, 18 Sep 2026 04:15:33 -0400 Message-ID: <20260918081543.139871-19-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne Introduce a generation number to track memory attribute modifications. This will allow KVM components to invalidate any assumptions they might have about guest's physical addresses and their permissions when memory attributes change. Like with memory slot updates, it's mandatory for components that access guest memory based on cached information to do so within a KVM SRCU read-side critical section, and that they validate the generation number before accessing memory. This, in combination with the synchronize_srcu() call within the memory attributes ioctl handler, ensures the following: - A memory attribute modification operation only returns after all users of outdated GPA data are done running. - Any component accessing cached data after the memory attribute modification returned will see the updated generation number. Additionally, loads/stores of the generation number have acquire/release semantics; which ensures all attribute writes are visible before updating the generation, and loads from attributes happen after having read the current generation number. Ultimately, since synchronize_srcu_expedited() is an expensive operation, only perform it when absolutely necessary. Do so if the introduced memory attribute is known to require synchronization or if the attribute being cleared contained a memory attribute that required synchronization. There shouldn't be any performance loss for memory attributes that don't require synchronization (and in general for any VMM that does not apply memory protections), because the attributes generation will always remain unchanged. Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- arch/x86/kvm/x86.c | 4 +- include/linux/kvm_host.h | 34 ++++++++++++++-- include/linux/kvm_types.h | 6 ++- include/trace/events/kvm.h | 14 +++++-- virt/kvm/kvm_main.c | 80 +++++++++++++++++++++++++++++++++----- virt/kvm/pfncache.c | 13 ++++--- 6 files changed, 124 insertions(+), 27 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 5b7ff5d9af75..f77b20e5d8f8 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2064,7 +2064,7 @@ static void record_steal_time(struct kvm_vcpu *vcpu) if (WARN_ON_ONCE(current->mm !=3D vcpu->kvm->mm)) return; =20 - if (unlikely(!kvm_check_gen(vcpu->kvm, ghc->generation) || + if (unlikely(!kvm_check_gen(vcpu->kvm, ghc->slots_generation, ghc->attrs_= generation) || gpa !=3D ghc->gpa || kvm_is_error_hva(ghc->hva) || !ghc->memslot)) { /* We rely on the fact that it fits in a single page. */ @@ -2640,7 +2640,7 @@ static void kvm_steal_time_set_preempted(struct kvm_v= cpu *vcpu) if (unlikely(current->mm !=3D vcpu->kvm->mm)) return; =20 - if (unlikely(!kvm_check_gen(vcpu->kvm, ghc->generation) || + if (unlikely(!kvm_check_gen(vcpu->kvm, ghc->slots_generation, ghc->attrs_= generation) || gpa !=3D ghc->gpa || kvm_is_error_hva(ghc->hva) || !ghc->memslot)) return; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 7599ab7ad7ef..ab62ce37bc18 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -878,6 +878,7 @@ struct kvm { struct { /* Protected by slots_lock (for writes) and RCU (for reads) */ struct xarray array; + u64 generation; } mem_attrs; #endif char stats_id[KVM_STATS_NAME_SIZE]; @@ -2592,13 +2593,31 @@ static inline bool kvm_mem_attributes_may_exec(u64 = attrs) #define KVM_MEMORY_ATTRIBUTE_PROT \ (KVM_MEMORY_ATTRIBUTE_NR | KVM_MEMORY_ATTRIBUTE_NW | KVM_MEMORY_ATTRIBUTE= _NX) =20 +#define KVM_MEMORY_ATTRIBUTE_NEEDS_SYNC_MASK KVM_MEMORY_ATTRIBUTE_PROT + static inline unsigned long kvm_get_memory_attributes(struct kvm *kvm, gfn= _t gfn) { return xa_to_value(xa_load(&kvm->mem_attrs.array, gfn)); } =20 +static inline u64 kvm_mem_attributes_generation(struct kvm *kvm) +{ + RCU_LOCKDEP_WARN(!lockdep_is_held(&kvm->slots_lock) && + !srcu_read_lock_held(&kvm->srcu), + "Suspicious memory attribute generation usage\n"); + + /* + * The acquire pairs with the release in kvm_vm_set_mem_attributes(). + * Memory attributes should only be queried _after_ storing the + * generation number. + */ + return smp_load_acquire(&kvm->mem_attrs.generation); +} + bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t e= nd, unsigned long mask, unsigned long attrs); +bool kvm_range_has_any_memory_attributes(struct kvm *kvm, gfn_t start, gfn= _t end, + unsigned long mask); bool kvm_arch_pre_set_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range); bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, @@ -2628,6 +2647,10 @@ static inline unsigned long kvm_get_memory_attribute= s(struct kvm *kvm, gfn_t gfn { return 0; } +static inline u64 kvm_mem_attributes_generation(struct kvm *kvm) +{ + return 0; +} #endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */ =20 static inline int kvm_mem_attributes_may_read_gfn(struct kvm *kvm, gfn_t g= fn) @@ -2644,16 +2667,19 @@ static inline int kvm_mem_attributes_may_write_gfn(= struct kvm *kvm, gfn_t gfn) return kvm_mem_attributes_may_write(attrs); } =20 -static inline bool kvm_memslots_check_gen(struct kvm *kvm, u64 slots_gener= ation, struct kvm_memslots **p_slots) +static inline bool kvm_memslots_check_gen(struct kvm *kvm, u64 slots_gener= ation, + u64 attrs_generation, struct kvm_memslots **p_slots) { struct kvm_memslots *slots =3D *p_slots =3D kvm_memslots(kvm); - return slots->generation =3D=3D slots_generation; + return slots->generation =3D=3D slots_generation && + kvm_mem_attributes_generation(kvm) =3D=3D attrs_generation; } =20 -static inline bool kvm_check_gen(struct kvm *kvm, u64 slots_generation) +static inline bool kvm_check_gen(struct kvm *kvm, u64 slots_generation, + u64 attrs_generation) { struct kvm_memslots *slots; - return kvm_memslots_check_gen(kvm, slots_generation, &slots); + return kvm_memslots_check_gen(kvm, slots_generation, attrs_generation, &s= lots); } =20 =20 diff --git a/include/linux/kvm_types.h b/include/linux/kvm_types.h index a568d8e6f4e8..7d911220e00d 100644 --- a/include/linux/kvm_types.h +++ b/include/linux/kvm_types.h @@ -74,7 +74,8 @@ typedef u64 hfn_t; typedef hfn_t kvm_pfn_t; =20 struct gfn_to_hva_cache { - u64 generation; + u64 slots_generation; + u64 attrs_generation; gpa_t gpa; unsigned long hva; unsigned long len; @@ -82,7 +83,8 @@ struct gfn_to_hva_cache { }; =20 struct gfn_to_pfn_cache { - u64 generation; + u64 slots_generation; + u64 attrs_generation; gpa_t gpa; unsigned long uhva; struct kvm_memory_slot *memslot; diff --git a/include/trace/events/kvm.h b/include/trace/events/kvm.h index b282e3a86769..a620131e9010 100644 --- a/include/trace/events/kvm.h +++ b/include/trace/events/kvm.h @@ -365,23 +365,29 @@ TRACE_EVENT(kvm_dirty_ring_exit, * @attr: The value of the attribute being set. */ TRACE_EVENT(kvm_vm_set_mem_attributes, - TP_PROTO(gfn_t start, gfn_t end, unsigned long attr), - TP_ARGS(start, end, attr), + TP_PROTO(gfn_t start, gfn_t end, unsigned long attr, bool sync, u64 gener= ation), + TP_ARGS(start, end, attr, sync, generation), =20 TP_STRUCT__entry( __field(gfn_t, start) __field(gfn_t, end) __field(unsigned long, attr) + __field(bool, sync) + __field(u64, generation) ), =20 TP_fast_assign( __entry->start =3D start; __entry->end =3D end; __entry->attr =3D attr; + __entry->sync =3D sync; + __entry->generation =3D generation; ), =20 - TP_printk("%#016llx -- %#016llx [0x%lx]", - __entry->start, __entry->end, __entry->attr) + TP_printk("%#016llx -- %#016llx [0x%lx], sync %d gen %llu", + __entry->start, __entry->end, __entry->attr, + __entry->sync, __entry->generation) + ); #endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */ =20 diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 89e79d3f27f0..ea9f3e0625cb 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1118,6 +1118,7 @@ static struct kvm *kvm_create_vm(unsigned long type, = const char *fdname) xa_init(&kvm->vcpu_array); #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES xa_init(&kvm->mem_attrs.array); + kvm->mem_attrs.generation =3D 0; #endif =20 INIT_LIST_HEAD(&kvm->gpc_list); @@ -2470,6 +2471,51 @@ bool kvm_range_has_memory_attributes(struct kvm *kvm= , gfn_t start, gfn_t end, return true; } =20 +/* + * Returns true if _any_ gfns in the range [@start, @end) have attributes = that + * match _any_ bit in @mask. + */ +bool kvm_range_has_any_memory_attributes(struct kvm *kvm, gfn_t start, gfn= _t end, + unsigned long mask) +{ + XA_STATE(xas, &kvm->mem_attrs.array, start); + void *entry; + + mask &=3D kvm_supported_mem_attributes(kvm); + if (!mask) + return false; + + if (end =3D=3D start + 1) + return !!(kvm_get_memory_attributes(kvm, start) & mask); + + guard(rcu)(); + for (;;) { + do { + entry =3D xas_next(&xas); + } while (xas_retry(&xas, entry)); + + if (xas.xa_index >=3D end) + break; + + if (xa_to_value(entry) & mask) + return true; + } + + return false; +} + +static bool kvm_range_memory_attributes_need_sync(struct kvm *kvm, + gfn_t start, gfn_t end, + unsigned long attributes) +{ + u64 mask =3D KVM_MEMORY_ATTRIBUTE_NEEDS_SYNC_MASK; + + if (attributes & mask) + return true; + + return kvm_range_has_any_memory_attributes(kvm, start, end, mask); +} + static __always_inline void kvm_handle_gfn_range(struct kvm *kvm, struct kvm_mmu_notifier_range *range) { @@ -2562,20 +2608,25 @@ static int kvm_vm_set_mem_attributes(struct kvm *kv= m, gfn_t start, gfn_t end, .on_lock =3D kvm_mmu_invalidate_end, .may_block =3D true, }; + bool sync =3D false; unsigned long i; void *entry; int r =3D 0; =20 entry =3D attributes ? xa_mk_value(attributes) : NULL; =20 - trace_kvm_vm_set_mem_attributes(start, end, attributes); - mutex_lock(&kvm->slots_lock); =20 /* Nothing to do if the entire range has the desired attributes. */ if (kvm_range_has_memory_attributes(kvm, start, end, ~0, attributes)) goto out_unlock; =20 + sync =3D kvm_range_memory_attributes_need_sync(kvm, start, end, + attributes); + + trace_kvm_vm_set_mem_attributes(start, end, attributes, sync, + kvm->mem_attrs.generation + 1); + /* * Reserve memory ahead of time to avoid having to deal with failures * partway through setting the new attributes. @@ -2597,10 +2648,16 @@ static int kvm_vm_set_mem_attributes(struct kvm *kv= m, gfn_t start, gfn_t end, cond_resched(); } =20 + /* Pairs with acquire in kvm_mem_attributes_generation() */ + smp_store_release(&kvm->mem_attrs.generation, + kvm->mem_attrs.generation + 1); + kvm_handle_gfn_range(kvm, &post_set_range); =20 out_unlock: mutex_unlock(&kvm->slots_lock); + if (sync) + synchronize_srcu_expedited(&kvm->srcu); =20 return r; } @@ -3470,7 +3527,8 @@ int kvm_vcpu_write_guest(struct kvm_vcpu *vcpu, gpa_t= gpa, const void *data, } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_vcpu_write_guest); =20 -static int __kvm_gfn_to_hva_cache_init(struct kvm_memslots *slots, +static int __kvm_gfn_to_hva_cache_init(struct kvm *kvm, + struct kvm_memslots *slots, struct gfn_to_hva_cache *ghc, gpa_t gpa, unsigned long len) { @@ -3480,8 +3538,8 @@ static int __kvm_gfn_to_hva_cache_init(struct kvm_mem= slots *slots, gfn_t nr_pages_needed =3D end_gfn - start_gfn + 1; gfn_t nr_pages_avail; =20 - /* Update ghc->generation before performing any error checks. */ - ghc->generation =3D slots->generation; + /* Update ghc->slots_generation before performing any error checks. */ + ghc->slots_generation =3D slots->generation; =20 if (start_gfn > end_gfn) { ghc->hva =3D KVM_HVA_ERR_BAD; @@ -3500,6 +3558,8 @@ static int __kvm_gfn_to_hva_cache_init(struct kvm_mem= slots *slots, return -EFAULT; } =20 + ghc->attrs_generation =3D kvm_mem_attributes_generation(kvm); + /* Use the slow path for cross page reads and writes. */ if (nr_pages_needed =3D=3D 1) ghc->hva +=3D offset; @@ -3515,7 +3575,7 @@ int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct= gfn_to_hva_cache *ghc, gpa_t gpa, unsigned long len) { struct kvm_memslots *slots =3D kvm_memslots(kvm); - return __kvm_gfn_to_hva_cache_init(slots, ghc, gpa, len); + return __kvm_gfn_to_hva_cache_init(kvm, slots, ghc, gpa, len); } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_gfn_to_hva_cache_init); =20 @@ -3530,8 +3590,8 @@ int kvm_write_guest_offset_cached(struct kvm *kvm, st= ruct gfn_to_hva_cache *ghc, if (WARN_ON_ONCE(len + offset > ghc->len)) return -EINVAL; =20 - if (unlikely(!kvm_memslots_check_gen(kvm, ghc->generation, &slots))) { - if (__kvm_gfn_to_hva_cache_init(slots, ghc, ghc->gpa, ghc->len)) + if (unlikely(!kvm_memslots_check_gen(kvm, ghc->slots_generation, ghc->att= rs_generation, &slots))) { + if (__kvm_gfn_to_hva_cache_init(kvm, slots, ghc, ghc->gpa, ghc->len)) return -EFAULT; } =20 @@ -3568,8 +3628,8 @@ int kvm_read_guest_offset_cached(struct kvm *kvm, str= uct gfn_to_hva_cache *ghc, if (WARN_ON_ONCE(len + offset > ghc->len)) return -EINVAL; =20 - if (unlikely(!kvm_memslots_check_gen(kvm, ghc->generation, &slots))) { - if (__kvm_gfn_to_hva_cache_init(slots, ghc, ghc->gpa, ghc->len)) + if (unlikely(!kvm_memslots_check_gen(kvm, ghc->slots_generation, ghc->att= rs_generation, &slots))) { + if (__kvm_gfn_to_hva_cache_init(kvm, slots, ghc, ghc->gpa, ghc->len)) return -EFAULT; } =20 diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index e09703d249bb..46ffae69fe77 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -76,10 +76,11 @@ bool kvm_gpc_check(struct gfn_to_pfn_cache *gpc, unsign= ed long len) return false; =20 /* - * If the page was cached from a memslot, make sure the memslots have - * not been re-configured. + * If the page was cached from a memslot, make sure the memslots nor + * memory attributes have not been re-configured. */ - if (!kvm_is_error_gpa(gpc->gpa) && !kvm_check_gen(gpc->kvm, gpc->generati= on)) + if (!kvm_is_error_gpa(gpc->gpa) && + !kvm_check_gen(gpc->kvm, gpc->slots_generation, gpc->attrs_generation= )) return false; =20 if (kvm_is_error_hva(gpc->uhva)) @@ -253,6 +254,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cac= he *gpc) =20 static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsi= gned long uhva) { + struct kvm *kvm =3D gpc->kvm; unsigned long page_offset; bool unmap_old =3D false; unsigned long old_uhva; @@ -292,12 +294,13 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache = *gpc, gpa_t gpa, unsigned l =20 page_offset =3D offset_in_page(gpa); =20 - if (!kvm_memslots_check_gen(gpc->kvm, gpc->generation, &slots) || + if (!kvm_memslots_check_gen(gpc->kvm, gpc->slots_generation, gpc->attrs_= generation, &slots) || gpc->gpa !=3D gpa || kvm_is_error_hva(gpc->uhva)) { gfn_t gfn =3D gpa_to_gfn(gpa); =20 + gpc->attrs_generation =3D kvm_mem_attributes_generation(kvm); gpc->gpa =3D gpa; - gpc->generation =3D slots->generation; + gpc->slots_generation =3D slots->generation; gpc->memslot =3D __gfn_to_memslot(slots, gfn); gpc->uhva =3D gfn_to_hva_memslot(gpc->memslot, gfn); =20 --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E82D4B0CA0 for ; Fri, 18 Sep 2026 08:16:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719372; cv=none; b=XWVPgYKeLrVeAQSjq7vqprrEnjXfCeqGzH/JiUsF1vQ1hpGTgcyd6gb/+CG9CWzD16Cs9elDsWNJu/03Gz6item/VqHhJMYASGN/yp5+1D9/GCj3Ghc7VJSr3cRaIhgztaVQB0yymWtfWg3MvZoqRQ8uzvvFqNqpdLBthqUgZfE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719372; c=relaxed/simple; bh=LG2/6fNzKM3D1tVIK83ssuQn0CPfTGoTKvQuc9FW7VE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=TfMO1qODu/BAPYvA5K0l4Rh49k6b4UYHZhdyt8yrcaHYWEG+ZTpgF3i8vRhfEXcNX755eUI4ZBmW2YUq5szCpHQkdOX+TcsCKoNM5zBkE+jN/cKxW4bhS/fLoKQuwtBynGjBwIdbJyP6BHFlKP1tfjbfnQEwmj3Ui4FxL1GcHBg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Mso9vqeA; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Mso9vqeA" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719365; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z7WjpcYWka36Pi/4Seg3ZdLwc3xMGEYD5XS7g2LXTR4=; b=Mso9vqeAx3oiZ4UghTvVsfkGDypvOa34Uuqc8FZAQBgTnPvlYbNgE9M0ujDdDHjIdiOROS bmpr0n+tf9qclQUCPrOAk358asAZYHEMiVVhh/+lnQBAnpudSzMzW7Gkmt9V5+he3N1Jp1 Or6m4Lnme+TwBxc3oaslFAtxNP+qpfk= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-96-4B6l90TEPqy0ch2d17d2bA-1; Fri, 18 Sep 2026 04:16:02 -0400 X-MC-Unique: 4B6l90TEPqy0ch2d17d2bA-1 X-Mimecast-MFC-AGG-ID: 4B6l90TEPqy0ch2d17d2bA_1789719361 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E4A69195419A; Fri, 18 Sep 2026 08:16:00 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 41CC130001A2; Fri, 18 Sep 2026 08:16:00 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 19/28] KVM: Take memory protections into account for accesses with cached gfn->hva Date: Fri, 18 Sep 2026 04:15:34 -0400 Message-ID: <20260918081543.139871-20-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne Account for memory attributes when accessing guest memory through kvm_get/put_guest(). This requires tracking the memory attributes generation as part of gfn_to_hva_cache's data, invalidate the cached information if the generation changes, and failing to refresh the cache if restrictive memory attributes are found within the GPA range. Similar to how gfn_to_hva_cache disallows caching gfns mapped within read-only memory slots, gfns marked as read-only by memory attributes will also fail to initialize. Unsurprisingly, the same behaviour applies to gfns mapped as non-accessible (NR/NW), while non-executable mappings are okay. Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- include/linux/kvm_host.h | 20 +++++++++++++++++--- virt/kvm/kvm_main.c | 15 +++++++++++---- 2 files changed, 28 insertions(+), 7 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ab62ce37bc18..e9c0932f150e 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1359,7 +1359,8 @@ int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct= gfn_to_hva_cache *ghc, typeof(v) __user *__uaddr =3D (typeof(__uaddr))(__addr + offset); \ int __ret =3D -EFAULT; \ \ - if (!kvm_is_error_hva(__addr)) \ + if (!kvm_is_error_hva(__addr) && \ + kvm_mem_attributes_may_read_gfn(kvm, gfn)) \ __ret =3D get_user(v, __uaddr); \ __ret; \ }) @@ -1379,7 +1380,8 @@ int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct= gfn_to_hva_cache *ghc, typeof(v) __user *__uaddr =3D (typeof(__uaddr))(__addr + offset); \ int __ret =3D -EFAULT; \ \ - if (!kvm_is_error_hva(__addr)) \ + if (!kvm_is_error_hva(__addr) && \ + kvm_mem_attributes_may_write_gfn(kvm, gfn)) \ __ret =3D put_user(v, __uaddr); \ if (!__ret) \ mark_page_dirty(kvm, gfn); \ @@ -2647,6 +2649,12 @@ static inline unsigned long kvm_get_memory_attribute= s(struct kvm *kvm, gfn_t gfn { return 0; } +static inline bool kvm_range_has_any_memory_attributes(struct kvm *kvm, + gfn_t start, gfn_t end, + unsigned long mask) +{ + return false; +} static inline u64 kvm_mem_attributes_generation(struct kvm *kvm) { return 0; @@ -2667,6 +2675,13 @@ static inline int kvm_mem_attributes_may_write_gfn(s= truct kvm *kvm, gfn_t gfn) return kvm_mem_attributes_may_write(attrs); } =20 +static inline bool kvm_range_has_rw_memory_protections(struct kvm *kvm, + gfn_t start, gfn_t end) +{ + return kvm_range_has_any_memory_attributes(kvm, start, end, + KVM_MEMORY_ATTRIBUTE_NR | KVM_MEMORY_ATTRIBUTE_NW); +} + static inline bool kvm_memslots_check_gen(struct kvm *kvm, u64 slots_gener= ation, u64 attrs_generation, struct kvm_memslots **p_slots) { @@ -2682,7 +2697,6 @@ static inline bool kvm_check_gen(struct kvm *kvm, u64= slots_generation, return kvm_memslots_check_gen(kvm, slots_generation, attrs_generation, &s= lots); } =20 - #ifdef CONFIG_KVM_GUEST_MEMFD int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, kvm_pfn_t *pfn, struct page **page, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ea9f3e0625cb..24f46dce192e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3550,15 +3550,22 @@ static int __kvm_gfn_to_hva_cache_init(struct kvm *= kvm, * If the requested region crosses two memslots, we still * verify that the entire region is valid here. */ - for ( ; start_gfn <=3D end_gfn; start_gfn +=3D nr_pages_avail) { - ghc->memslot =3D __gfn_to_memslot(slots, start_gfn); - ghc->hva =3D gfn_to_hva_many(ghc->memslot, start_gfn, - &nr_pages_avail); + for (gfn_t gfn =3D start_gfn ; gfn <=3D end_gfn; gfn +=3D nr_pages_avail)= { + ghc->memslot =3D __gfn_to_memslot(slots, gfn); + ghc->hva =3D gfn_to_hva_many(ghc->memslot, gfn, &nr_pages_avail); if (kvm_is_error_hva(ghc->hva)) return -EFAULT; } =20 + /* + * RW memory attributes are incompatible with GHC. The RW protection + * check has to happen after storing the generation number. + */ ghc->attrs_generation =3D kvm_mem_attributes_generation(kvm); + if (kvm_range_has_rw_memory_protections(kvm, start_gfn, end_gfn + 1)) { + ghc->hva =3D KVM_HVA_ERR_BAD; + return -EFAULT; + } =20 /* Use the slow path for cross page reads and writes. */ if (nr_pages_needed =3D=3D 1) --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 345314B0E42 for ; Fri, 18 Sep 2026 08:16:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719372; cv=none; b=IF0jo5mZXU8sv19Qkv4HWsSBfE03Wlt5K3HaaAM3kbQHOxkxR8Uv7QfiLo7TmL6awJbwAB6Z5VPkW5kmxtIEMBXRzLs7AiFObW7596fFB6cCYunkzGs8DKc73I3Tc6xsX3LBojxRUDvrDn0poh5Xs7FdqwHrCshCUB83ToWcLaY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719372; c=relaxed/simple; bh=ssbZx19SgT03PitfpW9iwEfiJ7Rq53NzaKL5KpbPZcg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BsU0S9iM/2BSBIYllUyg6ooSUGkzGoK6IiSU6jqaDCbmgW4XHQgsCkSVQszUh8KfarK1znyd2cImzfHLZ1eIMsPEOv7JEhEx6TR4b6olDS8iNs8uAkMp7MypD9x9CcxrJn/G/IO4fBpV34vOR76aS8LgRlCcqRJojPCuz/Z8L8Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=VlkiMKkC; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VlkiMKkC" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719366; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y6y8qshOMUMHy0hJtW9RAYJxeweVqNG/1k4529WtVRE=; b=VlkiMKkCcjhCfA7cQzUEIoR6BpJQkAWunXicjubRf3H3bsWMaGf/zr3iAxwW6fGocEvvea J8PNYuPuQPwHQF4UhgNu6ui/Up/E3JvPNFv4A/Tx0Yb+CNx6bWsiJMau3Zqsmiqe6LsBB4 eU1dTQNYeARFYs0gnokDb9pDZADbX58= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-185-fmIjR1VnNFiSxhOW6Af9Tg-1; Fri, 18 Sep 2026 04:16:03 -0400 X-MC-Unique: fmIjR1VnNFiSxhOW6Af9Tg-1 X-Mimecast-MFC-AGG-ID: fmIjR1VnNFiSxhOW6Af9Tg_1789719362 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 43BFD18011E9; Fri, 18 Sep 2026 08:16:02 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9A77C30001A2; Fri, 18 Sep 2026 08:16:01 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 20/28] KVM: pfncache: Fail to refresh if it contains memory protections Date: Fri, 18 Sep 2026 04:15:35 -0400 Message-ID: <20260918081543.139871-21-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne Similar to how pfncache disallows caching gfns mapped within read-only memory slots, gfns marked as read-only by memory attributes must fail to initialize. Unsurprisingly, the same behaviour applies to gfns mapped as non-accessible (NR/NW), while non-executable mappings are fine. Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- virt/kvm/pfncache.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 46ffae69fe77..39935740136d 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -298,7 +298,18 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *= gpc, gpa_t gpa, unsigned l gpc->gpa !=3D gpa || kvm_is_error_hva(gpc->uhva)) { gfn_t gfn =3D gpa_to_gfn(gpa); =20 + /* + * RW memory attributes are incompatible with GPC. The + * RW protection check has to happen after storing the + * generation number. + */ gpc->attrs_generation =3D kvm_mem_attributes_generation(kvm); + if (kvm_range_has_rw_memory_protections(kvm, gfn, gfn + 1)) { + gpc->uhva =3D KVM_HVA_ERR_BAD; + ret =3D -EFAULT; + goto out; + } + gpc->gpa =3D gpa; gpc->slots_generation =3D slots->generation; gpc->memslot =3D __gfn_to_memslot(slots, gfn); --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC7274B404B for ; Fri, 18 Sep 2026 08:16:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719376; cv=none; b=YW8GZN/OyXuxwZGh6SxpWDov+8OFmynwhURLKyidUA5Nn0SntZHT2BMeN3WN38xpjwrGim5FeqVyd9JMwlw+ZiiArIau0YHVGNz9NU0Vnx8UJvDSPyUjgK7uCZr6aPf2h9oBHNUiHgEZ7Ps3ZTXWWPDrqVTHVNfBp2GtLMehWAA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719376; c=relaxed/simple; bh=I5ibwMtY9EbomTpjerkx/2uNla7H7oHQYY11dx8+HWI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ObOR1StPqwgsz2wzVe4lijE9GnDQJb2yP73lsQmlSpiF+CjnNs+m2iCqz6rQEqkjQj1BtUtTtUqjX9det7OFqz5fE361jp6OX3e9DulyJceOoeqk+YMMsVg92bv6YcQ66GfcmqfjZp/Tq2SN0TGxHSRe3gBUvQCbBkBqDDNREGg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=WHyF429c; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WHyF429c" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719367; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jP3CCCr3me+qhU/3Ko3iK48MQgZC82F2uaDjJuPoHc4=; b=WHyF429cfGmjSRJEiopdUU78XSkBn08LvgP3TDsI0fsfn21jU8cKRo6l/IviltiEAE9xAL BjKGmdehvkbZBED75AJC/S4Cey9Zq0ViuatEYGCmc77E3CIMu4xVvNNOwcgTItcuitUogV K8NQtd+hFeD9U3S0zRxzy3U/LYBFMCU= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-280-CqLKpG1ENVCLudAaimEyZg-1; Fri, 18 Sep 2026 04:16:04 -0400 X-MC-Unique: CqLKpG1ENVCLudAaimEyZg-1 X-Mimecast-MFC-AGG-ID: CqLKpG1ENVCLudAaimEyZg_1789719363 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 146521977514; Fri, 18 Sep 2026 08:16:03 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6A5F830001A2; Fri, 18 Sep 2026 08:16:02 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 21/28] KVM: x86/mmu: Take memory protection attributes into account during faults Date: Fri, 18 Sep 2026 04:15:36 -0400 Message-ID: <20260918081543.139871-22-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne Take memory protection attributes when faulting guest memory. Prohibited memory accesses will cause a user-space -EFAULT exit just like private memory accesses. Userspace will either enable the access or bump it to the guest as some kind of exception (e.g. a VTL return). Since the struct kvm_page_fault already has the access type in PFERR_* format, the check is done via the kvm_page_format permissions table. This means that it supports naturally all page table format variants, and it can even handle mode-based memory protection when the host uses MBEC/GMET. The only thing that needs some care is to build the restricted ACC_* mask with the root page's own access mask as a base (and not ACC_ALL). Otherwise, supervisor mode execution would be handled incorrectly on AMD processors with GMET. To avoid spamming the trace buffer too much, the new trace event only kicks in if memory protection attributes are present for the faulted gfn. Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 49 +++++++++++++++++++++++++++++++++ arch/x86/kvm/mmu/mmu_internal.h | 19 +++++++++++++ arch/x86/kvm/mmu/mmutrace.h | 36 ++++++++++++++++++++++++ arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/spte.h | 11 ++------ 5 files changed, 107 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 797b18201d98..3fd4cc3c57a5 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4642,6 +4642,50 @@ static int kvm_mmu_faultin_pfn_gmem(struct kvm_vcpu = *vcpu, return RET_PF_CONTINUE; } =20 +static inline unsigned kvm_get_gfn_protections(struct kvm_vcpu *vcpu, gfn_= t gfn) +{ + struct kvm *kvm =3D vcpu->kvm; + unsigned int access =3D vcpu->arch.mmu->root_role.access; + unsigned long attrs =3D kvm_get_memory_attributes(kvm, gfn); + if (!attrs) + return access; + + WARN_ON_ONCE(!kvm_mem_attributes_valid(kvm, attrs)); + + if (!kvm_mem_attributes_may_read(attrs)) + access &=3D ~ACC_READ_MASK; + if (!kvm_mem_attributes_may_write(attrs)) + access &=3D ~ACC_WRITE_MASK; + if (!kvm_mem_attributes_may_exec(attrs)) { + access &=3D ~ACC_EXEC_MASK; + if (shadow_xu_mask) + access &=3D ~ACC_USER_EXEC_MASK; + } + + return access; +} + +static int kvm_faultin_memory_protections(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault) +{ + unsigned access; + + /* Memory attributes don't apply to MMIO regions */ + if (unlikely(!fault->slot)) + return RET_PF_CONTINUE; + + access =3D kvm_get_gfn_protections(vcpu, fault->gfn); + if (access =3D=3D ACC_ALL) + return RET_PF_CONTINUE; + + trace_kvm_faultin_memory_protections(vcpu, fault, access); + if (__permission_fault(vcpu->arch.mmu, access, fault)) + return -EFAULT; + + fault->host_access &=3D access; + return RET_PF_CONTINUE; +} + static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { @@ -4722,6 +4766,11 @@ static int kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, if (unlikely(!slot)) return kvm_handle_noslot_fault(vcpu, fault, access); =20 + if (kvm_faultin_memory_protections(vcpu, fault)) { + kvm_mmu_prepare_memory_fault_exit(vcpu, fault); + return -EFAULT; + } + /* * Retry the page fault if the gfn hit a memslot that is being deleted * or moved. This ensures any existing SPTEs for the old memslot will diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 00215b9f309f..0bad60c5aab6 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -290,6 +290,25 @@ struct kvm_page_fault { bool write_fault_to_shadow_pgtable; }; =20 +/* + * Returns true if the access indicated by @fault is forbidden by the exis= ting + * SPTE protections. + */ +static inline bool __permission_fault(struct kvm_mmu *mmu, unsigned access, + struct kvm_page_fault *fault) +{ + unsigned pfec; + + /* + * RSVD is handled elsewhere, and is used for SMAP in the context + * of accessing fmt.permissions[]. SPTEs never use PK or SS, as + * they are not supported for shadow paging and irrelevant for TDP. + */ + pfec =3D fault->error_code & ( + PFERR_WRITE_MASK | PFERR_USER_MASK | PFERR_FETCH_MASK); + return (mmu->fmt.permissions[pfec >> 1] >> access) & 1; +} + /* * Return values of handle_mmio_page_fault(), mmu.page_fault(), fast_page_= fault(), * and of course kvm_mmu_do_page_fault(). diff --git a/arch/x86/kvm/mmu/mmutrace.h b/arch/x86/kvm/mmu/mmutrace.h index 8354d9f39777..744f111644cb 100644 --- a/arch/x86/kvm/mmu/mmutrace.h +++ b/arch/x86/kvm/mmu/mmutrace.h @@ -8,6 +8,13 @@ #undef TRACE_SYSTEM #define TRACE_SYSTEM kvmmmu =20 +#ifdef CREATE_TRACE_POINTS +#define tracing_kvm_rip_read(vcpu) ({ \ + typeof(vcpu) __vcpu =3D vcpu; \ + __vcpu->arch.guest_state_protected ? 0 : kvm_rip_read(__vcpu); \ + }) +#endif + #define KVM_MMU_PAGE_FIELDS \ __field(__u8, mmu_valid_gen) \ __field(__u64, gfn) \ @@ -447,6 +454,35 @@ TRACE_EVENT( __entry->gfn, __entry->spte, __entry->level, __entry->errno) ); =20 +TRACE_EVENT(kvm_faultin_memory_protections, + TP_PROTO(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, + unsigned access), + TP_ARGS(vcpu, fault, access), + + TP_STRUCT__entry( + __field(unsigned int, vcpu_id) + __field(unsigned long, guest_rip) + __field(u64, fault_address) + __field(bool, write) + __field(bool, exec) + __field(unsigned, access) + ), + + TP_fast_assign( + __entry->vcpu_id =3D vcpu->vcpu_id; + __entry->guest_rip =3D tracing_kvm_rip_read(vcpu); + __entry->fault_address =3D fault->gfn; + __entry->write =3D fault->write; + __entry->exec =3D fault->exec; + __entry->access =3D access; + ), + + TP_printk("vcpu %d rip 0x%lx gfn 0x%016llx access %s protections 0x%x", + __entry->vcpu_id, __entry->guest_rip, __entry->fault_address, + __entry->exec ? "X" : (__entry->write ? "W" : "R"), + __entry->access) +); + #endif /* _TRACE_KVMMMU_H */ =20 #undef TRACE_INCLUDE_PATH diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index e6ec14165f40..b031123fcb36 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -992,7 +992,7 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, stru= ct kvm_mmu_page *sp, int =20 sptep =3D &sp->spt[i]; spte =3D *sptep; - host_access =3D ACC_ALL; + host_access =3D kvm_get_gfn_protections(vcpu, gfn); if (!(spte & shadow_host_writable_mask)) host_access &=3D ~ACC_WRITE_MASK; slot =3D kvm_vcpu_gfn_to_memslot(vcpu, gfn); diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 589f3954633e..14f322944671 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -491,7 +491,7 @@ static inline bool is_mmu_writable_spte(u64 spte) static inline bool spte_permission_fault(struct kvm_mmu *mmu, u64 spte, struct kvm_page_fault *fault) { - unsigned pfec, pte_access; + unsigned pte_access; =20 if (!is_shadow_present_pte(spte)) return true; @@ -511,14 +511,7 @@ static inline bool spte_permission_fault(struct kvm_mm= u *mmu, u64 spte, pte_access |=3D spte & shadow_xu_mask ? ACC_USER_EXEC_MASK : 0; } =20 - /* - * RSVD is handled elsewhere, and is used for SMAP in the context - * of accessing fmt.permissions[]. SPTEs never use PK or SS, as - * they are not supported for shadow paging and irrelevant for TDP. - */ - pfec =3D fault->error_code & ( - PFERR_WRITE_MASK | PFERR_USER_MASK | PFERR_FETCH_MASK); - return (mmu->fmt.permissions[pfec >> 1] >> pte_access) & 1; + return __permission_fault(mmu, pte_access, fault); } =20 /* --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79FF94B488C for ; Fri, 18 Sep 2026 08:16:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719379; cv=none; b=Pj6BIzNcUMJJR10PXr7HQsNmCvsOasrN+TrvVATDV8oVEWOqG2RH40qamXO0kEYzN6siNNltepj59eZLqZEGyK5O0sFzs7P6mu+XYTP8KIspnF5zlN6w1cbc72A4xRhM0XDtCAVbKwAsakxm3OLNq9XN0OFaXo7katfbmsFLQts= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719379; c=relaxed/simple; bh=0/G/zhm/TYcWJVnrrz3Po70HAzX5NNAVsCqtzhX+LHg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=cXAmy94gldQSbTISCiYc0wrHXDDG/7gRugTSvHdH8b5k5ww+UOETcug5Ur8HVc9Yecg8KRFJjI+7aaY6t4d5CQchjzahQO1hJA3H+ox5ZhAdgTX12d/ru0AxD936etg7KYVjKwoIcdXp7BztJduCLpmqn/wc+XehozdmD++4imM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=jB+CJO4G; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="jB+CJO4G" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719369; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GqXmK6gYxDQBh7MLxdL5kQ5+nKTLwxAUz7gyTtAit3A=; b=jB+CJO4GFNFm4JJCmz4glCwWJlzBz4/5iCtCau1j/3PqCqFJNOpRUYTR4PTZdQGEU7qM6K rCbwYU7gMC9MICJ3/oy+1iawmynVFTk7MXhuE2NwUNTy+y4c/dWUPCbkZ6te7uJ+2D60ov d1nkcVOsXHGXX+VxqTLpIeyILrl8BcQ= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-588-YDBJXWm-N_KlII8Unk4XLw-1; Fri, 18 Sep 2026 04:16:05 -0400 X-MC-Unique: YDBJXWm-N_KlII8Unk4XLw-1 X-Mimecast-MFC-AGG-ID: YDBJXWm-N_KlII8Unk4XLw_1789719364 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E2AEF1955D55; Fri, 18 Sep 2026 08:16:03 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 3A2AF300022B; Fri, 18 Sep 2026 08:16:03 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 22/28] KVM: x86/mmu: Issue memory fault exit if walk failed due to memory attribute Date: Fri, 18 Sep 2026 04:15:37 -0400 Message-ID: <20260918081543.139871-23-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne During execution, page table entries are subject to memory protection simply because their GPA is obtained with an EPT walk. During emulation, the checks need to be done by hand and result in an -EFAULT userspace exit. Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/paging_tmpl.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index b031123fcb36..d2836a327bc9 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -89,6 +89,7 @@ struct guest_walker { unsigned int pt_access[PT_MAX_FULL_LEVELS]; unsigned int pte_access; gfn_t gfn; + bool memory_attributes_fault; struct x86_exception fault; }; =20 @@ -342,6 +343,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker= *walker, =20 trace_kvm_mmu_pagetable_walk(addr, access); retry_walk: + walker->memory_attributes_fault =3D false; walker->level =3D w->cpu_role.base.level; pte =3D kvm_mmu_get_guest_pgd(vcpu, w); have_ad =3D PT_HAVE_ACCESSED_DIRTY(w); @@ -411,6 +413,12 @@ static int FNAME(walk_addr_generic)(struct guest_walke= r *walker, if (unlikely(kvm_is_error_hva(host_addr))) goto error; =20 + if (!kvm_mem_attributes_may_read_gfn(vcpu->kvm, gpa_to_gfn(real_gpa))) { + walker->memory_attributes_fault =3D true; + walker->gfn =3D gpa_to_gfn(real_gpa); + goto error; + } + ptep_user =3D (pt_element_t __user *)((void *)host_addr + offset); if (unlikely(get_user(pte, ptep_user))) goto error; @@ -820,6 +828,12 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, st= ruct kvm_page_fault *fault * The page is not mapped by the guest. Let the guest handle it. */ if (!r) { + if (walker.memory_attributes_fault) { + fault->gfn =3D walker.gfn; + kvm_mmu_prepare_memory_fault_exit(vcpu, fault); + return -EFAULT; + } + if (!fault->prefetch) __kvm_inject_emulated_page_fault(vcpu, &walker.fault, true); =20 --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55CEE4AF14B for ; Fri, 18 Sep 2026 08:16:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719374; cv=none; b=iA6QuncASulQ5FuziqBd4I5mCocMhk2Xb1wF79lnqXvAFKWY5Hyo1uADP+bhI+Vc1ZnWHhukaEFvq6mCuqL5wjvRc8iwZ7No4ungK43bfhzKY2Ftw89pZg16oaGK8VRz8mXBUpKIuFLdi8/IyDjb5sFuBIa+ibJK5WJZeWKIW8c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719374; c=relaxed/simple; bh=3fY4WlcZJ6wOE2Ax39CGRJ/RSbNMWT2vQvLgD0k4WK0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WUWm+VbUbEIn5iol3nGkGjdF8E7x5pPzX8M96RInh44CYOm0mkOBhVTGHcXTtdqJe4+5X2sLNF03KszaFrPU3bKeF4EVjmtXAqf6oA4PZCKBoHkOi+gyTzg8Eqo6w00g25X9nnX0VhU9qq3Gp04upWvKf5dljk3SKT3+AXHTz+Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=d086OQ/8; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="d086OQ/8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719367; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Jol8ennk1TZbPIOM6O5ZAfPzLlJYocGjoT0OUGC9e2Y=; b=d086OQ/88NXU+QNaLtoZ4mjnQFo/gixB0QxXytK1isBOSB2LqPY/nPucWZPVQ90AllIgiV s9+u7Jhcrk33K6s/eDHTIF5hhbpkTEQKecGr2xrbKpec/j+h2BZ9I3a7S9O5EpQ4vt59uc a93LS9458q1ir+x4goEu+sVwOfhR5tI= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-649-VtmDwBMrPVymEKG1956PdA-1; Fri, 18 Sep 2026 04:16:05 -0400 X-MC-Unique: VtmDwBMrPVymEKG1956PdA-1 X-Mimecast-MFC-AGG-ID: VtmDwBMrPVymEKG1956PdA_1789719364 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BD5DC1944DE2; Fri, 18 Sep 2026 08:16:04 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1598B30001A2; Fri, 18 Sep 2026 08:16:03 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 23/28] KVM: x86/mmu: Do not update accessed/dirty if guest PTE is read-only Date: Fri, 18 Sep 2026 04:15:38 -0400 Message-ID: <20260918081543.139871-24-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne In this case do not go all the way out to userspace and just stomp out the write. This is consistent with the way KVM handles read-only memory slots during emulation, though admittedly inconsistent with everything else: NPT for example always wants nested page table entries for the guest page tables to be writable, and will fault. Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/paging_tmpl.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index d2836a327bc9..2ff655edd57c 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -419,6 +419,9 @@ static int FNAME(walk_addr_generic)(struct guest_walker= *walker, goto error; } =20 + if (!kvm_mem_attributes_may_write_gfn(vcpu->kvm, gpa_to_gfn(real_gpa))) + walker->pte_writable[walker->level - 1] =3D false; + ptep_user =3D (pt_element_t __user *)((void *)host_addr + offset); if (unlikely(get_user(pte, ptep_user))) goto error; --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F8004A0EF5 for ; Fri, 18 Sep 2026 08:16:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719382; cv=none; b=jkN7Qm6OAJfGAau4Pl4RbOjta+W0JIfT8MxfI7rejv+vne2/2maGhql75oQkavYr4917yu5KJX2d0RzdH/hQhTw2CPBATONOPpKicULvBbBtCYaul7plH9Y5ihhiGuj0XwH5lMvbu0Sh/fq3Z3fepQenr7dCqylycqdcDTCgAlA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719382; c=relaxed/simple; bh=uhfwHcFTsphtgoGnq7FEk1hwkzaxHQToX+f3dsmAGd0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rJoCwOz+QazK1Tq2M9FjOUMM8BggHdXjYWELneH+sy8FwDw5tFDmnP/7YzM2iL4aN/JUAnPA20mHeKbT+FlUlVrvJDUa+En1mVwJlNqxDu3uTsGwq/QvBHnPjiNRZrQiXkW68aVRoNazeAGE0NXdJphSsLnMf5lFRZ2/VoIEeGU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=KsU2t9J6; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KsU2t9J6" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719372; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NUMowBI0tpa/MvWQQQhIIqurSd/Pmjk5XTM/6IbZmBE=; b=KsU2t9J6NS2GvYZ7uNarT9z0kJOh/2rInlZiriLRi8pfLXOBNdN41rmVZIDh4E/vHcYHjl QLaUPn9PMN14mVcK1d+VPp1gF0Wfq9jtoRYu17MUJ4Um27TPpQ2LBp2KS9CAjImiTT9bQr FlR1ZW2/SAL8GnpWcVJ6yTRxJ1WZeVY= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-660-622btM0fMt6LbPq2LuuNeA-1; Fri, 18 Sep 2026 04:16:06 -0400 X-MC-Unique: 622btM0fMt6LbPq2LuuNeA-1 X-Mimecast-MFC-AGG-ID: 622btM0fMt6LbPq2LuuNeA_1789719365 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 96AAE195419D; Fri, 18 Sep 2026 08:16:05 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E374E3000218; Fri, 18 Sep 2026 08:16:04 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 24/28] KVM: x86/mmu: Do not prefetch sptes on gfns backed by memory attributes Date: Fri, 18 Sep 2026 04:15:39 -0400 Message-ID: <20260918081543.139871-25-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" From: Nicolas Saenz Julienne Prefetched SPTEs are always given full access. Do not prefetch GFNs that have memory protections applied. Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 3fd4cc3c57a5..097095f71d60 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3187,6 +3187,11 @@ static bool kvm_mmu_prefetch_sptes(struct kvm_vcpu *= vcpu, gfn_t gfn, u64 *sptep, return false; =20 for (i =3D 0; i < nr_pages; i++, gfn++, sptep++) { + if (!kvm_get_memory_attributes(vcpu->kvm, gfn)) { + kvm_release_page_clean(pages[i]); + continue; + } + mmu_set_spte(vcpu, slot, sptep, access, gfn, page_to_pfn(pages[i]), NULL); =20 --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE0774BC00B for ; Fri, 18 Sep 2026 08:16:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719385; cv=none; b=T2fflj2jGCsFBIx8xiKj4+fCjSI44nVw9asDOumJ+kActWOAWKd75kGaOr9CEZXGURW7zSvJoNZ3BdvguFvqXdHLQkexjiKBEJ5hYGPlhND8MDgUNbVT/5NpYQGeQjCDwSnTrE2ng71iBXlggolsCBH2Vr4ue9w72z+aUcDubX8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719385; c=relaxed/simple; bh=zYdgoF19yrBAddmiz5U+pMMnjvnHH3FkMGSpyu7iIjE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=sPYxo4wMFgpQAO+7SaBemJ2TK8oSLAgHSjjcAgQEM+gtIdrFAm5DMw14++famfzau9d3psrYPNaNZ5JZ+Tp6DoUevFwlDvSHiRcQZnAgA3O01yG/D7RVOvG2BGhQfPA5iH/BDcg7HBFIBobM9f3OApIGriLIm/6H+JERWCdnAaI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Ugl2aLdq; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ugl2aLdq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719374; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c39yRW+h6dkqpPKLdQfPTPGhL/SUy3kZmIEvvwgjLgg=; b=Ugl2aLdq97vK/8l1ipMts+HX2YGiqlnwGTzVItqmmZ0B6cls26RX/gHUZGK8CacFo2oj0H L2VTPU+Uor0FtrwM4sFaQinTvNWGHHmnzaCqDyvp+aXK3B4AZ42+oseVsbo5Q5ZbrrDT7Y vw5peO3x0gcbQlc49nWXgY+X7Pp3VGo= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-392-0CefMwXNM0-QHQc_3Ag8KQ-1; Fri, 18 Sep 2026 04:16:07 -0400 X-MC-Unique: 0CefMwXNM0-QHQc_3Ag8KQ-1 X-Mimecast-MFC-AGG-ID: 0CefMwXNM0-QHQc_3Ag8KQ_1789719366 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 66F02195398D; Fri, 18 Sep 2026 08:16:06 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id BCFE030001A2; Fri, 18 Sep 2026 08:16:05 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 25/28] KVM: x86/mmu: Obsolete all roots if memattr contains gPTEs Date: Fri, 18 Sep 2026 04:15:40 -0400 Message-ID: <20260918081543.139871-26-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Note that the zapping of obsolete roots needs to happen after mmu_lock is dropped, so introduce a separate kvm_arch callback. The two map neatly to the "front" and "back" halves of kvm_mmu_zap_all_fast(). Co-developed-by: Nicolas Saenz Julienne Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 32 ++++++++++++++++++++++++++++++++ include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 7 +++++-- 3 files changed, 38 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 097095f71d60..d658cfe31cd9 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -8265,6 +8265,7 @@ bool kvm_arch_post_set_memory_attributes(struct kvm *= kvm, { unsigned long attrs =3D range->arg.attributes; struct kvm_memory_slot *slot =3D range->slot; + struct kvm_mmu_page *sp; int level; =20 lockdep_assert_held_write(&kvm->mmu_lock); @@ -8320,9 +8321,40 @@ bool kvm_arch_post_set_memory_attributes(struct kvm = *kvm, hugepage_set_mixed(slot, gfn, level); } } + + /* + * There are special considerations when applying an memory protection + * attibute against a GPTE page. If set read-only, access/dirty bits + * within that page shouldn't be updated. If set non-accesible, + * accessing a virtual address that requires traversing that GPTE page + * should fault. + * + * On TDP enabled guests, the CPU faults on the GPTE address upon + * detecting such a situation. + * + * On non-TDP, upon detecting this situation, and based on the fact it + * should be a rare occasion, invalidate all the mmu roots. + */ + for (gfn_t gfn =3D range->start; gfn < range->end; gfn++) { + for_each_gfn_valid_sp_with_gptes(kvm, sp, gfn) { + __kvm_mmu_zap_all_fast_front_half(kvm); + return false; + } + } return false; } =20 +void kvm_arch_post_set_memory_attributes_unlocked(struct kvm *kvm) +{ + /* + * This may also be called if no zap was done in + * kvm_arch_post_set_memory_attributes(). It is not an issue because + * __kvm_mmu_zap_all_fast_back_half() is cheap when it does not do + * anything. + */ + __kvm_mmu_zap_all_fast_back_half(kvm); +} + void kvm_mmu_init_memslot_memory_attributes(struct kvm *kvm, struct kvm_memory_slot *slot) { diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index e9c0932f150e..33f3199a855f 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2624,6 +2624,7 @@ bool kvm_arch_pre_set_memory_attributes(struct kvm *k= vm, struct kvm_gfn_range *range); bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range); +void kvm_arch_post_set_memory_attributes_unlocked(struct kvm *kvm); bool kvm_mem_attributes_valid(struct kvm *kvm, unsigned long attrs); u64 kvm_supported_mem_attributes(struct kvm *kvm); =20 diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 24f46dce192e..ccbec4796e4b 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2516,7 +2516,7 @@ static bool kvm_range_memory_attributes_need_sync(str= uct kvm *kvm, return kvm_range_has_any_memory_attributes(kvm, start, end, mask); } =20 -static __always_inline void kvm_handle_gfn_range(struct kvm *kvm, +static __always_inline bool kvm_handle_gfn_range(struct kvm *kvm, struct kvm_mmu_notifier_range *range) { struct kvm_gfn_range gfn_range; @@ -2566,6 +2566,8 @@ static __always_inline void kvm_handle_gfn_range(stru= ct kvm *kvm, =20 if (found_memslot) KVM_MMU_UNLOCK(kvm); + + return found_memslot; } =20 static bool kvm_pre_set_memory_attributes(struct kvm *kvm, @@ -2652,7 +2654,8 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm,= gfn_t start, gfn_t end, smp_store_release(&kvm->mem_attrs.generation, kvm->mem_attrs.generation + 1); =20 - kvm_handle_gfn_range(kvm, &post_set_range); + if (kvm_handle_gfn_range(kvm, &post_set_range)) + kvm_arch_post_set_memory_attributes_unlocked(kvm); =20 out_unlock: mutex_unlock(&kvm->slots_lock); --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E16554B7A2E for ; Fri, 18 Sep 2026 08:16:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719379; cv=none; b=HsXFX3B9ECayFV9MpW0wO1CCHmohHRX7raelJda6PKH3lPYlvXkf+tZ3L3S00GyAwN+zgvGi7/ZxQP+826LcSBnjaaWY//e1xI+qBXQtaFlc3vJrAmnTbSFhG6cyGFtBS4jC2Rcd6H3MZ8hRPejV2Q6NfIgo+lajky1wWSV1TmY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719379; c=relaxed/simple; bh=MaZ/eRKUgmtGb4wfswpYOSpDBuxxDv55LPe2qjyKFa4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rM5BVHT8AxHmkEoTB8G+ZBjpq+ffg+RjS8A5Wn7KxsYPoo7fiBuI1ipM0/ddS2ILhx0LVyN/1zH2UHb8n6nozmMJpi3569hljYSOsIDEYBnb1sfGAHODjgHexeYWpYfQHB2wQgS+pRl0096RQ3xweD8Mlg7S0Zl7B+rATCUnn0w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AQzwGGyH; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AQzwGGyH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719371; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8/a+bb5eRkxWO3LL9YfkFttk4jO33JNtYU1u5jCAemo=; b=AQzwGGyHVVsDKv4YctotrsDDxmVKs0tJkTlG3CfqUQKTmY1/1cebw30zwhAexKI2ZlbXHx IwvFwLolxQSVydnyQZVrNulTvm21V9jrAUGreG5xTjPJAK0h1HzBLVr/cfDc9CcpajIYKE my6G8DCrbIV+8mFV+cH8d2GtmExnhCc= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-435-5Ulniin5O-ySdly9MVeWJg-1; Fri, 18 Sep 2026 04:16:08 -0400 X-MC-Unique: 5Ulniin5O-ySdly9MVeWJg-1 X-Mimecast-MFC-AGG-ID: 5Ulniin5O-ySdly9MVeWJg_1789719367 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 36E2B1955D7A; Fri, 18 Sep 2026 08:16:07 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8D2E930001A2; Fri, 18 Sep 2026 08:16:06 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 26/28] KVM: x86: selftests: Introduce memory protection attributes test Date: Fri, 18 Sep 2026 04:15:41 -0400 Message-ID: <20260918081543.139871-27-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Introduce architecture-independent tests for memory attributes, using small architecture-specific snippets to handle the execution tests. Co-developed-by: Nicolas Saenz Julienne Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- tools/include/uapi/linux/kvm.h | 3 + tools/testing/selftests/kvm/Makefile.kvm | 1 + .../testing/selftests/kvm/include/kvm_util.h | 23 +- .../testing/selftests/kvm/memory_attributes.c | 373 ++++++++++++++++++ .../selftests/kvm/x86/memory_attributes.c | 43 ++ 5 files changed, 433 insertions(+), 10 deletions(-) create mode 100644 tools/testing/selftests/kvm/memory_attributes.c create mode 100644 tools/testing/selftests/kvm/x86/memory_attributes.c diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h index 419011097fa8..9bd6488adc62 100644 --- a/tools/include/uapi/linux/kvm.h +++ b/tools/include/uapi/linux/kvm.h @@ -1649,6 +1649,9 @@ struct kvm_memory_attributes { __u64 flags; }; =20 +#define KVM_MEMORY_ATTRIBUTE_NR (1ULL << 0) +#define KVM_MEMORY_ATTRIBUTE_NW (1ULL << 1) +#define KVM_MEMORY_ATTRIBUTE_NX (1ULL << 2) #define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3) =20 #define KVM_CREATE_GUEST_MEMFD _IOWR(KVMIO, 0xd4, struct kvm_create_guest= _memfd) diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selft= ests/kvm/Makefile.kvm index 6e2bbf743eaa..776b91c800d0 100644 --- a/tools/testing/selftests/kvm/Makefile.kvm +++ b/tools/testing/selftests/kvm/Makefile.kvm @@ -167,6 +167,7 @@ TEST_GEN_PROGS_x86 +=3D rseq_test TEST_GEN_PROGS_x86 +=3D steal_time TEST_GEN_PROGS_x86 +=3D system_counter_offset_test TEST_GEN_PROGS_x86 +=3D pre_fault_memory_test +TEST_GEN_PROGS_x86 +=3D memory_attributes =20 # Compiled outputs used by test targets TEST_GEN_PROGS_EXTENDED_x86 +=3D x86/nx_huge_pages_test diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing= /selftests/kvm/include/kvm_util.h index 5a7a455b5387..996909230ac7 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -418,24 +418,27 @@ static inline void vm_enable_cap(struct kvm_vm *vm, u= 32 cap, u64 arg0) vm_ioctl(vm, KVM_ENABLE_CAP, &enable_cap); } =20 -static inline void vm_set_memory_attributes(struct kvm_vm *vm, gpa_t gpa, - u64 size, u64 attributes) +static inline int __vm_set_memory_attributes(struct kvm_vm *vm, u64 gpa, + u64 size, u64 attributes, + u64 flags) { struct kvm_memory_attributes attr =3D { .attributes =3D attributes, .address =3D gpa, .size =3D size, - .flags =3D 0, + .flags =3D flags, }; =20 - /* - * KVM_SET_MEMORY_ATTRIBUTES overwrites _all_ attributes. These flows - * need significant enhancements to support multiple attributes. - */ - TEST_ASSERT(!attributes || attributes =3D=3D KVM_MEMORY_ATTRIBUTE_PRIVATE, - "Update me to support multiple attributes!"); + return __vm_ioctl(vm, KVM_SET_MEMORY_ATTRIBUTES, &attr); +} =20 - vm_ioctl(vm, KVM_SET_MEMORY_ATTRIBUTES, &attr); +static inline void vm_set_memory_attributes(struct kvm_vm *vm, gpa_t gpa, + u64 size, u64 attributes) +{ + int rc; + + rc =3D __vm_set_memory_attributes(vm, gpa, size, attributes, 0); + TEST_ASSERT_VM_VCPU_IOCTL(!rc, KVM_SET_MEMORY_ATTRIBUTES, rc, vm); } =20 =20 diff --git a/tools/testing/selftests/kvm/memory_attributes.c b/tools/testin= g/selftests/kvm/memory_attributes.c new file mode 100644 index 000000000000..1e6e78f43100 --- /dev/null +++ b/tools/testing/selftests/kvm/memory_attributes.c @@ -0,0 +1,373 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2024, Amazon.com, Inc. or its affiliates. All Rights Rese= rved + * + * Test for KVM_MEMORY_ATTRIBUTES + */ +#include +#include + +#include "test_util.h" +#include "ucall_common.h" +#include "kvm_util.h" +#include "processor.h" +#include "hyperv.h" +#include "apic.h" +#include "asm/pvclock-abi.h" + +#define KVM_MEMORY_ATTRIBUTE_NO_ACCESS \ + (KVM_MEMORY_ATTRIBUTE_NR | KVM_MEMORY_ATTRIBUTE_NW | \ + KVM_MEMORY_ATTRIBUTE_NX) + +#define TEST_MEM_SLOT 1 +#define TEST_MEM_GPA 0x80000000 + +#define MMIO_GPA 0x700000000 +#define MMIO_GVA MMIO_GPA + +enum { + TEST_OP_NOP, + TEST_OP_READ, + TEST_OP_WRITE, + TEST_OP_EXEC, + TEST_OP_EXIT, +}; + +const char *test_op_names[] =3D +{ + [TEST_OP_READ] =3D "Read", + [TEST_OP_WRITE] =3D "Write", + [TEST_OP_EXEC] =3D "Exec", + [TEST_OP_EXIT] =3D "Exit", +}; + +struct test_data { + uint8_t op; + int stage; + gva_t vaddr; + + struct kvm_vcpu *vcpu; +}; + +static struct test_data *test_data; + +static uint64_t arch_controlled_read(gva_t addr); +static void arch_controlled_write(gva_t addr, uint64_t val); +static void arch_controlled_exec(gva_t addr); +static void arch_write_return_insn(struct kvm_vm *vm, gpa_t vaddr); + +static void guest_code(void *data) +{ + struct test_data *test_data =3D data; + int stage =3D 1; + + while (true) { + gva_t vaddr =3D READ_ONCE(test_data->vaddr); + + switch(READ_ONCE(test_data->op)) { + case TEST_OP_READ: + (void) arch_controlled_read(vaddr); + GUEST_SYNC(stage++); + break; + case TEST_OP_WRITE: + arch_controlled_write(vaddr, 1); + GUEST_SYNC(stage++); + break; + case TEST_OP_EXEC: + arch_controlled_exec(vaddr); + GUEST_SYNC(stage++); + break; + default: + goto exit; + }; + } + +exit: + GUEST_DONE(); +} + +static void vcpu_run_and_inc_stage(struct kvm_vcpu *vcpu) +{ + struct ucall uc; + + vcpu_run(vcpu); + + TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_IO); + switch (get_ucall(vcpu, &uc)) { + case UCALL_SYNC: + TEST_ASSERT(uc.args[1] =3D=3D test_data->stage, + "Unexpected stage: %ld (%d expected)", + uc.args[1], test_data->stage); + break; + case UCALL_ABORT: + REPORT_GUEST_ASSERT(uc); + /* NOT REACHED */ + default: + TEST_FAIL("Unknown ucall %lu", uc.cmd); + } + + test_data->stage++; +} + +static void test_page_restricted(struct kvm_vcpu *vcpu, int op, + gva_t vaddr, gpa_t fault_paddr, + uint64_t fault_reason) +{ + struct kvm_vm *vm =3D vcpu->vm; + int rc; + + test_data->op =3D op; + test_data->vaddr =3D vaddr; + + rc =3D _vcpu_run(vcpu); + TEST_ASSERT(rc =3D=3D -1 && errno =3D=3D EFAULT, + "KVM_RUN IOCTL didn't return EFAULT on %s, rc %d, errno %d", + test_op_names[op], rc, errno); + TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_MEMORY_FAULT); + TEST_ASSERT_EQ(vcpu->run->memory_fault.gpa, fault_paddr); + TEST_ASSERT_EQ(vcpu->run->memory_fault.flags, fault_reason); + TEST_ASSERT_EQ(vcpu->run->memory_fault.size, vm->page_size); +} + +static void test_page_accessible(struct kvm_vcpu *vcpu, int op, gva_t vadd= r) +{ + test_data->op =3D op; + test_data->vaddr =3D vaddr; + vcpu_run_and_inc_stage(vcpu); +} + +#include "x86/memory_attributes.c" + +/* + * We want to test the following cases: + * - Sucessful access to GPAs backed by memory attributes (for ex. read ac= cess + * on an read-only page). + * - First fault after setting memory attributes, with unpopulated SPTEs/E= PTS. + * - Fault caused by an SPTE/EPT reflecting the memory attributes. + * + * The list of ops below tests the 3 situations for each memory attribute + * combination. + */ +const struct memory_access { + const char *name; + uint64_t attrs; + int ops[5]; +} access_array[] =3D { + { "all allowed", 0, { TEST_OP_READ, TEST_OP_WRITE, TEST_OP_EXEC } }, + { "no write (unmapped)", KVM_MEMORY_ATTRIBUTE_NW, + { TEST_OP_WRITE, TEST_OP_READ, TEST_OP_EXEC, TEST_OP_WRITE } }, + { "no write (mapped)", KVM_MEMORY_ATTRIBUTE_NW, + { TEST_OP_READ, TEST_OP_WRITE, TEST_OP_READ, TEST_OP_EXEC, TEST_OP_WRIT= E } }, + { "no exec (unmapped)", KVM_MEMORY_ATTRIBUTE_NX, + { TEST_OP_EXEC, TEST_OP_READ, TEST_OP_WRITE, TEST_OP_EXEC } }, + { "no exec (mapped)", KVM_MEMORY_ATTRIBUTE_NX, + { TEST_OP_READ, TEST_OP_EXEC, TEST_OP_READ, TEST_OP_WRITE, TEST_OP_EXEC= } }, + { "read only", KVM_MEMORY_ATTRIBUTE_NW | KVM_MEMORY_ATTRIBUTE_NX, + { TEST_OP_EXEC, TEST_OP_WRITE, TEST_OP_READ, TEST_OP_WRITE, + TEST_OP_EXEC } }, + { "no access (map on read)", KVM_MEMORY_ATTRIBUTE_NO_ACCESS, + { TEST_OP_READ, TEST_OP_WRITE, TEST_OP_EXEC } }, + { "no access (map on write)", KVM_MEMORY_ATTRIBUTE_NO_ACCESS, + { TEST_OP_WRITE, TEST_OP_READ, TEST_OP_EXEC } }, + { "no access (map on exec)", KVM_MEMORY_ATTRIBUTE_NO_ACCESS, + { TEST_OP_EXEC, TEST_OP_READ, TEST_OP_WRITE } }, + /* Verify everything is back to normal */ + { "all allowed (2)", 0, { TEST_OP_READ, TEST_OP_WRITE, TEST_OP_EXEC } }, +}; + +static void test_page_access(struct kvm_vcpu *vcpu, gva_t vaddr, + uint64_t attrs, const int ops[]) +{ + struct kvm_vm *vm =3D vcpu->vm; + gpa_t paddr =3D addr_gva2gpa(vm, vaddr); + + for (int i =3D 0; i < ARRAY_SIZE(access_array[0].ops); i++) { + int op =3D ops[i]; + + if (op =3D=3D TEST_OP_NOP) + continue; + + /* + * We're about to have the guest jump into 'vaddr', make it a + * 'ret' instruction so it returns right away. + */ + if (op =3D=3D TEST_OP_EXEC) + arch_write_return_insn(vm, paddr); + + vm_set_memory_attributes(vm, paddr, vm->page_size, attrs); + + /* + * Attributes are negated, a match means the operation should + * fail. + */ + if (attrs & BIT_ULL(op - 1)) { + test_page_restricted(vcpu, op, vaddr, paddr, BIT(op - 1)); + vm_set_memory_attributes(vm, paddr, vm->page_size, 0); + } + + test_page_accessible(vcpu, op, vaddr); + } + +} + +static void test_memory_access(struct kvm_vcpu *vcpu, gva_t test_vm_vaddr, + size_t size) +{ + struct kvm_vm *vm =3D vcpu->vm; + gpa_t test_vm_paddr =3D addr_gva2gpa(vm, test_vm_vaddr); + + printf("gva %lx gpa %lx\n", test_vm_vaddr, test_vm_paddr); + for (size_t i =3D 0; i < ARRAY_SIZE(access_array); i++) { + uint64_t attrs =3D access_array[i].attrs; + + printf("starting %s test...\n", access_array[i].name); + vm_set_memory_attributes(vm, test_vm_paddr, size, attrs); + + for (gva_t vaddr =3D test_vm_vaddr; + vaddr < test_vm_vaddr + size; vaddr +=3D PAGE_SIZE) { + test_page_access(vcpu, vaddr, attrs, access_array[i].ops); + } + } +} + +static void test_memattrs_ignore_mmio(struct kvm_vcpu *vcpu) +{ + struct kvm_vm *vm =3D vcpu->vm; + + vm_set_memory_attributes(vm, MMIO_GPA, vm->page_size, + KVM_MEMORY_ATTRIBUTE_NO_ACCESS); + + test_data->op =3D TEST_OP_READ; + test_data->vaddr =3D MMIO_GVA; + vcpu_run(vcpu); + TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_MMIO); + TEST_ASSERT_EQ(vcpu->run->mmio.phys_addr, MMIO_GPA); + TEST_ASSERT_EQ(vcpu->run->mmio.is_write, 0); + TEST_ASSERT_EQ(vcpu->run->mmio.len, 8); + vcpu_run_and_inc_stage(vcpu); + + test_data->op =3D TEST_OP_WRITE; + test_data->vaddr =3D MMIO_GVA; + vcpu_run(vcpu); + TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_MMIO); + TEST_ASSERT_EQ(vcpu->run->mmio.phys_addr, MMIO_GPA); + TEST_ASSERT_EQ(vcpu->run->mmio.is_write, 1); + TEST_ASSERT_EQ(vcpu->run->mmio.len, 8); + vcpu_run_and_inc_stage(vcpu); + + vm_set_memory_attributes(vm, MMIO_GPA, vm->page_size, 0); +} + +static void test_input_validation(struct kvm_vm *vm) +{ + uint64_t flags, gpa =3D 0, size =3D 0, attrs =3D 0; + int rc; + + /* 'flags' is unsupported */ + flags =3D BIT(0); + rc =3D __vm_set_memory_attributes(vm, gpa, size, attrs, flags); + TEST_ASSERT_VM_VCPU_IOCTL(rc =3D=3D -1 && errno =3D=3D EINVAL, + KVM_SET_MEMORY_ATTRIBUTES, rc, vm); + + /* 'size' can't be 0 */ + flags =3D 0; + rc =3D __vm_set_memory_attributes(vm, gpa, size, attrs, flags); + TEST_ASSERT_VM_VCPU_IOCTL(rc =3D=3D -1 && errno =3D=3D EINVAL, + KVM_SET_MEMORY_ATTRIBUTES, rc, vm); + + /* 'gpa' shouldn't overflow */ + gpa =3D 0ULL - vm->page_size; + size =3D vm->page_size; + rc =3D __vm_set_memory_attributes(vm, gpa, size, attrs, flags); + TEST_ASSERT_VM_VCPU_IOCTL(rc =3D=3D -1 && errno =3D=3D EINVAL, + KVM_SET_MEMORY_ATTRIBUTES, rc, vm); + + /* 'gpa' should be page aligned */ + gpa =3D 1; + rc =3D __vm_set_memory_attributes(vm, gpa, size, attrs, flags); + TEST_ASSERT_VM_VCPU_IOCTL(rc =3D=3D -1 && errno =3D=3D EINVAL, + KVM_SET_MEMORY_ATTRIBUTES, rc, vm); + + /* 'size' should be page aligned */ + gpa =3D 0; + size =3D 1; + rc =3D __vm_set_memory_attributes(vm, gpa, size, attrs, flags); + TEST_ASSERT_VM_VCPU_IOCTL(rc =3D=3D -1 && errno =3D=3D EINVAL, + KVM_SET_MEMORY_ATTRIBUTES, rc, vm); + + /* exec mappings require read access */ + size =3D vm->page_size; + attrs =3D KVM_MEMORY_ATTRIBUTE_NR | KVM_MEMORY_ATTRIBUTE_NW; + rc =3D __vm_set_memory_attributes(vm, gpa, size, attrs, flags); + TEST_ASSERT_VM_VCPU_IOCTL(rc =3D=3D -1 && errno =3D=3D EINVAL, + KVM_SET_MEMORY_ATTRIBUTES, rc, vm); + + /* write mappings require read access */ + size =3D vm->page_size; + attrs =3D KVM_MEMORY_ATTRIBUTE_NR | KVM_MEMORY_ATTRIBUTE_NX; + rc =3D __vm_set_memory_attributes(vm, gpa, size, attrs, flags); + TEST_ASSERT_VM_VCPU_IOCTL(rc =3D=3D -1 && errno =3D=3D EINVAL, + KVM_SET_MEMORY_ATTRIBUTES, rc, vm); + + /* private mappings are incompatible with access restrictions */ + attrs =3D KVM_MEMORY_ATTRIBUTE_NW | KVM_MEMORY_ATTRIBUTE_PRIVATE; + rc =3D __vm_set_memory_attributes(vm, gpa, size, attrs, flags); + TEST_ASSERT_VM_VCPU_IOCTL(rc =3D=3D -1 && errno =3D=3D EINVAL, + KVM_SET_MEMORY_ATTRIBUTES, rc, vm); +} + +static void test_finalize(struct kvm_vcpu *vcpu) +{ + test_data->op =3D TEST_OP_EXIT; + vcpu_run(vcpu); + TEST_ASSERT_EQ(get_ucall(vcpu, NULL), UCALL_DONE); +} + +static struct test_data *init_test_data(struct kvm_vcpu *vcpu) +{ + struct kvm_vm *vm =3D vcpu->vm; + gva_t test_data_vm_vaddr; + + test_data_vm_vaddr =3D vm_alloc_page(vm); + vcpu_args_set(vcpu, 1, test_data_vm_vaddr); + + test_data =3D addr_gva2hva(vm, test_data_vm_vaddr); + test_data->stage =3D 1; + test_data->vcpu =3D vcpu; + + return test_data; +} + +int main(int argc, char *argv[]) +{ + uint32_t guest_page_size =3D vm_guest_mode_params[VM_MODE_DEFAULT].page_s= ize; + unsigned int ptes_per_page =3D guest_page_size / 8; + size_t pages =3D ptes_per_page * 2; /* 2 huge-pages */ + size_t size =3D guest_page_size * pages; + struct kvm_vcpu *vcpu; + gva_t test_mem; + struct kvm_vm *vm; + + TEST_REQUIRE(kvm_check_cap(KVM_CAP_MEMORY_ATTRIBUTES) & + KVM_MEMORY_ATTRIBUTE_NO_ACCESS); + + vm =3D __vm_create_with_one_vcpu(&vcpu, pages, guest_code); + + /* add a few extra pages for use by tests */ + vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, + TEST_MEM_GPA, TEST_MEM_SLOT, + pages + 100, 0); + vm->memslots[MEM_REGION_TEST_DATA] =3D TEST_MEM_SLOT; + + test_mem =3D vm_alloc_pages(vm, pages); + virt_map(vcpu->vm, MMIO_GVA, MMIO_GPA, 1); + test_data =3D init_test_data(vcpu); + + test_input_validation(vm); + test_memory_access(vcpu, test_mem, size); + test_memattrs_ignore_mmio(vcpu); + test_finalize(vcpu); + + kvm_vm_free(vm); + return 0; +} diff --git a/tools/testing/selftests/kvm/x86/memory_attributes.c b/tools/te= sting/selftests/kvm/x86/memory_attributes.c new file mode 100644 index 000000000000..2e1148f5146d --- /dev/null +++ b/tools/testing/selftests/kvm/x86/memory_attributes.c @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2024, Amazon.com, Inc. or its affiliates. All Rights Rese= rved + * + * Test for KVM_MEMORY_ATTRIBUTES + */ +#include "kvm_util.h" +#include "apic.h" + +uint64_t arch_controlled_read(gva_t addr) +{ + uint64_t val; + + asm volatile("mov %[addr], %%rax \n\r" + "mov (%%rax), %[val] \n\r" + : [val] "=3Dr" (val) + : [addr] "m"(addr) + : "memory", "rax"); + + return val; +} + +void arch_controlled_write(gva_t addr, uint64_t val) +{ + asm volatile("mov %[addr], %%rax \n\r" + "mov %[val], %%rbx \n\r" + "mov %%rbx, (%%rax) \n\r" + :: [addr] "m" (addr), [val] "m" (val) + : "memory", "rax", "rbx"); +} + +void arch_controlled_exec(gva_t addr) +{ + asm volatile("mov %[addr], %%rax \n\r" + "call *%%rax \n\t" + :: [addr] "m"(addr) + : "memory", "rax"); +} + +void arch_write_return_insn(struct kvm_vm *vm, gpa_t vaddr) +{ + memset(addr_gpa2hva(vm, vaddr), 0xc3, 1); +} --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 057993DAADC for ; Fri, 18 Sep 2026 08:16:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719382; cv=none; b=rXbrlJQEs6/xgd/gSJANuLoCak6fxniTXop8zUDJIQDAj2EXl3LgegATVIh21HmM9CSMnPAKZkDKk+/k3XC9fDFB+Tih1lTVIS4wShJgss4C5naGmHjn6qG3O6j2CYf8kV5Hno2xURN+LpUA9OMPAi5JxkqvuqI3yzxKDiIbZ0Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719382; c=relaxed/simple; bh=9qvtd6HlMYOvHuR/UIrfN3fQJKi7NfdPz+uYYhP8Vxg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=VlgGmXdYCkz4C+RqcVeUv1dUwNbd/AYMhyP52xgTk7heWbMVeRajBQzs4DL71RzPR8vTCe0rcvKgXVIYSMr1uG38g7sE5fePbxYuRreAdlDHBx09jxSAAYpC+T7f4DaTADi/YqVQts6Xaw70+QtEL1RpTf2TEBH0vWcRMkbTJD0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=QVS8luAU; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QVS8luAU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719371; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Sm9axdAph78zaIk0C1MNqvYLnD47Sloq9YIbNoyapPg=; b=QVS8luAUr0eCmg8MkUGTP66/FY5/Ee83WVO34pvtSMsNM5xgFgy7DenQw/k25iRHrMnVdu uQ2n57nzzTobTl8R0MMoqR7JLh0JHzK++ljLj5U7/2S1I901WDCrW8wVUqsn5VkEpV/Z2Q iV7E7bxMtSzVGYIc4yPTnpGWefsvKVE= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-645-IxkWLatlNwec1syhIbtVQA-1; Fri, 18 Sep 2026 04:16:09 -0400 X-MC-Unique: IxkWLatlNwec1syhIbtVQA-1 X-Mimecast-MFC-AGG-ID: IxkWLatlNwec1syhIbtVQA_1789719368 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 05DDA18000A3; Fri, 18 Sep 2026 08:16:08 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5D0E430001A2; Fri, 18 Sep 2026 08:16:07 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 27/28] KVM: x86: selftests: Introduce memory attributes PTE test Date: Fri, 18 Sep 2026 04:15:42 -0400 Message-ID: <20260918081543.139871-28-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Add more memory attributes tests to check that memory protection restricts accesses even when installed on a page that holds guest page table entries. Notably two cases are taken into account: - The page is made non-accesible. In such case the next access to a virtual memory address translated by that paging structure should fault. - The page is made read-only. In such case the next access to a virtual memory address translated by that paging structure should either fault, or succeed yet not perform any writes into the PTE (accessed and dirty bits). Co-developed-by: Nicolas Saenz Julienne Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- .../testing/selftests/kvm/memory_attributes.c | 39 ++++- .../selftests/kvm/x86/memory_attributes.c | 144 +++++++++++++++++- 2 files changed, 177 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/kvm/memory_attributes.c b/tools/testin= g/selftests/kvm/memory_attributes.c index 1e6e78f43100..c703c5586d43 100644 --- a/tools/testing/selftests/kvm/memory_attributes.c +++ b/tools/testing/selftests/kvm/memory_attributes.c @@ -25,11 +25,16 @@ #define MMIO_GPA 0x700000000 #define MMIO_GVA MMIO_GPA =20 +#define PT_WRITABLE_MASK BIT_ULL(1) +#define PT_ACCESSED_MASK BIT_ULL(5) +#define PTE_VADDR 0x1000000000 + enum { TEST_OP_NOP, TEST_OP_READ, TEST_OP_WRITE, TEST_OP_EXEC, + TEST_OP_INVPLG, TEST_OP_EXIT, }; =20 @@ -38,6 +43,7 @@ const char *test_op_names[] =3D [TEST_OP_READ] =3D "Read", [TEST_OP_WRITE] =3D "Write", [TEST_OP_EXEC] =3D "Exec", + [TEST_OP_INVPLG] =3D "Invplg", [TEST_OP_EXIT] =3D "Exit", }; =20 @@ -45,6 +51,7 @@ struct test_data { uint8_t op; int stage; gva_t vaddr; + uint64_t expected_val; =20 struct kvm_vcpu *vcpu; }; @@ -54,7 +61,8 @@ static struct test_data *test_data; static uint64_t arch_controlled_read(gva_t addr); static void arch_controlled_write(gva_t addr, uint64_t val); static void arch_controlled_exec(gva_t addr); -static void arch_write_return_insn(struct kvm_vm *vm, gpa_t vaddr); +static void arch_write_return_insn(struct kvm_vm *vm, gpa_t paddr); +static bool arch_test_op(struct test_data *test_data); =20 static void guest_code(void *data) { @@ -62,6 +70,7 @@ static void guest_code(void *data) int stage =3D 1; =20 while (true) { + uint64_t expected_val =3D READ_ONCE(test_data->expected_val); gva_t vaddr =3D READ_ONCE(test_data->vaddr); =20 switch(READ_ONCE(test_data->op)) { @@ -70,7 +79,7 @@ static void guest_code(void *data) GUEST_SYNC(stage++); break; case TEST_OP_WRITE: - arch_controlled_write(vaddr, 1); + arch_controlled_write(vaddr, expected_val); GUEST_SYNC(stage++); break; case TEST_OP_EXEC: @@ -78,7 +87,10 @@ static void guest_code(void *data) GUEST_SYNC(stage++); break; default: - goto exit; + if (!arch_test_op(test_data)) + goto exit; + GUEST_SYNC(stage++); + break; }; } =20 @@ -109,6 +121,21 @@ static void vcpu_run_and_inc_stage(struct kvm_vcpu *vc= pu) test_data->stage++; } =20 +static int test_page(struct kvm_vcpu *vcpu, int op, gva_t vaddr) +{ + int rc; + + test_data->op =3D op; + test_data->vaddr =3D vaddr; + + rc =3D _vcpu_run(vcpu); + + if (rc >=3D 0) + test_data->stage++; + + return rc < 0 ? -errno : rc; +} + static void test_page_restricted(struct kvm_vcpu *vcpu, int op, gva_t vaddr, gpa_t fault_paddr, uint64_t fault_reason) @@ -125,7 +152,8 @@ static void test_page_restricted(struct kvm_vcpu *vcpu,= int op, test_op_names[op], rc, errno); TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_MEMORY_FAULT); TEST_ASSERT_EQ(vcpu->run->memory_fault.gpa, fault_paddr); - TEST_ASSERT_EQ(vcpu->run->memory_fault.flags, fault_reason); + if (fault_reason) + TEST_ASSERT_EQ(vcpu->run->memory_fault.flags, fault_reason); TEST_ASSERT_EQ(vcpu->run->memory_fault.size, vm->page_size); } =20 @@ -366,6 +394,9 @@ int main(int argc, char *argv[]) test_input_validation(vm); test_memory_access(vcpu, test_mem, size); test_memattrs_ignore_mmio(vcpu); +#ifdef __x86_64__ + arch_test_memory_access_pte(vcpu, test_mem); +#endif test_finalize(vcpu); =20 kvm_vm_free(vm); diff --git a/tools/testing/selftests/kvm/x86/memory_attributes.c b/tools/te= sting/selftests/kvm/x86/memory_attributes.c index 2e1148f5146d..12395feb6ac7 100644 --- a/tools/testing/selftests/kvm/x86/memory_attributes.c +++ b/tools/testing/selftests/kvm/x86/memory_attributes.c @@ -37,7 +37,147 @@ void arch_controlled_exec(gva_t addr) : "memory", "rax"); } =20 -void arch_write_return_insn(struct kvm_vm *vm, gpa_t vaddr) +void arch_write_return_insn(struct kvm_vm *vm, gpa_t paddr) { - memset(addr_gpa2hva(vm, vaddr), 0xc3, 1); + memset(addr_gpa2hva(vm, paddr), 0xc3, 1); +} + +bool arch_test_op(struct test_data *test_data) +{ + gva_t vaddr =3D READ_ONCE(test_data->vaddr); + + switch(READ_ONCE(test_data->op)) { + case TEST_OP_INVPLG: + asm volatile("invlpg (%0)" + :: "b" (vaddr): "memory"); + return true; + default: + return false; + } +} + +/* + * This test validates that, during a page walk, if the page a PTE is plac= ed in + * is read-only the accesss and dirty bits will not be written. Note There= 's a + * slight variation in behaviour between TDP and non-TDP VMs: + * - With TDP enabled, KVM issues a fault exit upon observing the non-wri= table + * page. + * - With non-TDP, the access bit is not set, but the walk succeeds. + * + * This is aligned with read-only memslots' behaviour. + */ +static void test_memory_access_pte_ro(struct kvm_vcpu *vcpu, gva_t vaddr) +{ + struct kvm_vm *vm =3D vcpu->vm; + gpa_t paddr; + u64 *pte; + const u64 accessed_mask =3D PTE_ACCESSED_MASK(&vm->mmu); + + pte =3D vm_get_pte(vm, vaddr); + paddr =3D addr_hva2gpa(vm, pte) & GENMASK(61, vm->page_shift); + + *pte &=3D ~accessed_mask; + vm_set_memory_attributes(vm, paddr, vm->page_size, KVM_MEMORY_ATTRIBUTE_N= W); + if (test_page(vcpu, TEST_OP_READ, vaddr) < 0) { + test_page_restricted(vcpu, TEST_OP_READ, vaddr, paddr, + /* write PTE's accessed bit */ + KVM_MEMORY_EXIT_FLAG_WRITE); + + vm_set_memory_attributes(vm, paddr, vm->page_size, 0); + test_page_accessible(vcpu, TEST_OP_READ, vaddr); + TEST_ASSERT_EQ(*pte & accessed_mask, accessed_mask); + + /* Re-run the test, now vaddr is backed by an EPT. */ + *pte &=3D ~accessed_mask; + vm_set_memory_attributes(vm, paddr, vm->page_size, + KVM_MEMORY_ATTRIBUTE_NW); + test_page_restricted(vcpu, TEST_OP_READ, vaddr, paddr, + /* write PTE's accessed bit */ + KVM_MEMORY_EXIT_FLAG_WRITE); + vm_set_memory_attributes(vm, paddr, vm->page_size, 0); + test_page_accessible(vcpu, TEST_OP_READ, vaddr); + } else { + TEST_ASSERT_EQ(*pte & accessed_mask, 0); + vm_set_memory_attributes(vm, paddr, vm->page_size, 0); + } +} + +/* + * This test validates that, during a page walk, if the page a PTE is plac= ed in + * is maked as non-accesible, KVM issues a fault exit. + */ +static void test_memory_access_pte_nr(struct kvm_vcpu *vcpu, gva_t vaddr) +{ + struct kvm_vm *vm =3D vcpu->vm; + gpa_t paddr; + uint64_t *pte; + + pte =3D vm_get_pte(vm, vaddr); + paddr =3D addr_hva2gpa(vm, pte) & GENMASK(61, vm->page_shift); + + vm_set_memory_attributes(vm, paddr, vm->page_size, + KVM_MEMORY_ATTRIBUTE_NO_ACCESS); + + test_page_restricted(vcpu, TEST_OP_READ, vaddr, paddr, 0); + + vm_set_memory_attributes(vm, paddr, vm->page_size, 0); + test_page_accessible(vcpu, TEST_OP_READ, vaddr); + + /* Re-run the test, now vaddr is backed by an SPTE. */ + vm_set_memory_attributes(vm, paddr, vm->page_size, + KVM_MEMORY_ATTRIBUTE_NO_ACCESS); + test_page_restricted(vcpu, TEST_OP_READ, vaddr, paddr, 0); + vm_set_memory_attributes(vm, paddr, vm->page_size, 0); + test_page_accessible(vcpu, TEST_OP_READ, vaddr); +} + +static void test_memory_access_sync_spte(struct kvm_vcpu *vcpu, gva_t vadd= r) +{ + struct kvm_vm *vm =3D vcpu->vm; + gpa_t paddr =3D addr_gva2gpa(vm, vaddr); + uint64_t *pte, old_pte, new_pte; + + pte =3D vm_get_pte(vm, vaddr); + gpa_t pte_paddr =3D addr_hva2gpa(vm, pte); + gpa_t pte_page_paddr =3D pte_paddr & GENMASK(61, vm->page_shift); + int pte_offset =3D pte_paddr - pte_page_paddr; + virt_pg_map(vm, PTE_VADDR, pte_page_paddr); + old_pte =3D *pte; + + /* Set vmaddr as non-executable */ + vm_set_memory_attributes(vm, paddr, vm->page_size, KVM_MEMORY_ATTRIBUTE_N= X); + + /* + * Make sure SPTEs are populated as previous op might have destroyed + * them. We new have a non-executable SPTE. + */ + test_page_accessible(vcpu, TEST_OP_READ, vaddr); + + /* + * Update PTE, make it non-writable and flush TLBs to make sure we go + * through the sync_spte path. This should update the SPTE and make it + * read-only. + */ + new_pte =3D (old_pte & ~PT_WRITABLE_MASK) | PT_ACCESSED_MASK; + test_data->expected_val =3D new_pte; + test_page_accessible(vcpu, TEST_OP_WRITE, PTE_VADDR + pte_offset); + TEST_ASSERT_EQ(*pte, new_pte); + test_page_accessible(vcpu, TEST_OP_INVPLG, vaddr); + + /* The not executable attrs remain valid */ + arch_write_return_insn(vm, paddr); + test_page_restricted(vcpu, TEST_OP_EXEC, vaddr, paddr, + KVM_MEMORY_EXIT_FLAG_EXEC); + + /* Cleanup */ + *pte =3D old_pte; + vm_set_memory_attributes(vm, paddr, vm->page_size, 0); + test_page_accessible(vcpu, TEST_OP_EXEC, vaddr); +} + +static void arch_test_memory_access_pte(struct kvm_vcpu *vcpu, gva_t vaddr) +{ + test_memory_access_pte_nr(vcpu, vaddr); + test_memory_access_pte_ro(vcpu, vaddr); + test_memory_access_sync_spte(vcpu, vaddr); } --=20 2.52.0 From nobody Fri Sep 25 00:40:33 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7782B4B95B4 for ; Fri, 18 Sep 2026 08:16:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719382; cv=none; b=T5PxZYQ+VmGWKyIqYhhEHB+QwPqTM1qntiAk592/yh4v8v+J3alFLHbK6RNqamxyWovvvvI8xsmvg4oMDIt1WLmNGFAy9G8ya1tIn84dT2ZaNbLa547vHrB5W4+TIfTmmVB5d3uSfLaPK/2OJbRzZEjFjzGAHtZyWh7wwK/BX+w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789719382; c=relaxed/simple; bh=ZeAs2RHWuVZQ4kVY9VrdUgPI0drlMcBQr3Sohd7nkUU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=C4ItkB6FkYS6Do5bEBNeLolJnTRCrYyYlF88uhoqjigIlfiObA6sVCyEtBfrRHAgncCgC6h6znkdES7ECQBnHApeiE7UFXWuKo21BnR7XHQZWKjOeXGGJeuBJIy1lYwb2vedCqGX1MFZQeu1fzDlB+CI8Fyf5Nv/Adw0fJFdecw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=A7dZuz/q; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="A7dZuz/q" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1789719372; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sQFKDtIIcyaHramE20aRDws/f2ayTgXSNylmL8x1hGc=; b=A7dZuz/qsrFvT9c75fRlF6Puk5MyXblYSDwwwJcO0utaQchWuc5efATblYJr94MvDeNBas G9yHt4+QvyjJO7yccztMRBu4KTob7v/TIfbTze4M/nu+7Zw0enDtU0xd8X3YfGB1nxrP1b 8eJs+ClgPeej9VZU+bxPnlxpn9TGB0A= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-7-uimsVkKpNpKXFMcuaY1olQ-1; Fri, 18 Sep 2026 04:16:09 -0400 X-MC-Unique: uimsVkKpNpKXFMcuaY1olQ-1 X-Mimecast-MFC-AGG-ID: uimsVkKpNpKXFMcuaY1olQ_1789719368 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CAC541802542; Fri, 18 Sep 2026 08:16:08 +0000 (UTC) Received: from virtlab1023.virt.eng.rdu2.dc.redhat.com (virtlab1023.virt.eng.rdu2.dc.redhat.com [10.18.48.26]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2D15730001A2; Fri, 18 Sep 2026 08:16:08 +0000 (UTC) From: Paolo Bonzini To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: nsaenz@amazon.com, vkuznets@redhat.com, snambakam@linux.microsoft.com Subject: [PATCH v2 28/28] KVM: x86: selftests: Introduce memory attributes side-channel tests Date: Fri, 18 Sep 2026 04:15:43 -0400 Message-ID: <20260918081543.139871-29-pbonzini@redhat.com> In-Reply-To: <20260918081543.139871-1-pbonzini@redhat.com> References: <20260918081543.139871-1-pbonzini@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" Introduce memory attributes selftests to catch any vulnerable side-channels. Memory attributes access restrictions are vulnerable to side-channel attacks. This means that any KVM operation initiated by the guest that requires guest memory access (which is the case for most para-virtualised interfaces) needs to consider memory attributes. The tests confirm this requirement is upheld for a variety of use-cases, exercising various kinds of guest memory access such as kvm_read/write_guest(), gfn_to_hva_cache and gfn_to_pfn_cache. Co-developed-by: Nicolas Saenz Julienne Signed-off-by: Nicolas Saenz Julienne Signed-off-by: Paolo Bonzini --- .../testing/selftests/kvm/include/kvm_util.h | 9 + .../selftests/kvm/include/x86/processor.h | 1 + .../testing/selftests/kvm/lib/x86/processor.c | 5 + .../testing/selftests/kvm/memory_attributes.c | 51 +++- .../selftests/kvm/x86/memory_attributes.c | 232 ++++++++++++++++++ 5 files changed, 297 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing= /selftests/kvm/include/kvm_util.h index 996909230ac7..b18af4f5f615 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -883,6 +883,15 @@ static inline int vcpu_get_stats_fd(struct kvm_vcpu *v= cpu) return fd; } =20 +static inline struct kvm_translation vcpu_translate(struct kvm_vcpu *vcpu, + u64 gva) +{ + struct kvm_translation tr =3D { .linear_address =3D gva }; + + vcpu_ioctl(vcpu, KVM_TRANSLATE, &tr); + return tr; +} + int __kvm_has_device_attr(int dev_fd, u32 group, u64 attr); =20 static inline void kvm_has_device_attr(int dev_fd, u32 group, u64 attr) diff --git a/tools/testing/selftests/kvm/include/x86/processor.h b/tools/te= sting/selftests/kvm/include/x86/processor.h index 6e6f70035508..398380157fa1 100644 --- a/tools/testing/selftests/kvm/include/x86/processor.h +++ b/tools/testing/selftests/kvm/include/x86/processor.h @@ -1472,6 +1472,7 @@ static inline bool kvm_is_lbrv_enabled(void) return !!get_kvm_amd_param_integer("lbrv"); } =20 +u64 *vm_get_pte_level(struct kvm_vm *vm, gva_t gva, int *level); u64 *vm_get_pte(struct kvm_vm *vm, gva_t gva); =20 u64 kvm_hypercall(u64 nr, u64 a0, u64 a1, u64 a2, u64 a3); diff --git a/tools/testing/selftests/kvm/lib/x86/processor.c b/tools/testin= g/selftests/kvm/lib/x86/processor.c index d31fa81ea075..f51b4929d668 100644 --- a/tools/testing/selftests/kvm/lib/x86/processor.c +++ b/tools/testing/selftests/kvm/lib/x86/processor.c @@ -400,6 +400,11 @@ u64 *tdp_get_pte(struct kvm_vm *vm, u64 l2_gpa) return __vm_get_page_table_entry(vm, &vm->stage2_mmu, l2_gpa, &level); } =20 +u64 *vm_get_pte_level(struct kvm_vm *vm, gva_t gva, int *level) +{ + return __vm_get_page_table_entry(vm, &vm->mmu, gva, level); +} + u64 *vm_get_pte(struct kvm_vm *vm, gva_t gva) { int level =3D PG_LEVEL_4K; diff --git a/tools/testing/selftests/kvm/memory_attributes.c b/tools/testin= g/selftests/kvm/memory_attributes.c index c703c5586d43..63dec49c2888 100644 --- a/tools/testing/selftests/kvm/memory_attributes.c +++ b/tools/testing/selftests/kvm/memory_attributes.c @@ -22,6 +22,8 @@ #define TEST_MEM_SLOT 1 #define TEST_MEM_GPA 0x80000000 =20 +#define HV_STATUS_INVALID_HYPERCALL_INPUT 3 + #define MMIO_GPA 0x700000000 #define MMIO_GVA MMIO_GPA =20 @@ -29,12 +31,35 @@ #define PT_ACCESSED_MASK BIT_ULL(5) #define PTE_VADDR 0x1000000000 =20 +static volatile uint64_t ipis_rcvd; + +static pthread_t vcpu_thread; + +struct hv_vpset { + u64 format; + u64 valid_bank_mask; + u64 bank_contents[2]; +}; + +enum HV_GENERIC_SET_FORMAT { + HV_GENERIC_SET_SPARSE_4K, + HV_GENERIC_SET_ALL, +}; + +struct hv_send_ipi_ex { + u32 vector; + u32 reserved; + struct hv_vpset vp_set; +}; + enum { TEST_OP_NOP, TEST_OP_READ, TEST_OP_WRITE, TEST_OP_EXEC, TEST_OP_INVPLG, + TEST_OP_HYPERV_HYPERCALL_INPUT, + TEST_OP_MONITOR_ADDRESS, TEST_OP_EXIT, }; =20 @@ -44,6 +69,8 @@ const char *test_op_names[] =3D [TEST_OP_WRITE] =3D "Write", [TEST_OP_EXEC] =3D "Exec", [TEST_OP_INVPLG] =3D "Invplg", + [TEST_OP_HYPERV_HYPERCALL_INPUT] =3D "HvHcall input", + [TEST_OP_MONITOR_ADDRESS] =3D "Monitor address", [TEST_OP_EXIT] =3D "Exit", }; =20 @@ -51,6 +78,8 @@ struct test_data { uint8_t op; int stage; gva_t vaddr; + gpa_t paddr; + uint8_t confirm_read; uint64_t expected_val; =20 struct kvm_vcpu *vcpu; @@ -63,19 +92,26 @@ static void arch_controlled_write(gva_t addr, uint64_t = val); static void arch_controlled_exec(gva_t addr); static void arch_write_return_insn(struct kvm_vm *vm, gpa_t paddr); static bool arch_test_op(struct test_data *test_data); +static void arch_guest_init(void); =20 static void guest_code(void *data) { struct test_data *test_data =3D data; int stage =3D 1; =20 + arch_guest_init(); + while (true) { uint64_t expected_val =3D READ_ONCE(test_data->expected_val); + bool confirm_read =3D READ_ONCE(test_data->confirm_read); gva_t vaddr =3D READ_ONCE(test_data->vaddr); + u64 val; =20 switch(READ_ONCE(test_data->op)) { case TEST_OP_READ: - (void) arch_controlled_read(vaddr); + val =3D arch_controlled_read(vaddr); + if (confirm_read) + GUEST_ASSERT_EQ(expected_val, val); GUEST_SYNC(stage++); break; case TEST_OP_WRITE: @@ -86,6 +122,14 @@ static void guest_code(void *data) arch_controlled_exec(vaddr); GUEST_SYNC(stage++); break; + case TEST_OP_MONITOR_ADDRESS: { + if (arch_controlled_read(vaddr) !=3D expected_val) + GUEST_SYNC(stage++); + break; + } + case TEST_OP_NOP: + GUEST_SYNC(stage++); + break; default: if (!arch_test_op(test_data)) goto exit; @@ -145,6 +189,7 @@ static void test_page_restricted(struct kvm_vcpu *vcpu,= int op, =20 test_data->op =3D op; test_data->vaddr =3D vaddr; + test_data->paddr =3D fault_paddr; =20 rc =3D _vcpu_run(vcpu); TEST_ASSERT(rc =3D=3D -1 && errno =3D=3D EFAULT, @@ -390,12 +435,16 @@ int main(int argc, char *argv[]) test_mem =3D vm_alloc_pages(vm, pages); virt_map(vcpu->vm, MMIO_GVA, MMIO_GPA, 1); test_data =3D init_test_data(vcpu); +#ifdef __x86_64__ + vcpu_set_hv_cpuid(vcpu); +#endif =20 test_input_validation(vm); test_memory_access(vcpu, test_mem, size); test_memattrs_ignore_mmio(vcpu); #ifdef __x86_64__ arch_test_memory_access_pte(vcpu, test_mem); + arch_test_side_channels(vcpu, test_mem, size); #endif test_finalize(vcpu); =20 diff --git a/tools/testing/selftests/kvm/x86/memory_attributes.c b/tools/te= sting/selftests/kvm/x86/memory_attributes.c index 12395feb6ac7..6ce7da5cb376 100644 --- a/tools/testing/selftests/kvm/x86/memory_attributes.c +++ b/tools/testing/selftests/kvm/x86/memory_attributes.c @@ -45,17 +45,28 @@ void arch_write_return_insn(struct kvm_vm *vm, gpa_t pa= ddr) bool arch_test_op(struct test_data *test_data) { gva_t vaddr =3D READ_ONCE(test_data->vaddr); + gpa_t paddr =3D READ_ONCE(test_data->paddr); =20 switch(READ_ONCE(test_data->op)) { case TEST_OP_INVPLG: asm volatile("invlpg (%0)" :: "b" (vaddr): "memory"); return true; + case TEST_OP_HYPERV_HYPERCALL_INPUT: + hyperv_hypercall(HVCALL_SEND_IPI_EX, paddr, 0); + asm volatile ("sti; hlt; cli;"); + GUEST_ASSERT_EQ(ipis_rcvd, 1); + return true; default: return false; } } =20 +void arch_guest_init(void) +{ + x2apic_enable(); +} + /* * This test validates that, during a page walk, if the page a PTE is plac= ed in * is read-only the accesss and dirty bits will not be written. Note There= 's a @@ -181,3 +192,224 @@ static void arch_test_memory_access_pte(struct kvm_vc= pu *vcpu, gva_t vaddr) test_memory_access_pte_ro(vcpu, vaddr); test_memory_access_sync_spte(vcpu, vaddr); } + +#define IPI_VECTOR 0xfe + +static void guest_ipi_handler_hv(struct ex_regs *regs) +{ + ipis_rcvd++; + wrmsr(HV_X64_MSR_EOI, 1); +} + +/* + * This test verifies that the Hyper-V hypercall exit handler takes memory + * attributes into account before accessing input data. It coordinates wit= h the + * guest through the 'TEST_OP_HYPERV_HYPERCALL_INPUT' operation and instru= cts + * the guest to issue two PV IPIs. The first PV IPI fails because the input + * data is held in read-protected memory. Subsequently, the memory protect= ion + * is lifted, and the second PV IPI succeeds. + */ +static void test_side_channel_hyperv_hypercall_inputs(struct kvm_vcpu *vcp= u, + gva_t vaddr, + size_t size) +{ + struct kvm_vm *vm =3D vcpu->vm; + struct hv_send_ipi_ex *ipi_ex =3D addr_gva2hva(vm, vaddr); + gpa_t paddr =3D addr_gva2gpa(vm, vaddr); + + if (!kvm_has_cap(KVM_CAP_HYPERV_SEND_IPI) || + !kvm_has_cap(KVM_CAP_HCALL_FAULT_EXIT)) + return; + + vm_enable_cap(vcpu->vm, KVM_CAP_HCALL_FAULT_EXIT, 1); + + ipis_rcvd =3D 0; + vm_install_exception_handler(vm, IPI_VECTOR, guest_ipi_handler_hv); + vcpu_set_msr(vcpu, HV_X64_MSR_GUEST_OS_ID, HYPERV_LINUX_OS_ID); + + *ipi_ex =3D (struct hv_send_ipi_ex){ + .vector =3D IPI_VECTOR, + .vp_set.format =3D HV_GENERIC_SET_ALL, + }; + + vm_set_memory_attributes(vm, paddr, vm->page_size, KVM_MEMORY_ATTRIBUTE_N= O_ACCESS); + test_page_restricted(vcpu, TEST_OP_HYPERV_HYPERCALL_INPUT, vaddr, paddr, + KVM_MEMORY_EXIT_FLAG_READ); + vm_set_memory_attributes(vm, paddr, vm->page_size, 0); + vcpu_run_and_inc_stage(vcpu); +} + +/* + * Verifies that the guest page table walker fails the walk if it encounte= rs a + * page table entry address read-protected by a memory attribute. + */ +static void test_side_channel_emul_page_walks(struct kvm_vcpu *vcpu, + gva_t test_vm_addr, + size_t size) +{ + const uint64_t pte_addr_mask =3D GENMASK(51, 12); + struct kvm_vm *vm =3D vcpu->vm; + struct kvm_translation tr; + int level =3D PG_LEVEL_1G; + gpa_t paddr; + uint64_t *pte; + + pte =3D vm_get_pte_level(vm, test_vm_addr, &level); + TEST_ASSERT_EQ(level, PG_LEVEL_1G); + paddr =3D *pte & pte_addr_mask; + + tr =3D vcpu_translate(vcpu, test_vm_addr); + TEST_ASSERT_EQ(tr.valid, true); + TEST_ASSERT_EQ(tr.physical_address, addr_gva2gpa(vm, test_vm_addr)); + + vm_set_memory_attributes(vm, paddr, vm->page_size, KVM_MEMORY_ATTRIBUTE_N= O_ACCESS); + tr =3D vcpu_translate(vcpu, test_vm_addr); + TEST_ASSERT_EQ(tr.valid, false); + + vm_set_memory_attributes(vm, paddr, vm->page_size, 0); +} + +static void vm_set_vapic_addr(struct kvm_vcpu *vcpu, uint64_t addr) +{ + struct kvm_vapic_addr va; + + va.vapic_addr =3D addr; + vcpu_ioctl(vcpu, KVM_SET_VAPIC_ADDR, &va); +} + +/* + * Perform a dummy regs update to issue a KVM_REQ_EVENT. This forces + * vapic to be synced before entering the guest. + */ +static void vcpu_force_vapic_update(struct kvm_vcpu *vcpu) +{ + struct kvm_regs regs; + + vcpu_regs_get(vcpu, ®s); + vcpu_regs_set(vcpu, ®s); +} + +/* + * Setup the vapic address on a GPA that is write-protected. Force an vapic + * update and validate its contents were not changes. Then, lift the write + * restriction and validate the page's contents are updated. + */ +static void test_side_channel_vapic_addr(struct kvm_vcpu *vcpu) +{ + struct kvm_vm *vm =3D vcpu->vm; + gva_t vaddr =3D vm_alloc_page(vm); + gpa_t paddr =3D addr_gva2gpa(vm, vaddr); + + vm_set_vapic_addr(vcpu, paddr); + test_data->op =3D TEST_OP_READ; + test_data->vaddr =3D vaddr; + test_data->confirm_read =3D 1; + test_data->expected_val =3D ~0ULL >> 32; + memset(addr_gva2hva(vm, vaddr), 0xff, sizeof(uint32_t)); + vm_set_memory_attributes(vm, paddr, vm->page_size, KVM_MEMORY_ATTRIBUTE_N= W); + vcpu_run_and_inc_stage(vcpu); + + vm_set_memory_attributes(vm, paddr, vm->page_size, 0); + vcpu_force_vapic_update(vcpu); + test_data->expected_val =3D 0ULL; + vcpu_run_and_inc_stage(vcpu); + + vm_set_vapic_addr(vcpu, 0); + test_data->confirm_read =3D 0; +} + +static void *vcpu_worker(void *data) +{ + struct test_data *test_data =3D data; + struct kvm_vcpu *vcpu =3D test_data->vcpu; + + vcpu_run_and_inc_stage(vcpu); + + return NULL; +} + +/* + * Set up the pvclock page and validate that KVM periodically updates the + * 'version' field. Subsequently, make the pvclock page non-writable and + * verify that the 'version' field is no longer updated. + */ +static void test_side_channel_pvclock(struct kvm_vcpu *vcpu) +{ + struct kvm_vm *vm =3D vcpu->vm; + gva_t vaddr =3D vm_alloc_page(vm); + gpa_t paddr =3D addr_gva2gpa(vm, vaddr); + struct pvclock_vcpu_time_info *pvclock =3D addr_gpa2hva(vm, paddr); + + pvclock->version =3D 0; + + test_data->op =3D TEST_OP_MONITOR_ADDRESS; + test_data->vaddr =3D vaddr + offsetof(struct pvclock_vcpu_time_info, vers= ion); + test_data->expected_val =3D pvclock->version; + + pthread_create(&vcpu_thread, NULL, vcpu_worker, test_data); + vcpu_set_msr(vcpu, MSR_KVM_SYSTEM_TIME_NEW, paddr | 0x1); + usleep(msecs_to_usecs(1000)); + TEST_ASSERT_EQ(pthread_tryjoin_np(vcpu_thread, NULL), 0); + vcpu_set_msr(vcpu, MSR_KVM_SYSTEM_TIME_NEW, 0); + + test_data->expected_val =3D pvclock->version; + vm_set_memory_attributes(vm, paddr, vm->page_size, KVM_MEMORY_ATTRIBUTE_N= W); + pthread_create(&vcpu_thread, NULL, vcpu_worker, test_data); + vcpu_set_msr(vcpu, MSR_KVM_SYSTEM_TIME_NEW, paddr | 0x1); + usleep(msecs_to_usecs(1000)); + TEST_ASSERT_EQ(pthread_tryjoin_np(vcpu_thread, NULL), EBUSY); + + /* Force the 'monitor_address' guest operation to finish */ + test_data->op =3D TEST_OP_NOP; + TEST_ASSERT_EQ(pthread_join(vcpu_thread, NULL), 0); + vm_set_memory_attributes(vm, paddr, vm->page_size, 0); + vcpu_set_msr(vcpu, MSR_KVM_SYSTEM_TIME_NEW, 0); +} + +/* + * Write to MSR_KVM_WALL_CLOCK_NEW and verify that the struct's 'version' = field + * is updated. Subsequently, make the target guest physical address + * non-writable, and verify the 'version' field isn't updated anymore. + */ +static void test_side_channel_wallclock(struct kvm_vcpu *vcpu) +{ + struct kvm_vm *vm =3D vcpu->vm; + gva_t vaddr =3D vm_alloc_page(vm); + gpa_t paddr =3D addr_gva2gpa(vm, vaddr); + struct pvclock_wall_clock *wc =3D addr_gva2hva(vm, vaddr); + + wc->version =3D 0x0; + vcpu_set_msr(vcpu, MSR_KVM_WALL_CLOCK_NEW, paddr); + TEST_ASSERT_EQ(READ_ONCE(wc->version), 2); + + vm_set_memory_attributes(vm, paddr, vm->page_size, KVM_MEMORY_ATTRIBUTE_N= W); + vcpu_set_msr(vcpu, MSR_KVM_WALL_CLOCK_NEW, paddr); + TEST_ASSERT_EQ(READ_ONCE(wc->version), 2); + + vm_set_memory_attributes(vm, paddr, vm->page_size, 0); +} + +/* + * Memory attributes are vulnerable to side-channel attacks. This means th= at + * any KVM operation initiated by the guest that requires guest memory acc= ess + * (which is the case for most pv-interfaces) needs to consider memory + * attributes. + * + * The following tests validate that this requirement is upheld for a vari= ety + * of use-cases. These test cases were selected to exercise specific appro= aches + * to accessing guest memory, including: + * + * - kvm_read/write_guest() + * - gfn_to_hva_cache + * - gfn_to_pfn_cache + * - Guest page walker + */ +static void arch_test_side_channels(struct kvm_vcpu *vcpu, gva_t test_vm_a= ddr, + size_t size) +{ + test_side_channel_hyperv_hypercall_inputs(vcpu, test_vm_addr, size); + test_side_channel_emul_page_walks(vcpu, test_vm_addr, size); + test_side_channel_vapic_addr(vcpu); + test_side_channel_wallclock(vcpu); + test_side_channel_pvclock(vcpu); +} --=20 2.52.0