From nobody Fri Dec 19 20:55:12 2025 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F19072080DC for ; Wed, 16 Apr 2025 15:26:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817217; cv=none; b=u9KiE69o+RxEGXHG9g9GZV6FSKv2f/4RFdJJmYIMknjD8YIfQxCztlgjnMkR25AcvLfpenkzVJc3XYL14xcSu3ev5wdcWGlVXlRHU2pIhgBwH8Iish0JhMoWBTvmqFymKFJvu9GnnJoT47pkmh7HzMvoKR3ZYZwcGJ4IK7q/zls= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817217; c=relaxed/simple; bh=+tI4PEfGQfafoe0FcQqYYwv3iwkEbI6CYxC3OLOKiy4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sZlmVvxk0WlXWTZwFpm/zpphLsyAuE32bs80TaLfzhgJwAjLXi/dlFRa+Te1Ucp5W8ERSQUIKZ89YpUWyWgdNNxnviAuNa/cLgsT5Lp+NM95pOI2ubFM0KA5WpPkG2z1yBR8+sOEVtZ2sKltLRzt21gOoCRWnqq8SefKLPKIG4Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=35vYyYP5; arc=none smtp.client-ip=209.85.208.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="35vYyYP5" Received: by mail-ed1-f73.google.com with SMTP id 4fb4d7f45d1cf-5e5d9682f53so5918890a12.2 for ; Wed, 16 Apr 2025 08:26:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744817213; x=1745422013; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=uzD2f0rxQ0+kK+W36kLcNjO+gtstze2fsnlGYTmtJfM=; b=35vYyYP5L6+sw3b90ckhDCjI6k+3M2oTHD60jf5nVTL4GVAiLv1VGrXBkKVYFAtQ0p xEb4hLrUJ7OrYvQnLPUDdBm1JFmJHYinbJYGOLwEpfk6sZwcYN2NahrjYj1/p+/OUvqd bxHxNZ8li9q/SrhFLJwcPIKq40fNheD+mjakPgFHevoPDbZI269034wQMSXO/U0jiuB+ H0SQpgpOEO7hcVofzHm9suMlnuew6otZv0hYAqWmGcY+cAh5xkgo1vqPihVaBV+HqGZl HI5KyM9TVt5nmoxskWvmPdzHfuK8MRJHwIpSO1BtbV9A3wpSSa4Z3iSjXu7u4CIX/fNT yAaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744817213; x=1745422013; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=uzD2f0rxQ0+kK+W36kLcNjO+gtstze2fsnlGYTmtJfM=; b=k07qRKhju12jZj6HcU0Tjh9S4ZNYFLeDfruMyVF221NU9uha1+mTdKBjchwYBrMXYW 8DDwUIyqQqH9tgTSZKO172mrjDqmWkV1i/B+4NKiLkqHrQzZ/9cpT0eMGOPd5W7q15w/ MKgBNrtbEqjF+Xyd3AAoo0YQpkXUC/bUVgOlIAKngOnnygEWwW2L/WN2OEcCQp+vqBTd HoV68sB+v5hD1c7CDn+TV89mGj2kN6PGwfcPmV9VlPXRy4E+R9k/qdg1zfxwXgp+9YYO Kd2CRXXypaoY+zZG9BygfJNbJWfVVUSspOOR76TUiz27pym2dNH+9rvO8kEo5JgS74TD B/8A== X-Forwarded-Encrypted: i=1; AJvYcCViOrnc22u6S8mWEFVvoshNx8hwkWptxXnCZ+mlUQJhzB8Nw15BvJJqYe0qjaxJndKkmgDQPaGJI65lvRY=@vger.kernel.org X-Gm-Message-State: AOJu0Yyli3tsM8lrLvnvutlIKOa66Wu3YXf1OtuFw3FM51jPcKyKKoA5 MgA5Nt5eZ3UGlFHRlKu1mHWuQAL2FSLqO1swIDzhX5/dsMyNA/chuMhoJCaNOF2GzILoriI9f5Z nbPcD8w== X-Google-Smtp-Source: AGHT+IHeDz4onV1W2s/vfCOCzAITZ8Cgy1MgbHMQ8Bx0jNC7ijt2U2LknXq2pGOSmZDw3fZM7GLT6soOihnk X-Received: from edsu15.prod.google.com ([2002:aa7:d98f:0:b0:5f4:ade4:88fd]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:35c2:b0:5e5:bbd5:676a with SMTP id 4fb4d7f45d1cf-5f4b75b2ce3mr1715528a12.22.1744817213096; Wed, 16 Apr 2025 08:26:53 -0700 (PDT) Date: Wed, 16 Apr 2025 15:26:41 +0000 In-Reply-To: <20250416152648.2982950-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250416152648.2982950-1-qperret@google.com> X-Mailer: git-send-email 2.49.0.604.gff1f9ca942-goog Message-ID: <20250416152648.2982950-2-qperret@google.com> Subject: [PATCH v2 1/7] KVM: arm64: Track SVE state in the hypervisor vcpu structure From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Fuad Tabba , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Fuad Tabba When dealing with a guest with SVE enabled, make sure the host SVE state is pinned at EL2 S1, and that the hypervisor vCPU state is correctly initialised (and then unpinned on teardown). Co-authored-by: Marc Zyngier Signed-off-by: Fuad Tabba Signed-off-by: Marc Zyngier Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_host.h | 12 ++++---- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 4 --- arch/arm64/kvm/hyp/nvhe/pkvm.c | 47 ++++++++++++++++++++++++++++-- 3 files changed, 51 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index e98cfe7855a6..931011b68819 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -971,20 +971,22 @@ struct kvm_vcpu_arch { #define vcpu_sve_zcr_elx(vcpu) \ (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) =20 -#define vcpu_sve_state_size(vcpu) ({ \ +#define sve_state_size_from_vl(sve_max_vl) ({ \ size_t __size_ret; \ - unsigned int __vcpu_vq; \ + unsigned int __vq; \ \ - if (WARN_ON(!sve_vl_valid((vcpu)->arch.sve_max_vl))) { \ + if (WARN_ON(!sve_vl_valid(sve_max_vl))) { \ __size_ret =3D 0; \ } else { \ - __vcpu_vq =3D vcpu_sve_max_vq(vcpu); \ - __size_ret =3D SVE_SIG_REGS_SIZE(__vcpu_vq); \ + __vq =3D sve_vq_from_vl(sve_max_vl); \ + __size_ret =3D SVE_SIG_REGS_SIZE(__vq); \ } \ \ __size_ret; \ }) =20 +#define vcpu_sve_state_size(vcpu) sve_state_size_from_vl((vcpu)->arch.sve_= max_vl) + #define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE | \ KVM_GUESTDBG_USE_SW_BP | \ KVM_GUESTDBG_USE_HW | \ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/h= yp-main.c index 2c37680d954c..59db9606e6e1 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -123,10 +123,6 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_v= cpu) =20 hyp_vcpu->vcpu.arch.ctxt =3D host_vcpu->arch.ctxt; =20 - hyp_vcpu->vcpu.arch.sve_state =3D kern_hyp_va(host_vcpu->arch.sve_state); - /* Limit guest vector length to the maximum supported by the host. */ - hyp_vcpu->vcpu.arch.sve_max_vl =3D min(host_vcpu->arch.sve_max_vl, kvm_ho= st_sve_max_vl); - hyp_vcpu->vcpu.arch.mdcr_el2 =3D host_vcpu->arch.mdcr_el2; hyp_vcpu->vcpu.arch.hcr_el2 &=3D ~(HCR_TWI | HCR_TWE); hyp_vcpu->vcpu.arch.hcr_el2 |=3D READ_ONCE(host_vcpu->arch.hcr_el2) & diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 5a335a51deca..338505cb0171 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -372,6 +372,18 @@ static void unpin_host_vcpu(struct kvm_vcpu *host_vcpu) hyp_unpin_shared_mem(host_vcpu, host_vcpu + 1); } =20 +static void unpin_host_sve_state(struct pkvm_hyp_vcpu *hyp_vcpu) +{ + void *sve_state; + + if (!vcpu_has_feature(&hyp_vcpu->vcpu, KVM_ARM_VCPU_SVE)) + return; + + sve_state =3D kern_hyp_va(hyp_vcpu->vcpu.arch.sve_state); + hyp_unpin_shared_mem(sve_state, + sve_state + vcpu_sve_state_size(&hyp_vcpu->vcpu)); +} + static void unpin_host_vcpus(struct pkvm_hyp_vcpu *hyp_vcpus[], unsigned int nr_vcpus) { @@ -384,6 +396,7 @@ static void unpin_host_vcpus(struct pkvm_hyp_vcpu *hyp_= vcpus[], continue; =20 unpin_host_vcpu(hyp_vcpu->host_vcpu); + unpin_host_sve_state(hyp_vcpu); } } =20 @@ -398,12 +411,40 @@ static void init_pkvm_hyp_vm(struct kvm *host_kvm, st= ruct pkvm_hyp_vm *hyp_vm, pkvm_init_features_from_host(hyp_vm, host_kvm); } =20 -static void pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp_vcpu, struct kvm_= vcpu *host_vcpu) +static int pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp_vcpu, struct kvm_v= cpu *host_vcpu) { struct kvm_vcpu *vcpu =3D &hyp_vcpu->vcpu; + unsigned int sve_max_vl; + size_t sve_state_size; + void *sve_state; + int ret =3D 0; =20 - if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) + if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) { vcpu_clear_flag(vcpu, VCPU_SVE_FINALIZED); + return 0; + } + + /* Limit guest vector length to the maximum supported by the host. */ + sve_max_vl =3D min(READ_ONCE(host_vcpu->arch.sve_max_vl), kvm_host_sve_ma= x_vl); + sve_state_size =3D sve_state_size_from_vl(sve_max_vl); + sve_state =3D kern_hyp_va(READ_ONCE(host_vcpu->arch.sve_state)); + + if (!sve_state || !sve_state_size) { + ret =3D -EINVAL; + goto err; + } + + ret =3D hyp_pin_shared_mem(sve_state, sve_state + sve_state_size); + if (ret) + goto err; + + vcpu->arch.sve_state =3D sve_state; + vcpu->arch.sve_max_vl =3D sve_max_vl; + + return 0; +err: + clear_bit(KVM_ARM_VCPU_SVE, vcpu->kvm->arch.vcpu_features); + return ret; } =20 static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu, @@ -432,7 +473,7 @@ static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp= _vcpu, if (ret) goto done; =20 - pkvm_vcpu_init_sve(hyp_vcpu, host_vcpu); + ret =3D pkvm_vcpu_init_sve(hyp_vcpu, host_vcpu); done: if (ret) unpin_host_vcpu(host_vcpu); --=20 2.49.0.604.gff1f9ca942-goog From nobody Fri Dec 19 20:55:12 2025 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BF052080DB for ; Wed, 16 Apr 2025 15:26:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817218; cv=none; b=rzDIDm/c/dcm0a8nFOYwhBlYy7wnJr1qY/pF7OpXq4DO1FalFrrmPY2nf5l2eMZ8Ly/N62N6EJeCWwRErbz2nfvcOKrPy1uAGYob2zV4prXHcAHLxqzn6EnmvZB+WR+bmV1X+rIfn7PC35aNkltupUL/qWVn5LqW41HfAuDXjDg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817218; c=relaxed/simple; bh=lw3AVtJbpuAwOe11LdCap/7mFJF4YAPAEl8JSKKuzOs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IyG/q5zta5SOc2cnYhAfmpGpIxNaIR4/MS4e1DRilTko8yIwCsa8Ff/TK2MgaOQkV04xSAfcCLhr7IwN1ADWuR88St/HH5G3UqVlOidnlnkgWXmCQxvYMLY6MzccLQrdj1JIDltgm3qe0TFim0uFKgbHOR2+AJcyFJPa93FuqUc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=hjsXrviy; arc=none smtp.client-ip=209.85.208.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hjsXrviy" Received: by mail-ed1-f73.google.com with SMTP id 4fb4d7f45d1cf-5e5c5ea184dso6420853a12.0 for ; Wed, 16 Apr 2025 08:26:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744817215; x=1745422015; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ySUALm4kAiEtFHHbP93/9gqyCVQ7SC9yKGUHGHBoiHU=; b=hjsXrviyhWBG+ylfCl1U5eKuoBg2ZsHTISJiytZKGG8YPrn9g4KPUOYC6B6t7Cg6BI B05cGtiAuKn9c0jzkNTOdj2CGHS1yHf+lyP7dwZU38z7Ez0D1BtroQl046flSr/v7ZZ2 ejPH2G0UzuP+1pYdUsjphUR1GTI5kv2ZAzCzgbIx0S6hkDBelErfxBu/FUJTCiwcOLNU GRFoqIDRDdUDnsNgJnoQkfgAyML2MjsYCFyG9Rhq9i5SWdKXPZ1ms2yhhcQioGsbd0gR qltWyuMFR5HJUBekmtJ8sfhwIJxoo3vwVaOq8isAefsa/sVrlACwhFGCvmxRuRSSIHeT mO6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744817215; x=1745422015; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ySUALm4kAiEtFHHbP93/9gqyCVQ7SC9yKGUHGHBoiHU=; b=Suk55neh9Hu1GmdJq4L6a+XyeIjtI/B7WCMM4XfXYFR8CRU1Gs9DMOqwTCpq2rmfuF 0aaXzsFqlqTfTVKEj6WLvIYvR22mGnQ8D5ivdqwHuwfHhBkzUo8T3MSCU83fqeYDtd9n 1Fo9/1cngfFudwzMocKNx1kKQBbcDME2ozacnwaZjRqJKXVtek2VfioFpfhF74XbbEVj 1WPy+JJxKlPBlK1w4oDqI2XfHeLDIYp2wvjBrDPjjRCIqZr8sxFuqZ/Kj01xluFhZ6gN L3V6zmPlqRmZ/YtKlUCfyOr8iAAVNMnZA3AXoNgvUwjGDB+z42aTmE8C0XMOqK6X9vH8 88Gg== X-Forwarded-Encrypted: i=1; AJvYcCXwY/WwlIwds01pE1o3GeLZEHw+StHdY9utGM9x03HzkNkogyJvYHqZ6CmCZRBihuhlnQG7/UiGDdq1VAA=@vger.kernel.org X-Gm-Message-State: AOJu0YyAy+Y5gzRAxqC9BYvwdSwO9CbFF7yy8cqgi1dgPafFR++e8iF1 WDEXD7mLunl5OgkMwY3unWX4xzIF7fWoSe/OlHOBOruVscLxhYW1FaaKFt9t0t9541YUYH2uUwO U+LqF1g== X-Google-Smtp-Source: AGHT+IFO6LqLQpchltIVKkcVMTkkcoAUlzOcNnFkY+QSrpAdIOkv/ajoQQVC0VjwY0/SmeO3qq2XvFKzCmMO X-Received: from edb17.prod.google.com ([2002:a05:6402:2391:b0:5f4:b71f:15a6]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:26c8:b0:5f4:c619:2392 with SMTP id 4fb4d7f45d1cf-5f4c6192660mr959365a12.1.1744817215420; Wed, 16 Apr 2025 08:26:55 -0700 (PDT) Date: Wed, 16 Apr 2025 15:26:42 +0000 In-Reply-To: <20250416152648.2982950-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250416152648.2982950-1-qperret@google.com> X-Mailer: git-send-email 2.49.0.604.gff1f9ca942-goog Message-ID: <20250416152648.2982950-3-qperret@google.com> Subject: [PATCH v2 2/7] KVM: arm64: Fix pKVM page-tracking comments From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Fuad Tabba , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Most of the comments relating to pKVM page-tracking in nvhe/memory.h are now either slightly outdated or outright wrong. Fix the comments. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/memory.h | 29 ++++++++++++++++-------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/memory.h b/arch/arm64/kvm/hyp/= include/nvhe/memory.h index 34233d586060..178028172c0b 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/memory.h +++ b/arch/arm64/kvm/hyp/include/nvhe/memory.h @@ -8,20 +8,29 @@ #include =20 /* - * Bits 0-1 are reserved to track the memory ownership state of each page: - * 00: The page is owned exclusively by the page-table owner. - * 01: The page is owned by the page-table owner, but is shared - * with another entity. - * 10: The page is shared with, but not owned by the page-table owner. - * 11: Reserved for future use (lending). + * Bits 0-1 are used to encode the memory ownership state of each page fro= m the + * point of view of a pKVM "component" (host, hyp, guest, ... see enum + * pkvm_component_id): + * 00: The page is owned and exclusively accessible by the component; + * 01: The page is owned and accessible by the component, but is also + * accessible by another component; + * 10: The page is accessible but not owned by the component; + * The storage of this state depends on the component: either in the + * hyp_vmemmap for the host state or in PTE software bits for the hypervis= or + * and guests. */ enum pkvm_page_state { PKVM_PAGE_OWNED =3D 0ULL, PKVM_PAGE_SHARED_OWNED =3D BIT(0), PKVM_PAGE_SHARED_BORROWED =3D BIT(1), - __PKVM_PAGE_RESERVED =3D BIT(0) | BIT(1), - - /* Meta-states which aren't encoded directly in the PTE's SW bits */ + __PKVM_PAGE_RESERVED =3D BIT(0) | BIT(1), + + /* + * 'Meta-states' are not stored directly in PTE SW bits for hyp and + * guest states, but inferred from the context (e.g. invalid PTE + * entries). For the host, meta-states are stored directly in the + * struct hyp_page. + */ PKVM_NOPAGE =3D BIT(2), }; #define PKVM_PAGE_META_STATES_MASK (~__PKVM_PAGE_RESERVED) @@ -44,7 +53,7 @@ struct hyp_page { u16 refcount; u8 order; =20 - /* Host (non-meta) state. Guarded by the host stage-2 lock. */ + /* Host state. Guarded by the host stage-2 lock. */ enum pkvm_page_state host_state : 8; =20 u32 host_share_guest_count; --=20 2.49.0.604.gff1f9ca942-goog From nobody Fri Dec 19 20:55:12 2025 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3AA522080F6 for ; Wed, 16 Apr 2025 15:26:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817221; cv=none; b=LUuNSsBM9a4B7tk0382cavK9Cmwr1k3/DV7tCYlNpwbB1AIw3n1k3Lz46sXBe8OvEi/+sDYdcaOW+ffR1J9of1DjQ/JVvi3TXrJ+YD5OLyn/nel755mpyRean9Y6RxNgwSfIk+HAHgPWmDLMVjOh2Y5k5QO2YgdJrJzn+2p7PC0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817221; c=relaxed/simple; bh=2VwDODIxhUffdZwjAx4r5VNASzG0d9BWhmRZ0Me6Fuo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=afwVsGrWbezC6gamcSOYMbD/cwBM3oMUXprOPENYONqqEBzwPKM1AU+J9ktlcJbZdKeV3+zuBbd0mknslGmRboKyW5vDK9+0DDmBo8Wrl0/BFz2KT99Z6tStKUiGUuZuAy9NlxyvXrUbJbSECA7k9ywINzo3TiYaLyfrvSYxcAI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=nX/t+EKq; arc=none smtp.client-ip=209.85.208.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="nX/t+EKq" Received: by mail-ed1-f74.google.com with SMTP id 4fb4d7f45d1cf-5e5d9682f53so5918925a12.2 for ; Wed, 16 Apr 2025 08:26:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744817217; x=1745422017; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=sWpGHRaShaKEt1EwgADnEDZtkiEBTsCp5JfVvOy2vqs=; b=nX/t+EKqf2wj6L+Pj5N3d48H2QZ4u6/HqG22XQvu/lDqGG+0RaKgkRMheHc74eymzF gwx7Td5vi7emtLeyzebo9Qvm5PVzUq12BgphvcDPKRI9E1w2YyM8+uKkv9fmu3LF4K8B sktMut8S38g/JA9nto158n/CbjhFm6cc6g92A2xmHYV5uQS1nLCmnlOtRxob6q3MiPES zU/VsEuWyv/ESBpNSbbcBh+Vzldl3I+gXYh2NJfg9B0/7jXwY/IlAaJhZMEIUyCkkPll RCKsvmeDrwnPbpvFmVlvPEYnAJ2Fx/nQOulWQewXL4bsk3xjd5j/5WTB2tdz5QYMmdR5 qI3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744817217; x=1745422017; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sWpGHRaShaKEt1EwgADnEDZtkiEBTsCp5JfVvOy2vqs=; b=cdjz4Mm3cBEeduYABH2qCvBYEj/axZWnTqpYY3IDruiFONy24/srZoKcDrsQxJtc4q 1mbSktb5cPf7PnyBrVSBVyorwuIG+pULNep4/8O96z74ilbfB+pnmfVjDdY+Kz/tEm+J epvY9E0hZpKySiEjgw1QugdQohID4ECkOmMRFd9d5dEY+Imu+g5JN7j3CU3a1MZGzCoJ RJSbtBUAQUtbDPOTzP+suryhR83i1qV/eDJlJ3VCsMvT0dqAxCazIIurtHccp2ExQ7xY RVNMdIiBcyxz+eqpWlnITn0FwdzqN6SWnU0j6cfL1e+vb1znQIzFbxkNxbL4E+Tx9ZVu KPdA== X-Forwarded-Encrypted: i=1; AJvYcCUWMlJg5F0o8HFOxYER1SbprsU52Bf5ESt9YCT9pFOCuKuL4qqGrlYGlhX3tMj0EtQXsaS9pYenm0PAJoM=@vger.kernel.org X-Gm-Message-State: AOJu0Yw30Sf+bBU7NN0xcLfXK442n/UtEQeVCCGxIYcdpYACagCPjI5c X4f7l5FW77nH9xvmlEJ2Y3C+irNVZxoUkS9SV4TrPY0/sd6iqDOC9FyMoYqRb79HGZmykVDck98 1kwm5nA== X-Google-Smtp-Source: AGHT+IEoG2AXXZ1tWBUhjh5KrFg0jiFxDBUl8QZtd9ikeeeP7z6vejMB73nvwGYxLNaTouC7ucA8ZutxaU7j X-Received: from ediq14.prod.google.com ([2002:a50:cc8e:0:b0:5e6:e69f:4a7f]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:34cb:b0:5e5:3643:c8b5 with SMTP id 4fb4d7f45d1cf-5f4b75c8c33mr1934847a12.30.1744817217576; Wed, 16 Apr 2025 08:26:57 -0700 (PDT) Date: Wed, 16 Apr 2025 15:26:43 +0000 In-Reply-To: <20250416152648.2982950-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250416152648.2982950-1-qperret@google.com> X-Mailer: git-send-email 2.49.0.604.gff1f9ca942-goog Message-ID: <20250416152648.2982950-4-qperret@google.com> Subject: [PATCH v2 3/7] KVM: arm64: Use 0b11 for encoding PKVM_NOPAGE From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Fuad Tabba , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The page ownership state encoded as 0b11 is currently considered reserved for future use, and PKVM_NOPAGE uses bit 2. In order to simplify the relocation of the hyp ownership state into the vmemmap in later patches, let's use the 'reserved' encoding for the PKVM_NOPAGE state. The struct hyp_page layout isn't guaranteed stable at all, so there is no real reason to have 'reserved' encodings. No functional changes intended. Reviewed-by: Marc Zyngier Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/memory.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/memory.h b/arch/arm64/kvm/hyp/= include/nvhe/memory.h index 178028172c0b..bf28f9f9de65 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/memory.h +++ b/arch/arm64/kvm/hyp/include/nvhe/memory.h @@ -23,7 +23,6 @@ enum pkvm_page_state { PKVM_PAGE_OWNED =3D 0ULL, PKVM_PAGE_SHARED_OWNED =3D BIT(0), PKVM_PAGE_SHARED_BORROWED =3D BIT(1), - __PKVM_PAGE_RESERVED =3D BIT(0) | BIT(1), =20 /* * 'Meta-states' are not stored directly in PTE SW bits for hyp and @@ -31,9 +30,8 @@ enum pkvm_page_state { * entries). For the host, meta-states are stored directly in the * struct hyp_page. */ - PKVM_NOPAGE =3D BIT(2), + PKVM_NOPAGE =3D BIT(0) | BIT(1), }; -#define PKVM_PAGE_META_STATES_MASK (~__PKVM_PAGE_RESERVED) =20 #define PKVM_PAGE_STATE_PROT_MASK (KVM_PGTABLE_PROT_SW0 | KVM_PGTABLE_PROT= _SW1) static inline enum kvm_pgtable_prot pkvm_mkstate(enum kvm_pgtable_prot pro= t, --=20 2.49.0.604.gff1f9ca942-goog From nobody Fri Dec 19 20:55:12 2025 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 997092080FF for ; Wed, 16 Apr 2025 15:27:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817223; cv=none; b=JeiBwgIUntpv3EkXW3i2UHbtwhGvhBCdVB3vhrdRLKHNiU+wxLh2dpUsaip8tS64IjbyFBcROyWzCoCSJjL09oZFAdYKRdz8dWfAhGlF0rv+Q3X3DB7PYWa3r9OYBNM7ut7oVd7rjGvwEyUXVGVGqv1WvssPWzIihrgHhqrzaiI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817223; c=relaxed/simple; bh=J9Yf6lxkoaWGjJfgEDe4MZh/KA3qtVyYK53UWzW0zEg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=n0gJYXtOc9FNmjMqVeVCHohXe36E4TBpEdI+ui13SSqBRQyUfpXQtNLO+WpEdCZiCY6PJRhS+98kosMq+QrzycsNnqKN+OZajagoITA0QOg9aMuAOAWtLBi3SQGwYdWHRfQOskIItL2WVsZ7YCqF0vFDToZYlLCf+Ig9ydT3MSY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=MjM/TY9A; arc=none smtp.client-ip=209.85.218.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MjM/TY9A" Received: by mail-ej1-f73.google.com with SMTP id a640c23a62f3a-ac2bb3ac7edso738376666b.2 for ; Wed, 16 Apr 2025 08:27:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744817220; x=1745422020; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=UXTqFvc4opPwLmL9HAQrXX66kUt2kBii3Ie6HubIYaY=; b=MjM/TY9ACHQE1ZmMKF5ofuuwpOTGhcn5KSFYjm9kEAY7knvQZ1QVG3JZW2lOLbQq+p TeSW8iCQbdfZoHziAX9XtwabfuoBjuTXXz0/33+qP2dj5LuOyuTqCoOCLYoEhHhmA17S ZEp8mvzepWmeCiXLjcRqu2B/880V7prvYArQfZUN0QBvJOWTne3Pvbqwe9jQPOPbQ2X3 21qmR3UL1Q2NnEn19qypIlSm9Uv6ttMl0Rpdnrk2VC7zWP0IFL+jY4ggUBbo2HwW2b+y dmcSv6UmgoKfCknCQ2WgneTZ2HC+cfuMQV4X3K7CjNgxSb5LJ2kalpsTHd1o9Llmwluq rQSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744817220; x=1745422020; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UXTqFvc4opPwLmL9HAQrXX66kUt2kBii3Ie6HubIYaY=; b=E5TSoR/z3m5GFzm5hOb7hwKs6n1S8Fcd9E+ry4SQpiUTuqzY06rb+L5mdPtVPfylC+ 3LY88BVD8V+IPQDDKWRKyY5NZ/j8AaP88Aw4LKKKsx6KW+cDyiIVfqMObhKppWFNB34Q tGQMQGXcJBrOBCMoS/hd70/SLHyFwa+v3kyoTgPUBUA6FqCKrUQ3XIARFj5fBmx2j3LD 9jORDb35sTJlhoXW7EegJ30CnVweMih0EzWv4kieqOUkbSNunF5iph9dWQfdpjml8Vz5 hY8kWXXxUzEBEjnytyZZdhigShAsl9W6Xqn0+zOR8aH+QzngVlwlSYEhjxomb3SoxgKL 4m0w== X-Forwarded-Encrypted: i=1; AJvYcCXbjinVh+rPTJNgIngVkuqkjYXHMclPAoorz3xSCSAlhrdT6IBsrLXlwzDtvlB8NoxiCzzHm92Ctzf4+Ls=@vger.kernel.org X-Gm-Message-State: AOJu0YyxYnn37j9xKYmQShwR05ohBLHRPJHYQ9nv5TXXMYLYE/nQ6dib GbizdSebrFglutT3bf7zPr7+DbD6Q0yT4Yejn+OjSDMSLKruqO66vIj/Z0S7a9xx+Kc2jaO34W6 LUdc3Jw== X-Google-Smtp-Source: AGHT+IEu+nQt3ZvaltJ8ile4BL4Mx8RMjIE5u1QNIcuD+qDhprPfJ5PQQJQVdc2cu/l2DfDtzKgEXuEQXODM X-Received: from edod3.prod.google.com ([2002:a50:ea83:0:b0:5ec:289d:754b]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:9408:b0:abf:6ec7:65e9 with SMTP id a640c23a62f3a-acb42b6cbddmr211529666b.43.1744817220022; Wed, 16 Apr 2025 08:27:00 -0700 (PDT) Date: Wed, 16 Apr 2025 15:26:44 +0000 In-Reply-To: <20250416152648.2982950-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250416152648.2982950-1-qperret@google.com> X-Mailer: git-send-email 2.49.0.604.gff1f9ca942-goog Message-ID: <20250416152648.2982950-5-qperret@google.com> Subject: [PATCH v2 4/7] KVM: arm64: Introduce {get,set}_host_state() helpers From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Fuad Tabba , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Instead of directly accessing the host_state member in struct hyp_page, introduce static inline accessors to do it. The future hyp_state member will follow the same pattern as it will need some logic in the accessors. Reviewed-by: Marc Zyngier Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/memory.h | 12 +++++++++++- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 14 +++++++------- arch/arm64/kvm/hyp/nvhe/setup.c | 4 ++-- 3 files changed, 20 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/memory.h b/arch/arm64/kvm/hyp/= include/nvhe/memory.h index bf28f9f9de65..a1754aecf8f8 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/memory.h +++ b/arch/arm64/kvm/hyp/include/nvhe/memory.h @@ -52,7 +52,7 @@ struct hyp_page { u8 order; =20 /* Host state. Guarded by the host stage-2 lock. */ - enum pkvm_page_state host_state : 8; + unsigned __host_state : 8; =20 u32 host_share_guest_count; }; @@ -89,6 +89,16 @@ static inline struct hyp_page *hyp_phys_to_page(phys_add= r_t phys) #define hyp_page_to_virt(page) __hyp_va(hyp_page_to_phys(page)) #define hyp_page_to_pool(page) (((struct hyp_page *)page)->pool) =20 +static inline enum pkvm_page_state get_host_state(phys_addr_t phys) +{ + return (enum pkvm_page_state)hyp_phys_to_page(phys)->__host_state; +} + +static inline void set_host_state(phys_addr_t phys, enum pkvm_page_state s= tate) +{ + hyp_phys_to_page(phys)->__host_state =3D state; +} + /* * Refcounting for 'struct hyp_page'. * hyp_pool::lock must be held if atomic access to the refcount is require= d. diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index 2a5284f749b4..e9060e6205cb 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -467,7 +467,7 @@ static int host_stage2_adjust_range(u64 addr, struct kv= m_mem_range *range) return -EAGAIN; =20 if (pte) { - WARN_ON(addr_is_memory(addr) && hyp_phys_to_page(addr)->host_state !=3D = PKVM_NOPAGE); + WARN_ON(addr_is_memory(addr) && get_host_state(addr) !=3D PKVM_NOPAGE); return -EPERM; } =20 @@ -496,7 +496,7 @@ static void __host_update_page_state(phys_addr_t addr, = u64 size, enum pkvm_page_ phys_addr_t end =3D addr + size; =20 for (; addr < end; addr +=3D PAGE_SIZE) - hyp_phys_to_page(addr)->host_state =3D state; + set_host_state(addr, state); } =20 int host_stage2_set_owner_locked(phys_addr_t addr, u64 size, u8 owner_id) @@ -627,7 +627,7 @@ static int __host_check_page_state_range(u64 addr, u64 = size, =20 hyp_assert_lock_held(&host_mmu.lock); for (; addr < end; addr +=3D PAGE_SIZE) { - if (hyp_phys_to_page(addr)->host_state !=3D state) + if (get_host_state(addr) !=3D state) return -EPERM; } =20 @@ -637,7 +637,7 @@ static int __host_check_page_state_range(u64 addr, u64 = size, static int __host_set_page_state_range(u64 addr, u64 size, enum pkvm_page_state state) { - if (hyp_phys_to_page(addr)->host_state =3D=3D PKVM_NOPAGE) { + if (get_host_state(addr) =3D=3D PKVM_NOPAGE) { int ret =3D host_stage2_idmap_locked(addr, size, PKVM_HOST_MEM_PROT); =20 if (ret) @@ -911,7 +911,7 @@ int __pkvm_host_share_guest(u64 pfn, u64 gfn, struct pk= vm_hyp_vcpu *vcpu, goto unlock; =20 page =3D hyp_phys_to_page(phys); - switch (page->host_state) { + switch (get_host_state(phys)) { case PKVM_PAGE_OWNED: WARN_ON(__host_set_page_state_range(phys, PAGE_SIZE, PKVM_PAGE_SHARED_OW= NED)); break; @@ -964,9 +964,9 @@ static int __check_host_shared_guest(struct pkvm_hyp_vm= *vm, u64 *__phys, u64 ip if (WARN_ON(ret)) return ret; =20 - page =3D hyp_phys_to_page(phys); - if (page->host_state !=3D PKVM_PAGE_SHARED_OWNED) + if (get_host_state(phys) !=3D PKVM_PAGE_SHARED_OWNED) return -EPERM; + page =3D hyp_phys_to_page(phys); if (WARN_ON(!page->host_share_guest_count)) return -EINVAL; =20 diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setu= p.c index d62bcb5634a2..1a414288fe8c 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -201,10 +201,10 @@ static int fix_host_ownership_walker(const struct kvm= _pgtable_visit_ctx *ctx, case PKVM_PAGE_OWNED: return host_stage2_set_owner_locked(phys, PAGE_SIZE, PKVM_ID_HYP); case PKVM_PAGE_SHARED_OWNED: - hyp_phys_to_page(phys)->host_state =3D PKVM_PAGE_SHARED_BORROWED; + set_host_state(phys, PKVM_PAGE_SHARED_BORROWED); break; case PKVM_PAGE_SHARED_BORROWED: - hyp_phys_to_page(phys)->host_state =3D PKVM_PAGE_SHARED_OWNED; + set_host_state(phys, PKVM_PAGE_SHARED_OWNED); break; default: return -EINVAL; --=20 2.49.0.604.gff1f9ca942-goog From nobody Fri Dec 19 20:55:12 2025 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF5EB211489 for ; Wed, 16 Apr 2025 15:27:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817225; cv=none; b=kMrQf5ZAIGvTbdvjhRSKiHn65ox93vYKD5YmWbgDH9kn7woC5VxSNaUkjU8lNViAnnUmm32kEeLLi+wk4uj/4pHb9y8jjDDTDcqRCfdcgIf2ubB4fh29w+7qqte9TT0VUUXbT1sjauxI6w8ZF2cF2b+ZZ2vrzp9UqbvDqsRCe6Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817225; c=relaxed/simple; bh=ybG6Mz+ruCzz/7s7l63e7fUUrL5yxQEllagsjQ9Cf3w=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=pq11VKTAys2uAJWUOQ7AlnybnQcfSaeceqAUETuvLINMODMhhrs9EVCX8IFY0dh2t5u5soDZxvlC2xAOYaePGClBEi50uYvf6orxyJm8sW6yzCsRysbLqDKXG5YMU43LDEN6V/CM/6sAwpKyliOL+NCw6hUv19CyncEZNwhaenI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JGm+rdDG; arc=none smtp.client-ip=209.85.208.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JGm+rdDG" Received: by mail-ed1-f74.google.com with SMTP id 4fb4d7f45d1cf-5e636b06d34so7140294a12.1 for ; Wed, 16 Apr 2025 08:27:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744817222; x=1745422022; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+H4roSy0ETO2xvleEvsVdzxRkBxmddA29rKMSpiaoY8=; b=JGm+rdDGJDLnquYOsY01Qj7TrbLmuKnTrZnvSHWQnGaFZZTbvMh9zNDioK9vIrbPS3 4AtMwrih0PEoAkwIIsGclGBvd1XpRHYLWl70YMZLoDGIv9BaLm6ssraqdvQzcdXe54hS 60U8UN5XHLBr9bRqn/SqK6r326u8S4iVZJOSTnZVyzBgMcVoTNl8XkM6iDGMhzsqrzVx 1X3WH47pdbj9tLNJ47AvlwxjwRfyhbHxtdsy7y7PW29Aj4ATqx+DRrWegfwmMZ8Men07 b/EZEgwLpA2HXetkNDAgGJQNNPDQlbvDPVmgD9zHSWVV492SgJd4TAHOKi/ykd41tlWi 3jrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744817222; x=1745422022; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+H4roSy0ETO2xvleEvsVdzxRkBxmddA29rKMSpiaoY8=; b=EB7tYVUcrUR5g7nNJoupqWXWurTiczdTY3zpaqce7EolQ36f/nNiz2JMaSXvyGX4Wn YzAgTlSIVbdz/27rh2V1YcZc9v+eHuItmCMo0WsGHtWbs1dQZkT1kdjAs46sPpYJWrKT UnfJQweiBxCBl+uBkpBtyUxLMhfDvO995rvvjP2ZDJSTdHv86SIMTelawngX4Q/za3RT bMu3ZfojCayGpqtLaJpaIjsOmv8AXjzL62CRQbNWKuk2how4mf0x5HHKlZJSqZSoWIzJ 8Wt8DfYcbYRHEdOgg+6yJAtA/r5eNe+8bLSrls0uI1wv+v8H596A6BG8jfkZYUV377lZ pH/Q== X-Forwarded-Encrypted: i=1; AJvYcCVt3gLk+ffHV+vxQLQvWs5EOP2NUquJLQb/9ydE/lqiUuRPFY/RLnkwVci22PV+OrV8YsrsK9xtRYjCFCA=@vger.kernel.org X-Gm-Message-State: AOJu0Yy4iEDfPFNEGjb5zCx/wmYo+8PtYw0lCnhYE1XmI+Gpw925tSeN z+7oFDUNOdZYO6G72xeKpQYb8TwF7RovEX34+cDtIHoxFKHtxch0Mtk9fnQK7dClLJpIdqHdKqZ vv1fRbA== X-Google-Smtp-Source: AGHT+IE3AJ5+KNd/wrKE6Yh8WQPTsdymaFnGvYhcC6y9WjjslUVSUYFVwO0K57lR6F2n9q3uVegAXV72ChYn X-Received: from ediq14.prod.google.com ([2002:a50:cc8e:0:b0:5e5:cbc8:77b7]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:3486:b0:5e8:c092:7a6 with SMTP id 4fb4d7f45d1cf-5f4b75d848cmr2056666a12.21.1744817222246; Wed, 16 Apr 2025 08:27:02 -0700 (PDT) Date: Wed, 16 Apr 2025 15:26:45 +0000 In-Reply-To: <20250416152648.2982950-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250416152648.2982950-1-qperret@google.com> X-Mailer: git-send-email 2.49.0.604.gff1f9ca942-goog Message-ID: <20250416152648.2982950-6-qperret@google.com> Subject: [PATCH v2 5/7] KVM: arm64: Move hyp state to hyp_vmemmap From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Fuad Tabba , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Tracking the hypervisor's ownership state into struct hyp_page has several benefits, including allowing far more efficient lookups (no page-table walk needed) and de-corelating the state from the presence of a mapping. This will later allow to map pages into EL2 stage-1 less proactively which is generally a good thing for security. And in the future this will help with tracking the state of pages mapped into the hypervisor's private range without requiring an alias into the 'linear map' range. Reviewed-by: Marc Zyngier Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/memory.h | 29 +++++++++++--- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 51 ++++++++++++------------ arch/arm64/kvm/hyp/nvhe/setup.c | 6 ++- 3 files changed, 53 insertions(+), 33 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/memory.h b/arch/arm64/kvm/hyp/= include/nvhe/memory.h index a1754aecf8f8..eb0c2ebd1743 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/memory.h +++ b/arch/arm64/kvm/hyp/include/nvhe/memory.h @@ -16,8 +16,7 @@ * accessible by another component; * 10: The page is accessible but not owned by the component; * The storage of this state depends on the component: either in the - * hyp_vmemmap for the host state or in PTE software bits for the hypervis= or - * and guests. + * hyp_vmemmap for the host and hyp states or in PTE software bits for gue= sts. */ enum pkvm_page_state { PKVM_PAGE_OWNED =3D 0ULL, @@ -25,13 +24,14 @@ enum pkvm_page_state { PKVM_PAGE_SHARED_BORROWED =3D BIT(1), =20 /* - * 'Meta-states' are not stored directly in PTE SW bits for hyp and - * guest states, but inferred from the context (e.g. invalid PTE - * entries). For the host, meta-states are stored directly in the + * 'Meta-states' are not stored directly in PTE SW bits for guest + * states, but inferred from the context (e.g. invalid PTE entries). + * For the host and hyp, meta-states are stored directly in the * struct hyp_page. */ PKVM_NOPAGE =3D BIT(0) | BIT(1), }; +#define PKVM_PAGE_STATE_MASK (BIT(0) | BIT(1)) =20 #define PKVM_PAGE_STATE_PROT_MASK (KVM_PGTABLE_PROT_SW0 | KVM_PGTABLE_PROT= _SW1) static inline enum kvm_pgtable_prot pkvm_mkstate(enum kvm_pgtable_prot pro= t, @@ -52,7 +52,14 @@ struct hyp_page { u8 order; =20 /* Host state. Guarded by the host stage-2 lock. */ - unsigned __host_state : 8; + unsigned __host_state : 4; + + /* + * Complement of the hyp state. Guarded by the hyp stage-1 lock. We use + * the complement so that the initial 0 in __hyp_state_comp (due to the + * entire vmemmap starting off zeroed) encodes PKVM_NOPAGE. + */ + unsigned __hyp_state_comp : 4; =20 u32 host_share_guest_count; }; @@ -99,6 +106,16 @@ static inline void set_host_state(phys_addr_t phys, enu= m pkvm_page_state state) hyp_phys_to_page(phys)->__host_state =3D state; } =20 +static inline enum pkvm_page_state get_hyp_state(phys_addr_t phys) +{ + return hyp_phys_to_page(phys)->__hyp_state_comp ^ PKVM_PAGE_STATE_MASK; +} + +static inline void set_hyp_state(phys_addr_t phys, enum pkvm_page_state st= ate) +{ + hyp_phys_to_page(phys)->__hyp_state_comp =3D state ^ PKVM_PAGE_STATE_MASK; +} + /* * Refcounting for 'struct hyp_page'. * hyp_pool::lock must be held if atomic access to the refcount is require= d. diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index e9060e6205cb..25ff84c053c1 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -649,24 +649,24 @@ static int __host_set_page_state_range(u64 addr, u64 = size, return 0; } =20 -static enum pkvm_page_state hyp_get_page_state(kvm_pte_t pte, u64 addr) +static void __hyp_set_page_state_range(phys_addr_t phys, u64 size, enum pk= vm_page_state state) { - if (!kvm_pte_valid(pte)) - return PKVM_NOPAGE; + phys_addr_t end =3D phys + size; =20 - return pkvm_getstate(kvm_pgtable_hyp_pte_prot(pte)); + for (; phys < end; phys +=3D PAGE_SIZE) + set_hyp_state(phys, state); } =20 -static int __hyp_check_page_state_range(u64 addr, u64 size, - enum pkvm_page_state state) +static int __hyp_check_page_state_range(phys_addr_t phys, u64 size, enum p= kvm_page_state state) { - struct check_walk_data d =3D { - .desired =3D state, - .get_page_state =3D hyp_get_page_state, - }; + phys_addr_t end =3D phys + size; + + for (; phys < end; phys +=3D PAGE_SIZE) { + if (get_hyp_state(phys) !=3D state) + return -EPERM; + } =20 - hyp_assert_lock_held(&pkvm_pgd_lock); - return check_page_state_range(&pkvm_pgtable, addr, size, &d); + return 0; } =20 static enum pkvm_page_state guest_get_page_state(kvm_pte_t pte, u64 addr) @@ -694,7 +694,6 @@ int __pkvm_host_share_hyp(u64 pfn) { u64 phys =3D hyp_pfn_to_phys(pfn); void *virt =3D __hyp_va(phys); - enum kvm_pgtable_prot prot; u64 size =3D PAGE_SIZE; int ret; =20 @@ -705,13 +704,13 @@ int __pkvm_host_share_hyp(u64 pfn) if (ret) goto unlock; if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) { - ret =3D __hyp_check_page_state_range((u64)virt, size, PKVM_NOPAGE); + ret =3D __hyp_check_page_state_range(phys, size, PKVM_NOPAGE); if (ret) goto unlock; } =20 - prot =3D pkvm_mkstate(PAGE_HYP, PKVM_PAGE_SHARED_BORROWED); - WARN_ON(pkvm_create_mappings_locked(virt, virt + size, prot)); + __hyp_set_page_state_range(phys, size, PKVM_PAGE_SHARED_BORROWED); + WARN_ON(pkvm_create_mappings_locked(virt, virt + size, PAGE_HYP)); WARN_ON(__host_set_page_state_range(phys, size, PKVM_PAGE_SHARED_OWNED)); =20 unlock: @@ -734,7 +733,7 @@ int __pkvm_host_unshare_hyp(u64 pfn) ret =3D __host_check_page_state_range(phys, size, PKVM_PAGE_SHARED_OWNED); if (ret) goto unlock; - ret =3D __hyp_check_page_state_range(virt, size, PKVM_PAGE_SHARED_BORROWE= D); + ret =3D __hyp_check_page_state_range(phys, size, PKVM_PAGE_SHARED_BORROWE= D); if (ret) goto unlock; if (hyp_page_count((void *)virt)) { @@ -742,6 +741,7 @@ int __pkvm_host_unshare_hyp(u64 pfn) goto unlock; } =20 + __hyp_set_page_state_range(phys, size, PKVM_NOPAGE); WARN_ON(kvm_pgtable_hyp_unmap(&pkvm_pgtable, virt, size) !=3D size); WARN_ON(__host_set_page_state_range(phys, size, PKVM_PAGE_OWNED)); =20 @@ -757,7 +757,6 @@ int __pkvm_host_donate_hyp(u64 pfn, u64 nr_pages) u64 phys =3D hyp_pfn_to_phys(pfn); u64 size =3D PAGE_SIZE * nr_pages; void *virt =3D __hyp_va(phys); - enum kvm_pgtable_prot prot; int ret; =20 host_lock_component(); @@ -767,13 +766,13 @@ int __pkvm_host_donate_hyp(u64 pfn, u64 nr_pages) if (ret) goto unlock; if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) { - ret =3D __hyp_check_page_state_range((u64)virt, size, PKVM_NOPAGE); + ret =3D __hyp_check_page_state_range(phys, size, PKVM_NOPAGE); if (ret) goto unlock; } =20 - prot =3D pkvm_mkstate(PAGE_HYP, PKVM_PAGE_OWNED); - WARN_ON(pkvm_create_mappings_locked(virt, virt + size, prot)); + __hyp_set_page_state_range(phys, size, PKVM_PAGE_OWNED); + WARN_ON(pkvm_create_mappings_locked(virt, virt + size, PAGE_HYP)); WARN_ON(host_stage2_set_owner_locked(phys, size, PKVM_ID_HYP)); =20 unlock: @@ -793,7 +792,7 @@ int __pkvm_hyp_donate_host(u64 pfn, u64 nr_pages) host_lock_component(); hyp_lock_component(); =20 - ret =3D __hyp_check_page_state_range(virt, size, PKVM_PAGE_OWNED); + ret =3D __hyp_check_page_state_range(phys, size, PKVM_PAGE_OWNED); if (ret) goto unlock; if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) { @@ -802,6 +801,7 @@ int __pkvm_hyp_donate_host(u64 pfn, u64 nr_pages) goto unlock; } =20 + __hyp_set_page_state_range(phys, size, PKVM_NOPAGE); WARN_ON(kvm_pgtable_hyp_unmap(&pkvm_pgtable, virt, size) !=3D size); WARN_ON(host_stage2_set_owner_locked(phys, size, PKVM_ID_HOST)); =20 @@ -816,19 +816,18 @@ int hyp_pin_shared_mem(void *from, void *to) { u64 cur, start =3D ALIGN_DOWN((u64)from, PAGE_SIZE); u64 end =3D PAGE_ALIGN((u64)to); + u64 phys =3D __hyp_pa(start); u64 size =3D end - start; int ret; =20 host_lock_component(); hyp_lock_component(); =20 - ret =3D __host_check_page_state_range(__hyp_pa(start), size, - PKVM_PAGE_SHARED_OWNED); + ret =3D __host_check_page_state_range(phys, size, PKVM_PAGE_SHARED_OWNED); if (ret) goto unlock; =20 - ret =3D __hyp_check_page_state_range(start, size, - PKVM_PAGE_SHARED_BORROWED); + ret =3D __hyp_check_page_state_range(phys, size, PKVM_PAGE_SHARED_BORROWE= D); if (ret) goto unlock; =20 diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setu= p.c index 1a414288fe8c..955c431af5d0 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -194,16 +194,20 @@ static int fix_host_ownership_walker(const struct kvm= _pgtable_visit_ctx *ctx, =20 /* * Adjust the host stage-2 mappings to match the ownership attributes - * configured in the hypervisor stage-1. + * configured in the hypervisor stage-1, and make sure to propagate them + * to the hyp_vmemmap state. */ state =3D pkvm_getstate(kvm_pgtable_hyp_pte_prot(ctx->old)); switch (state) { case PKVM_PAGE_OWNED: + set_hyp_state(phys, PKVM_PAGE_OWNED); return host_stage2_set_owner_locked(phys, PAGE_SIZE, PKVM_ID_HYP); case PKVM_PAGE_SHARED_OWNED: + set_hyp_state(phys, PKVM_PAGE_SHARED_OWNED); set_host_state(phys, PKVM_PAGE_SHARED_BORROWED); break; case PKVM_PAGE_SHARED_BORROWED: + set_hyp_state(phys, PKVM_PAGE_SHARED_BORROWED); set_host_state(phys, PKVM_PAGE_SHARED_OWNED); break; default: --=20 2.49.0.604.gff1f9ca942-goog From nobody Fri Dec 19 20:55:12 2025 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A9BD211715 for ; Wed, 16 Apr 2025 15:27:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817228; cv=none; b=TKzWQKOQ5hQQ2GaZW1vzKo9nlnDIlw1i+UhKTkn0ojjD7D64oh9D2wmj4mlMObFqN2cu7vH4eUbuctNsq0enlkjH0OlANccutt+wRSZURrjeq0rVtqMkwBwJOt/B4+F5X4mQbXxKIzj4WcjY/A/l7HhjmN+XXbB6b3SR5U/w8Ow= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817228; c=relaxed/simple; bh=mlPlfdc/NWwYqy2zjYwAzNRWIZhZ7ZZR38Y6Ldz2Cr0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Ee6AYKtVne4UMhgsjT1KGb2fSumSP+gZ2ozoHGDLIihZ0e557r2AA6e2L1rud/RsqEsBAP3mzqTYtF3jIiIGNvoRkByNd342HugbS1f5iVIK5+PsDZc9+trCm+OaAIA8mZ2haepRBkVOrPiM1MOlIIgB+jprV8tqgijOLcEOB3o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=rNayNeQD; arc=none smtp.client-ip=209.85.218.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rNayNeQD" Received: by mail-ej1-f74.google.com with SMTP id a640c23a62f3a-ac3dca41591so620226066b.1 for ; Wed, 16 Apr 2025 08:27:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744817224; x=1745422024; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=VYU32RrZwSVxBTFJfFZ/zcmulUROASDhu21QhFFvtpI=; b=rNayNeQDPSxQYMnciZNcdl7/kUmhK16bMPGV70Gxg5zzBD/MLjOXtAzRXFXk4tGkCI rImmHevMX1axjiPTW14omZr7+wKAGB0Qf4zWr7ZmQtbWGkD8tZzIxYEx56fIGYBH8lgb 4+8YLYILzeweoplpMqIGIOGF2ozD557XlXDUqWgyiwF0nQdmoAm2sbU40hNner/8BWsX wCx0XppMyPhUm0DpVB8vTy4rYv8qkB7HxYaoIKeIqNoevC3gLBHDuG0+DMbShP4Jl/7p BaGCuJdQi+Fsz7GSgK7yzaYuoYu6ABy1elFdI7AKPmEqoy8G/BwYWxhgNsDlTP6DkLTM VB3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744817224; x=1745422024; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VYU32RrZwSVxBTFJfFZ/zcmulUROASDhu21QhFFvtpI=; b=S4dOyaP6oDwjI+IZeQbcbQYCDwM0n5jl5DgKW1/CwUFYnPvVWOlEwqkl/1kwM5MMux 3OrgMVXqLm+XqISl6JXgSZTYMuKhhtRsWTEJQ+33+AVZMw3zNpOTzyG5FnSe5bxhm1Uh BQFxTgiuFcybkKynGwD2FHqcOyxveakBJ5JZEBinQNKlftvmO251do+zWbS6RuYrxkgI mJBO7+m8v/9uD13/8ZihXxEraE5hhJWSoUe2FNoDmmzbGE//h9APmFsAiSP9z2drFCau fjrWLAmAtgVkdLBaQ4TuKeQcJt1ED3FL9Y5l6oF+JW0lBfpS+OLklLNVXk1NwGHa/rOh 3pVQ== X-Forwarded-Encrypted: i=1; AJvYcCV3xyqOqCrjx5ylQo0Fzp2W+rDXWU8deL23Mj4nkFgIDrxzks4vzAY5cgpKUMBLHCKIMLlcwj2s9YpjnB8=@vger.kernel.org X-Gm-Message-State: AOJu0YxHSgtZSViLQO7O/1vlYcnx7nyreR2TngclXAXanBGuYTL/+0I7 TrgQ2PEu4Od8TXB1Y32sWmAlLRWPuKfr5CKBWQLdGNvEO/X9grF+EgaRLv9PhyZUFLtoII9LSl6 OSwtdKA== X-Google-Smtp-Source: AGHT+IFx/wZB15eiIdBs2Eh1Wi5DlZDuOgycITN7QmUrR/0Pho2JFdCK7PrHfR8mLm6W9F5WKdLs/Vk8CDxe X-Received: from edyc19.prod.google.com ([2002:aa7:df13:0:b0:5ec:dada:a73e]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a17:906:478a:b0:ac7:cfe0:3014 with SMTP id a640c23a62f3a-acb4296492fmr220940266b.25.1744817224502; Wed, 16 Apr 2025 08:27:04 -0700 (PDT) Date: Wed, 16 Apr 2025 15:26:46 +0000 In-Reply-To: <20250416152648.2982950-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250416152648.2982950-1-qperret@google.com> X-Mailer: git-send-email 2.49.0.604.gff1f9ca942-goog Message-ID: <20250416152648.2982950-7-qperret@google.com> Subject: [PATCH v2 6/7] KVM: arm64: Defer EL2 stage-1 mapping on share From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Fuad Tabba , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We currently blindly map into EL2 stage-1 *any* page passed to the __pkvm_host_share_hyp() HVC. This is less than ideal from a security perspective as it makes exploitation of potential hypervisor gadgets easier than it should be. But interestingly, pKVM should never need to access SHARED_BORROWED pages that it hasn't previously pinned, so there is no need to map the page before that. Reviewed-by: Marc Zyngier Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index 25ff84c053c1..91b757e3fb4c 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -693,7 +693,6 @@ static int __guest_check_page_state_range(struct pkvm_h= yp_vcpu *vcpu, u64 addr, int __pkvm_host_share_hyp(u64 pfn) { u64 phys =3D hyp_pfn_to_phys(pfn); - void *virt =3D __hyp_va(phys); u64 size =3D PAGE_SIZE; int ret; =20 @@ -710,7 +709,6 @@ int __pkvm_host_share_hyp(u64 pfn) } =20 __hyp_set_page_state_range(phys, size, PKVM_PAGE_SHARED_BORROWED); - WARN_ON(pkvm_create_mappings_locked(virt, virt + size, PAGE_HYP)); WARN_ON(__host_set_page_state_range(phys, size, PKVM_PAGE_SHARED_OWNED)); =20 unlock: @@ -742,7 +740,6 @@ int __pkvm_host_unshare_hyp(u64 pfn) } =20 __hyp_set_page_state_range(phys, size, PKVM_NOPAGE); - WARN_ON(kvm_pgtable_hyp_unmap(&pkvm_pgtable, virt, size) !=3D size); WARN_ON(__host_set_page_state_range(phys, size, PKVM_PAGE_OWNED)); =20 unlock: @@ -818,6 +815,7 @@ int hyp_pin_shared_mem(void *from, void *to) u64 end =3D PAGE_ALIGN((u64)to); u64 phys =3D __hyp_pa(start); u64 size =3D end - start; + struct hyp_page *p; int ret; =20 host_lock_component(); @@ -831,8 +829,14 @@ int hyp_pin_shared_mem(void *from, void *to) if (ret) goto unlock; =20 - for (cur =3D start; cur < end; cur +=3D PAGE_SIZE) - hyp_page_ref_inc(hyp_virt_to_page(cur)); + for (cur =3D start; cur < end; cur +=3D PAGE_SIZE) { + p =3D hyp_virt_to_page(cur); + hyp_page_ref_inc(p); + if (p->refcount =3D=3D 1) + WARN_ON(pkvm_create_mappings_locked((void *)cur, + (void *)cur + PAGE_SIZE, + PAGE_HYP)); + } =20 unlock: hyp_unlock_component(); @@ -845,12 +849,17 @@ void hyp_unpin_shared_mem(void *from, void *to) { u64 cur, start =3D ALIGN_DOWN((u64)from, PAGE_SIZE); u64 end =3D PAGE_ALIGN((u64)to); + struct hyp_page *p; =20 host_lock_component(); hyp_lock_component(); =20 - for (cur =3D start; cur < end; cur +=3D PAGE_SIZE) - hyp_page_ref_dec(hyp_virt_to_page(cur)); + for (cur =3D start; cur < end; cur +=3D PAGE_SIZE) { + p =3D hyp_virt_to_page(cur); + if (p->refcount =3D=3D 1) + WARN_ON(kvm_pgtable_hyp_unmap(&pkvm_pgtable, cur, PAGE_SIZE) !=3D PAGE_= SIZE); + hyp_page_ref_dec(p); + } =20 hyp_unlock_component(); host_unlock_component(); --=20 2.49.0.604.gff1f9ca942-goog From nobody Fri Dec 19 20:55:12 2025 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 764C2212B34 for ; Wed, 16 Apr 2025 15:27:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817230; cv=none; b=TviTdYc7HOMkRf9OFEcVoCbbumPzgvRkes9ZhjkkqqJ91c+dHEHS+p8d7qas+FR6KCfMx/L2eaokXurAcRliGZUFCvvK+RlraVlpOR7NUasV9K6UdL30lAwZKH0d44XjwuZrpYo2ev/FJ7oI7a7uPoP/jdqD+Kw+dWTDKCIqPR8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744817230; c=relaxed/simple; bh=uu6A7iueq6DtipNuS6Phk5LGlAI7BCi/IestivOyyuk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Cq8tmAk2GYwOKqGyztGMiGcWiFU/5G2b2DFsLS/Dx2vvltpLDmviztBU6w/Wloz74MDV/jxFM0wjX//moxWDYlO9eKBnPk6A8f5CA+6Ug7UGm8cyJcC3fZM00In1WotSPJlqo5IXqFTlp0Q7/0OhJp241GC2gJqSOIBck5jMKrc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=jLFhr8R0; arc=none smtp.client-ip=209.85.208.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jLFhr8R0" Received: by mail-ed1-f73.google.com with SMTP id 4fb4d7f45d1cf-5e6d978792dso6061657a12.1 for ; Wed, 16 Apr 2025 08:27:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744817227; x=1745422027; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=j+5woTKLEDxUzA3eOLn0hsAEJ5MJo4JdU/wMD9M47qc=; b=jLFhr8R0rjBD6ZkEjsLJUIBozZHCH+BRMLTH/b5pJ4x7nbenNmQUQc96V7NRpHp9AX B77OHDLOLKUNTOQTy4tdm1uvjCEfeK9A9kjHaF9VP+bVfJBmAroX2VQbgIYBYaog99es 50sMIl73YZq5DEZ2fyDFXwo8EkGIMBBn6YzBmvJhtpjr/aQl5Hf0cNA9Qayqyu7jjQF+ sndXB5GzGtpvFs/Ye28FJWrLMd86MvHDPcD77ppzZkzlYGZY+Ulv49lg2VAjTMtDH/Zp bNfPvOkYQynICMFUugi6Xi6CiqF8JRCmgUduVLDf7nCVadjKcGFPs2xJbjiEXzE8HXE1 oY7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744817227; x=1745422027; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=j+5woTKLEDxUzA3eOLn0hsAEJ5MJo4JdU/wMD9M47qc=; b=uxM5Pie3+oFLnBM0Iqn65XiciFCa4uTZOadsuCYau+rhsSMcrDrdJfXTDMSPcZJC3K xko6CRkQdOUQt3vqBQ2ItOrHw8XloGHKGnsbIKiOakmwcOJgR+FdqxYHpEdRkQfnXeVj a/fi0wwOdLFf0X7nKS85TG0VkgM+XlVFSXl5bqSnL6Nu33b1Ss33tnjsQU3u6/emIu6O eO8DFOpdg+0qZXJBE4xkn+s/tk6xRNvyGqlNwsw9qpT0VAecLKYoKT83ZZCZ0Ln0dpPT N4W4MIz5uivdZC4/Brf+ORIRjrTNu/CsxmHQx3PIVLcEJ0le6PB2VH1CcmPiZL5ldG/x vruQ== X-Forwarded-Encrypted: i=1; AJvYcCUpf96CmCOpFcYr3p+/DnKnZhAaVYjgf3DDkG44XvKkyw5QBIOZ+rvYguAr0S1QeLDMqMJSdMbwpQkjE7I=@vger.kernel.org X-Gm-Message-State: AOJu0YxMrZDAzjA0IHoEFTSbQwwuYyPIlk5e4SMoirbiumDuzhYECpgQ B/tesAVs7ggy9XY7TmKW9sN6RMpqCoBI39xlTCYfriHjE/UYty2P9SoNFt4dgGmf0JMAN0RdI2C kY7q0uw== X-Google-Smtp-Source: AGHT+IGCybzKX7dX809ygOrq+ajkf80dzTivmVZqG7Y0aKxNYJUDug4w5qsQZIOiGF/fQUZzevVuaJaQC1OG X-Received: from edqt19.prod.google.com ([2002:aa7:d713:0:b0:5e5:339d:60ab]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:42c5:b0:5eb:cc22:aa00 with SMTP id 4fb4d7f45d1cf-5f4b748ff90mr1896967a12.19.1744817226898; Wed, 16 Apr 2025 08:27:06 -0700 (PDT) Date: Wed, 16 Apr 2025 15:26:47 +0000 In-Reply-To: <20250416152648.2982950-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250416152648.2982950-1-qperret@google.com> X-Mailer: git-send-email 2.49.0.604.gff1f9ca942-goog Message-ID: <20250416152648.2982950-8-qperret@google.com> Subject: [PATCH v2 7/7] KVM: arm64: Unconditionally cross check hyp state From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Fuad Tabba , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the hypervisor's state is stored in the hyp_vmemmap, we no longer need an expensive page-table walk to read it. This means we can now afford to cross check the hyp-state during all memory ownership transitions where the hyp is involved unconditionally, hence avoiding problems such as [1]. [1] https://lore.kernel.org/kvmarm/20241128154406.602875-1-qperret@google.c= om/ Reviewed-by: Marc Zyngier Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 24 +++++++++--------------- 1 file changed, 9 insertions(+), 15 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index 91b757e3fb4c..709d286999a1 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -702,11 +702,9 @@ int __pkvm_host_share_hyp(u64 pfn) ret =3D __host_check_page_state_range(phys, size, PKVM_PAGE_OWNED); if (ret) goto unlock; - if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) { - ret =3D __hyp_check_page_state_range(phys, size, PKVM_NOPAGE); - if (ret) - goto unlock; - } + ret =3D __hyp_check_page_state_range(phys, size, PKVM_NOPAGE); + if (ret) + goto unlock; =20 __hyp_set_page_state_range(phys, size, PKVM_PAGE_SHARED_BORROWED); WARN_ON(__host_set_page_state_range(phys, size, PKVM_PAGE_SHARED_OWNED)); @@ -762,11 +760,9 @@ int __pkvm_host_donate_hyp(u64 pfn, u64 nr_pages) ret =3D __host_check_page_state_range(phys, size, PKVM_PAGE_OWNED); if (ret) goto unlock; - if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) { - ret =3D __hyp_check_page_state_range(phys, size, PKVM_NOPAGE); - if (ret) - goto unlock; - } + ret =3D __hyp_check_page_state_range(phys, size, PKVM_NOPAGE); + if (ret) + goto unlock; =20 __hyp_set_page_state_range(phys, size, PKVM_PAGE_OWNED); WARN_ON(pkvm_create_mappings_locked(virt, virt + size, PAGE_HYP)); @@ -792,11 +788,9 @@ int __pkvm_hyp_donate_host(u64 pfn, u64 nr_pages) ret =3D __hyp_check_page_state_range(phys, size, PKVM_PAGE_OWNED); if (ret) goto unlock; - if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) { - ret =3D __host_check_page_state_range(phys, size, PKVM_NOPAGE); - if (ret) - goto unlock; - } + ret =3D __host_check_page_state_range(phys, size, PKVM_NOPAGE); + if (ret) + goto unlock; =20 __hyp_set_page_state_range(phys, size, PKVM_NOPAGE); WARN_ON(kvm_pgtable_hyp_unmap(&pkvm_pgtable, virt, size) !=3D size); --=20 2.49.0.604.gff1f9ca942-goog