From nobody Sat Feb 7 18:21:07 2026 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4840D6FBF for ; Thu, 27 Feb 2025 00:33:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740616398; cv=none; b=awHcmUS2/X4NwHvhBCVS+rD9jMkefby1J8eprZY6nxyuEe7etjs2viWnFutv3u7YxVsh9e9dED8IQ1MU1hO6aiaxMQvH4zPM0ZZPL1kw28GmQZWsuXNhGJZV4+T2fEch5vuUr7TYTUqJTepXe/XukEpReOhc+s+rWok5jGmoYww= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740616398; c=relaxed/simple; bh=oJPtYm8a9NJewkREbq9VadrFsmWB8AXYZ5A8G3HTEtE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cCp6DhEJix+fvivz5C1BjQvdPZetk+FNo1fe2l0Y8QIwTLSJ9SGLA16ySwnyvbEGT81I7N92wpGOFsyjRJCi7Hccv9ngo2vGgaQ7uyMqPkIuyX5cCVw7UIo6pthkE/5KG1UF2sVOSKlQ3fx91nFmI1yxxJn/Nrrw9mefLFwRA60= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=eCLogDf1; arc=none smtp.client-ip=209.85.218.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eCLogDf1" Received: by mail-ej1-f73.google.com with SMTP id a640c23a62f3a-abb9d1c8113so33459666b.0 for ; Wed, 26 Feb 2025 16:33:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740616394; x=1741221194; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=cnSi0QZnCKAvL+73Iyi2doPe04c0Ox90ZdzmLMLkEvI=; b=eCLogDf1Sr5cjqIpCPfH3Be9bJftYUZVIaEPMBRqSgy4PyeWNiF/3Xhjl9Toihk3Hl goVz0/aWuipey5gkvKcu6Matuz2vWwGOb8OS+vmo//eDmIATYpOSfXM1NptVbyxcRUco io+0tkPMWa15U2h6r9czFDIe8rEgrji5oeGwjXAYRrEzV2PLHbiQng8yi70nRpGJZrdq w62pMv671HypDwYXI6IbOQE17C+O7NuZWaI08qt4Gu/1oB9TUJ70b/4dV4mWLo0JHWhD mhH8FyBItdn3aqFBqMehAIRjCk2l6h4/G7UKuMjAv1CIB2oOXLAlAAFAFfnPB33wxSWm QXXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740616394; x=1741221194; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=cnSi0QZnCKAvL+73Iyi2doPe04c0Ox90ZdzmLMLkEvI=; b=Ybfju6DG2Y9HsXQrChf/17H7CpUIuo7CE6s/0TnHGUubG9kXpgKN2qpWxvmEJcgpvJ GftOOZ49qTh+PP58jpJ3M5ruupu7TLEei8EnVa27plYQgdecohlzaJf+v4HB0TcXZWgP xBrRmA1Y2HTHcM+1CEFdJ/ANqOXXp316Yu98p/iT5NwG1mkqCVqvPKpsxvKN/R0ZDH6S yzSvCGPv+SOON29bMugEU5c1te6mBB6chjMy5HLIlaDLu4/HaHlg8H98jUEBvzST9pm+ i9I4QrCIZRZBTmsYVnGYAJ61Wbu7hMeYJm/xS98g3S6rqGo2PtT3mrtpBwjLXf0MFhyc sawg== X-Forwarded-Encrypted: i=1; AJvYcCVF0Qg9p3MUsg6xHZmVVly3uP3ml3nTHYAQ7g4Ds3MX1V8QQqUg+y9zNZ8wV20FzQ5gbFr6MNEl9ROYAo0=@vger.kernel.org X-Gm-Message-State: AOJu0YwHCFTY28bWqYfqqD/QonvX7SVmzPOfZs1z40sfU3+eCC/B/CEw rfhQPZV3sWlhuEdBUmrhcYkf1wvWaSTEhfAMMhU+VjcVwRTYCgRY0Xq/LjDq1vvZrDovbqB56RZ Mcj+VfA== X-Google-Smtp-Source: AGHT+IFnzWlAoWfe7gVv/3yG2kJPIQ7pfQ+/eZHRD7MANnsJ1+erytT4JEEh5E6KoQuNIQGX7Bdb2AUdqKfL X-Received: from ejcsk13.prod.google.com ([2002:a17:906:630d:b0:abe:e921:5690]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:cf86:b0:ab7:ec8b:c642 with SMTP id a640c23a62f3a-abeeecf6f79mr771155666b.5.1740616394645; Wed, 26 Feb 2025 16:33:14 -0800 (PST) Date: Thu, 27 Feb 2025 00:33:05 +0000 In-Reply-To: <20250227003310.367350-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227003310.367350-1-qperret@google.com> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog Message-ID: <20250227003310.367350-2-qperret@google.com> Subject: [PATCH 1/6] KVM: arm64: Track SVE state in the hypervisor vcpu structure From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Fuad Tabba When dealing with a guest with SVE enabled, make sure the host SVE state is pinned at EL2 S1, and that the hypervisor vCPU state is correctly initialised (and then unpinned on teardown). Co-authored-by: Marc Zyngier Signed-off-by: Fuad Tabba Signed-off-by: Marc Zyngier Signed-off-by: Quentin Perret Reviewed-by: Marc Zyngier --- arch/arm64/include/asm/kvm_host.h | 12 ++++--- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 4 --- arch/arm64/kvm/hyp/nvhe/pkvm.c | 54 +++++++++++++++++++++++++++--- 3 files changed, 56 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 3a7ec98ef123..90b58f87b107 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -930,20 +930,22 @@ struct kvm_vcpu_arch { #define vcpu_sve_zcr_elx(vcpu) \ (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) =20 -#define vcpu_sve_state_size(vcpu) ({ \ +#define sve_state_size_from_vl(sve_max_vl) ({ \ size_t __size_ret; \ - unsigned int __vcpu_vq; \ + unsigned int __vq; \ \ - if (WARN_ON(!sve_vl_valid((vcpu)->arch.sve_max_vl))) { \ + if (WARN_ON(!sve_vl_valid(sve_max_vl))) { \ __size_ret =3D 0; \ } else { \ - __vcpu_vq =3D vcpu_sve_max_vq(vcpu); \ - __size_ret =3D SVE_SIG_REGS_SIZE(__vcpu_vq); \ + __vq =3D sve_vq_from_vl(sve_max_vl); \ + __size_ret =3D SVE_SIG_REGS_SIZE(__vq); \ } \ \ __size_ret; \ }) =20 +#define vcpu_sve_state_size(vcpu) sve_state_size_from_vl((vcpu)->arch.sve_= max_vl) + #define KVM_GUESTDBG_VALID_MASK (KVM_GUESTDBG_ENABLE | \ KVM_GUESTDBG_USE_SW_BP | \ KVM_GUESTDBG_USE_HW | \ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/h= yp-main.c index 2c37680d954c..59db9606e6e1 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -123,10 +123,6 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_v= cpu) =20 hyp_vcpu->vcpu.arch.ctxt =3D host_vcpu->arch.ctxt; =20 - hyp_vcpu->vcpu.arch.sve_state =3D kern_hyp_va(host_vcpu->arch.sve_state); - /* Limit guest vector length to the maximum supported by the host. */ - hyp_vcpu->vcpu.arch.sve_max_vl =3D min(host_vcpu->arch.sve_max_vl, kvm_ho= st_sve_max_vl); - hyp_vcpu->vcpu.arch.mdcr_el2 =3D host_vcpu->arch.mdcr_el2; hyp_vcpu->vcpu.arch.hcr_el2 &=3D ~(HCR_TWI | HCR_TWE); hyp_vcpu->vcpu.arch.hcr_el2 |=3D READ_ONCE(host_vcpu->arch.hcr_el2) & diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 3927fe52a3dd..3ec27e12b043 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -356,13 +356,29 @@ static void unpin_host_vcpu(struct kvm_vcpu *host_vcp= u) hyp_unpin_shared_mem(host_vcpu, host_vcpu + 1); } =20 +static void unpin_host_sve_state(struct pkvm_hyp_vcpu *hyp_vcpu) +{ + void *sve_state; + + if (!vcpu_has_feature(&hyp_vcpu->vcpu, KVM_ARM_VCPU_SVE)) + return; + + sve_state =3D kern_hyp_va(hyp_vcpu->vcpu.arch.sve_state); + hyp_unpin_shared_mem(sve_state, + sve_state + vcpu_sve_state_size(&hyp_vcpu->vcpu)); +} + static void unpin_host_vcpus(struct pkvm_hyp_vcpu *hyp_vcpus[], unsigned int nr_vcpus) { int i; =20 - for (i =3D 0; i < nr_vcpus; i++) - unpin_host_vcpu(hyp_vcpus[i]->host_vcpu); + for (i =3D 0; i < nr_vcpus; i++) { + struct pkvm_hyp_vcpu *hyp_vcpu =3D hyp_vcpus[i]; + + unpin_host_vcpu(hyp_vcpu->host_vcpu); + unpin_host_sve_state(hyp_vcpu); + } } =20 static void init_pkvm_hyp_vm(struct kvm *host_kvm, struct pkvm_hyp_vm *hyp= _vm, @@ -376,12 +392,40 @@ static void init_pkvm_hyp_vm(struct kvm *host_kvm, st= ruct pkvm_hyp_vm *hyp_vm, pkvm_init_features_from_host(hyp_vm, host_kvm); } =20 -static void pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp_vcpu, struct kvm_= vcpu *host_vcpu) +static int pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp_vcpu, struct kvm_v= cpu *host_vcpu) { struct kvm_vcpu *vcpu =3D &hyp_vcpu->vcpu; + unsigned int sve_max_vl; + size_t sve_state_size; + void *sve_state; + int ret =3D 0; =20 - if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) + if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) { vcpu_clear_flag(vcpu, VCPU_SVE_FINALIZED); + return 0; + } + + /* Limit guest vector length to the maximum supported by the host. */ + sve_max_vl =3D min(READ_ONCE(host_vcpu->arch.sve_max_vl), kvm_host_sve_ma= x_vl); + sve_state_size =3D sve_state_size_from_vl(sve_max_vl); + sve_state =3D kern_hyp_va(READ_ONCE(host_vcpu->arch.sve_state)); + + if (!sve_state || !sve_state_size) { + ret =3D -EINVAL; + goto err; + } + + ret =3D hyp_pin_shared_mem(sve_state, sve_state + sve_state_size); + if (ret) + goto err; + + vcpu->arch.sve_state =3D sve_state; + vcpu->arch.sve_max_vl =3D sve_max_vl; + + return 0; +err: + clear_bit(KVM_ARM_VCPU_SVE, vcpu->kvm->arch.vcpu_features); + return ret; } =20 static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu, @@ -416,7 +460,7 @@ static int init_pkvm_hyp_vcpu(struct pkvm_hyp_vcpu *hyp= _vcpu, if (ret) goto done; =20 - pkvm_vcpu_init_sve(hyp_vcpu, host_vcpu); + ret =3D pkvm_vcpu_init_sve(hyp_vcpu, host_vcpu); done: if (ret) unpin_host_vcpu(host_vcpu); --=20 2.48.1.658.g4767266eb4-goog From nobody Sat Feb 7 18:21:07 2026 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B750EED8 for ; Thu, 27 Feb 2025 00:33:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740616400; cv=none; b=G4XOoPT1/yaSutTfrKRhAZj936MCkbThdySYz+xsBsG1dyIXYr5bQYIbtMwC0gMtgUm9dc5GXsirSlEob2NjnTPlwedj9BVMofD24ZH4+yBFHLYV4wH/O8MMq7IgpHoF9Cz7xgOcrRAQX8ZQs1BsM4cpDrCVyLDhUK4j5PBzjWc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740616400; c=relaxed/simple; bh=261ZPCgnrvvDXYcolm1gc3HyK99Q/XfdlKS2Gd7awrQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=G6HegHxg6IUIl5TA7MlIbo2/HRZWfXZoGhDU9ryk69gYbvTkUQSnL1drg1ZwvgxwTLEGL7hFjhdtkPZOPgoau3MD95EowrcIBGPPuvnK78+qnO/wAp+zJrLZHxxu+rGMwoD8QwsV7BkcqZubTBEbJ/zcU7XnZ7Yhv6HubgD9BPU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XsN5ERQX; arc=none smtp.client-ip=209.85.218.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XsN5ERQX" Received: by mail-ej1-f74.google.com with SMTP id a640c23a62f3a-ab68fbe53a4so34139766b.2 for ; Wed, 26 Feb 2025 16:33:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740616397; x=1741221197; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=nP/0ASv3gZv6Dq4JFLJ6V0xk++r2sqgnKrNUADZ27XI=; b=XsN5ERQXwm9d8W72NIbybPof42gz4GaMRHviYxZSVjJ0gV+f5YBAMbiQnGHobXGstw 6/tXbwAkJ13lNR4aeQOEQeWsHL7JLYKu2eHZohCJ6MDVHbb3QX/OqyDdwz0MV1zSi2dz aZMSqQB1eeh785JZgSWRrRgwVv+wHUdMW2ECcppYvdcu2CxJtoGuTj1F83cGw49poupp vvSubl1sxotVHHrabIbiUi2aAt6j2wabNvUQj6o8hPk10AY5TrCZx2D1pC6B+6Hi8NAq RIIEGJhfcobvEZ+I7MVi/MZ1NL3wlL2mjNIzFhfNInZ0TINOxkea4PCSExpCM3EkWbM3 MFAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740616397; x=1741221197; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=nP/0ASv3gZv6Dq4JFLJ6V0xk++r2sqgnKrNUADZ27XI=; b=cw3+lGFZupK3lXIkoeIIRpT1BIqkWRE1x2hPzw4dq88JDGZw84i2OY4FCfBWk1/R0e U1d2TPc6B6bbAtrG+QHc7pefdCkCtXepLi8WETOn8O20GekX0USJ3DflYvfCV6kLA/Mf Sp6cFYId4BbhMWGcEdoxfc1EvT6YW2nPISJnVkUjg/Y8M0qLIuLfPIeC7pZhjfoK77SD aMu5pkK3tPUnKbcWLNJxV1cip41eeopiBi2L9FdD9w77reeGEkpI2IRR3TJiQYNvHgn6 otCclln+u8bRP+C9dkETAn8x4UiZG3pV9D/AOctby4Qbo2oWjCdwIXNC4PTVXyGLrdSq zx1A== X-Forwarded-Encrypted: i=1; AJvYcCVnEDPwtnZ/0LvWnWpFQVZn64eYSot5KWAMxkW8303EQeHg/S/t7nFJ3/ym8SMpuboFQ6qdDu32IoTx91o=@vger.kernel.org X-Gm-Message-State: AOJu0YxUo/aZPEKDTfy0VkqvDOHODcv0/DcjLRKeZVm8ajwuGRTu1Qgs ki5L/l9Z0XvswugJCj2NobHUhvvaghEF4bKw2Qrnb7Pepe/aqq1ZbdvHH7TCJ4H5SM3/p578vIs Y1cPfyA== X-Google-Smtp-Source: AGHT+IG1YIW+GqBALCieMnU4BqI9X9ToFAuUlbxFKvAcEIATQsa5h/n36+QHjYaFBr4TgnPvu4HEXBtbRBDi X-Received: from ejcrr23.prod.google.com ([2002:a17:907:8997:b0:abb:9ace:21a3]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:7841:b0:aa6:5eae:7ece with SMTP id a640c23a62f3a-abed10179f1mr945776366b.43.1740616396839; Wed, 26 Feb 2025 16:33:16 -0800 (PST) Date: Thu, 27 Feb 2025 00:33:06 +0000 In-Reply-To: <20250227003310.367350-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227003310.367350-1-qperret@google.com> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog Message-ID: <20250227003310.367350-3-qperret@google.com> Subject: [PATCH 2/6] KVM: arm64: Use 0b11 for encoding PKVM_NOPAGE From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The page ownership state encoded as 0b11 is currently considered reserved for future use, and PKVM_NOPAGE uses bit 2. In order to simplify the relocation of the hyp ownership state into the vmemmap in later patches, let's use the 'reserved' encoding for the PKVM_NOPAGE state. The struct hyp_page layout isn't guaranteed stable at all, so there is no real reason to have 'reserved' encodings. No functional changes intended. Signed-off-by: Quentin Perret Reviewed-by: Marc Zyngier --- arch/arm64/kvm/hyp/include/nvhe/memory.h | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/memory.h b/arch/arm64/kvm/hyp/= include/nvhe/memory.h index 34233d586060..642b5e05fe77 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/memory.h +++ b/arch/arm64/kvm/hyp/include/nvhe/memory.h @@ -13,18 +13,15 @@ * 01: The page is owned by the page-table owner, but is shared * with another entity. * 10: The page is shared with, but not owned by the page-table owner. - * 11: Reserved for future use (lending). */ enum pkvm_page_state { PKVM_PAGE_OWNED =3D 0ULL, PKVM_PAGE_SHARED_OWNED =3D BIT(0), PKVM_PAGE_SHARED_BORROWED =3D BIT(1), - __PKVM_PAGE_RESERVED =3D BIT(0) | BIT(1), =20 /* Meta-states which aren't encoded directly in the PTE's SW bits */ - PKVM_NOPAGE =3D BIT(2), + PKVM_NOPAGE =3D BIT(0) | BIT(1), }; -#define PKVM_PAGE_META_STATES_MASK (~__PKVM_PAGE_RESERVED) =20 #define PKVM_PAGE_STATE_PROT_MASK (KVM_PGTABLE_PROT_SW0 | KVM_PGTABLE_PROT= _SW1) static inline enum kvm_pgtable_prot pkvm_mkstate(enum kvm_pgtable_prot pro= t, --=20 2.48.1.658.g4767266eb4-goog From nobody Sat Feb 7 18:21:07 2026 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8BC8327470 for ; Thu, 27 Feb 2025 00:33:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740616402; cv=none; b=hnxp+f+isinBdH6b290081wx+n8QDQ7bS8OeVR1fdkpdjhDGrK9a4282sWHQpL5CQAwrydByiCmJRgsoVw+urtJ26f4wjdAiBX3y1hjDBFWXgbiSWIw8tT4SmmYUsJ+KGUBcnI+uN7h5itg8CDciZIjnWQnrd7k8KTxGYSEHc2M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740616402; c=relaxed/simple; bh=fXqWM3SBGVlqwHL/vGbkHIEcyvaqkjwWBeVvjy3yeOg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bojZyJqqIqSAhsJEPanI2M9SXGad+AfOqwLNNRiT2XLDjnbY+APA3juAVl6LhZXA/xlZ6Zpi2Ypxb66oU1pArbIdgRweuhtTCDMWbJmRDL+J/z8e7ftqwSk2+4F0xJIdxoBsyOc8uVnZq2XMlWx3IE7F/0jWeJbN+nh9HEv9Tm8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bi8DtybN; arc=none smtp.client-ip=209.85.218.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bi8DtybN" Received: by mail-ej1-f74.google.com with SMTP id a640c23a62f3a-abb86503062so38239566b.2 for ; Wed, 26 Feb 2025 16:33:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740616399; x=1741221199; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ARvzDFB7zgeRMzJfEJ/MY3yWuG0D2YavMEjMtd8skDU=; b=bi8DtybN2kvWt0ROcYCpVUKp8Is5XXmxPNf0gsCJD2bflLAjkYIV8J9V07T5Fyl4hf jebkxRPTKQlPN0vO7TWjIjV3cwHsZBijaHXMgAVzA9O3C3PbHg9Gnyt11CQtSVAKE+sl OIuAtmH1Nb/gawjLOeav76QGyUoEWzmmH2KSsl/q5sLc8GPnJnCZBm7SJ9TxHrI5bde1 R8290OvhRxCz1JvoBjPr48YOrdz2TAPzQ/a8b8QFdU5rCNBFutlo/d3NU5rR7vbV+M4O Jbnly8dA8MFAk2fTsn+n/YALoi1Up0XJPdQnr2vxR3zBX+oVMsTDxgY0o9G5siRoUzh+ Hqpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740616399; x=1741221199; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ARvzDFB7zgeRMzJfEJ/MY3yWuG0D2YavMEjMtd8skDU=; b=inO1xhGBi4FnSqUZeLk3WXZxKiZ4Uu/WWdGk2ca/hmKc3SbaqALbCibZU0CtAe+94z XIoG1XReA4gfik0HXqB1hwmfNAOlMb1wTITKuEupWnFz1nkEpZIk83eYo/OqGd33MrHH lKzIyZPPPHiSHDA31M7jQpAh9qJ4HaUMwnrR2kdlupObwwirlNcjvlekofJ5sdm51Awb uJ7zsH9WHx05pPwAXJf3+Gr+0EPq2gHRc4c+AeBIAYnVTXjK9jPS9DDobLMn2rjYBi3K EMOqGiQ+EcppkRbSnujo7MvnV6wiHLMNn4igcUeNtbBwsy05rg+XeYiYEUfZDt7r3Jzo gR+A== X-Forwarded-Encrypted: i=1; AJvYcCUeJYotxvXAJlp0ukVJdyJvuvlC4RmL86011BecPPvZgogmK6GMO6SS4JLeM/7j5X1RjRYaFbFzWMrGiJE=@vger.kernel.org X-Gm-Message-State: AOJu0YxbAJUhZbgZMLxjdCyD/Anc3iHIIDSjBPUzxeMpdPzMbSx1Ic7l ywhsLVijqFydQoKGBlDK2j6mHjqzoOabl+wp7M4UAEWR7vyc6n+SpSmST/FZmd0XwZlPX8954Jv UVXqkrQ== X-Google-Smtp-Source: AGHT+IGb9U+m49213J1IZUi+y8W79DM6Us+fCqOcM3EVD4VeabxFV0pdhtYZr/kBDuUNuD4BTttJ023fUDlE X-Received: from ejcli4.prod.google.com ([2002:a17:907:1984:b0:abe:e707:63bf]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:d8a:b0:abb:aef3:6052 with SMTP id a640c23a62f3a-abeeef81b5emr630257266b.55.1740616399076; Wed, 26 Feb 2025 16:33:19 -0800 (PST) Date: Thu, 27 Feb 2025 00:33:07 +0000 In-Reply-To: <20250227003310.367350-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227003310.367350-1-qperret@google.com> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog Message-ID: <20250227003310.367350-4-qperret@google.com> Subject: [PATCH 3/6] KVM: arm64: Introduce {get,set}_host_state() helpers From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Instead of directly accessing the host_state member in struct hyp_page, introduce static inline accessors to do it. The future hyp_state member will follow the same pattern as it will need some logic in the accessors. Signed-off-by: Quentin Perret Reviewed-by: Marc Zyngier --- arch/arm64/kvm/hyp/include/nvhe/memory.h | 12 +++++++++++- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 14 +++++++------- arch/arm64/kvm/hyp/nvhe/setup.c | 4 ++-- 3 files changed, 20 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/memory.h b/arch/arm64/kvm/hyp/= include/nvhe/memory.h index 642b5e05fe77..4a3c55d26ef3 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/memory.h +++ b/arch/arm64/kvm/hyp/include/nvhe/memory.h @@ -42,7 +42,7 @@ struct hyp_page { u8 order; =20 /* Host (non-meta) state. Guarded by the host stage-2 lock. */ - enum pkvm_page_state host_state : 8; + unsigned __host_state : 8; =20 u32 host_share_guest_count; }; @@ -79,6 +79,16 @@ static inline struct hyp_page *hyp_phys_to_page(phys_add= r_t phys) #define hyp_page_to_virt(page) __hyp_va(hyp_page_to_phys(page)) #define hyp_page_to_pool(page) (((struct hyp_page *)page)->pool) =20 +static inline enum pkvm_page_state get_host_state(phys_addr_t phys) +{ + return (enum pkvm_page_state)hyp_phys_to_page(phys)->__host_state; +} + +static inline void set_host_state(phys_addr_t phys, enum pkvm_page_state s= tate) +{ + hyp_phys_to_page(phys)->__host_state =3D state; +} + /* * Refcounting for 'struct hyp_page'. * hyp_pool::lock must be held if atomic access to the refcount is require= d. diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index 19c3c631708c..a45ffdec7612 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -467,7 +467,7 @@ static int host_stage2_adjust_range(u64 addr, struct kv= m_mem_range *range) return -EAGAIN; =20 if (pte) { - WARN_ON(addr_is_memory(addr) && hyp_phys_to_page(addr)->host_state !=3D = PKVM_NOPAGE); + WARN_ON(addr_is_memory(addr) && get_host_state(addr) !=3D PKVM_NOPAGE); return -EPERM; } =20 @@ -496,7 +496,7 @@ static void __host_update_page_state(phys_addr_t addr, = u64 size, enum pkvm_page_ phys_addr_t end =3D addr + size; =20 for (; addr < end; addr +=3D PAGE_SIZE) - hyp_phys_to_page(addr)->host_state =3D state; + set_host_state(addr, state); } =20 int host_stage2_set_owner_locked(phys_addr_t addr, u64 size, u8 owner_id) @@ -620,7 +620,7 @@ static int __host_check_page_state_range(u64 addr, u64 = size, =20 hyp_assert_lock_held(&host_mmu.lock); for (; addr < end; addr +=3D PAGE_SIZE) { - if (hyp_phys_to_page(addr)->host_state !=3D state) + if (get_host_state(addr) !=3D state) return -EPERM; } =20 @@ -630,7 +630,7 @@ static int __host_check_page_state_range(u64 addr, u64 = size, static int __host_set_page_state_range(u64 addr, u64 size, enum pkvm_page_state state) { - if (hyp_phys_to_page(addr)->host_state =3D=3D PKVM_NOPAGE) { + if (get_host_state(addr) =3D=3D PKVM_NOPAGE) { int ret =3D host_stage2_idmap_locked(addr, size, PKVM_HOST_MEM_PROT); =20 if (ret) @@ -904,7 +904,7 @@ int __pkvm_host_share_guest(u64 pfn, u64 gfn, struct pk= vm_hyp_vcpu *vcpu, goto unlock; =20 page =3D hyp_phys_to_page(phys); - switch (page->host_state) { + switch (get_host_state(phys)) { case PKVM_PAGE_OWNED: WARN_ON(__host_set_page_state_range(phys, PAGE_SIZE, PKVM_PAGE_SHARED_OW= NED)); break; @@ -957,9 +957,9 @@ static int __check_host_shared_guest(struct pkvm_hyp_vm= *vm, u64 *__phys, u64 ip if (WARN_ON(ret)) return ret; =20 - page =3D hyp_phys_to_page(phys); - if (page->host_state !=3D PKVM_PAGE_SHARED_OWNED) + if (get_host_state(phys) !=3D PKVM_PAGE_SHARED_OWNED) return -EPERM; + page =3D hyp_phys_to_page(phys); if (WARN_ON(!page->host_share_guest_count)) return -EINVAL; =20 diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setu= p.c index d62bcb5634a2..1a414288fe8c 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -201,10 +201,10 @@ static int fix_host_ownership_walker(const struct kvm= _pgtable_visit_ctx *ctx, case PKVM_PAGE_OWNED: return host_stage2_set_owner_locked(phys, PAGE_SIZE, PKVM_ID_HYP); case PKVM_PAGE_SHARED_OWNED: - hyp_phys_to_page(phys)->host_state =3D PKVM_PAGE_SHARED_BORROWED; + set_host_state(phys, PKVM_PAGE_SHARED_BORROWED); break; case PKVM_PAGE_SHARED_BORROWED: - hyp_phys_to_page(phys)->host_state =3D PKVM_PAGE_SHARED_OWNED; + set_host_state(phys, PKVM_PAGE_SHARED_OWNED); break; default: return -EINVAL; --=20 2.48.1.658.g4767266eb4-goog From nobody Sat Feb 7 18:21:07 2026 Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E6EA51C5A for ; Thu, 27 Feb 2025 00:33:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740616404; cv=none; b=CXAA4qXBt+ojj/c1t86XaJeE+X/o67Hy2b6lQIM2FV316jmNXXB9jhap6IWZ4LR/rEs6AeX1uBZj9fFCOfwGe5srOMJd6MB9aRPFa6cBRrMqDRlFRtLTE+z6QWLfWj8QiLvrSUytmbk+Tj9yYGv2ut4Z2yYG/2zgFOGWzietLa0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740616404; c=relaxed/simple; bh=MOpMjw+yKamjIuBtGw2bZG0YkfXNv40u4X66Hipr9m8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=M3EGDwuV9AKinfxcrivpCHVLzvUpSm5ZxU5yN22L5Hr1hG16NHGHEgpjKzJlrzuCcKuaqk4fY7JTyPIHoPzrKAcjXjFgCr6KguXQRP0SZPCfST11b3flJzIQ6h03/U8NUIUh6ZWt9Q/mECreN/pnQ9DRt2OnLjgYbT/HVo/huyI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=D8ymix9c; arc=none smtp.client-ip=209.85.218.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="D8ymix9c" Received: by mail-ej1-f74.google.com with SMTP id a640c23a62f3a-abef9384a3bso26964366b.3 for ; Wed, 26 Feb 2025 16:33:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740616401; x=1741221201; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=y4cLgcKO1PUW4FpDNffQ+Ql+VwqpG2i64q1nOyfOcxQ=; b=D8ymix9cm2Ll7m7L4MH7t7VK7Tv6vW2iCXRHThNjpM9JtNvxeaixJiT/OSh7O+eF1Y IE4E+TOM+vMa6VzCAMKY1pd3da1YwkWlF12amqrxk62oda7slpzImQWPEBxwx8VbJzlz IuJj1shFrMcxOM8ejoY8HFh8+GP9kdVLTzMyc49wY2xXVWo98zsZJwnCPOtj75cU40// 6Q8VrnS9R7MrKbrYqMSiB1s2JN6THarNfGiwbuFz9VrxuRsGy12tbmq5jL8DYTiIXbs9 diLJ0mIkOb03N8T4Ca+dio6LqEKBbNMq18kvJJotRqamGE0UzJukA3stmRBWsWDdN6hQ kQ1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740616401; x=1741221201; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=y4cLgcKO1PUW4FpDNffQ+Ql+VwqpG2i64q1nOyfOcxQ=; b=IEhpzXj291GR2HL/JxygRUVRALLnHbjSDmOkqvZukQMEVG0BtEzb8wp1bq2T7CktSY WBoaOw/TcUsxg+r5eO+L05FdVXwoME8TOqIJyxViOwzI/ndUS7CKsxUuazdk/NwIs9XK RQuQv/XSggZYBITLVrY1sfJ1TCIMZ+GU40Fru6uQdaXCvurkmNep2XtupI0Ipjz5f8Ux LKY5nuzg4WvM2X+zM/gIrPwhrli6dEcO1nuryNV2eHc15CwlmPmH0Y6bWYTZTaSyY3bI IxuTgvBYXnmgB6EszvL27gK8ce03it6Imb2XG2zd7elK/bd/B7QtA2LAXgFnL3GS2nOu nMHA== X-Forwarded-Encrypted: i=1; AJvYcCXyr/y2W2sbaYzjp4fmKpG8EnAYcKwEemh5Db39pJPDn+/fowLHCbzqfb+fxdFGGStnXl1NLrkmaFm/ZQE=@vger.kernel.org X-Gm-Message-State: AOJu0YyTUtaqv7K0ggsDVS/1PboDUScBfdn2vyhyTrtYReYxldvhfReG hF9kYrGWSBuIlPKlLNsDTZIAtBwAD1OS8UywSJEEfhX/u6c8isBltQ5tHAXdqI2mPIqwEt1u5cL ZW9m3GA== X-Google-Smtp-Source: AGHT+IGWCpUoU+SAED/xQN8A5aBnRG3yloyg3hA6+NnceXAeQRYPTf0IxPy3Oss2RrNOg12kmMv5qagT6Wh9 X-Received: from ejcux9.prod.google.com ([2002:a17:907:cf89:b0:abb:7df3:8192]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:2ce4:b0:abb:bcef:837c with SMTP id a640c23a62f3a-abeeef81b48mr736438766b.56.1740616401210; Wed, 26 Feb 2025 16:33:21 -0800 (PST) Date: Thu, 27 Feb 2025 00:33:08 +0000 In-Reply-To: <20250227003310.367350-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227003310.367350-1-qperret@google.com> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog Message-ID: <20250227003310.367350-5-qperret@google.com> Subject: [PATCH 4/6] KVM: arm64: Move hyp state to hyp_vmemmap From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Tracking the hypervisor's ownership state into struct hyp_page has several benefits, including allowing far more efficient lookups (no page-table walk needed) and de-corelating the state from the presence of a mapping. This will later allow to map pages into EL2 stage-1 less proactively which is generally a good thing for security. And in the future this will help with tracking the state of pages mapped into the hypervisor's private range without requiring an alias into the 'linear map' range. Signed-off-by: Quentin Perret Reviewed-by: Marc Zyngier --- arch/arm64/kvm/hyp/include/nvhe/memory.h | 20 +++++++++- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 51 ++++++++++++------------ arch/arm64/kvm/hyp/nvhe/setup.c | 6 ++- 3 files changed, 49 insertions(+), 28 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/memory.h b/arch/arm64/kvm/hyp/= include/nvhe/memory.h index 4a3c55d26ef3..cc4c01158368 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/memory.h +++ b/arch/arm64/kvm/hyp/include/nvhe/memory.h @@ -22,6 +22,7 @@ enum pkvm_page_state { /* Meta-states which aren't encoded directly in the PTE's SW bits */ PKVM_NOPAGE =3D BIT(0) | BIT(1), }; +#define PKVM_PAGE_STATE_MASK (BIT(0) | BIT(1)) =20 #define PKVM_PAGE_STATE_PROT_MASK (KVM_PGTABLE_PROT_SW0 | KVM_PGTABLE_PROT= _SW1) static inline enum kvm_pgtable_prot pkvm_mkstate(enum kvm_pgtable_prot pro= t, @@ -42,7 +43,14 @@ struct hyp_page { u8 order; =20 /* Host (non-meta) state. Guarded by the host stage-2 lock. */ - unsigned __host_state : 8; + unsigned __host_state : 4; + + /* + * Complement of the hyp (non-meta) state. Guarded by the hyp stage-1 loc= k. We use the + * complement so that the initial 0 in __hyp_state_comp (due to the entir= e vmemmap starting + * off zeroed) encodes PKVM_NOPAGE. + */ + unsigned __hyp_state_comp : 4; =20 u32 host_share_guest_count; }; @@ -89,6 +97,16 @@ static inline void set_host_state(phys_addr_t phys, enum= pkvm_page_state state) hyp_phys_to_page(phys)->__host_state =3D state; } =20 +static inline enum pkvm_page_state get_hyp_state(phys_addr_t phys) +{ + return hyp_phys_to_page(phys)->__hyp_state_comp ^ PKVM_PAGE_STATE_MASK; +} + +static inline void set_hyp_state(phys_addr_t phys, enum pkvm_page_state st= ate) +{ + hyp_phys_to_page(phys)->__hyp_state_comp =3D state ^ PKVM_PAGE_STATE_MASK; +} + /* * Refcounting for 'struct hyp_page'. * hyp_pool::lock must be held if atomic access to the refcount is require= d. diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index a45ffdec7612..3ab8c81500c2 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -642,24 +642,24 @@ static int __host_set_page_state_range(u64 addr, u64 = size, return 0; } =20 -static enum pkvm_page_state hyp_get_page_state(kvm_pte_t pte, u64 addr) +static void __hyp_set_page_state_range(phys_addr_t phys, u64 size, enum pk= vm_page_state state) { - if (!kvm_pte_valid(pte)) - return PKVM_NOPAGE; + phys_addr_t end =3D phys + size; =20 - return pkvm_getstate(kvm_pgtable_hyp_pte_prot(pte)); + for (; phys < end; phys +=3D PAGE_SIZE) + set_hyp_state(phys, state); } =20 -static int __hyp_check_page_state_range(u64 addr, u64 size, - enum pkvm_page_state state) +static int __hyp_check_page_state_range(phys_addr_t phys, u64 size, enum p= kvm_page_state state) { - struct check_walk_data d =3D { - .desired =3D state, - .get_page_state =3D hyp_get_page_state, - }; + phys_addr_t end =3D phys + size; + + for (; phys < end; phys +=3D PAGE_SIZE) { + if (get_hyp_state(phys) !=3D state) + return -EPERM; + } =20 - hyp_assert_lock_held(&pkvm_pgd_lock); - return check_page_state_range(&pkvm_pgtable, addr, size, &d); + return 0; } =20 static enum pkvm_page_state guest_get_page_state(kvm_pte_t pte, u64 addr) @@ -687,7 +687,6 @@ int __pkvm_host_share_hyp(u64 pfn) { u64 phys =3D hyp_pfn_to_phys(pfn); void *virt =3D __hyp_va(phys); - enum kvm_pgtable_prot prot; u64 size =3D PAGE_SIZE; int ret; =20 @@ -698,13 +697,13 @@ int __pkvm_host_share_hyp(u64 pfn) if (ret) goto unlock; if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) { - ret =3D __hyp_check_page_state_range((u64)virt, size, PKVM_NOPAGE); + ret =3D __hyp_check_page_state_range(phys, size, PKVM_NOPAGE); if (ret) goto unlock; } =20 - prot =3D pkvm_mkstate(PAGE_HYP, PKVM_PAGE_SHARED_BORROWED); - WARN_ON(pkvm_create_mappings_locked(virt, virt + size, prot)); + __hyp_set_page_state_range(phys, size, PKVM_PAGE_SHARED_BORROWED); + WARN_ON(pkvm_create_mappings_locked(virt, virt + size, PAGE_HYP)); WARN_ON(__host_set_page_state_range(phys, size, PKVM_PAGE_SHARED_OWNED)); =20 unlock: @@ -727,7 +726,7 @@ int __pkvm_host_unshare_hyp(u64 pfn) ret =3D __host_check_page_state_range(phys, size, PKVM_PAGE_SHARED_OWNED); if (ret) goto unlock; - ret =3D __hyp_check_page_state_range(virt, size, PKVM_PAGE_SHARED_BORROWE= D); + ret =3D __hyp_check_page_state_range(phys, size, PKVM_PAGE_SHARED_BORROWE= D); if (ret) goto unlock; if (hyp_page_count((void *)virt)) { @@ -735,6 +734,7 @@ int __pkvm_host_unshare_hyp(u64 pfn) goto unlock; } =20 + __hyp_set_page_state_range(phys, size, PKVM_NOPAGE); WARN_ON(kvm_pgtable_hyp_unmap(&pkvm_pgtable, virt, size) !=3D size); WARN_ON(__host_set_page_state_range(phys, size, PKVM_PAGE_OWNED)); =20 @@ -750,7 +750,6 @@ int __pkvm_host_donate_hyp(u64 pfn, u64 nr_pages) u64 phys =3D hyp_pfn_to_phys(pfn); u64 size =3D PAGE_SIZE * nr_pages; void *virt =3D __hyp_va(phys); - enum kvm_pgtable_prot prot; int ret; =20 host_lock_component(); @@ -760,13 +759,13 @@ int __pkvm_host_donate_hyp(u64 pfn, u64 nr_pages) if (ret) goto unlock; if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) { - ret =3D __hyp_check_page_state_range((u64)virt, size, PKVM_NOPAGE); + ret =3D __hyp_check_page_state_range(phys, size, PKVM_NOPAGE); if (ret) goto unlock; } =20 - prot =3D pkvm_mkstate(PAGE_HYP, PKVM_PAGE_OWNED); - WARN_ON(pkvm_create_mappings_locked(virt, virt + size, prot)); + __hyp_set_page_state_range(phys, size, PKVM_PAGE_OWNED); + WARN_ON(pkvm_create_mappings_locked(virt, virt + size, PAGE_HYP)); WARN_ON(host_stage2_set_owner_locked(phys, size, PKVM_ID_HYP)); =20 unlock: @@ -786,7 +785,7 @@ int __pkvm_hyp_donate_host(u64 pfn, u64 nr_pages) host_lock_component(); hyp_lock_component(); =20 - ret =3D __hyp_check_page_state_range(virt, size, PKVM_PAGE_OWNED); + ret =3D __hyp_check_page_state_range(phys, size, PKVM_PAGE_OWNED); if (ret) goto unlock; if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) { @@ -795,6 +794,7 @@ int __pkvm_hyp_donate_host(u64 pfn, u64 nr_pages) goto unlock; } =20 + __hyp_set_page_state_range(phys, size, PKVM_NOPAGE); WARN_ON(kvm_pgtable_hyp_unmap(&pkvm_pgtable, virt, size) !=3D size); WARN_ON(host_stage2_set_owner_locked(phys, size, PKVM_ID_HOST)); =20 @@ -809,19 +809,18 @@ int hyp_pin_shared_mem(void *from, void *to) { u64 cur, start =3D ALIGN_DOWN((u64)from, PAGE_SIZE); u64 end =3D PAGE_ALIGN((u64)to); + u64 phys =3D __hyp_pa(start); u64 size =3D end - start; int ret; =20 host_lock_component(); hyp_lock_component(); =20 - ret =3D __host_check_page_state_range(__hyp_pa(start), size, - PKVM_PAGE_SHARED_OWNED); + ret =3D __host_check_page_state_range(phys, size, PKVM_PAGE_SHARED_OWNED); if (ret) goto unlock; =20 - ret =3D __hyp_check_page_state_range(start, size, - PKVM_PAGE_SHARED_BORROWED); + ret =3D __hyp_check_page_state_range(phys, size, PKVM_PAGE_SHARED_BORROWE= D); if (ret) goto unlock; =20 diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setu= p.c index 1a414288fe8c..955c431af5d0 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -194,16 +194,20 @@ static int fix_host_ownership_walker(const struct kvm= _pgtable_visit_ctx *ctx, =20 /* * Adjust the host stage-2 mappings to match the ownership attributes - * configured in the hypervisor stage-1. + * configured in the hypervisor stage-1, and make sure to propagate them + * to the hyp_vmemmap state. */ state =3D pkvm_getstate(kvm_pgtable_hyp_pte_prot(ctx->old)); switch (state) { case PKVM_PAGE_OWNED: + set_hyp_state(phys, PKVM_PAGE_OWNED); return host_stage2_set_owner_locked(phys, PAGE_SIZE, PKVM_ID_HYP); case PKVM_PAGE_SHARED_OWNED: + set_hyp_state(phys, PKVM_PAGE_SHARED_OWNED); set_host_state(phys, PKVM_PAGE_SHARED_BORROWED); break; case PKVM_PAGE_SHARED_BORROWED: + set_hyp_state(phys, PKVM_PAGE_SHARED_BORROWED); set_host_state(phys, PKVM_PAGE_SHARED_OWNED); break; default: --=20 2.48.1.658.g4767266eb4-goog From nobody Sat Feb 7 18:21:07 2026 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE3AC7E0E4 for ; Thu, 27 Feb 2025 00:33:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740616407; cv=none; b=fxEBovbmb6ZJCI0AL/l6DHL+SYdKnMHJBXnIQtcb6Mf6SF5Tr99Wyf3+Sy5SFIHncWwlnMbp94OLZwJ0huycbPDiVN291ZS5kFP/52wMtcQSTS5+j4zVmrEsRyGMR9Tvzb8KUNLZ/TXQwXnkixg3GXpMXgzZxDl6kGFx69PyiVY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740616407; c=relaxed/simple; bh=ytI77feCh85Kuzlfp0vkiYSCReLiPQumaOtjnCZz+Lw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=J6TOVJ64i67L8IWfRUGFXCDYmmFla0d+lo32JAfauooLYLtzM/qisZaRpZdr3nL2Kh4fC096oLW1sEZFHmAk5iBTtRJ0DIisoUPgzVqm/2DnFmTOmfC/hbtMp43SI9Irv8t65eqLrLuAkGbZuYOIPY5EHJ5Fgf2iPR6ejbpASRg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3UdIWWth; arc=none smtp.client-ip=209.85.208.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3UdIWWth" Received: by mail-ed1-f73.google.com with SMTP id 4fb4d7f45d1cf-5e4becb4582so349273a12.3 for ; Wed, 26 Feb 2025 16:33:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740616403; x=1741221203; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=DGmJYM3ZdXjed+VdIIVOBszfWrx1X1fG4+VBkIzQ0l4=; b=3UdIWWthex1/gAAU/yZJxS8LcwzWJ9Fm1SwcrXsa5om/XohAtVo/vA43dYiK2T3OqY 43wG235bhgxKMxpzlKAY1cLxhkPUVITy393+U75rkgqp+UxdZaqNQweBcN2+uk9yFv8/ Xrc6p899VUbSGSsOZQULb0nj1N8RxTN2NrKUmwaC6oRq8OhH7EoO4Th+YybHZOq6Wyid Hk02h+i+KZI/kllkvNKWVkhnl2H4ZSL6MVW+EtgUPyHdRYt7CC2p4g4nHx2j0+/clOjX 8J+otjvEs2rptGaxqA9f7LSaRaRff97DbTsUbKm5fKSF8dQtbPHRRRotUI3ty5r0idT6 iIpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740616403; x=1741221203; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DGmJYM3ZdXjed+VdIIVOBszfWrx1X1fG4+VBkIzQ0l4=; b=rxdnPQJJRm+WXz/m1p8VDlNqBH1PeYblHTGypqvuZSixlmMz7M62xuxr1T2Z2Vcoj0 3HtQbso11+5dl9LuJ1ON/my1TCwtsduyc7TE2Clvbbd4mNKBZ104PVaCuuSUng0XxJbM ZxT1SpO25Kh67rJwYB2JGHgZuQpRfRADTSfU19ZkD6JBKRBSlbMqu32xE8BiJI+eeSyS 16k4oilNim/6nl4KmNTX2wy6XLrSjsaPg/hdcn1sgf6gUm2vZaAXuPEkmBAboNBRQcvE ToFqPioeQW8JEPlz5+EAmkEs3yTlDk6d7/hsiLYIa+wjEowsWZkMdx/qbRHpuPDI2q2T Asbg== X-Forwarded-Encrypted: i=1; AJvYcCV7LaWE2f0ZpfE8Nu6W4gXNHYgkwSaWZrHmXWEaHM2XmWDmiBJBzUk1omVdMbk87zjZf0xWi3Qd00vUidk=@vger.kernel.org X-Gm-Message-State: AOJu0YwQzJUjvttVwPqUGy14wX/oAHignjD/pR+QP1w8SVKmGw8xuoJR qnlWowkhr1qkKSLgzGaGY23jex664rfLM0FuNWFsIiZJGeFPwcIvtkPK8Dx6NB0IaIju3+hkHUe D0BJsCw== X-Google-Smtp-Source: AGHT+IG/8xxJ0edTLXImN+8foUCiY3IiLHVItOY++3PsS94AT277t67Spxy60uaWfxFei8RcZX4NGH7I/+gI X-Received: from edbeo13.prod.google.com ([2002:a05:6402:530d:b0:5e4:befa:4847]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:280d:b0:5dc:7725:a0c7 with SMTP id 4fb4d7f45d1cf-5e44448139dmr11904494a12.3.1740616403358; Wed, 26 Feb 2025 16:33:23 -0800 (PST) Date: Thu, 27 Feb 2025 00:33:09 +0000 In-Reply-To: <20250227003310.367350-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227003310.367350-1-qperret@google.com> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog Message-ID: <20250227003310.367350-6-qperret@google.com> Subject: [PATCH 5/6] KVM: arm64: Defer EL2 stage-1 mapping on share From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We currently blindly map into EL2 stage-1 *any* page passed to the __pkvm_host_share_hyp() HVC. This is less than ideal from a security perspective as it makes exploitation of potential hypervisor gadgets easier than it should be. But interestingly, pKVM should never need to access pages that it hasn't previously pinned, so there is no need to map the page before that. Signed-off-by: Quentin Perret Reviewed-by: Marc Zyngier --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index 3ab8c81500c2..ae39d74be1f2 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -686,7 +686,6 @@ static int __guest_check_page_state_range(struct pkvm_h= yp_vcpu *vcpu, u64 addr, int __pkvm_host_share_hyp(u64 pfn) { u64 phys =3D hyp_pfn_to_phys(pfn); - void *virt =3D __hyp_va(phys); u64 size =3D PAGE_SIZE; int ret; =20 @@ -703,7 +702,6 @@ int __pkvm_host_share_hyp(u64 pfn) } =20 __hyp_set_page_state_range(phys, size, PKVM_PAGE_SHARED_BORROWED); - WARN_ON(pkvm_create_mappings_locked(virt, virt + size, PAGE_HYP)); WARN_ON(__host_set_page_state_range(phys, size, PKVM_PAGE_SHARED_OWNED)); =20 unlock: @@ -735,7 +733,6 @@ int __pkvm_host_unshare_hyp(u64 pfn) } =20 __hyp_set_page_state_range(phys, size, PKVM_NOPAGE); - WARN_ON(kvm_pgtable_hyp_unmap(&pkvm_pgtable, virt, size) !=3D size); WARN_ON(__host_set_page_state_range(phys, size, PKVM_PAGE_OWNED)); =20 unlock: @@ -811,6 +808,7 @@ int hyp_pin_shared_mem(void *from, void *to) u64 end =3D PAGE_ALIGN((u64)to); u64 phys =3D __hyp_pa(start); u64 size =3D end - start; + struct hyp_page *p; int ret; =20 host_lock_component(); @@ -824,8 +822,14 @@ int hyp_pin_shared_mem(void *from, void *to) if (ret) goto unlock; =20 - for (cur =3D start; cur < end; cur +=3D PAGE_SIZE) - hyp_page_ref_inc(hyp_virt_to_page(cur)); + for (cur =3D start; cur < end; cur +=3D PAGE_SIZE) { + p =3D hyp_virt_to_page(cur); + hyp_page_ref_inc(p); + if (p->refcount =3D=3D 1) + WARN_ON(pkvm_create_mappings_locked((void *)cur, + (void *)cur + PAGE_SIZE, + PAGE_HYP)); + } =20 unlock: hyp_unlock_component(); @@ -838,12 +842,17 @@ void hyp_unpin_shared_mem(void *from, void *to) { u64 cur, start =3D ALIGN_DOWN((u64)from, PAGE_SIZE); u64 end =3D PAGE_ALIGN((u64)to); + struct hyp_page *p; =20 host_lock_component(); hyp_lock_component(); =20 - for (cur =3D start; cur < end; cur +=3D PAGE_SIZE) - hyp_page_ref_dec(hyp_virt_to_page(cur)); + for (cur =3D start; cur < end; cur +=3D PAGE_SIZE) { + p =3D hyp_virt_to_page(cur); + if (p->refcount =3D=3D 1) + WARN_ON(kvm_pgtable_hyp_unmap(&pkvm_pgtable, cur, PAGE_SIZE) !=3D PAGE_= SIZE); + hyp_page_ref_dec(p); + } =20 hyp_unlock_component(); host_unlock_component(); --=20 2.48.1.658.g4767266eb4-goog From nobody Sat Feb 7 18:21:07 2026 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 077C113AD26 for ; Thu, 27 Feb 2025 00:33:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740616408; cv=none; b=I6Ah/A69Z6Zi2xe0AWzsN/3UPVzhhVp/HvSMGXjHnLDx5/nDgQLlOmdOOGwe7oXd6oBcgJdapMQcaGlTmrGz298M/GDKVo+ltRQXXxkjVWTDOWqikzDq0RYkuwK9M09s4jTScsfGEGV1kf6cZZt15YOTRTfA4Y/DbzRpj6YUJhc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740616408; c=relaxed/simple; bh=rp3IDhVnxkQh03oSaa4+o9TnUw4DyqtAF5BOWHgKtFI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=LJzo+17gh4n4hbLD/C3fJAYgHT/kB7h64yI6SkpUG8Auqd6gIV2ojpfRs7nylUoj7dLyaJXQ1YkSYhG9dwKtbMtXYKRrec5u95Sqky8eI/fzoLwlIdzstTyjPhxrF/jgG1ii4H2UOG72EtPEOLayOiz8ubp0v46Dph5cimle9gs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CAeRGQSR; arc=none smtp.client-ip=209.85.218.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--qperret.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CAeRGQSR" Received: by mail-ej1-f73.google.com with SMTP id a640c23a62f3a-abbf03205e6so27490066b.1 for ; Wed, 26 Feb 2025 16:33:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740616405; x=1741221205; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=go/rSw9bkr3V2Z7S0UpAa5FZKtjaGuIC5hzD90cfQFg=; b=CAeRGQSR8PfzSq/TZLd+Fr65LfN56ZjyI87GONnLbf9lbMvZMVUkiuXTV0aLh/ZBVg go2MwNSjNcBlQjnwLXji8QydsxYXGmoB7dyXpxmOMhzU6btLweuyG7uhvYQutPVM0Z+O iHiotlLJpgbDtbIOa+r5LNBSuaZotLnwh4TTem00KunxolNX1fgAvvPgg+aIv5YXUTd0 TS0cNjQih8lRaZ2/zqZDH/l+NfvcWK6fhNX9t5xW71K9rqnomue6ac2MQa/nZ4XBHGPn Y25fzAJWuIPIwHmb/2VwPNWGJ7mCYtytdOHw3JVVWErweg7Rfn5MUppDytAAb3wYCWQr 1DKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740616405; x=1741221205; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=go/rSw9bkr3V2Z7S0UpAa5FZKtjaGuIC5hzD90cfQFg=; b=GV02nUtApIxxLf6tqYt57X5izAH+aGOHewG84Y7q0C/eQfP7VurZu+qMHqQdhvNygR aVLgUaVQgH5b4v5EwkQsyuBzrFqKOiwpmzW1O7k74N7ZHNFygkMrWgBVFdVhHqyxsZAe lAfnDzL6eVubtmN4qtAT7dPJAgD179R1kifu0jd5bCa9FLf7BuECN/HAsyZppZ6v0A5v Ag8zkAD8HbWfaNHiliXEjbRS4hS1RnOwaALAejRT3t5YXP2mF1xZl6uaU2Nl8O7p+LG7 2juTD4/Fmrttp+jDXTxs8A201VlxIIxUY0BYxkEvuWO42JzArFMK/uicrodKeqs/748n Wyqw== X-Forwarded-Encrypted: i=1; AJvYcCVvaifmy1wiWgR7t7vLlNa/qIqqirF+xqaTzb4HttftHh8uYg7JjskqdvcgqKcrvPNmYxBcIPzAt4CqW0E=@vger.kernel.org X-Gm-Message-State: AOJu0Yy5ZuYK0Ygbb8uFVUDdNAHMnCaPeBZXpZdPAoy95IglaG4w3EIQ tZla+AumosypAVCjqiYIcSLXmoi2qYFWQhbN2hYNmPgqTGsrT/4pEGq/tbMmFOmm6J+RFxs09Qy ij6RkOg== X-Google-Smtp-Source: AGHT+IHoE0D/+VTxFu9mBffqWU8TmUIEm77P7VHQouobm1YsZoARZri0ZIO8rfS0b5y60ajdzwb57L0Mwq80 X-Received: from ejclm3.prod.google.com ([2002:a17:907:18c3:b0:abb:9896:c72f]) (user=qperret job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:5202:b0:5e0:8c55:4fa with SMTP id 4fb4d7f45d1cf-5e444853ef4mr25783357a12.6.1740616405565; Wed, 26 Feb 2025 16:33:25 -0800 (PST) Date: Thu, 27 Feb 2025 00:33:10 +0000 In-Reply-To: <20250227003310.367350-1-qperret@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250227003310.367350-1-qperret@google.com> X-Mailer: git-send-email 2.48.1.658.g4767266eb4-goog Message-ID: <20250227003310.367350-7-qperret@google.com> Subject: [PATCH 6/6] KVM: arm64: Unconditionally cross check hyp state From: Quentin Perret To: Marc Zyngier , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon Cc: Vincent Donnefort , Quentin Perret , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the hypervisor's state is stored in the hyp_vmemmap, we no longer need an expensive page-table walk to read it. This means we can now afford to cross check the hyp-state during all memory ownership transitions where the hyp is involved unconditionally, hence avoiding problems such as [1]. [1] https://lore.kernel.org/kvmarm/20241128154406.602875-1-qperret@google.c= om/ Signed-off-by: Quentin Perret Reviewed-by: Marc Zyngier --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 24 +++++++++--------------- 1 file changed, 9 insertions(+), 15 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index ae39d74be1f2..22a906c7973a 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -695,11 +695,9 @@ int __pkvm_host_share_hyp(u64 pfn) ret =3D __host_check_page_state_range(phys, size, PKVM_PAGE_OWNED); if (ret) goto unlock; - if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) { - ret =3D __hyp_check_page_state_range(phys, size, PKVM_NOPAGE); - if (ret) - goto unlock; - } + ret =3D __hyp_check_page_state_range(phys, size, PKVM_NOPAGE); + if (ret) + goto unlock; =20 __hyp_set_page_state_range(phys, size, PKVM_PAGE_SHARED_BORROWED); WARN_ON(__host_set_page_state_range(phys, size, PKVM_PAGE_SHARED_OWNED)); @@ -755,11 +753,9 @@ int __pkvm_host_donate_hyp(u64 pfn, u64 nr_pages) ret =3D __host_check_page_state_range(phys, size, PKVM_PAGE_OWNED); if (ret) goto unlock; - if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) { - ret =3D __hyp_check_page_state_range(phys, size, PKVM_NOPAGE); - if (ret) - goto unlock; - } + ret =3D __hyp_check_page_state_range(phys, size, PKVM_NOPAGE); + if (ret) + goto unlock; =20 __hyp_set_page_state_range(phys, size, PKVM_PAGE_OWNED); WARN_ON(pkvm_create_mappings_locked(virt, virt + size, PAGE_HYP)); @@ -785,11 +781,9 @@ int __pkvm_hyp_donate_host(u64 pfn, u64 nr_pages) ret =3D __hyp_check_page_state_range(phys, size, PKVM_PAGE_OWNED); if (ret) goto unlock; - if (IS_ENABLED(CONFIG_NVHE_EL2_DEBUG)) { - ret =3D __host_check_page_state_range(phys, size, PKVM_NOPAGE); - if (ret) - goto unlock; - } + ret =3D __host_check_page_state_range(phys, size, PKVM_NOPAGE); + if (ret) + goto unlock; =20 __hyp_set_page_state_range(phys, size, PKVM_NOPAGE); WARN_ON(kvm_pgtable_hyp_unmap(&pkvm_pgtable, virt, size) !=3D size); --=20 2.48.1.658.g4767266eb4-goog