From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B9D41CEAD8 for ; Thu, 10 Oct 2024 18:24:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584693; cv=none; b=adJL5rfLgOAJSeHM+qdvAgy3GUJK4GLOCFJDT6tJbeOM4bhfi8hELdcAy1HTOEfGKqkWo1KP2tztPC+yDS+mvjD7PNd1HZgqVWaZOvdio6iEjCBFd8GSnu6A0KQbXCgBVlOudsOorf4Z32e87crx8MX3MOvoQWY+W4BSWRNWc74= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584693; c=relaxed/simple; bh=VeqAt9xsRyQYjwTyqnYEHFfSJqQO2zWoId1ssBiaEMk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=JnCCt7zIB1e1jz15QP05RZX0DAewyf5YMAw4+VkeEXkNKaSPZ4zNXirKsd0mFIM8tWcnktAcztFuQMBpAZCoASG4TgB4AdRUsxcfHaynwNTkUaOY0S+E9xCSABZiAMJv+zONcNDa+zRUEBh3nX4VHjGRtG64Dncjv94tEJ9VX2Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=c6ulq1H/; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="c6ulq1H/" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e2de6496easo620780a91.1 for ; Thu, 10 Oct 2024 11:24:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584691; x=1729189491; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Om/x5O1cFoCu45xFQlZl7muX4OGvvwFhFfu/OqGssvQ=; b=c6ulq1H/+SVkO3snwnEpIDv6c51AZwFPT1LwqFNCVv4NoGTlUIwMtlx2CGv4OolC4z 5koadCL6jvV5FKPkcR7h2EcqLSGLIrxPFSpXWBBtr1Q7Qs67rbWfXAkZuygIXztR30jX vinelIxIHIA9mCJvKaK123LtWbg1NOboLMeiKUh0EpYwOSI95YlMnIXDtfqepFmlcWAS CnV3G1l9SuTBGmxsijDu+22rWmksmHX3fhLSJfqBck7S6jAczfD9MKD7ErhknpjpLCXD TxRiyYgzfFwgfGEmjn40VJXaAyNZX1rswlRL/erQ/34CTu9kx47GkXzSb8i1IPcncetk Akrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584691; x=1729189491; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Om/x5O1cFoCu45xFQlZl7muX4OGvvwFhFfu/OqGssvQ=; b=S5djvTD61pbQ7k41sDISzPMKujwE4kgI9AjHiBP6VrCoHsfRvZkDhvIBhzZCqWdgYv B+rlYktrE9adOv0OjgSjjBIYn7Ud3udh9C0XAj35eqISmROmpzO/9valsJBHzLgAT2Rf n3yGt0SQD7jJokXnC0l1UyRaeUpU++JcIgCsPOyWc+jEXKv/VU0CMYxyFkD1MJknOVjo CSyl0LqbyAKByxf/hrM8shISKe3TZGIECIVXNDyaI7EX6Z5wzbWNAW9mPeu4dmxDgKUt EYgIV4uJpWFWnwc/dKKGfjnnvBpWOfBlbeIh7cvgmIYMurTwpMAm0cP+hOnLan/cKCjb qqKQ== X-Forwarded-Encrypted: i=1; AJvYcCW4VMR5NXcDAZQyve3Ai8rrv/hj9P+hxgkjyUrEiw8mSrzSkEr4iEzWMrwZ4nqasJSSjoeN3LIsGAm35wA=@vger.kernel.org X-Gm-Message-State: AOJu0YzG6HHC7RypE/3FfJbt++3G3XHkzwW32Uzt10jctWQgxsPhM6rG wx0wRQrqhFDqK/lXObOIHQEXSw/nvzl9N1b20pwSiwpNFbE2TMxhBTK3FwJtRNgnFeE1/ggfzE2 RTg== X-Google-Smtp-Source: AGHT+IG89v4lIFRhnNqxe9hFvIIxqDvldHGwH//6ib2L4odxYyk8SJA2JGOjXIHuxBeh8ylp7HgA6abaBbM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:eac4:b0:2e2:8f4d:457 with SMTP id 98e67ed59e1d1-2e2c80b5015mr7266a91.2.1728584689990; Thu, 10 Oct 2024 11:24:49 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:03 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-2-seanjc@google.com> Subject: [PATCH v13 01/85] KVM: Drop KVM_ERR_PTR_BAD_PAGE and instead return NULL to indicate an error From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove KVM_ERR_PTR_BAD_PAGE and instead return NULL, as "bad page" is just a leftover bit of weirdness from days of old when KVM stuffed a "bad" page into the guest instead of actually handling missing pages. See commit cea7bb21280e ("KVM: MMU: Make gfn_to_page() always safe"). Reviewed-by: Alex Benn=C3=A9e Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/powerpc/kvm/book3s_pr.c | 2 +- arch/powerpc/kvm/book3s_xive_native.c | 2 +- arch/s390/kvm/vsie.c | 2 +- arch/x86/kvm/lapic.c | 2 +- include/linux/kvm_host.h | 7 ------- virt/kvm/kvm_main.c | 15 ++++++--------- 6 files changed, 10 insertions(+), 20 deletions(-) diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index 7b8ae509328f..d7721297b9b6 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -645,7 +645,7 @@ static void kvmppc_patch_dcbz(struct kvm_vcpu *vcpu, st= ruct kvmppc_pte *pte) int i; =20 hpage =3D gfn_to_page(vcpu->kvm, pte->raddr >> PAGE_SHIFT); - if (is_error_page(hpage)) + if (!hpage) return; =20 hpage_offset =3D pte->raddr & ~PAGE_MASK; diff --git a/arch/powerpc/kvm/book3s_xive_native.c b/arch/powerpc/kvm/book3= s_xive_native.c index 6e2ebbd8aaac..d9bf1bc3ff61 100644 --- a/arch/powerpc/kvm/book3s_xive_native.c +++ b/arch/powerpc/kvm/book3s_xive_native.c @@ -654,7 +654,7 @@ static int kvmppc_xive_native_set_queue_config(struct k= vmppc_xive *xive, } =20 page =3D gfn_to_page(kvm, gfn); - if (is_error_page(page)) { + if (!page) { srcu_read_unlock(&kvm->srcu, srcu_idx); pr_err("Couldn't get queue page %llx!\n", kvm_eq.qaddr); return -EINVAL; diff --git a/arch/s390/kvm/vsie.c b/arch/s390/kvm/vsie.c index 89cafea4c41f..763a070f5955 100644 --- a/arch/s390/kvm/vsie.c +++ b/arch/s390/kvm/vsie.c @@ -661,7 +661,7 @@ static int pin_guest_page(struct kvm *kvm, gpa_t gpa, h= pa_t *hpa) struct page *page; =20 page =3D gfn_to_page(kvm, gpa_to_gfn(gpa)); - if (is_error_page(page)) + if (!page) return -EINVAL; *hpa =3D (hpa_t)page_to_phys(page) + (gpa & ~PAGE_MASK); return 0; diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 2098dc689088..20526e4d6c62 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2664,7 +2664,7 @@ int kvm_alloc_apic_access_page(struct kvm *kvm) } =20 page =3D gfn_to_page(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); - if (is_error_page(page)) { + if (!page) { ret =3D -EFAULT; goto out; } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index db567d26f7b9..ee186a1fbaad 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -153,13 +153,6 @@ static inline bool kvm_is_error_gpa(gpa_t gpa) return gpa =3D=3D INVALID_GPA; } =20 -#define KVM_ERR_PTR_BAD_PAGE (ERR_PTR(-ENOENT)) - -static inline bool is_error_page(struct page *page) -{ - return IS_ERR(page); -} - #define KVM_REQUEST_MASK GENMASK(7,0) #define KVM_REQUEST_NO_WAKEUP BIT(8) #define KVM_REQUEST_WAIT BIT(9) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 05cbb2548d99..4b659a649dfa 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3078,19 +3078,14 @@ EXPORT_SYMBOL_GPL(gfn_to_page_many_atomic); */ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) { - struct page *page; kvm_pfn_t pfn; =20 pfn =3D gfn_to_pfn(kvm, gfn); =20 if (is_error_noslot_pfn(pfn)) - return KVM_ERR_PTR_BAD_PAGE; + return NULL; =20 - page =3D kvm_pfn_to_refcounted_page(pfn); - if (!page) - return KVM_ERR_PTR_BAD_PAGE; - - return page; + return kvm_pfn_to_refcounted_page(pfn); } EXPORT_SYMBOL_GPL(gfn_to_page); =20 @@ -3184,7 +3179,8 @@ static void kvm_set_page_accessed(struct page *page) =20 void kvm_release_page_clean(struct page *page) { - WARN_ON(is_error_page(page)); + if (WARN_ON(!page)) + return; =20 kvm_set_page_accessed(page); put_page(page); @@ -3208,7 +3204,8 @@ EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); =20 void kvm_release_page_dirty(struct page *page) { - WARN_ON(is_error_page(page)); + if (WARN_ON(!page)) + return; =20 kvm_set_page_dirty(page); kvm_release_page_clean(page); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A57F41CEEB9 for ; Thu, 10 Oct 2024 18:24:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584695; cv=none; b=A+L/qWxYJ4DfAxx+wVYSe5X3wviRg0lToH6hGSJfhLEQpcjAABXXgMcxB0ycRRBaW8uZGv8gggEdx4yg7do7sYvBAefNEpQMybhcPsEaOTHzkd3+Xo1llknZuIOeaPaJoCiIuoYpCmWAUcu2lxLaOH0stKmsR0jqqCXovkeBJjA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584695; c=relaxed/simple; bh=GKQ1+9zVatIXCwDEsM6DNkbKcmF2enzdpw2dYi1+r2s=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=auVa3MOmcK593IE1U/A3HzVDtADJou73M0JSjDvHlKQ3OU7EFLCjJx7SxSI6RWzXl/qWKl2fqxByza5hBU1V9nsCEhhTTtD+jNeP0D/LRvLOfe3s0DMYNIHSfNUXTdz/wPLohgLokNt8ac468m6BDz2uMNrX3wNVTKpXqmaNhgc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LbsR3s+e; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LbsR3s+e" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-20b6144cc2aso12164445ad.0 for ; Thu, 10 Oct 2024 11:24:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584693; x=1729189493; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=yUMe47Ku2d+lU53n+UV6eA4G9zH9pOaBnB7PUsg7n0E=; b=LbsR3s+eI6WfEnRbM2uT3vT6R0rDFtwfUQWzpB0nPc/rlV2FAnol+zYCnI0BRzeuc9 6dvAAbxGV4LpiWM+0lW7vLCrRS/LH1TIj9aZ6qsziigEi04cwRAF6RIz64LkI/oouB/x v8nk2+H9lL8+D3jP+Pb7CYde9KT6zgK6VQyFbvnUfqmJel6dqRVM0mYatQEmtKjpv3I7 n6sf8oQlLqPt2heDQ2EePF2CK5+5jZiCSmRGtjwlxxgMXznVLy1FP6MirDNLNTbto9Ys ZOZtRZ1DreJkK526CJoZ1fyGnLu5fiFs1pj6y4ni5JpaLLTAkcgHbH+XaO5ny1q8AxUS Cliw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584693; x=1729189493; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=yUMe47Ku2d+lU53n+UV6eA4G9zH9pOaBnB7PUsg7n0E=; b=mrkQXicZ4/n89f/LWKsSyF3o9zEcY7mmmJ3vM/MkGyyyMRA087mVL9ja/ZoGXNdwtN qFMVODtmoQHUlumcs0IqS8wszjrBhIyYlBoJFf8SThXy1MfB10SbckHQB9E8OoMZoC1/ 5glLDXldw4dDrIU+9DOFrc38dEE9ingjaWDbtnfNjZie3Gr/orW0ceNnupWFsmLnuBCH FuJ99vxbFtib9cLpa2A0ogkcGNYQFvdDwXGbt+m5QrQ4b57jojBlWDF2PI4lctPUVpek gZ2SPdsYhIA2faUDfRe9W3GrmdtW7np6sBweo3rceIvWxZI2VkyA2GmXiBqLJYdK/kwd DD8Q== X-Forwarded-Encrypted: i=1; AJvYcCXzdlOxeOO3KOVwc/92BQbb9dsG9ZTgQZN9B/ts6Ff4tb3+Stscy99ajNO1GbE54shW57Zrllt+ZOEvu/4=@vger.kernel.org X-Gm-Message-State: AOJu0YyvAMLrZw7NWVpU+Pk1Qf09oSRPFhrSgfOfv8t6VGGXJGbWvXnE ZrNmim5hR56M/bPLJLywOXlPvRL0J9DkRBlQyTGRKY3xXL0+jdAU51XSidQQ96hpfjWRD4B6OpM b2w== X-Google-Smtp-Source: AGHT+IHgBvu4/6UpOEaDnS659CK1Rn1/E5r8mTk00ZsUtMRUIXJafj8Ignnkw5WxXqtPYIrr50vXY4UIGHU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:ff02:b0:205:58ee:1567 with SMTP id d9443c01a7336-20c80362278mr43175ad.0.1728584692824; Thu, 10 Oct 2024 11:24:52 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:04 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-3-seanjc@google.com> Subject: [PATCH v13 02/85] KVM: Allow calling kvm_release_page_{clean,dirty}() on a NULL page pointer From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Allow passing a NULL @page to kvm_release_page_{clean,dirty}(), there's no tangible benefit to forcing the callers to pre-check @page, and it ends up generating a lot of duplicate boilerplate code. Reviewed-by: Alex Benn=C3=A9e Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 4b659a649dfa..2032292df0b0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3179,7 +3179,7 @@ static void kvm_set_page_accessed(struct page *page) =20 void kvm_release_page_clean(struct page *page) { - if (WARN_ON(!page)) + if (!page) return; =20 kvm_set_page_accessed(page); @@ -3204,7 +3204,7 @@ EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); =20 void kvm_release_page_dirty(struct page *page) { - if (WARN_ON(!page)) + if (!page) return; =20 kvm_set_page_dirty(page); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB1CD1CF7BA for ; Thu, 10 Oct 2024 18:24:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584697; cv=none; b=dKCTxD+WnxTlm/gEaaDPd84RaflOjngcMwhEFCWg4/LH11Ga5Etp68qo863+imQ7mWWPNdRBeYF5pJfZliyYZzd1x1c/uQxS8GvzBlXwRECHzgDBFA5xq6mH7ILpqi2OYLusmKKfVkX9yUGk1be1t34YLOlZAvqzhdAh0Jt1QbY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584697; c=relaxed/simple; bh=iBKdL35kBHi2YuWkjigWDtnk5mPkldKXaU7GHff52mQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uonubBpkXoXIME0P0qlY+O8o75NWNYdLPCAS7MFLAhrXPympCleMrgjulSKWBNlNw9IFUJRp/Ye54OY6CPGua5hn92ozFW4IMJOv5G3jbdwrHjIII08qq9FmzhE5Mxucmq2F2CeribrWgWwigvPc3tXV3EBiB+FfqxapEHE5GYY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YefjeM7N; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YefjeM7N" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e3245ed6b8so23817557b3.0 for ; Thu, 10 Oct 2024 11:24:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584695; x=1729189495; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=8PiI9IzFtQVAZLsCzTZ7ta7WGdR9XnSo4A5BSh1uxRg=; b=YefjeM7NlL5WJTyheSu3K4yw2vp2CkS7uAx7Z40RaC/a/T9mf5Ut339LIWtkQ2XuXQ 17CbefMXqiqBMTUljRvZMHgszGwbaPFU/tkQky5Clu/5W9+/PGYR9hUCa+QIZ1madUv6 GhU3c/TE8J0FNncSSvv8YuqrNrxaNbhW9YiVKnl29law8WUX4fWW35jK/IILY2eu6SQ7 snOZPsHIyng5WefoSyBjOo1ogSwiFFA5eg3ef7Npa6sKnfD/cEcNQbDShB+D61sKl22R AeIaMhHY8Bfyfnv2ZxOyGX3QoGGAPDZXoT3sLgxyIdmnWJQG6fwCEFa2beItHdM20glS GA2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584695; x=1729189495; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=8PiI9IzFtQVAZLsCzTZ7ta7WGdR9XnSo4A5BSh1uxRg=; b=CkmCgvoThOp31N/kW+pgyrUaPuEayKWh+95xkhAAMXtFJ+9yEkDfUzdSklODbsQc/7 yhI8ErK+1RAAmqCa6i7b+0gKxF4+dldzpNuOdbgBfiY0eFhkFMSf1J4JLtFsv1erj/xP hxmlXKtDKiL1ZbqbEEyASTaCSicRJrOZ1M8gg07Ww1TnPUTErcotrwlqZUlpp714j8cD 0kYIAO26zlIzOQd8XoyO+JC9o+o2oLWKbhXY/A5xlxdI2Bii2Z2D7aHjmWN+yk9uZQWC q6LSX1TjtjkcGoH9QRi7vTLXPKFIE9aSmGOiwNCTmsD2fDArdAwR/xvSdz3u24G6l0B6 l8Yg== X-Forwarded-Encrypted: i=1; AJvYcCVA+cz9tvAH7pViSwRboawjIrtMDNEz7P2DKERw+SHh1Rb/144iVAIRD/emzA6XOgsBmIwpv++BE7liC/Y=@vger.kernel.org X-Gm-Message-State: AOJu0YxL/dIhmfHvNofhCRES1jtzgB9gvrqaXBneFpJUQhO/Wkt6tqnd J/HnFF2qd5AmoKsTG9HCOKNuainOWWrASCjhd6O4OYmvOdJsDxP5RI+HrLo/erSpR/JyDCbFRRF BmA== X-Google-Smtp-Source: AGHT+IGz6mODxZTH5agCMYiGZey+8PwFJu50se4JqpXBNG0SwyKrN9TWGGQJqqmZYCdc2Fnc7eka1IGyLLE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a81:ad16:0:b0:6c1:298e:5a7 with SMTP id 00721157ae682-6e3224662ffmr1632867b3.5.1728584694780; Thu, 10 Oct 2024 11:24:54 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:05 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-4-seanjc@google.com> Subject: [PATCH v13 03/85] KVM: Add kvm_release_page_unused() API to put pages that KVM never consumes From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add an API to release an unused page, i.e. to put a page without marking it accessed or dirty. The API will be used when KVM faults-in a page but bails before installing the guest mapping (and other similar flows). Reviewed-by: Alex Benn=C3=A9e Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- include/linux/kvm_host.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ee186a1fbaad..ab4485b2bddc 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1216,6 +1216,15 @@ unsigned long gfn_to_hva_prot(struct kvm *kvm, gfn_t= gfn, bool *writable); unsigned long gfn_to_hva_memslot(struct kvm_memory_slot *slot, gfn_t gfn); unsigned long gfn_to_hva_memslot_prot(struct kvm_memory_slot *slot, gfn_t = gfn, bool *writable); + +static inline void kvm_release_page_unused(struct page *page) +{ + if (!page) + return; + + put_page(page); +} + void kvm_release_page_clean(struct page *page); void kvm_release_page_dirty(struct page *page); =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E14061D041A for ; Thu, 10 Oct 2024 18:24:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584699; cv=none; b=J0SA1GQ4MPFjhUGXpN04GvwLekriZYc06iWNTaH6kf5Zlj1JaJqxVyAlDl93oRZuwCXrgpgz/hhNEMrgFkBXYv8VJmosdQOQ+01oEOeqBOFmppasHTuzO0LWMVDsyhvcz1cc9/ZNLRyyVonQF5ZdM3iR02GNj4VFzPsKBX2VymY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584699; c=relaxed/simple; bh=/+UXWBjoqTQ+9Qx4e881Uho0Hr1R01lQsdRPnyHLl9M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Qlu/VQHnx+cINKpJrZg/JMUtqkwpJaXChuzMPohdU6av0cSmEdShrC8chyXSsoFOHlNA7vOL4RJgljdtF5bb4QyTvXrsFCpZf+rlpwOZnSeBXiLr9huIliFrrVxQndL9LDL1/PglcY1V2EGG86D5ta9iXzhlL3x7a0wUobRbvS4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=f+oxcvBw; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="f+oxcvBw" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-690404fd230so25155627b3.3 for ; Thu, 10 Oct 2024 11:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584697; x=1729189497; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=f+oxcvBwyIf0CPK4Z3H2v2Mp1oKcg8M4xtBPnL3mEbTKwPFddA2gYnQvOwqwhMPqcs 6tTJPsf3R4LXc1m3EABExF2dXv+LHXuszre/1aFscEjO6qmtW3xxqqIATZx3rvdG8rsT 1J3bs69TevQpiRIDsRnNKqiW/W7gMd1WhJDESUTWDMC5TzfnghveyjVNiiFf2Y1HjFiQ vPehYSYcIUDZNDcFnYRqNJsZTdr7XltICzxxbDvDzqP29NLHv3HJTa/ykSCOdCs7Xdvo siw+q8s3hy1UsZky6mk+tSYSCcSAXsDzMU8KrWmL1toG+e2nbzaxdcSmAXg/WDWx5nyc 3r6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584697; x=1729189497; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=EgVEBqqH9OZc/bIigFGZk1JbUEM33npBXbO8sAOHol/SbmYti7/mROj3b2a7+4F+49 XPh2xNmu6PSTv0uqnzO0yEaVTpywebRuqY5ErhVqeq5D6IiH4nPWAgiWb9NHMeogLQWR PXYuaLRZQdffamwa82rqCKAcbCVKudRmGtbF02bpCfS7J+3bkRB49BRh0yo8x4SykDJ9 vAP37uRq0r0DrWP0VUUWX/XYm2hv1+skzTECbMtQ4IDqV7mqWz0ezXzFESr0brK2Na2g LJwVJufczYeQ4EPth+jHsYNtDKVaT9NXQI/m8+E3gUWvMeEMmXwjgjUpwo3OCq8Pt/q/ 2qKA== X-Forwarded-Encrypted: i=1; AJvYcCWqC1cIDODv88E6mZX9OkrE1F+S2dym8ieeinIH3fTXGtMyG2ICMm4xnJq9rhpQo/t9quzjaTjye2w8DNY=@vger.kernel.org X-Gm-Message-State: AOJu0Yxo+UGX6dR24ikrHkxDLVEFCctZanPcdJQBC6k5T6VFsGRhXuF7 /dlkTAneJvEcC8EaYEzxqwOTH1DyEl/hdtp7/8QpfuPErkAfPvpUJVgrHn6XCng3vL2nNeHGrVS 27A== X-Google-Smtp-Source: AGHT+IGrULPD5dT5NWC9dFAfgTsoL9JIWGzHU9t8+6e7flVyO5QHPCRlr/jDUZCu+nhXBHYl+WGzejB3KuQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a0d:e486:0:b0:6e2:371f:4aef with SMTP id 00721157ae682-6e322168931mr204147b3.3.1728584696651; Thu, 10 Oct 2024 11:24:56 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:06 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-5-seanjc@google.com> Subject: [PATCH v13 04/85] KVM: x86/mmu: Skip the "try unsync" path iff the old SPTE was a leaf SPTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Apply make_spte()'s optimization to skip trying to unsync shadow pages if and only if the old SPTE was a leaf SPTE, as non-leaf SPTEs in direct MMUs are always writable, i.e. could trigger a false positive and incorrectly lead to KVM creating a SPTE without write-protecting or marking shadow pages unsync. This bug only affects the TDP MMU, as the shadow MMU only overwrites a shadow-present SPTE when synchronizing SPTEs (and only 4KiB SPTEs can be unsync). Specifically, mmu_set_spte() drops any non-leaf SPTEs *before* calling make_spte(), whereas the TDP MMU can do a direct replacement of a page table with the leaf SPTE. Opportunistically update the comment to explain why skipping the unsync stuff is safe, as opposed to simply saying "it's someone else's problem". Cc: stable@vger.kernel.org Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/spte.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 8f7eb3ad88fc..5521608077ec 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -226,12 +226,20 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_= page *sp, spte |=3D PT_WRITABLE_MASK | shadow_mmu_writable_mask; =20 /* - * Optimization: for pte sync, if spte was writable the hash - * lookup is unnecessary (and expensive). Write protection - * is responsibility of kvm_mmu_get_page / kvm_mmu_sync_roots. - * Same reasoning can be applied to dirty page accounting. + * When overwriting an existing leaf SPTE, and the old SPTE was + * writable, skip trying to unsync shadow pages as any relevant + * shadow pages must already be unsync, i.e. the hash lookup is + * unnecessary (and expensive). + * + * The same reasoning applies to dirty page/folio accounting; + * KVM will mark the folio dirty using the old SPTE, thus + * there's no need to immediately mark the new SPTE as dirty. + * + * Note, both cases rely on KVM not changing PFNs without first + * zapping the old SPTE, which is guaranteed by both the shadow + * MMU and the TDP MMU. */ - if (is_writable_pte(old_spte)) + if (is_last_spte(old_spte, level) && is_writable_pte(old_spte)) goto out; =20 /* --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBD1E1E2029 for ; Thu, 10 Oct 2024 18:25:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584703; cv=none; b=rqLreTi3IQZW/LxV5Fw4hH7kc13w37MUtj9w/O4OOu9OMq/Je8axow9J8MVeGqEtVN8pPTxDm0XJi8HHeJ70NFYyWBbOCFoIyzgQL7wO9hFypkW2FxC67YxVstnZaptQdaNYghsyXhXWWqAzHHW2ng78eJTP4H2FE9upSPp412Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584703; c=relaxed/simple; bh=uAiI5uzFAMXWntu4OPmR0EhFBczJTAI1tjFERMQUX4o=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WGWG7BiKRhcE98heRO2hbluyTADLHOQsuba4CaXFTMovUGh9RX87Vfk3yowLQCAaqeK2WQcATk2SBfaAJR/mQey1C3BCL0NE61uDzCFeCfxzi6kcwrsyCRFmvtHOVG3fLJXebPeCUpfaFiCboqLtnI26uXB6xmLaP3w/1mkz1E4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=zZnV9RMn; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="zZnV9RMn" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e32b43e053so18000127b3.1 for ; Thu, 10 Oct 2024 11:25:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584701; x=1729189501; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=pPAj0F/7+N3G0fgp8rSBmKqwV8uR3Qu6XgX2gyD+fUo=; b=zZnV9RMn+Omcj8cSLLQ1OO0nfSJBegsKEALf2M2ho0vTZxhMugmk84GGd9BSEy8lfi 5bXdzgRyXEp9OOV73dHe9K+/mvumfArmQlwdAIIf098Xo0lDHqhVeyJufcDy6upSTUCg eJa0l/phgkxFHgmSe5dUQRSSAhpuDVME8CPxLTUOLKDPpIH8StTUDV6Gj9Aj32V4pyB7 hUXbYov9HOCq3yRTRJegHiHEPel6bi7MvYy78Fmk4PNgStPCSNCbeQWI874/CL75Nylh b8mC6E7bFg+6jKx8vU6UyFAv3zFXpZ6ePXNmjV/tqGGnu0kMfhBsyo2PJWG6sUybPPoq 1hOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584701; x=1729189501; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pPAj0F/7+N3G0fgp8rSBmKqwV8uR3Qu6XgX2gyD+fUo=; b=eWbtZ3/edZgo0N+d0jdw0GHq253h3NbNkgLD5nkXzZM903GqtlxHD04nKmupIFM4VZ Zvg9IfUsqBEFqRCTlw3hd7yLjez1nITB4dLi8iscf0JiwJjYS6EXQmWJbBLe9vgtyEQo oitd9vzoljxyygBu0LQLZsSua0hu02rPs4xEHSiAv5vuCUABeYkxVQjZhNHKY3FHFa5e kG8H3uyLk53rHNBOXabQnzmfh5YlBA65PlKZp0Bw1wieTTtOlDx4CJxG7N+EkMZgfnkf WraH6eBdItqrfLKdi8E72uBFKPRiLCB3RFl7QJ2DVzxjoAqRo/ABLbyHvjmhMW+O8whg EyVg== X-Forwarded-Encrypted: i=1; AJvYcCWPUnHhV7aSrIjMXBeiFnXRz7W82iIyWmrRp+5cLkbaiHeoFwcDBX3EdA4OvJvtNMZ1IHhX8hoiexRClxc=@vger.kernel.org X-Gm-Message-State: AOJu0YwPUS4hHsEdKOjkIAaD8mxS6eSaqyMLo0PX2Do3VM2YjV0Nm9yK ho1LZHAMRkmOfuoXYf3mbnWmRnxIxfJMCmXRSqiRMmRdwjFbbPTvtv7sgkyVJ57YxO97yFvtHnt w5w== X-Google-Smtp-Source: AGHT+IHQgMXiDY3ifHkkle2NgOnYNSR5nXtws6LnwLsJ4J6Fqip1dtdD5j5lAqBgp/4MJdCcEXFALInKEMI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:69c3:0:b0:e1a:9ed2:67f4 with SMTP id 3f1490d57ef6-e28fe4313ffmr4949276.2.1728584700890; Thu, 10 Oct 2024 11:25:00 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:07 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-6-seanjc@google.com> Subject: [PATCH v13 05/85] KVM: x86/mmu: Don't overwrite shadow-present MMU SPTEs when prefaulting From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Treat attempts to prefetch/prefault MMU SPTEs as spurious if there's an existing shadow-present SPTE, as overwriting a SPTE that may have been create by a "real" fault is at best confusing, and at worst potentially harmful. E.g. mmu_try_to_unsync_pages() doesn't unsync when prefetching, which creates a scenario where KVM could try to replace a Writable SPTE with a !Writable SPTE, as sp->unsync is checked prior to acquiring mmu_unsync_pages_lock. Note, this applies to three of the four flavors of "prefetch" in KVM: - KVM_PRE_FAULT_MEMORY - Async #PF (host or PV) - Prefetching The fourth flavor, SPTE synchronization, i.e. FNAME(sync_spte), _only_ overwrites shadow-present SPTEs when calling make_spte(). But SPTE synchronization specifically uses mmu_spte_update(), and so naturally avoids the @prefetch check in mmu_set_spte(). Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 3 +++ arch/x86/kvm/mmu/tdp_mmu.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a9a23e058555..a8c64069aa89 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2919,6 +2919,9 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct= kvm_memory_slot *slot, } =20 if (is_shadow_present_pte(*sptep)) { + if (prefetch) + return RET_PF_SPURIOUS; + /* * If we overwrite a PTE page pointer with a 2MB PMD, unlink * the parent of the now unreachable PTE. diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 3b996c1fdaab..3c6583468742 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1026,6 +1026,9 @@ static int tdp_mmu_map_handle_target_level(struct kvm= _vcpu *vcpu, if (WARN_ON_ONCE(sp->role.level !=3D fault->goal_level)) return RET_PF_RETRY; =20 + if (fault->prefetch && is_shadow_present_pte(iter->old_spte)) + return RET_PF_SPURIOUS; + if (unlikely(!fault->slot)) new_spte =3D make_mmio_spte(vcpu, iter->gfn, ACC_ALL); else --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E84F1E3784 for ; Thu, 10 Oct 2024 18:25:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584707; cv=none; b=VqWgFERg5FfjjHGKuQ9JEilMWPQZsUsMY6JtqVo1r4CKkNkH9Yyut914k4TKS8pJgOeyEtAYfFn+vNBIYPWjaXw7VvLBWPgV8D9irMaLthQ2TsEGAFEwUifYjivDCCXivSE8bYyM5W8BGIIxqb76OZqhNR1mB4Nv8RZ9ATtSV3M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584707; c=relaxed/simple; bh=V8Pz/YRM4TKMSvyoWeXbxNdYc7yxSwxWSxDHm25abTE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=D4323BWlNDUSce7u8kPBJCAXE8YnUSePpl1h9PHKJZqMXPBZd7DA6Oq2iVArI95fFhvxOvjoGUQ6qBaeEwS0DczX1Xpy+iV8Y2Xv/6XzeCPGzELkEukjPeN0W8f92ahTpqzi7+OVk4RRN7b55fR0m6bx0OduN8MgEjCuDvy50M0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=2Sdd9oap; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="2Sdd9oap" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e165fc5d94fso1874979276.2 for ; Thu, 10 Oct 2024 11:25:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584704; x=1729189504; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=zmCOkT+wuJeT9H1MjXHCrAkGhLQoiMWK376kHF4kwCg=; b=2Sdd9oap+q0MyhyBJfN/XXaLaeugLyBeVKfhNG2OdkEb/qrt0NkuxHYVoDsDdUZ9e1 pmV7SmQhjF9bAwgYB0J8zvebqLiGSyv20F8v9j0thW0q6ArX1s4TwXm8L3IHPzem3f8y DwxbUzTbCpK28hZiB1VxvTVS9uIMZOjdMOhgijGcTvQ/wCKV0SUyqlU1dK5YJ8IjBDbg b/lM5r6hSEmGcKJbLuF7of6iZwO86ElPJvMvme5CnHpYMePTLHn6R0KeLtgTbtl3JJy2 5t78epXtg7vnXBYurXwLP6RwMrqQnpppEpVK40dF3oMv/Bjakbx1TqmR49X4kE7/FaxV fqbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584704; x=1729189504; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zmCOkT+wuJeT9H1MjXHCrAkGhLQoiMWK376kHF4kwCg=; b=q2nwsldDeiihld+9mJ/azQOU3KPndXRNZfHj/MOrNqfr9SpMF4+c8mV7zdbwdZOW/7 zrgYQPDt36vhJ+zjzRsoLaD9JZ9zgMLidMTQpScMqDPaI+8koesrEOkfP73ItuulkAPk E525kqVvw+28ie0eQPyXlmffXI8hf7zHDannvrMHo/6laWMaiVtJt6WgW0Kml2z0KR+0 O0DdfS3vnf2Gga2i5Q92tZZIOChoEQ8wYcj4hjyU6mAGr7GYdgov2IqDDwdPSIlrUdOR d29cMO3ulLfdRk3SZ3yJDaMSqKw+1gPIU1vnP6eprJWXxoatZRDKH7M9gW3AUesZFR6a PA2A== X-Forwarded-Encrypted: i=1; AJvYcCVPWaUcABuyDGWYR61oEO2+roxRCEqlcFAaVpyTSuubO8hkbkcbYIJdj9uf+2qo1LWxD9gsDqOlS337GkY=@vger.kernel.org X-Gm-Message-State: AOJu0Yz8D9QLhnprILH8Ub7WAPQdOfDbaDl3cd6LBAC5ccUPw2o36u3U k/Czg1uxjruL3eoSOQEYb09xOKvxjwEfYwwd7vcyTwM9iFoKEpLfnwX4h1T55nsmcBOpeVvNoEK Cxw== X-Google-Smtp-Source: AGHT+IHLS32ZnUy4bY0gdVqRjJSlR5N1zmb3UxYk5drKGwVtxhq/Y8NIZIwMp1duovDzYwXPQsOfO5WBu4o= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6902:1812:b0:e24:9f58:dd17 with SMTP id 3f1490d57ef6-e28fe32f042mr66754276.1.1728584702932; Thu, 10 Oct 2024 11:25:02 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:08 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-7-seanjc@google.com> Subject: [PATCH v13 06/85] KVM: x86/mmu: Invert @can_unsync and renamed to @synchronizing From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Invert the polarity of "can_unsync" and rename the parameter to "synchronizing" to allow a future change to set the Accessed bit if KVM is synchronizing an existing SPTE. Querying "can_unsync" in that case is nonsensical, as the fact that KVM can't unsync SPTEs doesn't provide any justification for setting the Accessed bit. Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 12 ++++++------ arch/x86/kvm/mmu/mmu_internal.h | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/spte.c | 4 ++-- arch/x86/kvm/mmu/spte.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 4 ++-- 6 files changed, 13 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a8c64069aa89..0f21d6f76cab 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2795,7 +2795,7 @@ static void kvm_unsync_page(struct kvm *kvm, struct k= vm_mmu_page *sp) * be write-protected. */ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot = *slot, - gfn_t gfn, bool can_unsync, bool prefetch) + gfn_t gfn, bool synchronizing, bool prefetch) { struct kvm_mmu_page *sp; bool locked =3D false; @@ -2810,12 +2810,12 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const = struct kvm_memory_slot *slot, =20 /* * The page is not write-tracked, mark existing shadow pages unsync - * unless KVM is synchronizing an unsync SP (can_unsync =3D false). In - * that case, KVM must complete emulation of the guest TLB flush before - * allowing shadow pages to become unsync (writable by the guest). + * unless KVM is synchronizing an unsync SP. In that case, KVM must + * complete emulation of the guest TLB flush before allowing shadow + * pages to become unsync (writable by the guest). */ for_each_gfn_valid_sp_with_gptes(kvm, sp, gfn) { - if (!can_unsync) + if (synchronizing) return -EPERM; =20 if (sp->unsync) @@ -2941,7 +2941,7 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct= kvm_memory_slot *slot, } =20 wrprot =3D make_spte(vcpu, sp, slot, pte_access, gfn, pfn, *sptep, prefet= ch, - true, host_writable, &spte); + false, host_writable, &spte); =20 if (*sptep =3D=3D spte) { ret =3D RET_PF_SPURIOUS; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index c98827840e07..4da83544c4e1 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -164,7 +164,7 @@ static inline gfn_t gfn_round_for_level(gfn_t gfn, int = level) } =20 int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot = *slot, - gfn_t gfn, bool can_unsync, bool prefetch); + gfn_t gfn, bool synchronizing, bool prefetch); =20 void kvm_mmu_gfn_disallow_lpage(const struct kvm_memory_slot *slot, gfn_t = gfn); void kvm_mmu_gfn_allow_lpage(const struct kvm_memory_slot *slot, gfn_t gfn= ); diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index ae7d39ff2d07..6e7bd8921c6f 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -963,7 +963,7 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, stru= ct kvm_mmu_page *sp, int host_writable =3D spte & shadow_host_writable_mask; slot =3D kvm_vcpu_gfn_to_memslot(vcpu, gfn); make_spte(vcpu, sp, slot, pte_access, gfn, - spte_to_pfn(spte), spte, true, false, + spte_to_pfn(spte), spte, true, true, host_writable, &spte); =20 return mmu_spte_update(sptep, spte); diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 5521608077ec..0e47fea1a2d9 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -157,7 +157,7 @@ bool spte_has_volatile_bits(u64 spte) bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, const struct kvm_memory_slot *slot, unsigned int pte_access, gfn_t gfn, kvm_pfn_t pfn, - u64 old_spte, bool prefetch, bool can_unsync, + u64 old_spte, bool prefetch, bool synchronizing, bool host_writable, u64 *new_spte) { int level =3D sp->role.level; @@ -248,7 +248,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_pa= ge *sp, * e.g. it's write-tracked (upper-level SPs) or has one or more * shadow pages and unsync'ing pages is not allowed. */ - if (mmu_try_to_unsync_pages(vcpu->kvm, slot, gfn, can_unsync, prefetch))= { + if (mmu_try_to_unsync_pages(vcpu->kvm, slot, gfn, synchronizing, prefetc= h)) { wrprot =3D true; pte_access &=3D ~ACC_WRITE_MASK; spte &=3D ~(PT_WRITABLE_MASK | shadow_mmu_writable_mask); diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 2cb816ea2430..c81cac9358e0 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -499,7 +499,7 @@ bool spte_has_volatile_bits(u64 spte); bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, const struct kvm_memory_slot *slot, unsigned int pte_access, gfn_t gfn, kvm_pfn_t pfn, - u64 old_spte, bool prefetch, bool can_unsync, + u64 old_spte, bool prefetch, bool synchronizing, bool host_writable, u64 *new_spte); u64 make_huge_page_split_spte(struct kvm *kvm, u64 huge_spte, union kvm_mmu_page_role role, int index); diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 3c6583468742..76bca7a726c1 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1033,8 +1033,8 @@ static int tdp_mmu_map_handle_target_level(struct kvm= _vcpu *vcpu, new_spte =3D make_mmio_spte(vcpu, iter->gfn, ACC_ALL); else wrprot =3D make_spte(vcpu, sp, fault->slot, ACC_ALL, iter->gfn, - fault->pfn, iter->old_spte, fault->prefetch, true, - fault->map_writable, &new_spte); + fault->pfn, iter->old_spte, fault->prefetch, + false, fault->map_writable, &new_spte); =20 if (new_spte =3D=3D iter->old_spte) ret =3D RET_PF_SPURIOUS; --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 46F8D1E490B for ; Thu, 10 Oct 2024 18:25:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584709; cv=none; b=YB92T3JFAPogJtJEhL7upiPGH/4GQWrGU3IEtwStaIzVzcL/4lW7FS1hcrFeTmMGhCKUgjmXe+HPfBD8QmigFkZ4teGzlXz+c1RFGvVq8ziKLXrX+/haYHPLugLvtCee08NI7vZG6k+bFNyDpECS8Hpq4zONy4MfCBJbCZ262Ew= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584709; c=relaxed/simple; bh=zZ6qnR3p2Ej0Et0EkgjdDF1APVKDfMggaSwqeFMbAB0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=s7gTXsV6tlu8MsBR1ho/V/tXfIAMVmQfFzV9/EJen1Pu/MkTFYCdmPkcMxTVVES+8o/F7LfjmQbEbCMF0/o0MkrB0KCjsBrI1pnfT0ls7TqrsZhyjWsXSO/MfZZdRIgwCn926s3JGPyS5fESklqkJgtelb/+riVfiDS4oTYhB4U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=IDkmO6HR; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="IDkmO6HR" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e295559c37so1096521a91.0 for ; Thu, 10 Oct 2024 11:25:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584707; x=1729189507; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=GZoCfd92gqYHwqsSYOai9KQfQ7K6dxh+JWKccmiJTPU=; b=IDkmO6HRVmabgOlLjqCh29oQFT1bfBQbkp3F1tpeC5Cl5RH4yKJjEzu/fRdCPtY2LJ 8alxmLe/e6YwD8u37nx2cwxVWQGc05WHH8rhdocEO3m0ZILY2McAVy6C7ivAMxZD63lA 5rm83irlLcGgCTPE12ygB+5N8x12ZhG5hm21exX/zmXXDmPJm1UnrJS16qHVBYfIHr2O AobpACM+F+Sg7RjUSd+L/3V/uhbi11gyOc4mNHGNjscaAcXfUW6mW3Qzc8IqRRoIOJrz mQEaeoC3Oe07Txd1htL8K8SnhqjKEfbT3WNRYv4Ca+fTxlJk2zb0kfgWcERlJ9Sm7PG8 lEzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584707; x=1729189507; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=GZoCfd92gqYHwqsSYOai9KQfQ7K6dxh+JWKccmiJTPU=; b=WFOGAnuQbfTHF1kh4ikIH2+c0wGKjiMxMpmSwX5E+TTgzHlj6rOO4FpvTZDQTWzGIH uXu4Zs30j8pgrWk2118BBghHGP4RpaBJdPttMbJKDrqfFEzSifVqUUXVhBKvoy2VPmMg 5ASxBmqFZUR1I/qGImBA8yQY3tcuAOTOTVdlYLzDlEsIk9o/ANt0FAxYjIbpIdqRIRHp JOOUx9JaP3JSppvWcA57uP/vy/oLYl2+171zLyTXfWIdkg0xi1WRpB062gFKaQZQXQY8 YZtdtaYNBfMLyHaA5HyrLOJQmKJwVJ8fHb7NggSuOdc5hu6y29uVEEuw8Bz3HH1Uc+iV FUXw== X-Forwarded-Encrypted: i=1; AJvYcCUI8KD55XIHSwolEtnP9zgFwHBGMW8cX1WzEKH/TM0zhuNQyxSAN/DLKx+vj3S7j3yxqVCuw/dy67e+1yQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yx1o+bUYtZFz7MhoLFqQEZPpQwJYm2MYXiXhu51doeRl0ihDaXW sI63fhwlIr/i8dITQpBpXTLoJSd8IRnC4IW+ItUr9W6nWaKj4bmG3zuCQVK3ZWmBglF1cDAyuM2 PrA== X-Google-Smtp-Source: AGHT+IH8UDk7ruln14ZDsSWqbj1uDcx7028dIle1wpc1As3ec3XNWN4cizl8I1jAsp9rsgJdiTjtCNc1iNw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:fa84:b0:2e2:da81:40c6 with SMTP id 98e67ed59e1d1-2e2f0a524d4mr44a91.2.1728584705592; Thu, 10 Oct 2024 11:25:05 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:09 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-8-seanjc@google.com> Subject: [PATCH v13 07/85] KVM: x86/mmu: Mark new SPTE as Accessed when synchronizing existing SPTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Set the Accessed bit when making a "new" SPTE during SPTE synchronization, as _clearing_ the Accessed bit is counter-productive, and even if the Accessed bit wasn't set in the old SPTE, odds are very good the guest will access the page in the near future, as the most common case where KVM synchronizes a shadow-present SPTE is when the guest is making the gPTE read-only for Copy-on-Write (CoW). Preserving the Accessed bit will allow dropping the logic that propagates the Accessed bit to the underlying struct page when overwriting an existing SPTE, without undue risk of regressing page aging. Note, KVM's current behavior is very deliberate, as SPTE synchronization was the only "speculative" access type as of commit 947da5383069 ("KVM: MMU: Set the accessed bit on non-speculative shadow ptes"). But, much has changed since 2008, and more changes are on the horizon. Spurious clearing of the Accessed (and Dirty) was mitigated by commit e6722d9211b2 ("KVM: x86/mmu: Reduce the update to the spte in FNAME(sync_spte)"), which changed FNAME(sync_spte) to only overwrite SPTEs if the protections are actually changing. I.e. KVM is already preserving Accessed information for SPTEs that aren't dropping protections. And with the aforementioned future change to NOT mark the page/folio as accessed, KVM's SPTEs will become the "source of truth" so to speak, in which case clearing the Accessed bit outside of page aging becomes very undesirable. Suggested-by: Yan Zhao Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/spte.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 0e47fea1a2d9..618059b30b8b 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -178,7 +178,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_pa= ge *sp, spte |=3D SPTE_TDP_AD_WRPROT_ONLY; =20 spte |=3D shadow_present_mask; - if (!prefetch) + if (!prefetch || synchronizing) spte |=3D spte_shadow_accessed_mask(spte); =20 /* @@ -259,7 +259,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_pa= ge *sp, spte |=3D spte_shadow_dirty_mask(spte); =20 out: - if (prefetch) + if (prefetch && !synchronizing) spte =3D mark_spte_for_access_track(spte); =20 WARN_ONCE(is_rsvd_spte(&vcpu->arch.mmu->shadow_zero_check, spte, level), --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 434ED1CDA31 for ; Thu, 10 Oct 2024 18:25:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584712; cv=none; b=eLM8UOvOll3b/l4nwnMFC/CyovnoRpi6ymy1EeYDhr0xVR+D3B9DSGclh3YA70skC+Gavq6fa9pf46J7kkIxk9ZJzZjgWaIpP4NnSJ9g7aVzfhcTWz83o/mTv2Zdko1/3cBq0jNbUCBCPAO3+ON2Gh1/tBhfZbjgiOBuaLLOsco= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584712; c=relaxed/simple; bh=VkZarLC+Eu//G+0PeVKNSmckYM6o8fYMsW2puWWES3Q=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=g20mxCQdv9FB44HkEXLAu5CiK5igxrY5WF6tKBaTMSgySAdqTfaRXpNmEnY9vwOK1eT53/U/DGPwxn59iq/uhT9oMegsJONMFftXgE77tm78aDnMMQLvmlKdMSHwVEdyRTiS519GQQ18LBz4lyWyV08prz42PzLMpw48C/J12yo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Kbknugbz; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Kbknugbz" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7c6a4d99dbdso297127a12.0 for ; Thu, 10 Oct 2024 11:25:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584709; x=1729189509; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=e3d7DBw/sg5cuh+IKhcM0BkpGN0EJ0d/2DHG844pD6k=; b=KbknugbzdNUXpNoKARsIRLpx8PRqlV/FblB8LCRY3jXCEL8YQWp2+5e1SPcnpU1anA /Un1XGepnDYvRE8uMn1M8sNXDKThkOYuLMTMY4dvsf3kPUPmzir5ncOXQFn7+bv1p4hy H2jRB4kxAnETZzMSlESoEbGU/tyiL4jS3pm3WzUHXkieibovGZ1NPgTsV7+P+Cesmr1y Fs2xH6iupE8taJ7d1TWULgnebCuHG6dRZWs4+QPnv+cXQLB2p6C1WBwO4XLulqfa6dO3 cUBm6987ZwMn/Cpfa5BrC69Oqg5CqfZ6FFkwpWMqFmwwGgxbzBJ359J9aVrp/AaSSVAN 8DSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584709; x=1729189509; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=e3d7DBw/sg5cuh+IKhcM0BkpGN0EJ0d/2DHG844pD6k=; b=H++Q92TVWPdEeEXDWlzOtjEs00M7SK9lBKOTAfHaxI+Kt1oSa8MBJdfmt72oS4G83N 08Z18WaDIpX0o9XKZpSQbwJephifD3gXMHCf8KbZeQg2fs/QnEQQ1yfB35cJKiRe3dwe vV8ojQBuFWTEjqv9CVAloZKaaiPF+cRxtXEz05FcoLonZim1DLttJIpfrK487SWngpBK 66DN1/sUC2Q+e2paBEbPyEfmUljxjrcq4dypBdmKr2mm/1YNQWlXMGiml89bSLyfT9To fX3WeVufgfvSEc7l89rGRHgv6l51ImM8zAlrNLGzt8EANE74z2D3kr0Zwp2fkaHj67IL bHcw== X-Forwarded-Encrypted: i=1; AJvYcCXDg3kdB17mbw2o5I6o9G/cLMI7kbUO8wvxeBOYEe3BXko0cxTdIgplxl1jH0kXLABQ/jo3gXSo2WOCfHs=@vger.kernel.org X-Gm-Message-State: AOJu0Yx39GFk5xlamDL4Lrc1ed0N+0XZvL4XN03XtjH5hJKH8wtP5LGk eLQsjm9/3u5wlLm0uEhki323mcaVUdDke+QaKW9XIJb8sVbbkop7WEcKKZA/S3Q2eXPH3ZzqE81 ong== X-Google-Smtp-Source: AGHT+IFvOj1YmixVIm2giqrxikolLV70YR7/zWjxlJNmhfeSXdlwSACqKd64r4Is1alzH3q275ZQuugRXZo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90b:153:b0:2e2:af66:c33e with SMTP id 98e67ed59e1d1-2e2f0ae73f3mr37a91.1.1728584708343; Thu, 10 Oct 2024 11:25:08 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:10 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-9-seanjc@google.com> Subject: [PATCH v13 08/85] KVM: x86/mmu: Mark folio dirty when creating SPTE, not when zapping/modifying From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark pages/folios dirty when creating SPTEs to map PFNs into the guest, not when zapping or modifying SPTEs, as marking folios dirty when zapping or modifying SPTEs can be extremely inefficient. E.g. when KVM is zapping collapsible SPTEs to reconstitute a hugepage after disbling dirty logging, KVM will mark every 4KiB pfn as dirty, even though _at least_ 512 pfns are guaranteed to be in a single folio (the SPTE couldn't potentially be huge if that weren't the case). The problem only becomes worse for 1GiB HugeTLB pages, as KVM can mark a single folio dirty 512*512 times. Marking a folio dirty when mapping is functionally safe as KVM drops all relevant SPTEs in response to an mmu_notifier invalidation, i.e. ensures that the guest can't dirty a folio after access has been removed. And because KVM already marks folios dirty when zapping/modifying SPTEs for KVM reasons, i.e. not in response to an mmu_notifier invalidation, there is no danger of "prematurely" marking a folio dirty. E.g. if a filesystems cleans a folio without first removing write access, then there already exists races where KVM could mark a folio dirty before remote TLBs are flushed, i.e. before guest writes are guaranteed to stop. Furthermore, x86 is literally the only architecture that marks folios dirty on the backend; every other KVM architecture marks folios dirty at map time. x86's unique behavior likely stems from the fact that x86's MMU predates mmu_notifiers. Long, long ago, before mmu_notifiers were added, marking pages dirty when zapping SPTEs was logical, and perhaps even necessary, as KVM held references to pages, i.e. kept a page's refcount elevated while the page was mapped into the guest. At the time, KVM's rmap_remove() simply did: if (is_writeble_pte(*spte)) kvm_release_pfn_dirty(pfn); else kvm_release_pfn_clean(pfn); i.e. dropped the refcount and marked the page dirty at the same time. After mmu_notifiers were introduced, commit acb66dd051d0 ("KVM: MMU: don't hold pagecount reference for mapped sptes pages") removed the refcount logic, but kept the dirty logic, i.e. converted the above to: if (is_writeble_pte(*spte)) kvm_release_pfn_dirty(pfn); And for KVM x86, that's essentially how things have stayed over the last ~15 years, without anyone revisiting *why* KVM marks pages/folios dirty at zap/modification time, e.g. the behavior was blindly carried forward to the TDP MMU. Practically speaking, the only downside to marking a folio dirty during mapping is that KVM could trigger writeback of memory that was never actually written. Except that can't actually happen if KVM marks folios dirty if and only if a writable SPTE is created (as done here), because KVM always marks writable SPTEs as dirty during make_spte(). See commit 9b51a63024bd ("KVM: MMU: Explicitly set D-bit for writable spte."), circa 2015. Note, KVM's access tracking logic for prefetched SPTEs is a bit odd. If a guest PTE is dirty and writable, KVM will create a writable SPTE, but then mark the SPTE for access tracking. Which isn't wrong, just a bit odd, as it results in _more_ precise dirty tracking for MMUs _without_ A/D bits. To keep things simple, mark the folio dirty before access tracking comes into play, as an access-tracked SPTE can be restored in the fast page fault path, i.e. without holding mmu_lock. While writing SPTEs and accessing memslots outside of mmu_lock is safe, marking a folio dirty is not. E.g. if the fast path gets interrupted _just_ after setting a SPTE, the primary MMU could theoretically invalidate and free a folio before KVM marks it dirty. Unlike the shadow MMU, which waits for CPUs to respond to an IPI, the TDP MMU only guarantees the page tables themselves won't be freed (via RCU). Opportunistically update a few stale comments. Cc: David Matlack Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 30 ++++-------------------------- arch/x86/kvm/mmu/paging_tmpl.h | 6 +++--- arch/x86/kvm/mmu/spte.c | 20 ++++++++++++++++++-- arch/x86/kvm/mmu/tdp_mmu.c | 12 ------------ 4 files changed, 25 insertions(+), 43 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0f21d6f76cab..1ae823ebd12b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -547,10 +547,8 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte) kvm_set_pfn_accessed(spte_to_pfn(old_spte)); } =20 - if (is_dirty_spte(old_spte) && !is_dirty_spte(new_spte)) { + if (is_dirty_spte(old_spte) && !is_dirty_spte(new_spte)) flush =3D true; - kvm_set_pfn_dirty(spte_to_pfn(old_spte)); - } =20 return flush; } @@ -593,9 +591,6 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u= 64 *sptep) if (is_accessed_spte(old_spte)) kvm_set_pfn_accessed(pfn); =20 - if (is_dirty_spte(old_spte)) - kvm_set_pfn_dirty(pfn); - return old_spte; } =20 @@ -1250,16 +1245,6 @@ static bool spte_clear_dirty(u64 *sptep) return mmu_spte_update(sptep, spte); } =20 -static bool spte_wrprot_for_clear_dirty(u64 *sptep) -{ - bool was_writable =3D test_and_clear_bit(PT_WRITABLE_SHIFT, - (unsigned long *)sptep); - if (was_writable && !spte_ad_enabled(*sptep)) - kvm_set_pfn_dirty(spte_to_pfn(*sptep)); - - return was_writable; -} - /* * Gets the GFN ready for another round of dirty logging by clearing the * - D bit on ad-enabled SPTEs, and @@ -1275,7 +1260,8 @@ static bool __rmap_clear_dirty(struct kvm *kvm, struc= t kvm_rmap_head *rmap_head, =20 for_each_rmap_spte(rmap_head, &iter, sptep) if (spte_ad_need_write_protect(*sptep)) - flush |=3D spte_wrprot_for_clear_dirty(sptep); + flush |=3D test_and_clear_bit(PT_WRITABLE_SHIFT, + (unsigned long *)sptep); else flush |=3D spte_clear_dirty(sptep); =20 @@ -1628,14 +1614,6 @@ static bool kvm_rmap_age_gfn_range(struct kvm *kvm, clear_bit((ffs(shadow_accessed_mask) - 1), (unsigned long *)sptep); } else { - /* - * Capture the dirty status of the page, so that - * it doesn't get lost when the SPTE is marked - * for access tracking. - */ - if (is_writable_pte(spte)) - kvm_set_pfn_dirty(spte_to_pfn(spte)); - spte =3D mark_spte_for_access_track(spte); mmu_spte_update_no_track(sptep, spte); } @@ -3415,7 +3393,7 @@ static bool fast_pf_fix_direct_spte(struct kvm_vcpu *= vcpu, * harm. This also avoids the TLB flush needed after setting dirty bit * so non-PML cases won't be impacted. * - * Compare with set_spte where instead shadow_dirty_mask is set. + * Compare with make_spte() where instead shadow_dirty_mask is set. */ if (!try_cmpxchg64(sptep, &old_spte, new_spte)) return false; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 6e7bd8921c6f..fbaae040218b 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -892,9 +892,9 @@ static gpa_t FNAME(gva_to_gpa)(struct kvm_vcpu *vcpu, s= truct kvm_mmu *mmu, =20 /* * Using the information in sp->shadowed_translation (kvm_mmu_page_get_gfn= ()) is - * safe because: - * - The spte has a reference to the struct page, so the pfn for a given g= fn - * can't change unless all sptes pointing to it are nuked first. + * safe because SPTEs are protected by mmu_notifiers and memslot generatio= ns, so + * the pfn for a given gfn can't change unless all SPTEs pointing to the g= fn are + * nuked first. * * Returns * < 0: failed to sync spte diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 618059b30b8b..8e8d6ee79c8b 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -232,8 +232,8 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_pa= ge *sp, * unnecessary (and expensive). * * The same reasoning applies to dirty page/folio accounting; - * KVM will mark the folio dirty using the old SPTE, thus - * there's no need to immediately mark the new SPTE as dirty. + * KVM marked the folio dirty when the old SPTE was created, + * thus there's no need to mark the folio dirty again. * * Note, both cases rely on KVM not changing PFNs without first * zapping the old SPTE, which is guaranteed by both the shadow @@ -266,12 +266,28 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_= page *sp, "spte =3D 0x%llx, level =3D %d, rsvd bits =3D 0x%llx", spte, level, get_rsvd_bits(&vcpu->arch.mmu->shadow_zero_check, spte, level)); =20 + /* + * Mark the memslot dirty *after* modifying it for access tracking. + * Unlike folios, memslots can be safely marked dirty out of mmu_lock, + * i.e. in the fast page fault handler. + */ if ((spte & PT_WRITABLE_MASK) && kvm_slot_dirty_track_enabled(slot)) { /* Enforced by kvm_mmu_hugepage_adjust. */ WARN_ON_ONCE(level > PG_LEVEL_4K); mark_page_dirty_in_slot(vcpu->kvm, slot, gfn); } =20 + /* + * If the page that KVM got from the primary MMU is writable, i.e. if + * it's host-writable, mark the page/folio dirty. As alluded to above, + * folios can't be safely marked dirty in the fast page fault handler, + * and so KVM must (somewhat) speculatively mark the folio dirty even + * though it isn't guaranteed to be written as KVM won't mark the folio + * dirty if/when the SPTE is made writable. + */ + if (host_writable) + kvm_set_pfn_dirty(pfn); + *new_spte =3D spte; return wrprot; } diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 76bca7a726c1..517b384473c1 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -511,10 +511,6 @@ static void handle_changed_spte(struct kvm *kvm, int a= s_id, gfn_t gfn, if (is_leaf !=3D was_leaf) kvm_update_page_stats(kvm, level, is_leaf ? 1 : -1); =20 - if (was_leaf && is_dirty_spte(old_spte) && - (!is_present || !is_dirty_spte(new_spte) || pfn_changed)) - kvm_set_pfn_dirty(spte_to_pfn(old_spte)); - /* * Recursively handle child PTs if the change removed a subtree from * the paging structure. Note the WARN on the PFN changing without the @@ -1249,13 +1245,6 @@ static bool age_gfn_range(struct kvm *kvm, struct td= p_iter *iter, iter->level); new_spte =3D iter->old_spte & ~shadow_accessed_mask; } else { - /* - * Capture the dirty status of the page, so that it doesn't get - * lost when the SPTE is marked for access tracking. - */ - if (is_writable_pte(iter->old_spte)) - kvm_set_pfn_dirty(spte_to_pfn(iter->old_spte)); - new_spte =3D mark_spte_for_access_track(iter->old_spte); iter->old_spte =3D kvm_tdp_mmu_write_spte(iter->sptep, iter->old_spte, new_spte, @@ -1596,7 +1585,6 @@ static void clear_dirty_pt_masked(struct kvm *kvm, st= ruct kvm_mmu_page *root, trace_kvm_tdp_mmu_spte_changed(iter.as_id, iter.gfn, iter.level, iter.old_spte, iter.old_spte & ~dbit); - kvm_set_pfn_dirty(spte_to_pfn(iter.old_spte)); } =20 rcu_read_unlock(); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0EB5A1E8821 for ; Thu, 10 Oct 2024 18:25:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584713; cv=none; b=XLLuWZb7iVeT9tdZqToGEbqO7pyJkl+Ui6I/w9Z7YuTRktRpLeLQxkyt/zWRTYmLiDbmV34h5x3o+GRk/oDKLvt4HawlblKkquRhWg4X/jl8uLZMjm3tN2Riye7js7y91gOjMQCuJLqQj17LuCyzL8OHKjon9dv6RHCkJSv7t08= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584713; c=relaxed/simple; bh=sB5G3PQtHd5Wp6fJfSwA75+Ou8nv+Xyz2zynkW7I+sM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=T5+ItIuwLos3qwA8dpKGPySXmNtyUBK8s4mtr4EGbzW24mGKxXWY/I4io3TUPXYRMT9MPRLI4zuLpfZCkvP4RxIgueLTjLhzwEt6/c+pbTtGx5AuIRMiNJCZ7ZwuexvoxRdzNKKVqL5+e0BsP6IPLZmuMMydQz+/dM91qnwsq2I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=YFnRBGee; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="YFnRBGee" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e2b049b64aso20288147b3.3 for ; Thu, 10 Oct 2024 11:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584711; x=1729189511; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=0ix/Xs4/Qn/2DvOT3Bh000Q598gzWwOGIiO61AJ4Bto=; b=YFnRBGeeymk+2pmMBdw0wl0Yy+Pdr75lMEEb46Qj2R/ZMELxdBUC0VhXMaFoLMRZyw lM5BZATdJOfXIHC7Ufn9e5g1CVYCsM89XBgpEbeWnjle3YVc0JrKT5IY3d3WFOKNc03i tQo8sG3IUbpyxFHlOMrb7ZHLDldQnJqNa2RjhIHLewNO7ZU5NmsXMw8G548D7wuNl5BX KAXxb5IgcpvzTluv7+tQS9iUf8SuC1dnEWalfACIcBmgOeNyW3QjD8rsI9do0p1/7fwV MXEidCQsM6goBjTy2GmzJ7UCSunVjhh4Ev1K313o4BihBPVtWlNKHeeWHalpT8ZOONki 6nMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584711; x=1729189511; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=0ix/Xs4/Qn/2DvOT3Bh000Q598gzWwOGIiO61AJ4Bto=; b=ZpjTBJ/sQlMls9Mo03NTbAGCnHMFhqoB1uRdoA7O0OTmbDMa6m36HUWlbOPBD4ySDk YsDjt+5VGETMpRIOPJ8UPyVDvNSETPP0HxUtwv76AIXYmxLed48uq5auCh9fX4Xc4Xj5 H6D1vk9rlXteajxG7/uLSsNYnTwyD6l5yE4D02NjoUjVvy0/IeQaf2N52pnvyauOPzJi SxGmE/NsGWoeR/ZEimBoRXjxkC81KEQommdDaAmXUYPJgOFOWKvxeNpJET7Qwqwaq+P2 uBNNU/SMrGDf2sB1kEgzFq2jkhFm8s/zI0el9VgXMlRwkXMwjqebU7SlQx79IQq7FKSg BbQw== X-Forwarded-Encrypted: i=1; AJvYcCXAXFQ7u4VYNljoR+Rn2je/uEuseGUFCCPODYseXE/R6Am2abgLiLv+TOWkzb+C8lnLjven8I/jv8T4WQ8=@vger.kernel.org X-Gm-Message-State: AOJu0YzOqSxDHKAO19kYv5bnR780YVVfUseoJsZCfxxj7M8LjYzLm70r oq/jsgqWGvzR0iGCx1nwomRrQKjTFiNYY9T/P8fVbFJTERd+oaXK69S7M5YWVdXw/Wx6+DbfATj jDw== X-Google-Smtp-Source: AGHT+IGD2TfP2dloLIlDYfRlI1lvmWPD7YUtu8EYW3dxRXY7JxJbZQIAJnuRF8jcBfCYGhk25T1qJs7AAuY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:5b41:b0:6db:c6ac:62a0 with SMTP id 00721157ae682-6e322305467mr243067b3.5.1728584711093; Thu, 10 Oct 2024 11:25:11 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:11 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-10-seanjc@google.com> Subject: [PATCH v13 09/85] KVM: x86/mmu: Mark page/folio accessed only when zapping leaf SPTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that KVM doesn't clobber Accessed bits of shadow-present SPTEs, e.g. when prefetching, mark folios as accessed only when zapping leaf SPTEs, which is a rough heuristic for "only in response to an mmu_notifier invalidation". Page aging and LRUs are tolerant of false negatives, i.e. KVM doesn't need to be precise for correctness, and re-marking folios as accessed when zapping entire roots or when zapping collapsible SPTEs is expensive and adds very little value. E.g. when a VM is dying, all of its memory is being freed; marking folios accessed at that time provides no known value. Similarly, because KVM marks folios as accessed when creating SPTEs, marking all folios as accessed when userspace happens to delete a memslot doesn't add value. The folio was marked access when the old SPTE was created, and will be marked accessed yet again if a vCPU accesses the pfn again after reloading a new root. Zapping collapsible SPTEs is a similar story; marking folios accessed just because userspace disable dirty logging is a side effect of KVM behavior, not a deliberate goal. As an intermediate step, a.k.a. bisection point, towards *never* marking folios accessed when dropping SPTEs, mark folios accessed when the primary MMU might be invalidating mappings, as such zappings are not KVM initiated, i.e. might actually be related to page aging and LRU activity. Note, x86 is the only KVM architecture that "double dips"; every other arch marks pfns as accessed only when mapping into the guest, not when mapping into the guest _and_ when removing from the guest. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- Documentation/virt/kvm/locking.rst | 76 +++++++++++++++--------------- arch/x86/kvm/mmu/mmu.c | 4 +- arch/x86/kvm/mmu/tdp_mmu.c | 7 ++- 3 files changed, 43 insertions(+), 44 deletions(-) diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/lo= cking.rst index 20a9a37d1cdd..3d8bf40ca448 100644 --- a/Documentation/virt/kvm/locking.rst +++ b/Documentation/virt/kvm/locking.rst @@ -147,49 +147,51 @@ Then, we can ensure the dirty bitmaps is correctly se= t for a gfn. =20 2) Dirty bit tracking =20 -In the origin code, the spte can be fast updated (non-atomically) if the +In the original code, the spte can be fast updated (non-atomically) if the spte is read-only and the Accessed bit has already been set since the Accessed bit and Dirty bit can not be lost. =20 But it is not true after fast page fault since the spte can be marked writable between reading spte and updating spte. Like below case: =20 -+------------------------------------------------------------------------+ -| At the beginning:: | -| | -| spte.W =3D 0 | -| spte.Accessed =3D 1 | -+------------------------------------+-----------------------------------+ -| CPU 0: | CPU 1: | -+------------------------------------+-----------------------------------+ -| In mmu_spte_clear_track_bits():: | | -| | | -| old_spte =3D *spte; | = | -| | | -| | | -| /* 'if' condition is satisfied. */| | -| if (old_spte.Accessed =3D=3D 1 && | = | -| old_spte.W =3D=3D 0) | = | -| spte =3D 0ull; | = | -+------------------------------------+-----------------------------------+ -| | on fast page fault path:: | -| | | -| | spte.W =3D 1 = | -| | | -| | memory write on the spte:: | -| | | -| | spte.Dirty =3D 1 = | -+------------------------------------+-----------------------------------+ -| :: | | -| | | -| else | | -| old_spte =3D xchg(spte, 0ull) | = | -| if (old_spte.Accessed =3D=3D 1) | = | -| kvm_set_pfn_accessed(spte.pfn);| | -| if (old_spte.Dirty =3D=3D 1) | = | -| kvm_set_pfn_dirty(spte.pfn); | | -| OOPS!!! | | -+------------------------------------+-----------------------------------+ ++-------------------------------------------------------------------------+ +| At the beginning:: | +| | +| spte.W =3D 0 = | +| spte.Accessed =3D 1 = | ++-------------------------------------+-----------------------------------+ +| CPU 0: | CPU 1: | ++-------------------------------------+-----------------------------------+ +| In mmu_spte_update():: | | +| | | +| old_spte =3D *spte; | = | +| | | +| | | +| /* 'if' condition is satisfied. */ | | +| if (old_spte.Accessed =3D=3D 1 && | = | +| old_spte.W =3D=3D 0) | = | +| spte =3D new_spte; | = | ++-------------------------------------+-----------------------------------+ +| | on fast page fault path:: | +| | | +| | spte.W =3D 1 = | +| | | +| | memory write on the spte:: | +| | | +| | spte.Dirty =3D 1 = | ++-------------------------------------+-----------------------------------+ +| :: | | +| | | +| else | | +| old_spte =3D xchg(spte, new_spte);| = | +| if (old_spte.Accessed && | | +| !new_spte.Accessed) | | +| flush =3D true; | = | +| if (old_spte.Dirty && | | +| !new_spte.Dirty) | | +| flush =3D true; | = | +| OOPS!!! | | ++-------------------------------------+-----------------------------------+ =20 The Dirty bit is lost in this case. =20 diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 1ae823ebd12b..04228a7da69a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -542,10 +542,8 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte) * to guarantee consistency between TLB and page tables. */ =20 - if (is_accessed_spte(old_spte) && !is_accessed_spte(new_spte)) { + if (is_accessed_spte(old_spte) && !is_accessed_spte(new_spte)) flush =3D true; - kvm_set_pfn_accessed(spte_to_pfn(old_spte)); - } =20 if (is_dirty_spte(old_spte) && !is_dirty_spte(new_spte)) flush =3D true; diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 517b384473c1..8aa0d7a7602b 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -520,10 +520,6 @@ static void handle_changed_spte(struct kvm *kvm, int a= s_id, gfn_t gfn, if (was_present && !was_leaf && (is_leaf || !is_present || WARN_ON_ONCE(pfn_changed))) handle_removed_pt(kvm, spte_to_child_pt(old_spte, level), shared); - - if (was_leaf && is_accessed_spte(old_spte) && - (!is_present || !is_accessed_spte(new_spte) || pfn_changed)) - kvm_set_pfn_accessed(spte_to_pfn(old_spte)); } =20 static inline int __must_check __tdp_mmu_set_spte_atomic(struct tdp_iter *= iter, @@ -865,6 +861,9 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct k= vm_mmu_page *root, =20 tdp_mmu_iter_set_spte(kvm, &iter, SHADOW_NONPRESENT_VALUE); =20 + if (is_accessed_spte(iter.old_spte)) + kvm_set_pfn_accessed(spte_to_pfn(iter.old_spte)); + /* * Zappings SPTEs in invalid roots doesn't require a TLB flush, * see kvm_tdp_mmu_zap_invalidated_roots() for details. --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B64A1E9064 for ; Thu, 10 Oct 2024 18:25:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584715; cv=none; b=K3vVpNE7Z87kKtknljsfxw2toXnINVv6Yk+MpqK925GjliOHcpwlk7KXvSxlEQNZZP8Vqz/MxCZwgWUXrGrEgFAcDTll2ZrgXxugqKFwKzwPMAx8Izy89YtSGJusiNsGsCwa1VSCtYt3CuLs1Slu/1Zj+h8jeAFZWM+MNk7GkPs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584715; c=relaxed/simple; bh=FKY/uzFyW8/uk7MX2MD3fh26WMEEPHDBpnIM2/GXcs8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=i7b2Ii3kgWz1+TdqGvBRHJGQSShkg2yEiZk9Bj3sQkEkruNYhcnkl+abc/ezSfajA2xNhPrzYN4Jyo24XgX4VltUsdV/YtfKtvuW2EO+S2uLjRxd4PvKo4Ny7wmxe3Viy0Tl8MWqL5EIV9FIDaQUkTrYTakcWNlT2QczPTShhe8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=w5tvPSoS; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="w5tvPSoS" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e2d17b0e86so988042a91.3 for ; Thu, 10 Oct 2024 11:25:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584713; x=1729189513; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=k3/xcOX0JmpZSdxUAeDWhRU0v4mqVA9cM67JfDYoNsA=; b=w5tvPSoSWJn+x5q8Oi0ZuA+6nHh716+OIuMRO32iJPXf1iRYDrVPRgGrl1ZJENQvwc mz1HBZPp97M1QM9TizwsEctlYX2RZjDkJzAVpK2ZQIgTitnr1b7IytQF678iRdMz5tHh FUUvh1DdiXPRS4vweuQEsdZZ3uyd14jwGvFphrv6qIvKpgPgKpu0kl1URoW7+fcvcr81 nju8c/yF+vmfJpUJVd3TRkyahxIySMZEM0QnjUZq6q+VDmv8lr9b9PnwJtV3Y0LsQFCN Kh43SB85epl5iaJqPu/GSek9p6Db9/pYBgGQeM4RcUQbVYdV2ebPlFBqlLfYqVf3l6HF sB7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584713; x=1729189513; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=k3/xcOX0JmpZSdxUAeDWhRU0v4mqVA9cM67JfDYoNsA=; b=bk8pR8JJ/+kdCimfLMccthiX1T2Ancl96D6GOzwWq4CTyNAFJjWIQ198tk1qLaxlkR S85cYfhnob5tUCpvzz9a8NeHEpUYp+lk5gYHbbsHgtVJVPTFRPBe6OEi03tFOnvSLSkJ TkSJeQZhDvbrcGo2zP0M7k3U1gJFJSzXk3GgiI5FJbNcFRqTUt+HeS+TEGa1Eq4NstfD JdX+a1PtjHvnusHWyCGYHK3t6Peo3g1ijjs0yiHOuuY9mtivsFvRe5QyZoJ+mB9Yk9ku yAucXSbVW9MrM6R5UXqPz2amiu95GJdkE8h+LkTUZZMQO5AnxVk56fJm9wuqgoAndm8r k9KA== X-Forwarded-Encrypted: i=1; AJvYcCWiKf6kmErR/p27FpEZ5LyRSlAPm2TbwfsNAxY7E3iWp+NuqMtaMHC524giUYkplM0cD2jjePy3WRYy46U=@vger.kernel.org X-Gm-Message-State: AOJu0YwngKD08JcN1qrWe2+u6IEPGZZXZAo8hFXVvWzP71gqdYN2CSAo FIIxoCLuY8hcpnj8NgcYvc9xrZjS4dUVg1IzvHs/elLedq1ySykGRkUdTuZ1O1BUjrG9vszKwDB Gsg== X-Google-Smtp-Source: AGHT+IEzZdX7U43FtBqfYwOSCvhQrFRs7MoNj08p1am6l1FbD4QaVhgMc/3KfB78OlyTOzXtpjGPG3KIosU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:ea0e:b0:2e2:d843:4880 with SMTP id 98e67ed59e1d1-2e2f0c82a2fmr30a91.5.1728584713017; Thu, 10 Oct 2024 11:25:13 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:12 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-11-seanjc@google.com> Subject: [PATCH v13 10/85] KVM: x86/mmu: Use gfn_to_page_many_atomic() when prefetching indirect PTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use gfn_to_page_many_atomic() instead of gfn_to_pfn_memslot_atomic() when prefetching indirect PTEs (direct_pte_prefetch_many() already uses the "to page" APIS). Functionally, the two are subtly equivalent, as the "to pfn" API short-circuits hva_to_pfn() if hva_to_pfn_fast() fails, i.e. is just a wrapper for get_user_page_fast_only()/get_user_pages_fast_only(). Switching to the "to page" API will allow dropping the @atomic parameter from the entire hva_to_pfn() callchain. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/paging_tmpl.h | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index fbaae040218b..36b2607280f0 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -535,8 +535,8 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_= mmu_page *sp, { struct kvm_memory_slot *slot; unsigned pte_access; + struct page *page; gfn_t gfn; - kvm_pfn_t pfn; =20 if (FNAME(prefetch_invalid_gpte)(vcpu, sp, spte, gpte)) return false; @@ -549,12 +549,11 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kv= m_mmu_page *sp, if (!slot) return false; =20 - pfn =3D gfn_to_pfn_memslot_atomic(slot, gfn); - if (is_error_pfn(pfn)) + if (gfn_to_page_many_atomic(slot, gfn, &page, 1) !=3D 1) return false; =20 - mmu_set_spte(vcpu, slot, spte, pte_access, gfn, pfn, NULL); - kvm_release_pfn_clean(pfn); + mmu_set_spte(vcpu, slot, spte, pte_access, gfn, page_to_pfn(page), NULL); + kvm_release_page_clean(page); return true; } =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C6571E909C for ; Thu, 10 Oct 2024 18:25:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584717; cv=none; b=I35GD5DbBQgmlIu+9tOmYoWHDX6iCQlLg3pEESmavhh137VoYawbtZdp1TZEk5Yjs+D+BYj+iaCrYgxBjmWBXYnnfvGFJ4Nbn7KCN/RCBDgEwFuO+1BIak8JDMpVMt5/OjMYxIxwQcS8mGpW8kL2/5+MQ+6OeTUZTwRzkF1aX1Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584717; c=relaxed/simple; bh=bXBvqUXIo6rFfz7xJp3m3CL9RbZG3WivseimjKRiNcE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=BKeZI6o35Z3yhAwReDeJh25lsL/fd41gYPBeVuFaj2kdguQFZ90nlRxwGBWAOrYceoECvYqzOpWZiaJZVnX/cEPtcbymOabxj1V7vAWucIWVk3pzqOLQ/c664nvWcrUS3NmP52fBcZ+DztsxxvsLhloKQZN7tzQEZsT5b+kkYd8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=msluruMB; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="msluruMB" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-20b6144cc2aso12169015ad.0 for ; Thu, 10 Oct 2024 11:25:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584715; x=1729189515; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=WDoARMTIexYqUKDMQLAC2OQWFTeknat7PpvqQx990/M=; b=msluruMB47yV+eMqqJKQqQc3MtTMadYZ8sWgl6QRmMr09qa3+feEPjjvGyBY+uif8W qpS0VwqWhyL0czQJzqH+B9niYon2tyJam8T8Wx431OsNSh0byA40a4gCFG9E/iQ4vl7a 6UoZSJP5zfbt83Sm9xMLY1LQdgh7R2IqI7So3Y4ciWiXWgSST91zKUp2NWSuxui2Udu2 XYV5f67I+k1MuchiObUt1AkpUZwZ9d/5lXZxbM2nMVGYJ3xTOCQSP/NlphrkSb4KDTES lsmR21Ah+UemyDSlKJ7g2DcTy675DQzG6u8F3vgdUqFS9ZvBqFBbQWAyPrrqdAw2BlVR 7WoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584715; x=1729189515; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=WDoARMTIexYqUKDMQLAC2OQWFTeknat7PpvqQx990/M=; b=ExrfrDXFJbA4pkB68p6zRVmLaiQMX0jz2ca7C2O7UCPqPGTgLsB5KE/MQxwEireYaA 8eEmIJ2Zb+UJF3ARXXhQvlJRqlO4rRKbjcRE9NYvuwuus/lMQ94lcjwChXbQ7fcMANlU m0yZlg1pinUsIGpHaEgkXT1gMV8K7VUo8DRHbjAPFLpqP+o3/BrzPd53Rb4zE3W4j6ki qM7rrnY8HRFh7DmAVJDltBb42MnrtabxhX5WQT+YACwnDW5gojkNFsyz4eh9IX3tujXG MRcSPd+0n3L+wqjwVTMvYsls/GFg89eb4fhDsYrUoIIMLwUSnVW9K9KqZI0VRkwFic9U 0nQQ== X-Forwarded-Encrypted: i=1; AJvYcCXRbTOMKIt/y4Dr5JP7bmRrPy/1g/x1JA23nbtD+lwLZp8hmoZgM2JEZJzQBpYGKMCG3jy1cSjh1HBiwGs=@vger.kernel.org X-Gm-Message-State: AOJu0YzYc1NHlua5mfgN44+blp3suEZTV2j4EDphycDzxAT1vuv8ybw6 MtIedadUOhT6n1VETWHnl8W5IxgA80aOANuNQa6z3tAJcemGN1/HRCRJMntnyib/3nt4vKDuC/e TcQ== X-Google-Smtp-Source: AGHT+IFdLsiMiQR9ssEjYgvfJ+TOiOlvv8h202qp7bAbI4MCMFrC9IRsz+IufRcGyfoerq3Ms7MkoMKw8Uo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:f143:b0:20c:5beb:9c6a with SMTP id d9443c01a7336-20c80510505mr43225ad.4.1728584715291; Thu, 10 Oct 2024 11:25:15 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:13 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-12-seanjc@google.com> Subject: [PATCH v13 11/85] KVM: Rename gfn_to_page_many_atomic() to kvm_prefetch_pages() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rename gfn_to_page_many_atomic() to kvm_prefetch_pages() to try and communicate its true purpose, as the "atomic" aspect is essentially a side effect of the fact that x86 uses the API while holding mmu_lock. E.g. even if mmu_lock weren't held, KVM wouldn't want to fault-in pages, as the goal is to opportunistically grab surrounding pages that have already been accessed and/or dirtied by the host, and to do so quickly. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- include/linux/kvm_host.h | 4 ++-- virt/kvm/kvm_main.c | 6 +++--- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 04228a7da69a..5fe45ab0e818 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2958,7 +2958,7 @@ static int direct_pte_prefetch_many(struct kvm_vcpu *= vcpu, if (!slot) return -1; =20 - ret =3D gfn_to_page_many_atomic(slot, gfn, pages, end - start); + ret =3D kvm_prefetch_pages(slot, gfn, pages, end - start); if (ret <=3D 0) return -1; =20 diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 36b2607280f0..143b7e9f26dc 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -549,7 +549,7 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_= mmu_page *sp, if (!slot) return false; =20 - if (gfn_to_page_many_atomic(slot, gfn, &page, 1) !=3D 1) + if (kvm_prefetch_pages(slot, gfn, &page, 1) !=3D 1) return false; =20 mmu_set_spte(vcpu, slot, spte, pte_access, gfn, page_to_pfn(page), NULL); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ab4485b2bddc..56e7cde8c8b8 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1207,8 +1207,8 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm); void kvm_arch_flush_shadow_memslot(struct kvm *kvm, struct kvm_memory_slot *slot); =20 -int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, - struct page **pages, int nr_pages); +int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, + struct page **pages, int nr_pages); =20 struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn); unsigned long gfn_to_hva(struct kvm *kvm, gfn_t gfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 2032292df0b0..957b4a6c9254 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3053,8 +3053,8 @@ kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, = gfn_t gfn) } EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); =20 -int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, - struct page **pages, int nr_pages) +int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, + struct page **pages, int nr_pages) { unsigned long addr; gfn_t entry =3D 0; @@ -3068,7 +3068,7 @@ int gfn_to_page_many_atomic(struct kvm_memory_slot *s= lot, gfn_t gfn, =20 return get_user_pages_fast_only(addr, nr_pages, FOLL_WRITE, pages); } -EXPORT_SYMBOL_GPL(gfn_to_page_many_atomic); +EXPORT_SYMBOL_GPL(kvm_prefetch_pages); =20 /* * Do not use this helper unless you are absolutely certain the gfn _must_= be --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 167F31EABCF for ; Thu, 10 Oct 2024 18:25:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584720; cv=none; b=SATiZI+qGlXGhe/X/KTDbxeubctNnpnJ3KII7dKNd8iY9T1KjUNsxDgjOal8VYWZESBjcbgXeAd9SJZXGmEUGD7h4ppYMh1T+s/H1gRDRE8Rj0FuW1B+YQPLhFjeWvME6q8GMc4UHZQWuzwlubiXsuj8VOqahzbJzIKy/CthKGs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584720; c=relaxed/simple; bh=NgBQk0YRvUNmc13dbVZlntKigL70MLaajTeU3NGo9fA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fs9atesEcDy2Oe9UjqnGZa4nvvYRYFGNKSc30ZUY/oi9HODrpFh+A85bWGOIsh9fpmJ5PvA04LQQrMmcNizoM14ZxAPk1mMUFfBD72ZfVROB+6oK81Ebhe75k1dVBJkE7fjGkUI4y1+EkG06yPv0Z547vDqew4D9hHJceyqR+8k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=eihI10jr; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eihI10jr" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-7174c6cbdbaso1578869b3a.2 for ; Thu, 10 Oct 2024 11:25:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584717; x=1729189517; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=IMk6Ksq+DkJF8Cmr0W3poBl1lc0GSthNwEC5P5onZMQ=; b=eihI10jru2n8ukWDIrc4V1BUWuR7kg0e1MklavXglc6950CPFgvqStVWPROKStYTp3 hNXNXKKlqyzDuLLGxwwWxlgdHdmPGCfXDNUg1NmPaTS44X9jIVXe/ExDP544smBEGPaF hk8AtmMAPGvv8BKjmHvXS6tiKMnZmeVClyLoX/BaV/Blwx/GCI6B2Yu3V02I2crBiNsk 1u0NCDnwPePOmtM2eaOQH2672Om7WlhN2DEagdo+me81XgGdIVBDEjPXYHoKNWdjMqCl Z1oc4pvNwUbuOmrWza21vh57XnX3eqw7MGRoIVnvl5bwzDuQdR41ncqUkNktv/loOESb L5dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584717; x=1729189517; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=IMk6Ksq+DkJF8Cmr0W3poBl1lc0GSthNwEC5P5onZMQ=; b=ggBzfQi0CzKy/TJq/Uyhc0slVHGW/SEetLCGvRFCeF4qq62lO7fsZJTs+3z/kCvhOd fmcETz0h6/QOrk/uH/8lXZwrS3OtmwMkF5hkFMFEnqaeIXaayZMib7vf4l4PsnYpTmgV lrbEfHpehha4BrwJIW/IwVeVZbAp7DXrh2Dz42JzSH5ek47zSITXC4b5UYSenfbBXGTW 0YJio8qUkM+hXTkKW2m9Hl2TqNZ19jeCnjxYpyboPE6xFUGhLS+YIr6kDtE1xU1pnFop he7DOshNjBG4o7esPd/lh+ffJWQ7tg96FVCCBzIA1yMQK8Wgd6a685+/TJ4fHDmTHwx/ 9lOA== X-Forwarded-Encrypted: i=1; AJvYcCWQsoWVNsY1MzdAv0XixEXnBK6r0GIzpV2ftGH9eEtYpNa8v+rKGv+ORjfoDz4BNS1QKSvNGiZrMfH7Dfs=@vger.kernel.org X-Gm-Message-State: AOJu0YxeP1tWlHyvMb1blAPYn1RBqKJwT0bA21/ZUPUjbX/8S2d8nZAG PxowN7yCJukdJyGHKYnbtnY7eZpa/ysL3Z4qtJAkXymo2YgHfdIY5fbMckmjWNTBrehCkSI9yGY brg== X-Google-Smtp-Source: AGHT+IEgBAPk5NggidzeRIz4/KASJ69N4B17cGxFjG2BQTaLRrpMtimTBEcirbwLQ+H7ew0HtHwcTgFgXoI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2d8a:b0:71d:fc7c:4a34 with SMTP id d2e1a72fcca58-71e1dbff219mr24371b3a.6.1728584717336; Thu, 10 Oct 2024 11:25:17 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:14 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-13-seanjc@google.com> Subject: [PATCH v13 12/85] KVM: Drop @atomic param from gfn=>pfn and hva=>pfn APIs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop @atomic from the myriad "to_pfn" APIs now that all callers pass "false", and remove a comment blurb about KVM running only the "GUP fast" part in atomic context. No functional change intended. Reviewed-by: Alex Benn=C3=A9e Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- Documentation/virt/kvm/locking.rst | 4 +-- arch/arm64/kvm/mmu.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 +- arch/x86/kvm/mmu/mmu.c | 12 ++++---- include/linux/kvm_host.h | 4 +-- virt/kvm/kvm_main.c | 39 ++++++-------------------- virt/kvm/kvm_mm.h | 4 +-- virt/kvm/pfncache.c | 2 +- 9 files changed, 23 insertions(+), 48 deletions(-) diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/lo= cking.rst index 3d8bf40ca448..f463ac42ac7a 100644 --- a/Documentation/virt/kvm/locking.rst +++ b/Documentation/virt/kvm/locking.rst @@ -135,8 +135,8 @@ We dirty-log for gfn1, that means gfn2 is lost in dirty= -bitmap. For direct sp, we can easily avoid it since the spte of direct sp is fixed to gfn. For indirect sp, we disabled fast page fault for simplicity. =20 -A solution for indirect sp could be to pin the gfn, for example via -kvm_vcpu_gfn_to_pfn_atomic, before the cmpxchg. After the pinning: +A solution for indirect sp could be to pin the gfn before the cmpxchg. Af= ter +the pinning: =20 - We have held the refcount of pfn; that means the pfn can not be freed and be reused for another gfn. diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index a509b63bd4dd..a6e62cc9015c 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1569,7 +1569,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys= _addr_t fault_ipa, mmu_seq =3D vcpu->kvm->mmu_invalidate_seq; mmap_read_unlock(current->mm); =20 - pfn =3D __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, + pfn =3D __gfn_to_pfn_memslot(memslot, gfn, false, NULL, write_fault, &writable, NULL); if (pfn =3D=3D KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_= 64_mmu_hv.c index 1b51b1c4713b..8cd02ca4b1b8 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -613,7 +613,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu, write_ok =3D true; } else { /* Call KVM generic code to do the slow-path check */ - pfn =3D __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, + pfn =3D __gfn_to_pfn_memslot(memslot, gfn, false, NULL, writing, &write_ok, NULL); if (is_error_noslot_pfn(pfn)) return -EFAULT; diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book= 3s_64_mmu_radix.c index 408d98f8a514..26a969e935e3 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -852,7 +852,7 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcp= u, unsigned long pfn; =20 /* Call KVM generic code to do the slow-path check */ - pfn =3D __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, + pfn =3D __gfn_to_pfn_memslot(memslot, gfn, false, NULL, writing, upgrade_p, NULL); if (is_error_noslot_pfn(pfn)) return -EFAULT; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5fe45ab0e818..0e235f276ee5 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4380,9 +4380,9 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault return kvm_faultin_pfn_private(vcpu, fault); =20 async =3D false; - fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, false, - &async, fault->write, - &fault->map_writable, &fault->hva); + fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, &asyn= c, + fault->write, &fault->map_writable, + &fault->hva); if (!async) return RET_PF_CONTINUE; /* *pfn has correct page already */ =20 @@ -4402,9 +4402,9 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault * to wait for IO. Note, gup always bails if it is unable to quickly * get a page and a fatal signal, i.e. SIGKILL, is pending. */ - fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, - NULL, fault->write, - &fault->map_writable, &fault->hva); + fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, NULL, + fault->write, &fault->map_writable, + &fault->hva); return RET_PF_CONTINUE; } =20 diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 56e7cde8c8b8..2faafc7a56ae 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1232,9 +1232,8 @@ kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn= ); -kvm_pfn_t gfn_to_pfn_memslot_atomic(const struct kvm_memory_slot *slot, gf= n_t gfn); kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t g= fn, - bool atomic, bool interruptible, bool *async, + bool interruptible, bool *async, bool write_fault, bool *writable, hva_t *hva); =20 void kvm_release_pfn_clean(kvm_pfn_t pfn); @@ -1315,7 +1314,6 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn); =20 struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn= _t gfn); -kvm_pfn_t kvm_vcpu_gfn_to_pfn_atomic(struct kvm_vcpu *vcpu, gfn_t gfn); kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); int kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *ma= p); void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool = dirty); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 957b4a6c9254..0bc077213d3e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2756,8 +2756,7 @@ static inline int check_user_page_hwpoison(unsigned l= ong addr) =20 /* * The fast path to get the writable pfn which will be stored in @pfn, - * true indicates success, otherwise false is returned. It's also the - * only part that runs if we can in atomic context. + * true indicates success, otherwise false is returned. */ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, bool *writable, kvm_pfn_t *pfn) @@ -2922,7 +2921,6 @@ static int hva_to_pfn_remapped(struct vm_area_struct = *vma, /* * Pin guest page in memory and return its pfn. * @addr: host virtual address which maps memory to the guest - * @atomic: whether this function is forbidden from sleeping * @interruptible: whether the process can be interrupted by non-fatal sig= nals * @async: whether this function need to wait IO complete if the * host page is not in the memory @@ -2934,22 +2932,16 @@ static int hva_to_pfn_remapped(struct vm_area_struc= t *vma, * 2): @write_fault =3D false && @writable, @writable will tell the caller * whether the mapping is writable. */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool interruptible, - bool *async, bool write_fault, bool *writable) +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, + bool write_fault, bool *writable) { struct vm_area_struct *vma; kvm_pfn_t pfn; int npages, r; =20 - /* we can do it either atomically or asynchronously, not both */ - BUG_ON(atomic && async); - if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) return pfn; =20 - if (atomic) - return KVM_PFN_ERR_FAULT; - npages =3D hva_to_pfn_slow(addr, async, write_fault, interruptible, writable, &pfn); if (npages =3D=3D 1) @@ -2986,7 +2978,7 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic,= bool interruptible, } =20 kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t g= fn, - bool atomic, bool interruptible, bool *async, + bool interruptible, bool *async, bool write_fault, bool *writable, hva_t *hva) { unsigned long addr =3D __gfn_to_hva_many(slot, gfn, NULL, write_fault); @@ -3008,39 +3000,24 @@ kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_mem= ory_slot *slot, gfn_t gfn, writable =3D NULL; } =20 - return hva_to_pfn(addr, atomic, interruptible, async, write_fault, - writable); + return hva_to_pfn(addr, interruptible, async, write_fault, writable); } EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); =20 kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { - return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, - NULL, write_fault, writable, NULL); + return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, NULL, + write_fault, writable, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); =20 kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, false, NULL, true, - NULL, NULL); + return __gfn_to_pfn_memslot(slot, gfn, false, NULL, true, NULL, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); =20 -kvm_pfn_t gfn_to_pfn_memslot_atomic(const struct kvm_memory_slot *slot, gf= n_t gfn) -{ - return __gfn_to_pfn_memslot(slot, gfn, true, false, NULL, true, - NULL, NULL); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot_atomic); - -kvm_pfn_t kvm_vcpu_gfn_to_pfn_atomic(struct kvm_vcpu *vcpu, gfn_t gfn) -{ - return gfn_to_pfn_memslot_atomic(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn); -} -EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn_atomic); - kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn) { return gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index 715f19669d01..a3fa86f60d6c 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -20,8 +20,8 @@ #define KVM_MMU_UNLOCK(kvm) spin_unlock(&(kvm)->mmu_lock) #endif /* KVM_HAVE_MMU_RWLOCK */ =20 -kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool interruptible, - bool *async, bool write_fault, bool *writable); +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, + bool write_fault, bool *writable); =20 #ifdef CONFIG_HAVE_KVM_PFNCACHE void gfn_to_pfn_cache_invalidate_start(struct kvm *kvm, diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index f0039efb9e1e..58c706a610e5 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -198,7 +198,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cac= he *gpc) } =20 /* We always request a writeable mapping */ - new_pfn =3D hva_to_pfn(gpc->uhva, false, false, NULL, true, NULL); + new_pfn =3D hva_to_pfn(gpc->uhva, false, NULL, true, NULL); if (is_error_noslot_pfn(new_pfn)) goto out_error; =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33DD71EBA06 for ; Thu, 10 Oct 2024 18:25:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584722; cv=none; b=GrTKTMmRjXWviJRqsWoEB1sXlisLtf45tgZVnZQDy5/1PPRwO3UdJn+wA+XL3QGXYJNXPXQwHWUehmklZhROaUBpQYNy+HeFw4i1bBIOPWW3gXgREhlmtxtXn/NvyD/aYsnOaRKQh9CB+XFxU+o4T5piX8CQWzCa4PjwBsBGwok= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584722; c=relaxed/simple; bh=IF5BVNwy5lfsZS2zGuSAAv+TGmQDGgo7CuXbuO4ZMc4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OgtgR/nEnrelRHzkHRCS+417MJ3QnWBFr6P/dTedOHes1KXBK/zG/58Vyzm1u7DdZzerEy8IlNCmnOpdfKWlrU4fDTXVX5r/RY/vx9+11AJedIxViCl7WsF05hHmU+X32baX65P8E7HZAylKa0dxGIo3IsC4CT+l7hLiPnKqkug= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=OrSuHW3L; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="OrSuHW3L" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e284982a31so22425127b3.3 for ; Thu, 10 Oct 2024 11:25:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584719; x=1729189519; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=F0+KYw/6HzWsmGATvSxbQuKdqCcRS3waqcwxHrAen8Y=; b=OrSuHW3LuAFGFcLFdBetAruGipimy71Kdd23mnkCvSb3kzL/ihIej8uTFHbxMiLtnA 6YXSI2J+unla/LKg7JIEqX4mCyvrr3VQ406BwPbBG3PiTMk+2hz8bnoRevRPodlhpmMn aYJOYXJWz/3t+SqCWv1rhWrcN4c2Ngbiu7CrMUkXV7A/egH9oRwfDsMuE+3yorA8VAOz djCQlkkb9mc9WG+6oBsuPymdYZpYY26x8ENW3hOt+jiqZzezffl3p0vFDG1wHBHrBrbq D5swTKZ9IT9IBSz41WC4R7QS/s7rDjGVXDZH7rNvgqH9bS1bMtz43t3X2ef5ejrnse9y 5+Ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584719; x=1729189519; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=F0+KYw/6HzWsmGATvSxbQuKdqCcRS3waqcwxHrAen8Y=; b=SyEyQJU1NrPwfqwD6gxmFsdxq5vbe2F87pCKiKtCtmCzUAN71NfXdjALG1MESOBjA2 PbyDdPBscnXrIazmhdR5Zw96XSitDZd2uBg2oEF3l4kMOkzXDPVuZ8TUgy6KHA0Qxomt ad3P32GznrUYXQ3330Q/ECeGNMKEhmOeMy8XSEzPlCDJuBI+wepn/CksHl7/CpLav6BZ JiMLHJFJq7SfUwtEJAjCxcnbipOgNIXFG1szPTDEwqM74KG0XPOQf+4eolpW3xCsR24G ns0c1e+99bH+X5q/S/G0ttYns8pqTH8oBlfhGY+47jgtKIPY2tfbGVbc4cawyX2b3hNt M3eg== X-Forwarded-Encrypted: i=1; AJvYcCXIn0FcJfCrXnT1mXEI6GjJL5kZN+67jJzdzuOGy+4n+obF3wwQuGM9D506tvlAyFtyz3jT1K0l+Qjpcyg=@vger.kernel.org X-Gm-Message-State: AOJu0YxT7zT5CYWEUMkJCIdZ6bmyzhy+K8DNsjJ2xPNTbD8EQX9dvdmi mupaHix8tIOUbQM7vH1A04qU+jWhL1rG7iCrI7L4csiQrO+uctScTZimcQtYgnZGBEjN/RhzmIb u+A== X-Google-Smtp-Source: AGHT+IEeBu92qjGJ6/8zutZUBi8WOanw4UupdtztXpUfLE+vAeLHZHNkVuFOksmpHMU427MoGYN+Q9YZlwU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:3204:b0:62c:f976:a763 with SMTP id 00721157ae682-6e322185b22mr897567b3.1.1728584719334; Thu, 10 Oct 2024 11:25:19 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:15 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-14-seanjc@google.com> Subject: [PATCH v13 13/85] KVM: Annotate that all paths in hva_to_pfn() might sleep From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that hva_to_pfn() no longer supports being called in atomic context, move the might_sleep() annotation from hva_to_pfn_slow() to hva_to_pfn(). Reviewed-by: Alex Benn=C3=A9e Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 0bc077213d3e..17acc75990a5 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2804,8 +2804,6 @@ static int hva_to_pfn_slow(unsigned long addr, bool *= async, bool write_fault, struct page *page; int npages; =20 - might_sleep(); - if (writable) *writable =3D write_fault; =20 @@ -2939,6 +2937,8 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interru= ptible, bool *async, kvm_pfn_t pfn; int npages, r; =20 + might_sleep(); + if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) return pfn; =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9E201EF0A0 for ; Thu, 10 Oct 2024 18:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584725; cv=none; b=hZ8jI/5DJy7L009qibkEBv1tVc0PeYWRVVt0jWoMIqSkCpehTt1hso8e3YfZ9TXZK5/gfAGhUX1TRERiWrHqE4y79fD9Qz5XCOuM/t8TopXYqJlPm2O3w7senbxTMzdVJtu7GksgzL/2yUR/G+SoTA3EQngCK4QEg3TtMTr/R14= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584725; c=relaxed/simple; bh=rFmsStC1d1lztIwj7oxh4zyq4MQj9meeqHdDcxeq/wk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Q/dg6eiatbSddyOo9LAKJhoIjCFFBasoy8w4Vmh4GRvnATG3INl2R8gH8YNuR4JF7fF9UEhufquQTqlvJasKt7Q/K95SbG04g/93hptK2+pN3qrrfWfasipqScHDBYuuZdCcRHeei+DnTz8D181S2nV596hMmfpunkU0mMwxJ40= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=zBhsl50q; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="zBhsl50q" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7d4dee4dfdcso1264255a12.2 for ; Thu, 10 Oct 2024 11:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584722; x=1729189522; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RdgAxVyxp31c7WibHLiyIF43paER00mxZOAIHhsdwhk=; b=zBhsl50qhIyYlq5VgEtE/+OL3pX+IkWGN8RxGVp/QOIATD1A51kLNeCudHlw6tpDB0 TTnMuZ0YrHvKBYD5A8G+ycj8IUIKyFCIAvMqpCIDMUUrDrOdy+ab52MBoxiszJHwHzOm sj7/CT59ErmtRPL1Q4YUop0G579u9pMO68H1UQD9Q5tvvttPtuceH0HBENkiQygrEN+L UXFheIKzvN3cuQNMasUDogdtF4XkKlByJg1oeWEuvwjQVYTXh4u+Ukk61xBxYSl0CUBn zAffTJ/B4qMDCsY9MtrgvulhfXkVKD4XfiY8/udxWcKlNZIr+Ec1bJeX/OtZ25Pola2P ZJog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584722; x=1729189522; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RdgAxVyxp31c7WibHLiyIF43paER00mxZOAIHhsdwhk=; b=B9M517QGKU9Xyy2/tqI19d54S91xUGtoJd3hAEu0vQID1tNJkeNtjgnK5egDgeRxEN IAtMUjPZE4bzTCNLQwjA+nfRUZrxq8OzH+ifw6fgLW7H1V7zmUC0N/ua1ktMNKZ9Sp0F pmQYgkVcId944ruC8MBkIAfNAt2LHbJjpUbzWG4ovnnhVO150RF14r3TU2AFjWd23JrK isE4kvldbYaruZ3TvBnpCLU26oBRSV+/lPjZ5/3MhO9bpbZdH8L3zyz6MZYTaTC83Q+Z QAnBe1wwXyyrB+E2cmqjFEuF89iYnS0/0dWBi8/b9OH1FV4ATtB0a4SVZBKMoRL2Jv5/ nYOw== X-Forwarded-Encrypted: i=1; AJvYcCVeKuYVq9uMuZHG8+yYV8o1Dcnz2sKXIuvpxr4MEaXqIFuv+rShDCykpQ8ZsiVBRfXJ//wVXHLhqZlPiBo=@vger.kernel.org X-Gm-Message-State: AOJu0YypMbnKJ75HHkOz/AmwzM/u515DZ8tQjB+ms6DcA6/tfYbWF9So goPmT4/otgih3c45bgdcfdDGbcNPVTV+T/1Zbwad+VBrxno4wm6xwD5cEpA6YijvtzexBQ6YAAk 1kg== X-Google-Smtp-Source: AGHT+IH031wjBK6zBugJiyQwbgM1S3WkZumhuZG0/47H7WWzE24O4tbIqX0eqBdD0TzKWzHgCchY3qP4ics= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:9a94:b0:2e2:da81:40c1 with SMTP id 98e67ed59e1d1-2e2f09f2280mr107a91.1.1728584720854; Thu, 10 Oct 2024 11:25:20 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:16 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-15-seanjc@google.com> Subject: [PATCH v13 14/85] KVM: Return ERR_SIGPENDING from hva_to_pfn() if GUP returns -EGAIN From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Treat an -EAGAIN return from GUP the same as -EINTR and immediately report to the caller that a signal is pending. GUP only returns -EAGAIN if the _initial_ mmap_read_lock_killable() fails, which in turn onnly fails if a signal is pending Note, rwsem_down_read_slowpath() actually returns -EINTR, so GUP is really just making life harder than it needs to be. And the call to mmap_read_lock_killable() in the retry path returns its -errno verbatim, i.e. GUP (and thus KVM) is already handling locking failure this way, but only some of the time. Suggested-by: Paolo Bonzini Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 17acc75990a5..ebba5d22db2d 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2946,7 +2946,7 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interru= ptible, bool *async, writable, &pfn); if (npages =3D=3D 1) return pfn; - if (npages =3D=3D -EINTR) + if (npages =3D=3D -EINTR || npages =3D=3D -EAGAIN) return KVM_PFN_ERR_SIGPENDING; =20 mmap_read_lock(current->mm); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A24DC1EF92F for ; Thu, 10 Oct 2024 18:25:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584726; cv=none; b=cwNVgq5gpLOVtbaRmz/5hbNKay5RfKsavnVqqV6IdqNboZsbuKSlXlTETwQ8dUTu9yRqLTVSUPv262vPNBkgSg5JaZAgqHbvjxieqj6CODiO6ZyT6a08DRrIGOVL8tU7tT10hoeCX6Fv+bdQ/hWZB/4BJque+8USmd01o8ntuYc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584726; c=relaxed/simple; bh=xKxLUn1wM/pJucT4/ZxvVNmNhzhyHMbiLj3YVf1FNHQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hIF4vwXmjczPA+1dll0kHaeh5kgazP1q6uL3i5pTc0MzEgKw6quKNj9hT3KWnJ6Z6zrQGfRxL9FMEEMJqtGz1nBJWStdRnb39ECFAqVQKEtm84uIvXkT2bTvuAMnUi6SAARTMNPmYukZuOsQi+M3wcGT/d+dZmqE6+a9QFRjMjk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TYQRQk+w; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TYQRQk+w" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e2270a147aso21755317b3.0 for ; Thu, 10 Oct 2024 11:25:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584724; x=1729189524; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=86tNdlyX8HXR/WHFYCj/131p34DPDBwWgQrhOuSjETM=; b=TYQRQk+w7SZ6hxu000ERFTE/19nobGDZp4qfI4lUGaKSkx65llk5i8OtSPaWktaPe+ 5XozsAEpjOmnyoNqatnrYMAZ60RWiIscI44p1xR028Kq4YBFHeziOBq1g5YMW369+ewe DBbnBBj7uyWKhzvNHIsKPh1xm7J41dcuSCTEYZDqgPyXGvJRHeKaxnjlU+Q7xVjJlTj+ Py4wWC7F64nR2FkwHexXSLRXrQRj5eudodYY+PhBwB9w0LCVub3wGLZ+EE1KRXqo1yqd P1/WIlzQG/PMilWwZm1sDFH2cYInsL65Bp0RatNOaVo01u0iAbnToWRI8vRt9jBA3Mti z40w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584724; x=1729189524; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=86tNdlyX8HXR/WHFYCj/131p34DPDBwWgQrhOuSjETM=; b=qvoQNyrfkrOzUXJpiVoUl+KdVTbNK6BNUloh2sN4vK6nxlkB77Mh8sHjQLXfNJwQc/ f7Qv2smP3oaRyEa4X85mbLWiTQ5In7s622+LzsN/BJOaKTAn3MqNk6gPvrmIcu3vUgqC lxoue8vwuc36BES447Ic5OK9jHqV0mWKEt22ZTP6BuRO84/m6TiBM3BGYj9G8bEaB8rD HX/NJB5Q2o8chQT3/KGGCFr1dhAePwQ7yRdJvkui11SsHusxal3zTmWdLHvdmNKc4zjH DVr3RHXDlhcB3j42rzm9c+w+pXlsT3ke4Sb8kXKLI21ps4ckqsY7+gC0KfasVDh5r8dE B2Pg== X-Forwarded-Encrypted: i=1; AJvYcCVKvw5pptXjrI9UOrKFoD6jR8SSyuSQrin2wReSVhxopEFwHcsosUfk/F4Jxt02SgMlrWDEKeqXulbEUZk=@vger.kernel.org X-Gm-Message-State: AOJu0Yxxp3+kMUa2zh87C7v7onTGxogmG12NeNOURT1OIatwodSgygqs w5S3ojzUfGEk7vlRZAB9FTYKajG9VAfImFCy0c+zbf3j0xQiLcfGpbq3M1D3JSsvhRiJuDKi2u0 b+Q== X-Google-Smtp-Source: AGHT+IGf82b/hxj2IWFe54/TAnEDS4XyhakmYGUWxryxrCLTzb83+cUU5J1DZhSXbLBNZQvipdjBbhmVkR8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:8e09:0:b0:e28:ef25:5f13 with SMTP id 3f1490d57ef6-e28fe0e7352mr200743276.0.1728584723790; Thu, 10 Oct 2024 11:25:23 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:17 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-16-seanjc@google.com> Subject: [PATCH v13 15/85] KVM: Drop extra GUP (via check_user_page_hwpoison()) to detect poisoned page From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove check_user_page_hwpoison() as it's effectively dead code. Prior to commit 234b239bea39 ("kvm: Faults which trigger IO release the mmap_sem"), hva_to_pfn_slow() wasn't actually a slow path in all cases, i.e. would do get_user_pages_fast() without ever doing slow GUP with FOLL_HWPOISON. Now that hva_to_pfn_slow() is a straight shot to get_user_pages_unlocked(), and unconditionally passes FOLL_HWPOISON, it is impossible for hva_to_pfn() to get an -errno that needs to be morphed to -EHWPOISON. There are essentially four cases in KVM: - npages =3D=3D 0, then FOLL_NOWAIT, a.k.a. @async, must be true, and thus check_user_page_hwpoison() will not be called - npages =3D=3D 1 || npages =3D=3D -EHWPOISON, all good - npages =3D=3D -EINTR || npages =3D=3D -EAGAIN, bail early, all good - everything else, including -EFAULT, can go down the vma_lookup() path, as npages < 0 means KVM went through hva_to_pfn_slow() which passes FOLL_HWPOISON Suggested-by: Paolo Bonzini Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ebba5d22db2d..87f81e74cbc0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2746,14 +2746,6 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vc= pu *vcpu, gfn_t gfn, bool *w return gfn_to_hva_memslot_prot(slot, gfn, writable); } =20 -static inline int check_user_page_hwpoison(unsigned long addr) -{ - int rc, flags =3D FOLL_HWPOISON | FOLL_WRITE; - - rc =3D get_user_pages(addr, 1, flags, NULL); - return rc =3D=3D -EHWPOISON; -} - /* * The fast path to get the writable pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. @@ -2948,14 +2940,10 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool inter= ruptible, bool *async, return pfn; if (npages =3D=3D -EINTR || npages =3D=3D -EAGAIN) return KVM_PFN_ERR_SIGPENDING; + if (npages =3D=3D -EHWPOISON) + return KVM_PFN_ERR_HWPOISON; =20 mmap_read_lock(current->mm); - if (npages =3D=3D -EHWPOISON || - (!async && check_user_page_hwpoison(addr))) { - pfn =3D KVM_PFN_ERR_HWPOISON; - goto exit; - } - retry: vma =3D vma_lookup(current->mm, addr); =20 @@ -2972,7 +2960,6 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interru= ptible, bool *async, *async =3D true; pfn =3D KVM_PFN_ERR_FAULT; } -exit: mmap_read_unlock(current->mm); return pfn; } --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B16141F12E2 for ; Thu, 10 Oct 2024 18:25:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584728; cv=none; b=JKPQh5UPnpmKP4O0113DzCmM27Z+C0vnzHF26vgqNG1rwFLoJ/xxnKaW98QvcRw7/JlR2XKnwSS/1qyzS7UcHZzfQ9zmybR+2a0vdVrjH5ZSk794GWqkzPUhmtm/VJmxfR/mnrg8NJnF79O1EE2d4Y3cyKaj1dybcVonsylCkFg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584728; c=relaxed/simple; bh=42VPy3Va09DWIWyeI7bQhF4N8yIw+oK2ibHTdiPXTHw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=u4OLq5JR3BA0VFI7D7vFol9aBwOxC87TDwCUb+M63tPleHpjO+7tAIr9/sEL16nWF14ma9j+n8D+nrMtyExn184AKrw5snmSqkoL//Vi5nkyEUQe/vrmZs9MeASeSzI4H1PGFZqr2QPK8RQTtMMU2/HiDLZYVrzvQNi2bvjWOmo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WvslKqNS; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WvslKqNS" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71e048c1595so1304411b3a.1 for ; Thu, 10 Oct 2024 11:25:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584726; x=1729189526; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=H+RZpPtsk3Pe/RqqhWN1oE7ZGzOesPmt7slwmkU09yw=; b=WvslKqNSO5DKemaosV5Ekaavvk/7eArebNvJycUuLBkZGB/XJaMFX6aV1iKkGUFwP0 ZNwz0bY08AVY+Ai+JzYMIfwVM/iwhmIFh6ypp19yQJ7xlLlH4E1/h5/i4lkEIEswxRX6 G1V1+MDdkMXxuUeQhS2FytA/DTRX5CMGC7SlmlvH24EV5qDMrezu16iTJ094fO53XMoJ CGl31UkFsqipIwaYzOCagq4DAomkEhANFb6QoDU2IIOMnJOpIdKn8veU7TKRO0XUkEuU U8S1Qx2j8B7VTqF/nKa6l80LAoqMwmyO08nUTDqCHn4jvA8Xb+kpNd8Sq1I3C8YK6lcV TOiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584726; x=1729189526; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=H+RZpPtsk3Pe/RqqhWN1oE7ZGzOesPmt7slwmkU09yw=; b=xI3UTuX4vkN9Vy0ZCyZbXkw8lSykhIss7Ov2ztiKLDv0c5FohUqzmobALlNNp7EUR0 HtlChVmogvHURATnKd4FyKgAtR9Gke4ndy2y+IMDniU2iGmmn0Anyva/KNByK31bXNWS c0gVkqiUlPGUTDffjFCo+IrfrtBzendHuogglnePw6QT4weKqiKaKCaprIMes/zuMkLz ixZZFnipP+i1z7nSQVzdgBF8VIfsLUiDl1oeAi1xnG6xpFCZLhUSdD6J+lmEn46Fn9I2 rxQAUZIJ34kayb6/YadSjt5ToStOMWx3o+4vEYAoC3YjnuIFcVX6tck/4T4BC8lQ7Vhe eNvw== X-Forwarded-Encrypted: i=1; AJvYcCX4tZFDlyyliI6A6lDSnWWf49UljBZDRjIrVN+fNjT0OdP5hCoHXkdgBujt1bXrbE9fAVWXME3gqY93JS8=@vger.kernel.org X-Gm-Message-State: AOJu0Yzv2FSKGhr0FZtyVCUmGNcjet6ICyW1fZr485sbKwa9GCPU5PQq O4LNBi4FdP6qH5qBlyCCKaZSMUXdk7XhLw87t/YSk173cIaDatdSVGqy96usmZ68Qxa3r3XXdDh HmA== X-Google-Smtp-Source: AGHT+IEgY3dn5eo0Xr6pp3rAg0uVd1WuwbDag/Dk23UVvT7MMuxyKVAtWc2T42TlMp1H8D9GJ1Oltmo921o= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:8c17:b0:71d:f452:ee99 with SMTP id d2e1a72fcca58-71e1dbcac44mr6866b3a.3.1728584725637; Thu, 10 Oct 2024 11:25:25 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:18 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-17-seanjc@google.com> Subject: [PATCH v13 16/85] KVM: Replace "async" pointer in gfn=>pfn with "no_wait" and error code From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Stevens Add a pfn error code to communicate that hva_to_pfn() failed because I/O was needed and disallowed, and convert @async to a constant @no_wait boolean. This will allow eliminating the @no_wait param by having callers pass in FOLL_NOWAIT along with other FOLL_* flags. Tested-by: Alex Benn=C3=A9e Signed-off-by: David Stevens Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 18 +++++++++++------- include/linux/kvm_host.h | 3 ++- virt/kvm/kvm_main.c | 27 ++++++++++++++------------- virt/kvm/kvm_mm.h | 2 +- virt/kvm/pfncache.c | 4 ++-- 5 files changed, 30 insertions(+), 24 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0e235f276ee5..fa8f3fb7c14b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4374,17 +4374,21 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu = *vcpu, =20 static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault = *fault) { - bool async; - if (fault->is_private) return kvm_faultin_pfn_private(vcpu, fault); =20 - async =3D false; - fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, &asyn= c, + fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, fault->write, &fault->map_writable, &fault->hva); - if (!async) - return RET_PF_CONTINUE; /* *pfn has correct page already */ + + /* + * If resolving the page failed because I/O is needed to fault-in the + * page, then either set up an asynchronous #PF to do the I/O, or if + * doing an async #PF isn't possible, retry with I/O allowed. All + * other failures are terminal, i.e. retrying won't help. + */ + if (fault->pfn !=3D KVM_PFN_ERR_NEEDS_IO) + return RET_PF_CONTINUE; =20 if (!fault->prefetch && kvm_can_do_async_pf(vcpu)) { trace_kvm_try_async_get_page(fault->addr, fault->gfn); @@ -4402,7 +4406,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault * to wait for IO. Note, gup always bails if it is unable to quickly * get a page and a fatal signal, i.e. SIGKILL, is pending. */ - fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, NULL, + fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, fault->write, &fault->map_writable, &fault->hva); return RET_PF_CONTINUE; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 2faafc7a56ae..071a0a1f1c60 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -97,6 +97,7 @@ #define KVM_PFN_ERR_HWPOISON (KVM_PFN_ERR_MASK + 1) #define KVM_PFN_ERR_RO_FAULT (KVM_PFN_ERR_MASK + 2) #define KVM_PFN_ERR_SIGPENDING (KVM_PFN_ERR_MASK + 3) +#define KVM_PFN_ERR_NEEDS_IO (KVM_PFN_ERR_MASK + 4) =20 /* * error pfns indicate that the gfn is in slot but faild to @@ -1233,7 +1234,7 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn,= bool write_fault, bool *writable); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn= ); kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t g= fn, - bool interruptible, bool *async, + bool interruptible, bool no_wait, bool write_fault, bool *writable, hva_t *hva); =20 void kvm_release_pfn_clean(kvm_pfn_t pfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 87f81e74cbc0..dd5839abef6c 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2778,7 +2778,7 @@ static bool hva_to_pfn_fast(unsigned long addr, bool = write_fault, * The slow path to get the pfn of the specified host virtual address, * 1 indicates success, -errno is returned if error is detected. */ -static int hva_to_pfn_slow(unsigned long addr, bool *async, bool write_fau= lt, +static int hva_to_pfn_slow(unsigned long addr, bool no_wait, bool write_fa= ult, bool interruptible, bool *writable, kvm_pfn_t *pfn) { /* @@ -2801,7 +2801,7 @@ static int hva_to_pfn_slow(unsigned long addr, bool *= async, bool write_fault, =20 if (write_fault) flags |=3D FOLL_WRITE; - if (async) + if (no_wait) flags |=3D FOLL_NOWAIT; if (interruptible) flags |=3D FOLL_INTERRUPTIBLE; @@ -2912,8 +2912,8 @@ static int hva_to_pfn_remapped(struct vm_area_struct = *vma, * Pin guest page in memory and return its pfn. * @addr: host virtual address which maps memory to the guest * @interruptible: whether the process can be interrupted by non-fatal sig= nals - * @async: whether this function need to wait IO complete if the - * host page is not in the memory + * @no_wait: whether or not this function need to wait IO complete if the + * host page is not in the memory * @write_fault: whether we should get a writable host page * @writable: whether it allows to map a writable host page for !@write_fa= ult * @@ -2922,7 +2922,7 @@ static int hva_to_pfn_remapped(struct vm_area_struct = *vma, * 2): @write_fault =3D false && @writable, @writable will tell the caller * whether the mapping is writable. */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, bool write_fault, bool *writable) { struct vm_area_struct *vma; @@ -2934,7 +2934,7 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interru= ptible, bool *async, if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) return pfn; =20 - npages =3D hva_to_pfn_slow(addr, async, write_fault, interruptible, + npages =3D hva_to_pfn_slow(addr, no_wait, write_fault, interruptible, writable, &pfn); if (npages =3D=3D 1) return pfn; @@ -2956,16 +2956,17 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool inter= ruptible, bool *async, if (r < 0) pfn =3D KVM_PFN_ERR_FAULT; } else { - if (async && vma_is_valid(vma, write_fault)) - *async =3D true; - pfn =3D KVM_PFN_ERR_FAULT; + if (no_wait && vma_is_valid(vma, write_fault)) + pfn =3D KVM_PFN_ERR_NEEDS_IO; + else + pfn =3D KVM_PFN_ERR_FAULT; } mmap_read_unlock(current->mm); return pfn; } =20 kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t g= fn, - bool interruptible, bool *async, + bool interruptible, bool no_wait, bool write_fault, bool *writable, hva_t *hva) { unsigned long addr =3D __gfn_to_hva_many(slot, gfn, NULL, write_fault); @@ -2987,21 +2988,21 @@ kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_mem= ory_slot *slot, gfn_t gfn, writable =3D NULL; } =20 - return hva_to_pfn(addr, interruptible, async, write_fault, writable); + return hva_to_pfn(addr, interruptible, no_wait, write_fault, writable); } EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); =20 kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { - return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, NULL, + return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, write_fault, writable, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); =20 kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, NULL, true, NULL, NULL); + return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); =20 diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index a3fa86f60d6c..51f3fee4ca3f 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -20,7 +20,7 @@ #define KVM_MMU_UNLOCK(kvm) spin_unlock(&(kvm)->mmu_lock) #endif /* KVM_HAVE_MMU_RWLOCK */ =20 -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, bool write_fault, bool *writable); =20 #ifdef CONFIG_HAVE_KVM_PFNCACHE diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 58c706a610e5..32dc61f48c81 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -197,8 +197,8 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cac= he *gpc) cond_resched(); } =20 - /* We always request a writeable mapping */ - new_pfn =3D hva_to_pfn(gpc->uhva, false, NULL, true, NULL); + /* We always request a writable mapping */ + new_pfn =3D hva_to_pfn(gpc->uhva, false, false, true, NULL); if (is_error_noslot_pfn(new_pfn)) goto out_error; =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 718401F1310 for ; Thu, 10 Oct 2024 18:25:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584730; cv=none; b=QT/fBHi3zVZVXFP8hODIIBk+QSK/6DlQh+cgrvecz+pBHQhRGof0iNbQBCE9K0naCpfYPEeco1oS3eeD2+mklq0sjJbQ6XvQKlx4207lphRWBy1/O3CCVtEGkRm/XNSQnO1nz4oBEz+U5855vd1Ql/sRvcYSNFn3JmariDVCDac= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584730; c=relaxed/simple; bh=vtDT6oLoOmF91AiioFXu5/ol9RZL+XplRHPVYvyqLOQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=tXY2z1L3biN0pqJX9EgTIg1Czq9fuyHaet8DSB44dvmfiSofgx7RptE16cSuxRueBWtmlbj7IsbP0VzwLWwWsv807lt1s7irWbXzDVwfoytavkLERJxgJdqYdcczYdHzeixRwUXhcPQmq25nYRXTVdoho1+oMBuZXpIxBLiN1I0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Dha/A36a; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Dha/A36a" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-20c7ea6b060so15661845ad.1 for ; Thu, 10 Oct 2024 11:25:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584728; x=1729189528; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=5N9i4qcLR1M4w0icoJwy+0qPwifBeFycVmWqg2S4YD0=; b=Dha/A36ak1juWSy53/C5jnDKwMB3ibLmNWOqN2CaIPVTvc2bOsO+vIG4ji5VUNB+Fr gAcZshJlAnKB9IGnefOr8J2/sIsOnWzbaUSaKAYAfY0GdCKdUUDLPpPmylbsAc5fiJtQ e+FQSoUCLb8PwxOQTC2Wp4MvrQOZvR+GL6RpTuUF2Ws2zSXOwcR2KcfTnxlrY2LApBDX 1jDtMaJUCphK2q+yi25T6nAEQFpfTOKBCtnS6RvfK2LDAc5h9R7TmHsLd/jFQ02K/iU9 XMkUMOtKJ9vnvdqixw1TS95uc48IbgO3uWWv/2SyHoFjG6zpnpjdrM+rk6uPgJaCnthV Nb3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584728; x=1729189528; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=5N9i4qcLR1M4w0icoJwy+0qPwifBeFycVmWqg2S4YD0=; b=RCcBQm6ucRZ/HUAavuDLhjas4e6jbxUOBknOPzGq1MY9IIj7ekOotTjWA2HD0coegW mVxe9U09e/gzyyWpgCz2LyLDvImiC6sVmh2qjuaZAFMmSQ6jmwUkAWYIG9bPmE2ly3Sv oK+lF2pjcI595Bgu0kfbEz9VY/LeOSegwyVLelpyZQCqCCkKY80QgdNN09rWzTSLxRA0 w95Rh5ESJzzJMzDeCvogHrMplOhniOVTdi4jcw3FYPNmTN51Mhz1f3db+gx8oJyeeQ2I 0b0Q4657qVBtJOUGAVpG9hxqvLn3ygVgAiwlgLCK3xpYr4/lVTHKnrWXgK7AiaOdIBq6 qBDg== X-Forwarded-Encrypted: i=1; AJvYcCWlqs+Fk94vnmWkeuYlY8zpPm5hVQaPBugJrEPJdUPpnesFKS3dli+XDFQ44nNueN9GWYNfhQTzk7esZNU=@vger.kernel.org X-Gm-Message-State: AOJu0Ywt0wJ/MghDoNsUHstitT5mgyVJWiQvNZH6Twi++oIjao+bZbvD +W7sbWvlfmGynKUQHCIBu7BT6xTB0Ri+UNGSdDq2PSjUAf0thZbZa6UHmEzPxiq9Uc5gh9nbpAo Rlw== X-Google-Smtp-Source: AGHT+IHx3wOCIkhIacFyS0KkUCfogOtyOnmQIp9NOdA3rn83blMwYV/7exbhoH/uHIfky6alS3vCfOjmfpM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:d502:b0:20b:7ece:3225 with SMTP id d9443c01a7336-20c6358fdd3mr1169885ad.0.1728584727745; Thu, 10 Oct 2024 11:25:27 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:19 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-18-seanjc@google.com> Subject: [PATCH v13 17/85] KVM: x86/mmu: Drop kvm_page_fault.hva, i.e. don't track intermediate hva From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove kvm_page_fault.hva as it is never read, only written. This will allow removing the @hva param from __gfn_to_pfn_memslot(). Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 5 ++--- arch/x86/kvm/mmu/mmu_internal.h | 2 -- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index fa8f3fb7c14b..c67228b46bd5 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3294,7 +3294,6 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *v= cpu, fault->slot =3D NULL; fault->pfn =3D KVM_PFN_NOSLOT; fault->map_writable =3D false; - fault->hva =3D KVM_HVA_ERR_BAD; =20 /* * If MMIO caching is disabled, emulate immediately without @@ -4379,7 +4378,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault =20 fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, fault->write, &fault->map_writable, - &fault->hva); + NULL); =20 /* * If resolving the page failed because I/O is needed to fault-in the @@ -4408,7 +4407,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault */ fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, fault->write, &fault->map_writable, - &fault->hva); + NULL); return RET_PF_CONTINUE; } =20 diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 4da83544c4e1..633aedec3c2e 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -238,7 +238,6 @@ struct kvm_page_fault { /* Outputs of kvm_faultin_pfn. */ unsigned long mmu_seq; kvm_pfn_t pfn; - hva_t hva; bool map_writable; =20 /* @@ -313,7 +312,6 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu= *vcpu, gpa_t cr2_or_gpa, .is_private =3D err & PFERR_PRIVATE_ACCESS, =20 .pfn =3D KVM_PFN_ERR_FAULT, - .hva =3D KVM_HVA_ERR_BAD, }; int r; =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0275D1F4FAA for ; Thu, 10 Oct 2024 18:25:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584733; cv=none; b=VrRw74tiG9FlsScMMAscftaT2IQtiEIzPEbmVxTLpTGNIO/Ebp/bDkrGpe1vIHwoyzVVCA8y03UA9i8CAnRaJ/tO4nkDgf4BED1eLzh8J/MiUrXW8OHvpsQXkwzf9nPhwFqvhl5ZUGCOi/4q/8/Bya6Opwkqerwc2275UdP7OFc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584733; c=relaxed/simple; bh=gORZY+kwKqxFhTI7jRiw4mW6KKJknMWDgC2YkpetElw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fOb38K0ptUaVHqSA7tD7Zligjj4RaAV6jLPm63Th5xbDof3M543YXm9VSk1O0XBnPpSuuKXgnSg0OmxLO5JZYgD8PIIoTinP3vVkRPaEVP3YzR5UiSdtNw3Dmx5rVTwGrEHeWNUzsVU9hoD/Nno807pxe3rDICdu1tU0ma384Pg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=eIGj+Y80; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eIGj+Y80" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e29facda92so1239607a91.1 for ; Thu, 10 Oct 2024 11:25:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584731; x=1729189531; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=KxFe7qyo4eHIzhTxczhq6OF5aBNVD8CbU/vq6HHGwtQ=; b=eIGj+Y80OKxaBgravZE7eiui0FBQFj4LLKuhGsYIejJ/ru27Nu4bo6PYyo0RqdW6jx mlEsjNvzm4bt/om7CPDzDCIitHXfOAawPomD6hoIfarHR+jb/Vpik0cM7RCURqHtp9Th KS+pp/789ua5N645ObTpGNCJ5zY5VYGSoas2pjsYOGcz4D//yVxmivdlBgrIuPOsZsuD PsIcOCklR+rODFwgTw/JLAwVlh79y7eUR2dCyqq7sfL4r4jGaiVeE7naHt9ulZ5giDzT SQCuZV4v5mlU7zeVE9DYptsrrR1n7XGueT0FUJrtCmHgSugT/5LXEjrWUt41nO2Mw821 sr8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584731; x=1729189531; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=KxFe7qyo4eHIzhTxczhq6OF5aBNVD8CbU/vq6HHGwtQ=; b=U5tJVwdLf3qA+TdC6cD8U4oyF0woeGH2ROB2+qkHcrdPeFl7wYkJ/+gtFRP5q1DkEK AApIdv+7pWwD0oJDwC8kIe7x1cjxp5fwIQG2TZUq4A2ZjKFJr6ZH1ixrzQnf4NEKJZv9 kXZEE5QT3EebFWkfVxsaQpGjPsYX3rLd0ZnRccgp4HS+3FNc+6yIo68bk0v+HBG2C6Zl SOFPPcKPXiLgxW1Ua9DSxCklzvuIA/CL+l8NHMsPxWaNztNTNxeEDY/nz57jvS7O0F5X b8FaaygEIb0/mQq5ybOTvUWvrSKY0dcCB310/Nn1yU5bqorGWp24fLcZBZkSKkehLMvb kaPg== X-Forwarded-Encrypted: i=1; AJvYcCW4hLaCEUURuBotF8/vNGPMmdPo60zijLusC/gsfneByHLcFVyKti9Rt2/hhMtMED4tgDxRnIFYvyJr6gs=@vger.kernel.org X-Gm-Message-State: AOJu0YzeTH+/knD5DoHHDYe78q8xdZzMX30fl9dm+V5JQxx59+blFnj1 42g2IwG01o4mCU/4VlrYAEPntaK5leZ5fy74fC5yZDRBzYGGrhF1wUOUsd+8J5K3bXw0kd0EcTp +oA== X-Google-Smtp-Source: AGHT+IGFsZ8G+8oqHMXYnd6l/b9oSo3/s957bEBlaeuh+YsQIWv/Tzia2RQhXQw4+rIMPn/3epGKWmgCn2Q= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:5107:b0:2e2:ca3e:10fe with SMTP id 98e67ed59e1d1-2e2f0f88a54mr20a91.8.1728584729847; Thu, 10 Oct 2024 11:25:29 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:20 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-19-seanjc@google.com> Subject: [PATCH v13 18/85] KVM: Drop unused "hva" pointer from __gfn_to_pfn_memslot() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop @hva from __gfn_to_pfn_memslot() now that all callers pass NULL. No functional change intended. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/arm64/kvm/mmu.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 +- arch/x86/kvm/mmu/mmu.c | 6 ++---- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 9 +++------ 6 files changed, 9 insertions(+), 14 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index a6e62cc9015c..dd221587fcca 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1570,7 +1570,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys= _addr_t fault_ipa, mmap_read_unlock(current->mm); =20 pfn =3D __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - write_fault, &writable, NULL); + write_fault, &writable); if (pfn =3D=3D KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); return 0; diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_= 64_mmu_hv.c index 8cd02ca4b1b8..2f1d58984b41 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -614,7 +614,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu, } else { /* Call KVM generic code to do the slow-path check */ pfn =3D __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, &write_ok, NULL); + writing, &write_ok); if (is_error_noslot_pfn(pfn)) return -EFAULT; page =3D NULL; diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book= 3s_64_mmu_radix.c index 26a969e935e3..8304b6f8fe45 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -853,7 +853,7 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcp= u, =20 /* Call KVM generic code to do the slow-path check */ pfn =3D __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, upgrade_p, NULL); + writing, upgrade_p); if (is_error_noslot_pfn(pfn)) return -EFAULT; page =3D NULL; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index c67228b46bd5..28f2b842d6ca 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4377,8 +4377,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault return kvm_faultin_pfn_private(vcpu, fault); =20 fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, - fault->write, &fault->map_writable, - NULL); + fault->write, &fault->map_writable); =20 /* * If resolving the page failed because I/O is needed to fault-in the @@ -4406,8 +4405,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault * get a page and a fatal signal, i.e. SIGKILL, is pending. */ fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, - fault->write, &fault->map_writable, - NULL); + fault->write, &fault->map_writable); return RET_PF_CONTINUE; } =20 diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 071a0a1f1c60..cbc7b9c04c14 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1235,7 +1235,7 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn,= bool write_fault, kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn= ); kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t g= fn, bool interruptible, bool no_wait, - bool write_fault, bool *writable, hva_t *hva); + bool write_fault, bool *writable); =20 void kvm_release_pfn_clean(kvm_pfn_t pfn); void kvm_release_pfn_dirty(kvm_pfn_t pfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index dd5839abef6c..10071f31b2ca 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2967,13 +2967,10 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool inter= ruptible, bool no_wait, =20 kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t g= fn, bool interruptible, bool no_wait, - bool write_fault, bool *writable, hva_t *hva) + bool write_fault, bool *writable) { unsigned long addr =3D __gfn_to_hva_many(slot, gfn, NULL, write_fault); =20 - if (hva) - *hva =3D addr; - if (kvm_is_error_hva(addr)) { if (writable) *writable =3D false; @@ -2996,13 +2993,13 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gf= n, bool write_fault, bool *writable) { return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, - write_fault, writable, NULL); + write_fault, writable); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); =20 kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL, NULL); + return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C9391F4FC8 for ; Thu, 10 Oct 2024 18:25:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584737; cv=none; b=kzJF5sHRv1lnFJHEYaOz76eIy2nOTGkmHnr7/4vJRQSAeiCVZVMsaQXLKM7daIDfyvMjivuB0FYAcz1+qjR3WmmvyLBIqVh1Z6LmRNc7Xf0oDSehmfyYU0sUyp11c4EEsG6LntotXjIsRB5xf4NCSy7tgBLaahfrLfVg9ghdet8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584737; c=relaxed/simple; bh=B6DKB14GzdU+gSziUKXJ1u64FTzTfv1AIh0aTxVhDrQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fbYdsg979CP3qYJpfIEuqzEYL7HETiN9bo49IgPL7csZbY1mahaAaEz1derPJHCOfS2TzWWIXJ6jV7oNV5MRpD3YX7jVKl9E0I9vWNpE/WsdXUra0liknghRCDBazwf0K6BSsSsChZvguz9/uSABFGpA3zUOrMl4x4njIIull8E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=g1s0phY3; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="g1s0phY3" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e278b2a5c4so21557237b3.2 for ; Thu, 10 Oct 2024 11:25:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584733; x=1729189533; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=GUnoK8hes+aZNWzzso7nvh51KinEVhA6U4mspQ9USyg=; b=g1s0phY3wpDJ54yACpNQxa2jJ5a2WcyMg39r6mcL0Y5WvOsI7TPZmKy87sNNnninoG zinstTd6W23/g/DTDy9vblMtUSS/Mw0aCsL5WxcWrXK9YHs7OO15+K5dRJE45N88mZ5k bMgGbOPlYZoLPGfXxDDp3SS2NsWU9F9TnainvF/SLhaC9R/T8+lj/750qjFxMw/9Xfiy Zwf1LPVrkRC722UWTl+lDusREgs7UPSV2fkeMFMKdlJAT0pFqDWDX48hjEcLhHzWX+ly PthT23TBgnEc1ZqcJZE7l64Sk6YgJ5BG3NR4GOREQp1+pZoy+Ws+Z8Vq7urpfzDZJtdV ARmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584733; x=1729189533; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=GUnoK8hes+aZNWzzso7nvh51KinEVhA6U4mspQ9USyg=; b=MFIOCLmhqGLKP7sN6ZRb1vmeUqWXV0+nN9ikF+J42vn0oreihOseyhCXRdvH1PMJ/q SLYFcxETmJF0t27ZAi2kvVBWLFqjRapA3VJ2jYSe/8olxPU2LegSyIn6f1R1hvfAhKkd fGDy+J8RNSyjS07D3nb17gpHqq0a5GU4UPIciADSxno0abjo2rkIlA0zJClN8fH+0Xan YYcroSphR0hZ8hj7OD/2tYgU7sgYo37I1WEElf2Xo5wwfQhvb7zTAy+m2iyU55Ar5qHK heT2V7DpAhGluPHrtOkXAoSUZPT/Tb64OJ0HpjagO3N5CZt8JCmBeFGIOfS9iTtxYisU KkFQ== X-Forwarded-Encrypted: i=1; AJvYcCXwMG3uqWjcAxdcxTmNg3BvHR1sVxDVqw38et0n0sH9xsMkU/wRYS/iC2qod9pYiaJFc3CNgHFCuDHlSeQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yy+N7EwLz0BkU54GwaOUcKpAsPZ5+2hXT0mzLSh8+NOK+ACLxOT 9nz9UnvIKpXaIi2IovepXzw7gAFG6RU8EWcMfa2BOEe7LgAU6YvQe1I2awWlvizPFJGtySYIbkR VHg== X-Google-Smtp-Source: AGHT+IHJGx3vYA/ciscOnZoyMGBDqN0A5fNCXYXCqQ4jImN8v4VZ+ud/0qiiFEbLMXTR4U3Rg3+GOt3HIs4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2c01:b0:6db:d257:b98 with SMTP id 00721157ae682-6e32215253bmr563917b3.3.1728584733164; Thu, 10 Oct 2024 11:25:33 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:21 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-20-seanjc@google.com> Subject: [PATCH v13 19/85] KVM: Introduce kvm_follow_pfn() to eventually replace "gfn_to_pfn" APIs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Stevens Introduce kvm_follow_pfn() to eventually supplant the various "gfn_to_pfn" APIs, albeit by adding more wrappers. The primary motivation of the new helper is to pass a structure instead of an ever changing set of parameters, e.g. so that tweaking the behavior, inputs, and/or outputs of the "to pfn" helpers doesn't require churning half of KVM. In the more distant future, the APIs exposed to arch code could also follow suit, e.g. by adding something akin to x86's "struct kvm_page_fault" when faulting in guest memory. But for now, the goal is purely to clean up KVM's "internal" MMU code. As part of the conversion, replace the write_fault, interruptible, and no-wait boolean flags with FOLL_WRITE, FOLL_INTERRUPTIBLE, and FOLL_NOWAIT respectively. Collecting the various FOLL_* flags into a single field will again ease the pain of passing new flags. Tested-by: Alex Benn=C3=A9e Signed-off-by: David Stevens Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 162 +++++++++++++++++++++++--------------------- virt/kvm/kvm_mm.h | 20 +++++- virt/kvm/pfncache.c | 9 ++- 3 files changed, 109 insertions(+), 82 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 10071f31b2ca..52629ac26119 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2750,8 +2750,7 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcp= u *vcpu, gfn_t gfn, bool *w * The fast path to get the writable pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. */ -static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, - bool *writable, kvm_pfn_t *pfn) +static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { struct page *page[1]; =20 @@ -2760,14 +2759,13 @@ static bool hva_to_pfn_fast(unsigned long addr, boo= l write_fault, * or the caller allows to map a writable pfn for a read fault * request. */ - if (!(write_fault || writable)) + if (!((kfp->flags & FOLL_WRITE) || kfp->map_writable)) return false; =20 - if (get_user_page_fast_only(addr, FOLL_WRITE, page)) { + if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, page)) { *pfn =3D page_to_pfn(page[0]); - - if (writable) - *writable =3D true; + if (kfp->map_writable) + *kfp->map_writable =3D true; return true; } =20 @@ -2778,8 +2776,7 @@ static bool hva_to_pfn_fast(unsigned long addr, bool = write_fault, * The slow path to get the pfn of the specified host virtual address, * 1 indicates success, -errno is returned if error is detected. */ -static int hva_to_pfn_slow(unsigned long addr, bool no_wait, bool write_fa= ult, - bool interruptible, bool *writable, kvm_pfn_t *pfn) +static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { /* * When a VCPU accesses a page that is not mapped into the secondary @@ -2792,34 +2789,30 @@ static int hva_to_pfn_slow(unsigned long addr, bool= no_wait, bool write_fault, * Note that get_user_page_fast_only() and FOLL_WRITE for now * implicitly honor NUMA hinting faults and don't need this flag. */ - unsigned int flags =3D FOLL_HWPOISON | FOLL_HONOR_NUMA_FAULT; - struct page *page; + unsigned int flags =3D FOLL_HWPOISON | FOLL_HONOR_NUMA_FAULT | kfp->flags; + struct page *page, *wpage; int npages; =20 - if (writable) - *writable =3D write_fault; - - if (write_fault) - flags |=3D FOLL_WRITE; - if (no_wait) - flags |=3D FOLL_NOWAIT; - if (interruptible) - flags |=3D FOLL_INTERRUPTIBLE; - - npages =3D get_user_pages_unlocked(addr, 1, &page, flags); + npages =3D get_user_pages_unlocked(kfp->hva, 1, &page, flags); if (npages !=3D 1) return npages; =20 + if (!kfp->map_writable) + goto out; + + if (kfp->flags & FOLL_WRITE) { + *kfp->map_writable =3D true; + goto out; + } + /* map read fault as writable if possible */ - if (unlikely(!write_fault) && writable) { - struct page *wpage; - - if (get_user_page_fast_only(addr, FOLL_WRITE, &wpage)) { - *writable =3D true; - put_page(page); - page =3D wpage; - } + if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { + *kfp->map_writable =3D true; + put_page(page); + page =3D wpage; } + +out: *pfn =3D page_to_pfn(page); return npages; } @@ -2846,10 +2839,10 @@ static int kvm_try_get_pfn(kvm_pfn_t pfn) } =20 static int hva_to_pfn_remapped(struct vm_area_struct *vma, - unsigned long addr, bool write_fault, - bool *writable, kvm_pfn_t *p_pfn) + struct kvm_follow_pfn *kfp, kvm_pfn_t *p_pfn) { - struct follow_pfnmap_args args =3D { .vma =3D vma, .address =3D addr }; + struct follow_pfnmap_args args =3D { .vma =3D vma, .address =3D kfp->hva = }; + bool write_fault =3D kfp->flags & FOLL_WRITE; kvm_pfn_t pfn; int r; =20 @@ -2860,7 +2853,7 @@ static int hva_to_pfn_remapped(struct vm_area_struct = *vma, * not call the fault handler, so do it here. */ bool unlocked =3D false; - r =3D fixup_user_fault(current->mm, addr, + r =3D fixup_user_fault(current->mm, kfp->hva, (write_fault ? FAULT_FLAG_WRITE : 0), &unlocked); if (unlocked) @@ -2878,8 +2871,8 @@ static int hva_to_pfn_remapped(struct vm_area_struct = *vma, goto out; } =20 - if (writable) - *writable =3D args.writable; + if (kfp->map_writable) + *kfp->map_writable =3D args.writable; pfn =3D args.pfn; =20 /* @@ -2908,22 +2901,7 @@ static int hva_to_pfn_remapped(struct vm_area_struct= *vma, return r; } =20 -/* - * Pin guest page in memory and return its pfn. - * @addr: host virtual address which maps memory to the guest - * @interruptible: whether the process can be interrupted by non-fatal sig= nals - * @no_wait: whether or not this function need to wait IO complete if the - * host page is not in the memory - * @write_fault: whether we should get a writable host page - * @writable: whether it allows to map a writable host page for !@write_fa= ult - * - * The function will map a writable host page for these two cases: - * 1): @write_fault =3D true - * 2): @write_fault =3D false && @writable, @writable will tell the caller - * whether the mapping is writable. - */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, - bool write_fault, bool *writable) +kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp) { struct vm_area_struct *vma; kvm_pfn_t pfn; @@ -2931,11 +2909,10 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool inter= ruptible, bool no_wait, =20 might_sleep(); =20 - if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) + if (hva_to_pfn_fast(kfp, &pfn)) return pfn; =20 - npages =3D hva_to_pfn_slow(addr, no_wait, write_fault, interruptible, - writable, &pfn); + npages =3D hva_to_pfn_slow(kfp, &pfn); if (npages =3D=3D 1) return pfn; if (npages =3D=3D -EINTR || npages =3D=3D -EAGAIN) @@ -2945,18 +2922,19 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool inter= ruptible, bool no_wait, =20 mmap_read_lock(current->mm); retry: - vma =3D vma_lookup(current->mm, addr); + vma =3D vma_lookup(current->mm, kfp->hva); =20 if (vma =3D=3D NULL) pfn =3D KVM_PFN_ERR_FAULT; else if (vma->vm_flags & (VM_IO | VM_PFNMAP)) { - r =3D hva_to_pfn_remapped(vma, addr, write_fault, writable, &pfn); + r =3D hva_to_pfn_remapped(vma, kfp, &pfn); if (r =3D=3D -EAGAIN) goto retry; if (r < 0) pfn =3D KVM_PFN_ERR_FAULT; } else { - if (no_wait && vma_is_valid(vma, write_fault)) + if ((kfp->flags & FOLL_NOWAIT) && + vma_is_valid(vma, kfp->flags & FOLL_WRITE)) pfn =3D KVM_PFN_ERR_NEEDS_IO; else pfn =3D KVM_PFN_ERR_FAULT; @@ -2965,41 +2943,69 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool inter= ruptible, bool no_wait, return pfn; } =20 +static kvm_pfn_t kvm_follow_pfn(struct kvm_follow_pfn *kfp) +{ + kfp->hva =3D __gfn_to_hva_many(kfp->slot, kfp->gfn, NULL, + kfp->flags & FOLL_WRITE); + + if (kfp->hva =3D=3D KVM_HVA_ERR_RO_BAD) + return KVM_PFN_ERR_RO_FAULT; + + if (kvm_is_error_hva(kfp->hva)) + return KVM_PFN_NOSLOT; + + if (memslot_is_readonly(kfp->slot) && kfp->map_writable) { + *kfp->map_writable =3D false; + kfp->map_writable =3D NULL; + } + + return hva_to_pfn(kfp); +} + kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t g= fn, bool interruptible, bool no_wait, bool write_fault, bool *writable) { - unsigned long addr =3D __gfn_to_hva_many(slot, gfn, NULL, write_fault); - - if (kvm_is_error_hva(addr)) { - if (writable) - *writable =3D false; - - return addr =3D=3D KVM_HVA_ERR_RO_BAD ? KVM_PFN_ERR_RO_FAULT : - KVM_PFN_NOSLOT; - } - - /* Do not map writable pfn in the readonly memslot. */ - if (writable && memslot_is_readonly(slot)) { - *writable =3D false; - writable =3D NULL; - } - - return hva_to_pfn(addr, interruptible, no_wait, write_fault, writable); + struct kvm_follow_pfn kfp =3D { + .slot =3D slot, + .gfn =3D gfn, + .map_writable =3D writable, + }; + + if (write_fault) + kfp.flags |=3D FOLL_WRITE; + if (no_wait) + kfp.flags |=3D FOLL_NOWAIT; + if (interruptible) + kfp.flags |=3D FOLL_INTERRUPTIBLE; + + return kvm_follow_pfn(&kfp); } EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); =20 kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { - return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, - write_fault, writable); + struct kvm_follow_pfn kfp =3D { + .slot =3D gfn_to_memslot(kvm, gfn), + .gfn =3D gfn, + .flags =3D write_fault ? FOLL_WRITE : 0, + .map_writable =3D writable, + }; + + return kvm_follow_pfn(&kfp); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); =20 kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL); + struct kvm_follow_pfn kfp =3D { + .slot =3D slot, + .gfn =3D gfn, + .flags =3D FOLL_WRITE, + }; + + return kvm_follow_pfn(&kfp); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); =20 diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index 51f3fee4ca3f..d5a215958f06 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -20,8 +20,24 @@ #define KVM_MMU_UNLOCK(kvm) spin_unlock(&(kvm)->mmu_lock) #endif /* KVM_HAVE_MMU_RWLOCK */ =20 -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, - bool write_fault, bool *writable); + +struct kvm_follow_pfn { + const struct kvm_memory_slot *slot; + const gfn_t gfn; + + unsigned long hva; + + /* FOLL_* flags modifying lookup behavior, e.g. FOLL_WRITE. */ + unsigned int flags; + + /* + * If non-NULL, try to get a writable mapping even for a read fault. + * Set to true if a writable mapping was obtained. + */ + bool *map_writable; +}; + +kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp); =20 #ifdef CONFIG_HAVE_KVM_PFNCACHE void gfn_to_pfn_cache_invalidate_start(struct kvm *kvm, diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 32dc61f48c81..067daf9ad6ef 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -159,6 +159,12 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_ca= che *gpc) kvm_pfn_t new_pfn =3D KVM_PFN_ERR_FAULT; void *new_khva =3D NULL; unsigned long mmu_seq; + struct kvm_follow_pfn kfp =3D { + .slot =3D gpc->memslot, + .gfn =3D gpa_to_gfn(gpc->gpa), + .flags =3D FOLL_WRITE, + .hva =3D gpc->uhva, + }; =20 lockdep_assert_held(&gpc->refresh_lock); =20 @@ -197,8 +203,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cac= he *gpc) cond_resched(); } =20 - /* We always request a writable mapping */ - new_pfn =3D hva_to_pfn(gpc->uhva, false, false, true, NULL); + new_pfn =3D hva_to_pfn(&kfp); if (is_error_noslot_pfn(new_pfn)) goto out_error; =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B28751F4FD7 for ; Thu, 10 Oct 2024 18:25:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584737; cv=none; b=q3m18nYasY8GgRBZ1W9ba5RnTru9MC1iraL4DCSqXtmii4y3pgTet+YKyLaIdkAsrdDU0aoPxOGSAjomeQ4U7I+Y7nwSvntMj6FIi//vfDvED4upTXM+6oCvGXts1Kv0vpQpsAz1U5zWA28mgN188VKYIo5GV7nbuOtov4TPiG0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584737; c=relaxed/simple; bh=KvCWyyHWjrlulYIoOxtt/Dqt8bcDl+mOtHXiCiz5pRo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oyzqa8IVmkttopyw/qz7u07fCv6L+ayZuUcDRj3nhftXph6440cZbImweBtRSq1GxcC0RcldlQLs+ujlzZNG1/KtcHcLLwfikFuFRlGB87Rb2mST0D54iWjPszjb4Ss3KZnqyZZsu3PEeAeiNRANKYh/vYHPGmrNsDIX6hPIAKU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=aZapr+Qp; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="aZapr+Qp" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e2dcea0f03so997752a91.1 for ; Thu, 10 Oct 2024 11:25:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584735; x=1729189535; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=LNPEWcjaz+dEwpcBQF1ZcKP3HAxUc/LzxURpjzN7wyY=; b=aZapr+QpZhS7r/MQaYKqDvjfYRBORnxInsXEL92L+scGg5hKtUtQE7o8nOHIrYusqm 5EmvoFkDqXFzl2HpKgkfAiP5lENBwGYDxj5waWmwhFK3gp6/VX/zM/witRAv91QhuvDh hk9MOt+wYPDWmF57nqd7mgAbHZd24J/E0TKSFO7QbNz5i+Ba9ZoHcPZ0IONz8KHYSAGl 5BXGeZ4jCqP2voRrg95+ZwUWfHnXmR2Cq5/ywlxqLxLBk2lAB2BuSmj2RKdIIYKnlJk1 iEfZhsO1MnnSNipUT/G2uAr44IdPub7PgGYmTru8gHSf1ad+5Bp4aUAXkTFpHx/6ALns SViA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584735; x=1729189535; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=LNPEWcjaz+dEwpcBQF1ZcKP3HAxUc/LzxURpjzN7wyY=; b=acLagjuKItMz+1VxZ2hF5HuyHK0h+Pn1H6WTSeIqNwBqQUfWgtmQom80G56e/PdVWZ HZykmDRX9zaleZkkAU2RW3Lwydd89eUYR03yC8uLjIb5/80xLIl/+I8FwgDKCbMEzX5E cIqnRShLTBtnCxyX/iroVixgYYBbEu8tKL1cMDB1Bp09GrBHwfWg/8BZ072uzV3+vWfE RfWTI9/u/16ocgyomtY+YsTzTLPsV6rYgCAOnicrDJ+sjExGK4Ld7BnEiMNqwYp4AVRb 0fbJ3sL1aqK+wPjPyqDoJFsv0cLgov0H0hgEqdNvQXpfli6OgKtRc22SakMA+cq3wnRO C15w== X-Forwarded-Encrypted: i=1; AJvYcCVXUtuE0aT4sPu7a2ttzxorHC0r7lZiTbx15XeteyKxgoqo9DorsQ7ol4xspQOhn5TRLV8Rvp2q6s/AIY8=@vger.kernel.org X-Gm-Message-State: AOJu0YzV4Ho2SlCECC3yk6DLXhWD6pfx416AszkVdXt94+AF/IJNFLTo YpMCmZMwgrBV9JOIxHX4b+cynaxKHzfHiYsavO9fzPs4wi3qRw7snH29hWu8WKyYApJTCh9VdzO wLQ== X-Google-Smtp-Source: AGHT+IEnZEGZphhdHlQgb0iOcJfNRdGKMtxwSRuFd5MkDGpZIOhDcg8ohYIeF4ySSwO5bqKUu8yK4lYjZ+4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:8595:b0:2e2:d600:3862 with SMTP id 98e67ed59e1d1-2e2f0d9dbfemr89a91.8.1728584735013; Thu, 10 Oct 2024 11:25:35 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:22 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-21-seanjc@google.com> Subject: [PATCH v13 20/85] KVM: Remove pointless sanity check on @map param to kvm_vcpu_(un)map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop kvm_vcpu_{,un}map()'s useless checks on @map being non-NULL. The map is 100% kernel controlled, any caller that passes a NULL pointer is broken and needs to be fixed, i.e. a crash due to a NULL pointer dereference is desirable (though obviously not as desirable as not having a bug in the first place). Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 52629ac26119..c7691bc40389 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3071,9 +3071,6 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, st= ruct kvm_host_map *map) void *hva =3D NULL; struct page *page =3D KVM_UNMAPPED_PAGE; =20 - if (!map) - return -EINVAL; - pfn =3D gfn_to_pfn(vcpu->kvm, gfn); if (is_error_noslot_pfn(pfn)) return -EINVAL; @@ -3101,9 +3098,6 @@ EXPORT_SYMBOL_GPL(kvm_vcpu_map); =20 void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool = dirty) { - if (!map) - return; - if (!map->hva) return; =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E2FD61F7088 for ; Thu, 10 Oct 2024 18:25:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584739; cv=none; b=RXlWsNg5hlfQOkdUTvv51MPxQEDNfZhDTUdxjxisJpHafUr5trPMlenouV/z0vKDgmtIqX2Is8rLvgUtWQbvGWrRBHWKGT/lmgpk0QZ6bj4ghDWWCHS5VrdWWD7dZpmAPptEbrfS4fEwAyR7RrIYoWYHdPHnQhJ3o8r6CXxVrdI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584739; c=relaxed/simple; bh=w6CtoHT4iiBJ8oKluaibldhSnM94wRba+mvcEkROyJo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=liPoGHhPf0hAr1yR8rrDIuu5MDNOpTLJe9qZz5C5oq0TGj6Nyd/R8G89XLNfVT2r9RPJH9BEbhpp4qek5BwDjyPQ4k1eID5hSbqWqXeVZ8fAtzc3ibk3vUhYst1Nhi7GE+fDTSOkxD4pLlh3tib2tNXW54Fo0hItfit3Ki2Gcws= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=B4VD+bqQ; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="B4VD+bqQ" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e2605ce4276so2179755276.3 for ; Thu, 10 Oct 2024 11:25:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584737; x=1729189537; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=IMgO/JDFhWAWu7Wkr7+zBWYfApHhI977J8rKw7ML51Y=; b=B4VD+bqQ1bv6Pf45zN8plQZ+1nlQIV+x2YTm+XE8/QAFyOT6pF5ri3atTDy7wmWK1W cBuNjj+RZcYpDljbhbOPdKgUEc6sxDbKrQJ1fcQGXI1ipHd31NNylsUJEh9353nqzmBe xErqAc5drpMhDXL6T2FAUVJtAVGGhL9gh8F57tSlbW7u9huTn8DSiVUEoNHxzP9tsAAL App72uklOszMQmpZcdIaLkjDcH7P4uuH5tw33U+NlFCh5D1Z5egpssNiIn5bx95a2D6u zh0dl5vUT4MDK4fiWa6tECFRTbzyA11CMn2YXNF224VvrXZcgmYLwkzQLjyWsiB0wLbP KAFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584737; x=1729189537; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=IMgO/JDFhWAWu7Wkr7+zBWYfApHhI977J8rKw7ML51Y=; b=qrdaEc2BHgkmTJvcYVSAUmoillbOiO3S7GlwTWOv/pwJEkp6CpsU5KEudB8P1aq5Hd Pt5Y3ljkdTZ9EEts5c7dMemn5whwiewSgrexpG6HZa4/TH2V6RyBF4rMieav9Nh/kT7q pBE3TlG7qw8a3jNdTDu4DaD/HBMO0JLfxKiibMq2BopdDz1sWE0oCHhW2mSalVBVju8b VGcMTpdIl9OprSLgiUdBNdb2Of9S5LHDt183PStMDJoFafLw04p5IjNNVK09CBM4lHGh QZJ+Ad9eFpWp+YyzQIOVZhluqQO1KOGGdwFp/oy7/LibaSMDLyMFw6zYOlTrpwUQ1l23 pqqg== X-Forwarded-Encrypted: i=1; AJvYcCU+asxwIrS7DJU3GHmIlr5f+lUNbHSXkphYT5NVvsPdb5YdH5H+47uIVIZ0jRCWLgFMk/0leQlrLT4700o=@vger.kernel.org X-Gm-Message-State: AOJu0YyVw7AbK6+R/jkxbKahjMSWs3ao14+DzPbeh0udGNUsw1W2KgmZ n8+0GpDAy9ZjlOCzyQFXphw2vznyiMY1/9ef3/ojxlQwsVl8rWoiovlZ3D9httJrEvqW2g2lXbu o4g== X-Google-Smtp-Source: AGHT+IGlSN4X6DbrP5DPPe6+MPgoNlxMoOT3GZzpJV2jr+xnXsAJlzTjTlsnBYkwN+CsyuU9UJbLGW89hxk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:c70a:0:b0:e28:eae7:f84a with SMTP id 3f1490d57ef6-e28fe0feaecmr5507276.0.1728584736880; Thu, 10 Oct 2024 11:25:36 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:23 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-22-seanjc@google.com> Subject: [PATCH v13 21/85] KVM: Explicitly initialize all fields at the start of kvm_vcpu_map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Explicitly initialize the entire kvm_host_map structure when mapping a pfn, as some callers declare their struct on the stack, i.e. don't zero-initialize the struct, which makes the map->hva in kvm_vcpu_unmap() *very* suspect. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 40 ++++++++++++++++------------------------ 1 file changed, 16 insertions(+), 24 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index c7691bc40389..f1c9a781315c 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3067,32 +3067,24 @@ void kvm_release_pfn(kvm_pfn_t pfn, bool dirty) =20 int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *ma= p) { - kvm_pfn_t pfn; - void *hva =3D NULL; - struct page *page =3D KVM_UNMAPPED_PAGE; - - pfn =3D gfn_to_pfn(vcpu->kvm, gfn); - if (is_error_noslot_pfn(pfn)) - return -EINVAL; - - if (pfn_valid(pfn)) { - page =3D pfn_to_page(pfn); - hva =3D kmap(page); -#ifdef CONFIG_HAS_IOMEM - } else { - hva =3D memremap(pfn_to_hpa(pfn), PAGE_SIZE, MEMREMAP_WB); -#endif - } - - if (!hva) - return -EFAULT; - - map->page =3D page; - map->hva =3D hva; - map->pfn =3D pfn; + map->page =3D KVM_UNMAPPED_PAGE; + map->hva =3D NULL; map->gfn =3D gfn; =20 - return 0; + map->pfn =3D gfn_to_pfn(vcpu->kvm, gfn); + if (is_error_noslot_pfn(map->pfn)) + return -EINVAL; + + if (pfn_valid(map->pfn)) { + map->page =3D pfn_to_page(map->pfn); + map->hva =3D kmap(map->page); +#ifdef CONFIG_HAS_IOMEM + } else { + map->hva =3D memremap(pfn_to_hpa(map->pfn), PAGE_SIZE, MEMREMAP_WB); +#endif + } + + return map->hva ? 0 : -EFAULT; } EXPORT_SYMBOL_GPL(kvm_vcpu_map); =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E54F61CFEC9 for ; Thu, 10 Oct 2024 18:25:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584741; cv=none; b=CmSaoeWb5hDZyVTzd4ys2XezxRswXPgvqHVPc2Gi3jimPIDhC/tA7Cj1dxc2DS2IoYmOZMcthKDjcpYUDxTgxwAUGJexGSHnlcUXzXGGQSxRR3GeOLMqDL2CZA9VosYxxEyrtUJudzh/3U6UaJv7EGGKK6lfG2ed/13RzN3Qeu8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584741; c=relaxed/simple; bh=ZlT2BBZrHMf0x3c0Y2iKgnaJviq4ilGlr1zNhk4Vq/U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=a18oQNaqJOWw+KDMvIPh0IX8VREdWkM9Q/5GknviX//ARaHPz9DlIAS2mRiCVxSf74XrPOcFyLuM2DMrFyRn6CmTAMHw3dYNAQKerk8NUhFN8KfOLDjgDBI6G+5QCRFErSyLNU0csUCB4PirsawF/vfI/qDopl4TDw5jvEkTltY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CjJawX0g; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CjJawX0g" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e2906efc0eeso1518981276.3 for ; Thu, 10 Oct 2024 11:25:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584739; x=1729189539; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=wU+jX/zRq5GV793BchOJ2bBXpO8AjDudJh47DKJg70M=; b=CjJawX0gL3DlpHXrFl3UettsKvgKZhWv4SqQA90BYqhykfLcxkE6Wu1vh/woj9cKRe WBeA5xaSPtiO2n25gFk5BFhGTdTAIXI9+cKopSHheHDsvMSXgv9GRVTZNsFw8UHjUIyO bgz5EqJfA36MxGkk/wKO81Pr6UfjRrL4/iYWSOGDvEZr2PKMKXa9vGQ9qv7HktdxPuOA rYvrX4Mva7y9IcOyZ2keW7/rxD4tkbLD7VQOI/NAXhIe4EeyM29lHpbclTyhvbd6iFmV yktNhWnPAFFG1WSt+zWdIjGhy2UW8v7gJwCEuCC8hBDIry/zGPL/EdBbYHZWYL9FX1C3 sl0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584739; x=1729189539; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=wU+jX/zRq5GV793BchOJ2bBXpO8AjDudJh47DKJg70M=; b=ixAa4tOHm8ZApLBkUSymaHlmEwFmnTmZQ0+kdodGMVq8jUYZ/XI0XGgebZ7+tSpx+x LVs7+QdpD2Y2Y0gtqnocsIM+UqHPGJiJdY11JoSXCit+8vQ4AeqH1vlNe8MPljn0/Cjj DCD9Dw1VsMP4iKT58DAsA6jeR4qREikMTvKsCso9PvDpSr2JUNA28VtSP9Z4/liAg791 uhjzM/OXOl5NKEB5Nc7x+rqGMrd/fTvNhRja5bFp2YuS1mLhdNNhHauARc8ClFHSvhEc 664gHfrVKqOPRM6rgrPyQWQeS43i4kbKBTfSBPDH+VrFtVVQuLPKsF9LoRjn2zXAixEZ c/nA== X-Forwarded-Encrypted: i=1; AJvYcCXkWppFU1TR9fLVBT17yiQLXokB2j+uyqnUKriMhMRxPYMt1kPJ9qJLBsNSo69y/6gFtkEQSzMmXSCqeg8=@vger.kernel.org X-Gm-Message-State: AOJu0Yyk/1XmVzbrnQrvvmbwzu9Vv4pE0kE68goB2x1zNaPMfTUZUCSy 2b6IhP/lWqIA6NIrhz2Vb3B0eJhazJLEdBKs3ffzGvBonpRtwxPLuI9VYQwp12XH7m/QxlfGtP4 kZA== X-Google-Smtp-Source: AGHT+IGZ+3mddEgfwOZrB7Mg/yEB+clUKORcl3gwcx/GeFypDRI64NMuGnBtjzJ0gTzObJ64BsofGpKHxtA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:ab6d:0:b0:e28:f8e6:f4c6 with SMTP id 3f1490d57ef6-e28fe465652mr58758276.2.1728584738891; Thu, 10 Oct 2024 11:25:38 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:24 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-23-seanjc@google.com> Subject: [PATCH v13 22/85] KVM: Use NULL for struct page pointer to indicate mremapped memory From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop yet another unnecessary magic page value from KVM, as there's zero reason to use a poisoned pointer to indicate "no page". If KVM uses a NULL page pointer, the kernel will explode just as quickly as if KVM uses a poisoned pointer. Never mind the fact that such usage would be a blatant and egregious KVM bug. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- include/linux/kvm_host.h | 4 ---- virt/kvm/kvm_main.c | 4 ++-- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index cbc7b9c04c14..e3c01cbbc41a 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -273,16 +273,12 @@ enum { READING_SHADOW_PAGE_TABLES, }; =20 -#define KVM_UNMAPPED_PAGE ((void *) 0x500 + POISON_POINTER_DELTA) - struct kvm_host_map { /* * Only valid if the 'pfn' is managed by the host kernel (i.e. There is * a 'struct page' for it. When using mem=3D kernel parameter some memory * can be used as guest memory but they are not managed by host * kernel). - * If 'pfn' is not managed by the host kernel, this field is - * initialized to KVM_UNMAPPED_PAGE. */ struct page *page; void *hva; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index f1c9a781315c..7acb1a8af2e4 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3067,7 +3067,7 @@ void kvm_release_pfn(kvm_pfn_t pfn, bool dirty) =20 int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *ma= p) { - map->page =3D KVM_UNMAPPED_PAGE; + map->page =3D NULL; map->hva =3D NULL; map->gfn =3D gfn; =20 @@ -3093,7 +3093,7 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm= _host_map *map, bool dirty) if (!map->hva) return; =20 - if (map->page !=3D KVM_UNMAPPED_PAGE) + if (map->page) kunmap(map->page); #ifdef CONFIG_HAS_IOMEM else --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86EF81F890C for ; Thu, 10 Oct 2024 18:25:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584742; cv=none; b=HHcZ9e2I8LKjLjkn9qRPsYvZSgOvzX7e45lVK87Z3xf0j1TRQtxAchswprH9wkFW2a9TOR+BV5yf04s2gFbv8keSJFYTW4kJlQzrWVS+2fUpm1OINUxMju8Cdg6ZCFbZr9D1CVDfxlaLWnI93zuxUikI1r3tz64Cd1PxCgmV+DE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584742; c=relaxed/simple; bh=Z6RyJQ8NNQkeYyg1tN/wL4j+YrBUDlf6269dIZP90as=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=mIGHjjzFNkX5STw5sHapJynjvJE9e2FLDsglFTr9BET53TOXTHqVrkUchZ+9sOiw5C211Z13pq/oXlzgSJFukfw6w8GgYPKbX7t3Om6v/K0v+Niymy+AN6c35TYl5JkgQ7i3mHXTu/Dz/QO2VXIBgiK2Toq8BDESySQNK+dPKc4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qoM+EiSr; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qoM+EiSr" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-20c8d7f9128so11191695ad.1 for ; Thu, 10 Oct 2024 11:25:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584741; x=1729189541; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=UxQVf2h1B1ZgTBX9bfQB+O4u0WEdBy2E8BNaXtNQtEc=; b=qoM+EiSrBnuwmjbt/H8tKBvxJwaIx0hkbfvehONsdCk9vjqOswD4mvEj0K1OvuyMS8 Xefy/51XOxUXr3lsl7Gxr0tNwtj+ZPlYKogfR3q1PQN1U7OSpY3Nf2KyigCmie4XlfOu 2xoglyZSTIcyp+jlQO9uqGd3hDo0QHuTSL4kl7jncKIHd+kyxUPUGGTYKg6YzSg7q58o hz32KEqkByEhfBdPCd4ourTLtS05BCrpdXQYuWlkGQ0tWMIgXq7FyYT20RKpzbGXrhJW eTw/7JHF+f6c5jdj0TmxNhUAaWushq/8RPGlbiYwR+r7c/2SWrqGoB8HMLioJXH2uYJ/ aKWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584741; x=1729189541; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=UxQVf2h1B1ZgTBX9bfQB+O4u0WEdBy2E8BNaXtNQtEc=; b=hp2QwPl83XHUpb8hmx/PWWFpKopq//SLp/nuGca8f6sveuTqiwNsXKZtEKHNf1kqX+ K9Eil/rOYnT8TopKa15K6RrPfQ89xlSayXMqyarYYeJsqRIno9IXTkdO44aHrwXvtCPv YhOOJdN6oz/sJFrcJJCdpaj2J5ciAFX6sGAgWUDmQOFtoDnjyP9S0XZ4t4wwP1XDZxdU MusVVn/p2/U0pb+RbnmiwYUu+9WwnD5lupso2WZ35AFexuFgGBosOT+gHBXyajRBHoD9 LYHblvuE8Zbd66SO6C46PvZAxRNwJTOzhde/8ukRXtQ9pzAKwmsvsypoAFSeX1w6qqnE yBSA== X-Forwarded-Encrypted: i=1; AJvYcCXPQijcyqLFRk1TgDCOx8cv4ETGlun4Rj8aSEOLVD4hTRaVLcJ0hA78iOe0t3ah6WOEonsqQhplPPHmCb0=@vger.kernel.org X-Gm-Message-State: AOJu0YxS4b6Tc3c9HyXsWeLrcQ6PwfwVSECPc4k6YaPeQxDwCPFkzPsF o2iwxdY6oehMzJxgC+Uz/TkWCaW7TdrE77W7WAdNmGBNCK4NSV+ahbpfpyFYQ1PKTAHeAZqqdb+ /0A== X-Google-Smtp-Source: AGHT+IFSsaj7YGs0xXvL7KvHpo4OHZS23TfW901WnGn1dNBw4UEq8HUBbyPz7tDSJuBwPv672jv+MkN6+4M= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:2349:b0:20b:9365:e6e4 with SMTP id d9443c01a7336-20c63782bdcmr1094885ad.9.1728584740809; Thu, 10 Oct 2024 11:25:40 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:25 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-24-seanjc@google.com> Subject: [PATCH v13 23/85] KVM: nVMX: Rely on kvm_vcpu_unmap() to track validity of eVMCS mapping From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove the explicit evmptr12 validity check when deciding whether or not to unmap the eVMCS pointer, and instead rely on kvm_vcpu_unmap() to play nice with a NULL map->hva, i.e. to do nothing if the map is invalid. Note, vmx->nested.hv_evmcs_map is zero-allocated along with the rest of vcpu_vmx, i.e. the map starts out invalid/NULL. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/vmx/nested.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index a8e7bc04d9bf..e94a25373a59 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -231,11 +231,8 @@ static inline void nested_release_evmcs(struct kvm_vcp= u *vcpu) struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); struct vcpu_vmx *vmx =3D to_vmx(vcpu); =20 - if (nested_vmx_is_evmptr12_valid(vmx)) { - kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map, true); - vmx->nested.hv_evmcs =3D NULL; - } - + kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map, true); + vmx->nested.hv_evmcs =3D NULL; vmx->nested.hv_evmcs_vmptr =3D EVMPTR_INVALID; =20 if (hv_vcpu) { --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3A871D042A for ; Thu, 10 Oct 2024 18:25:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584746; cv=none; b=K35eaoU/TYoy8Nrp3ddiaexUP6TGWVSnEHKN7HTlsNUbJtnj6SnyQF+piJ4w71R5RFtf4txNJvyMmfiyYmS1rXRVKcFGuXIYYSDgFwJpFZg72Eyeaqoh8hFKkFfPySI+bwDLJ5KJoVYC4mPMGhyccua1aBL1gXf6Oab4EvhR+mo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584746; c=relaxed/simple; bh=QVyL+E+J2Fu+8nHNQtozyIIO+ywn/WvnTCVSYx092RU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=J+q9rPL5zP7jUAnuid+Zi8nj6N+GtDF1y2mRtGVs7M2wPMk2CzU5GBtej4s8WiqXRE2p1ZGiEEZkCggqxaB2gtoCoDqWnC5HraWRhwKiqmhwneAWbcWj17SSSCCYAt+DMp3/g1W61L46T5hD9zlsiMmWcwKyiJbyACZr9Z9d948= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=o9hLcB8T; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="o9hLcB8T" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e290a9a294fso1853539276.2 for ; Thu, 10 Oct 2024 11:25:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584743; x=1729189543; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=eHAZpPaCzYnE+TRrbh6ni4wh+Yys2pChavpSnHk2YEU=; b=o9hLcB8TcgrAYeRwjVSZvm39dElO8cyvKzpeBbUmWSYLS3W2vdIeqdtka+VJm68gJL Mt1xP26lna3GmuKsA5M6XaLxLrcnmZpwiPqK97xslR+y8/OD8joZ6hTsWJkepTSytJ4/ 5qQ/cvFl9Lhgl/9BAV4EsrHv5625y4MIqZDYmniqgzndR8v9BPPB25oMftqfGBWKejna 3Sh6VKQ7Q19iuz1MdTuXO7tL450UYOL13DDXwlw+nZBSCcl5e+WElcxqudAEceZSwbPv HeAmyiYjnu6f8gH9Kx4SOq0jxKrIFi4JcIiq3wGhRqCUNlPtIfkJbZnXdKTR6rDaWD7o bGSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584743; x=1729189543; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=eHAZpPaCzYnE+TRrbh6ni4wh+Yys2pChavpSnHk2YEU=; b=R/facJxMTEnvGYGlg5ynLHADcLkBn/5CIjy8n7C9PPqjGjkfpX9LocQlOSR0TIl2NR 6R1OF0O2Insa7NPQxdSitK6S6h01UKJTHk1FuC8NF9lsPR26yqvaf8MXGQaA5Mao75X9 4s92jrjZVy2RJUtsEfgdyRb0j2xkdpl0syIPi+7sfhPR3Honwu2tVSP3A8ji2buXzlS8 m6kf8lyRWcb7CTe7MxZzD/VxjgzXrJB7ORaqCogGLD05BCLMdB4/sdVz8JbXaGVZ0wCQ 7FaxoXBoPhigEmCRJhaYlXpfH3HxWOohyiPpK0cD9Ho37R9EJXoEjEy3/fE5oflWqP// gQwQ== X-Forwarded-Encrypted: i=1; AJvYcCW2FRf6OcEeVBfWyyK2qm3+Bj4meNJXKBV1jyBeuoiOLn9tWohA95XyYRl3FyuhFWufaqaJ5Ds08lm6gyM=@vger.kernel.org X-Gm-Message-State: AOJu0YzNcXfrCIL0zT+x9oEsvwt5qePnuclnphggAYSOp5O9LNFQjNyQ cukZyWpTw0qtSVeaHP5AF+Cn60d9Du5bNH0MuJPxaDuxVAe0HZnOrkDcpMst3VgQS6heXDS+2gz NIg== X-Google-Smtp-Source: AGHT+IGUAGxjHNELfO1gnQBoxyShlnhLiinRslmifWPsD1igdN7s2BU3B9tdatRPL4jwPkA1Z0qI9X+JjYQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:9346:0:b0:e28:fc1b:66bb with SMTP id 3f1490d57ef6-e28fe4f0fddmr4836276.6.1728584742775; Thu, 10 Oct 2024 11:25:42 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:26 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-25-seanjc@google.com> Subject: [PATCH v13 24/85] KVM: nVMX: Drop pointless msr_bitmap_map field from struct nested_vmx From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove vcpu_vmx.msr_bitmap_map and instead use an on-stack structure in the one function that uses the map, nested_vmx_prepare_msr_bitmap(). Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/vmx/nested.c | 8 ++++---- arch/x86/kvm/vmx/vmx.h | 2 -- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index e94a25373a59..fb37658b62c9 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -621,7 +621,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct= kvm_vcpu *vcpu, int msr; unsigned long *msr_bitmap_l1; unsigned long *msr_bitmap_l0 =3D vmx->nested.vmcs02.msr_bitmap; - struct kvm_host_map *map =3D &vmx->nested.msr_bitmap_map; + struct kvm_host_map msr_bitmap_map; =20 /* Nothing to do if the MSR bitmap is not in use. */ if (!cpu_has_vmx_msr_bitmap() || @@ -644,10 +644,10 @@ static inline bool nested_vmx_prepare_msr_bitmap(stru= ct kvm_vcpu *vcpu, return true; } =20 - if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), map)) + if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), &msr_bitmap_map)) return false; =20 - msr_bitmap_l1 =3D (unsigned long *)map->hva; + msr_bitmap_l1 =3D (unsigned long *)msr_bitmap_map.hva; =20 /* * To keep the control flow simple, pay eight 8-byte writes (sixteen @@ -711,7 +711,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct= kvm_vcpu *vcpu, nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, MSR_IA32_FLUSH_CMD, MSR_TYPE_W); =20 - kvm_vcpu_unmap(vcpu, &vmx->nested.msr_bitmap_map, false); + kvm_vcpu_unmap(vcpu, &msr_bitmap_map, false); =20 vmx->nested.force_msr_bitmap_recalc =3D false; =20 diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 2325f773a20b..40303b43da6c 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -200,8 +200,6 @@ struct nested_vmx { struct kvm_host_map virtual_apic_map; struct kvm_host_map pi_desc_map; =20 - struct kvm_host_map msr_bitmap_map; - struct pi_desc *pi_desc; bool pi_pending; u16 posted_intr_nv; --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 910C61F941F for ; Thu, 10 Oct 2024 18:25:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584747; cv=none; b=XUQaeZHEVj8ygN12yIMqTO3jgdrLj4CMOcyOWB+QsQaYGPg5nc6gF1FwIAoRPJgyHTEvDauSkRhErMGnHT+EFt9kcNwtbtVPX+5YK5VnXEgV4sYW1TfgE2w0wbNS9fYRma03hZfDAMUR+QYAAl09pRUsCFImNTbH1PUcpDvyq5o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584747; c=relaxed/simple; bh=H3zY/Xt0t+c4LMWpCCUO1umdZ2mHGDI4Sis4Gpw2f4E=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=V0HI4gfCzEjFycC/+wOYIV22+VWWgInbTRcNeORlSqKsMJe75U4kZuxLXoV/PUWCLsTOniKXswhiV6ywrioK/mageKeMZ69iSQyA/r6SAKvnRpHIbnpWUUgC7TIZYshL7UX3BJJyCDytBp+kZXyWBqNGR4Z4S6je//iJgY3ApZs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dOB3fNA4; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dOB3fNA4" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e25b39871fso27044087b3.0 for ; Thu, 10 Oct 2024 11:25:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584745; x=1729189545; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=xnti8P2TQudE7lvlkZdCl8Wq+TvGW4tpGYbCykNWsXw=; b=dOB3fNA47Ys/t2RsrBoFMYei+S0bW216ulRwWrnfXEdN8iMvrCGNtKMihXfXcvqKyO UwHBwAHc8CUabZz6ufuXBq9dDxqCduiUDhRaekZSGdsbD//eJIWik8hOLt/7s3rfOiEJ z1Uug1P8XEYf+oxXnQpeR8VnmbIOdQv4uB4mOAhxzWuOX9ZERNYGDzAES/UqOG5jy5PV rtiiw/BOtjrl63GJm9GZUYSl6rhmyGpAthr4RNSdMa+mvLJhz3Sbj/1MdqYhqoqW/Pe4 JhLDjNtSaQNO119lhq8ZFAG0O0b9ck7954dyAKa4UuBRfnmamD6eqSzGHFWMXD7lGYjg mLVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584745; x=1729189545; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=xnti8P2TQudE7lvlkZdCl8Wq+TvGW4tpGYbCykNWsXw=; b=o1FMQoNlm2gLaU9Afg9JwNGBm/K3jE70DCB30kc3YlrZcUUWlbPuDAJqZQL/am7KDl PXBTa3e9EmtBC5kdIw5myGv/+Kiywl5kk/aiiCjYtvJBjq4ipE0mS2dPZ3WJOzvtVmb2 1AcPLDpuLv7fz2je0QMBghcDhoJLbEV4UOMl48SkHZvXQ7w9nqlpIT7LU03LqkGRIXNC fcJel92E+UhRLl+ZDS4/iN1jwZZveW3abkjqfFDuNjra1p3q8xI3BxuiezfR4PJ8GEAa PoybdDhDGJDxUssahCUaGB1dlU5l/nrIWgHu2bi0IrY0AbDo5XKlGwcCrexRYinbnyZZ tvtA== X-Forwarded-Encrypted: i=1; AJvYcCWPoXVrJa8C5wmS83ximp+OZAWLPQQg+WREzVf4cofXjp0Zb271Nw4fhuicGusgfWhbfz+RULMPOAK6p1U=@vger.kernel.org X-Gm-Message-State: AOJu0YzwkS4EKbjeJQiNcbmnlbFKJxgHrqSBauSUeL2uj4b76GtcR7H5 qhLiKjfONgvkEhtlWXqY//d7C88jaeEY2pnwsFVCqk1H4bFDofR0P5QLwkTAOE3vK9O6d5OJyTB QOw== X-Google-Smtp-Source: AGHT+IEdX//xUJq0LlzHSFsWnzcNANrEE9KtnndGvFzP5BAu23zXYFVXA6eeBi51F5M5i08FgAsxiLe5HbY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:300b:b0:6e3:f32:5fc8 with SMTP id 00721157ae682-6e32217bbe0mr502817b3.1.1728584744665; Thu, 10 Oct 2024 11:25:44 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:27 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-26-seanjc@google.com> Subject: [PATCH v13 25/85] KVM: nVMX: Add helper to put (unmap) vmcs12 pages From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a helper to dedup unmapping the vmcs12 pages. This will reduce the amount of churn when a future patch refactors the kvm_vcpu_unmap() API. No functional change intended. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/vmx/nested.c | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index fb37658b62c9..81865db18e12 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -314,6 +314,21 @@ static void vmx_switch_vmcs(struct kvm_vcpu *vcpu, str= uct loaded_vmcs *vmcs) vcpu->arch.regs_dirty =3D 0; } =20 +static void nested_put_vmcs12_pages(struct kvm_vcpu *vcpu) +{ + struct vcpu_vmx *vmx =3D to_vmx(vcpu); + + /* + * Unpin physical memory we referred to in the vmcs02. The APIC access + * page's backing page (yeah, confusing) shouldn't actually be accessed, + * and if it is written, the contents are irrelevant. + */ + kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); + kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); + kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); + vmx->nested.pi_desc =3D NULL; +} + /* * Free whatever needs to be freed from vmx->nested when L1 goes down, or * just stops using VMX. @@ -346,15 +361,8 @@ static void free_nested(struct kvm_vcpu *vcpu) vmx->nested.cached_vmcs12 =3D NULL; kfree(vmx->nested.cached_shadow_vmcs12); vmx->nested.cached_shadow_vmcs12 =3D NULL; - /* - * Unpin physical memory we referred to in the vmcs02. The APIC access - * page's backing page (yeah, confusing) shouldn't actually be accessed, - * and if it is written, the contents are irrelevant. - */ - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); - kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); - vmx->nested.pi_desc =3D NULL; + + nested_put_vmcs12_pages(vcpu); =20 kvm_mmu_free_roots(vcpu->kvm, &vcpu->arch.guest_mmu, KVM_MMU_ROOTS_ALL); =20 @@ -5010,11 +5018,7 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm= _exit_reason, vmx_update_cpu_dirty_logging(vcpu); } =20 - /* Unpin physical memory we referred to in vmcs02 */ - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); - kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); - vmx->nested.pi_desc =3D NULL; + nested_put_vmcs12_pages(vcpu); =20 if (vmx->nested.reload_vmcs01_apic_access_page) { vmx->nested.reload_vmcs01_apic_access_page =3D false; --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D7DF41F9A8C for ; Thu, 10 Oct 2024 18:25:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584749; cv=none; b=uGhzhFAe+bp/lx1j9fOiEnzd3GsGQVX2vsHNhSJvm2HoT26aIC/Iucfr13fPAGddIJth/KM8mrfY7tCMQAfQjY76SdBAYFIW92Tm9ZoK0LN0sUwo5btB/pT/bDNbYrObAiX77NUf0tiyy3a64r4sLYIe93+06BHkGmFnRLsMPzE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584749; c=relaxed/simple; bh=wQkcrwrrK4R8UDZylZK1jjbxKHOYfzFAR77NBuuXUsk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=G+vssBwMNzvTFAwA8dcRQZTJMGHu994MUUuNOk7zMOrDXwUQm89xRO9Ges+a9FTYJj91WBSV/RSoXwKoNvf4OIiOR5OsmyJKGWVhBM+W+fjy4e41oKtvH7qRoX+I+mCmABOGgBGSM4bsIjKXXgndG5qq6JuXa0RnolCNmRkdv9w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=be7NmGNy; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="be7NmGNy" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2e148d4550bso1322249a91.1 for ; Thu, 10 Oct 2024 11:25:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584747; x=1729189547; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=/W6JK9yAcorM7mqEFQq2hRBqvFCoz2C4pz3yOEmiGek=; b=be7NmGNypoYKlIQHzeI5qmlCcUnBhnj66Rq1h3NtOxbOPwC+w2QWEewadPkTALHHw3 xPJKAp195KEPkglLi++7I4Q8+YOmOUjNKY8vER3KsZeeMIagBJvsGt5Pobsm74JYsbU6 LTrXc4Yk9chzFDVCyczjOYxiAQl3ChwlolnTSQxMAaBK84r8sAvRfe70ej1wxuwkkAso K642YYl0QtppHrwBVSCdBwaujah/UYJ3uFn3uGnTv6BDtvRNYAJCNnt+vptRV4XD1YyB zB9Nu1ry3UFXALvaKzFf8mG173wYfV1cR9LVTLoNYadPVMufgcCdJQL3tQsG1XncstE1 EjmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584747; x=1729189547; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/W6JK9yAcorM7mqEFQq2hRBqvFCoz2C4pz3yOEmiGek=; b=ZU+qOzkWuI13iB6m5SgBDkRBjEgSTNVBq/Y7H7G1mZV/xrHOepj3pP4c2jPwO4fijd d2RiZKI/cuaaAqdT+SG4iHrdjvzyK3Hbb1u0xVCoQsGfsbmpA41Lf1x9+5vjtOb2lP0p D8PgqBBS78Ah6z1xLG5yYa4tccJYogbKetkj6oVciPsA74la9653MJ+DvGh0RAx/5vta p1m8aYSjtBxc4PtOF8HstEOMbW6hfj5vnWprMQL60gzRT3SnPhDAE+KDUceg0gkXTXNx xsybK42DhUxH4tc9spUByGibD0Jgw8gN7eCtmeV4TXk9t2JE8zg77RLR8o1JtvjugDu0 q1TQ== X-Forwarded-Encrypted: i=1; AJvYcCWmiqyttUfZEaKtuXqCAHhkSzzRnmDkY0nGQOC8/WROnSaPk3Dmv/jMmV6kavcbGHp+8dJBhHJECFnHO0k=@vger.kernel.org X-Gm-Message-State: AOJu0YwyzPehBXdtAXJwU/tzJcJLglW2JPQvHNced1Dpb7yJQc91jtVY Z6ogD4NiY6/DgCKBzeaBKPqpppbpu92axE9++ZmqkSTBHmrx2ppuBCF2Nag4RgHkqhZh1mozH19 2lg== X-Google-Smtp-Source: AGHT+IFGWHaFUEsQRxn4rcRWabj8vzxqpLHdNOXqOmiiQlX4dD6MZqF5BpShV2hVfQIOX/PTyNMVtEMSya8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:9a94:b0:2e2:da81:40c1 with SMTP id 98e67ed59e1d1-2e2f09f2280mr109a91.1.1728584746508; Thu, 10 Oct 2024 11:25:46 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:28 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-27-seanjc@google.com> Subject: [PATCH v13 26/85] KVM: Use plain "struct page" pointer instead of single-entry array From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use a single pointer instead of a single-entry array for the struct page pointer in hva_to_pfn_fast(). Using an array makes the code unnecessarily annoying to read and update. No functional change intended. Reviewed-by: Alex Benn=C3=A9e Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 7acb1a8af2e4..d3e48fcc4fb0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2752,7 +2752,7 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcp= u *vcpu, gfn_t gfn, bool *w */ static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { - struct page *page[1]; + struct page *page; =20 /* * Fast pin a writable pfn only if it is a write fault request @@ -2762,8 +2762,8 @@ static bool hva_to_pfn_fast(struct kvm_follow_pfn *kf= p, kvm_pfn_t *pfn) if (!((kfp->flags & FOLL_WRITE) || kfp->map_writable)) return false; =20 - if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, page)) { - *pfn =3D page_to_pfn(page[0]); + if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page)) { + *pfn =3D page_to_pfn(page); if (kfp->map_writable) *kfp->map_writable =3D true; return true; --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BDAC1D042A for ; Thu, 10 Oct 2024 18:25:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584751; cv=none; b=IM/NAnH8CIHw7cDb4N7MCW2xWl50H+F0hkHaeXIcRm9c6w+ceytW5mAG9vmVIdng8Jh2j863TuSF7keoh9uqiFVIHz+cx7Nh3KVnhGjUMtBKomxBnVw0/svCe4vLEVj/OV77/8mwpWPxz/d++IGClcP0B9shoRBT5EzvVvtsi74= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584751; c=relaxed/simple; bh=5U+Xe396ie07CWKh5LcgsN1ZYAsnCq4iDqWBxnyRpWQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=T748L4GXYhIctk+D2KUVyijE04K/1K++b1Myqq+9S+Oo0fvSHdHjqbHT3kwWV7aeBiw9m69EmXlEgmpf0laRv43vtCpjstzQBk5QdwgFtI++Nq1Z6dOD+FwWCaO+PVyOgFJOaS+CmF75C2Cocq9gNbeqIY3YFA+unnczt+KACmg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lFnqB2HJ; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lFnqB2HJ" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71e01bfe040so1421850b3a.1 for ; Thu, 10 Oct 2024 11:25:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584749; x=1729189549; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=txhtY6exXDaGFQAWYGvmt09bjeoYz319F03IR2k/C3M=; b=lFnqB2HJNeC6aFRXmR03XZEiiImTYVHbWU/GkqwSXJ1GV6qh+onF9ea8pHau54nnJI JYMZo2Bdtnz9StiMiAh+3XeXwXPsCsBo4ISZ0LWerbxkuqPYHmQr81UeEuHQk2phaVBp K3Id19I5VP8U2GRXqNwXNPzdSdh7wcepp8PkASs/yhBCM17kjsTnGy8UkCWFYvjGO+kp nSgfjfAQrxTaWsofKYmr1+Cp2YVqzRjb56ONlO5cVt9OP7ILypd5IJCunw89Zzld0TPK EwYre895aXW+Dy/WBtL1VNQxnolmV9TTXJPy5ansjxuuDOLiuEvSknJMQiId1fkNjfnI McUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584749; x=1729189549; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=txhtY6exXDaGFQAWYGvmt09bjeoYz319F03IR2k/C3M=; b=reLjDEqisWmsuUOgBj4izxsl1bSytDwQqSnYzO1cUXAyUBxsT5QR43eckE9cLyIoaG CIaAL2CauHFKaL9M1lqF/pmqLqFy+1OEQRQfuHtmRS0jVQhQx6DPYtwXtzMUWi56Z1yW MTVlAe2I2tm5RPR7ODOSIEsK6Trq5DpKY9qFqwr01LSlb1bchpMxF6nJgGgEqu5ox31r uys1pCar+kkoZKLdFAtqzEBiwnRl7pmNeG9rc9yZPUriPv2z2YshqBDD10158NP2kXeH TYrb1H5RiFhJkcdBfrA03QUvTds9mQPFK17iv6sk7huk6uvl00QirFYUek/KoPQz+fSP yqbQ== X-Forwarded-Encrypted: i=1; AJvYcCXS8a1YrALiCpHKWt0mW25Xi5uJEuwnKqYjf6pUCvBMceT5tRKBgS055HdPKUsteamuIbyPRoNTd+LpWXo=@vger.kernel.org X-Gm-Message-State: AOJu0YylDvmGyEU8bII+xytIDY+ujk9dw0e5FiPYlcxdfy4F5xW+ZsiU +1Ukudm5gO9wb/+nKlgsjEVKqvLBuIxkOa6CYif1s7iI4i0wKDi+9oGFMymWkoCQDzN2Xn0BZJi Jrg== X-Google-Smtp-Source: AGHT+IE/uLa8aldzd4vJMnq+De69DiThAuVt+3q8zF6rqxlBj2/ZKo7+FHCoJyy45GpCD4mZjJfGV3U05Tk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2d8a:b0:71d:f1f9:b982 with SMTP id d2e1a72fcca58-71e1dc00b9cmr38199b3a.6.1728584749206; Thu, 10 Oct 2024 11:25:49 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:29 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-28-seanjc@google.com> Subject: [PATCH v13 27/85] KVM: Provide refcounted page as output field in struct kvm_follow_pfn From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add kvm_follow_pfn.refcounted_page as an output for the "to pfn" APIs to "return" the struct page that is associated with the returned pfn (if KVM acquired a reference to the page). This will eventually allow removing KVM's hacky kvm_pfn_to_refcounted_page() code, which is error prone and can't detect pfns that are valid, but aren't (currently) refcounted. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 99 +++++++++++++++++++++------------------------ virt/kvm/kvm_mm.h | 9 +++++ 2 files changed, 56 insertions(+), 52 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index d3e48fcc4fb0..e29f78ed6f48 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2746,6 +2746,46 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vc= pu *vcpu, gfn_t gfn, bool *w return gfn_to_hva_memslot_prot(slot, gfn, writable); } =20 +static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *= page, + struct follow_pfnmap_args *map, bool writable) +{ + kvm_pfn_t pfn; + + WARN_ON_ONCE(!!page =3D=3D !!map); + + if (kfp->map_writable) + *kfp->map_writable =3D writable; + + /* + * FIXME: Remove this once KVM no longer blindly calls put_page() on + * every pfn that points at a struct page. + * + * Get a reference for follow_pte() pfns if they happen to point at a + * struct page, as KVM will ultimately call kvm_release_pfn_clean() on + * the returned pfn, i.e. KVM expects to have a reference. + * + * Certain IO or PFNMAP mappings can be backed with valid struct pages, + * but be allocated without refcounting, e.g. tail pages of + * non-compound higher order allocations. Grabbing and putting a + * reference to such pages would cause KVM to prematurely free a page + * it doesn't own (KVM gets and puts the one and only reference). + * Don't allow those pages until the FIXME is resolved. + */ + if (map) { + pfn =3D map->pfn; + page =3D kvm_pfn_to_refcounted_page(pfn); + if (page && !get_page_unless_zero(page)) + return KVM_PFN_ERR_FAULT; + } else { + pfn =3D page_to_pfn(page); + } + + if (kfp->refcounted_page) + *kfp->refcounted_page =3D page; + + return pfn; +} + /* * The fast path to get the writable pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. @@ -2763,9 +2803,7 @@ static bool hva_to_pfn_fast(struct kvm_follow_pfn *kf= p, kvm_pfn_t *pfn) return false; =20 if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page)) { - *pfn =3D page_to_pfn(page); - if (kfp->map_writable) - *kfp->map_writable =3D true; + *pfn =3D kvm_resolve_pfn(kfp, page, NULL, true); return true; } =20 @@ -2797,23 +2835,15 @@ static int hva_to_pfn_slow(struct kvm_follow_pfn *k= fp, kvm_pfn_t *pfn) if (npages !=3D 1) return npages; =20 - if (!kfp->map_writable) - goto out; - - if (kfp->flags & FOLL_WRITE) { - *kfp->map_writable =3D true; - goto out; - } - /* map read fault as writable if possible */ - if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { - *kfp->map_writable =3D true; + if (!(flags & FOLL_WRITE) && kfp->map_writable && + get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { put_page(page); page =3D wpage; + flags |=3D FOLL_WRITE; } =20 -out: - *pfn =3D page_to_pfn(page); + *pfn =3D kvm_resolve_pfn(kfp, page, NULL, flags & FOLL_WRITE); return npages; } =20 @@ -2828,22 +2858,11 @@ static bool vma_is_valid(struct vm_area_struct *vma= , bool write_fault) return true; } =20 -static int kvm_try_get_pfn(kvm_pfn_t pfn) -{ - struct page *page =3D kvm_pfn_to_refcounted_page(pfn); - - if (!page) - return 1; - - return get_page_unless_zero(page); -} - static int hva_to_pfn_remapped(struct vm_area_struct *vma, struct kvm_follow_pfn *kfp, kvm_pfn_t *p_pfn) { struct follow_pfnmap_args args =3D { .vma =3D vma, .address =3D kfp->hva = }; bool write_fault =3D kfp->flags & FOLL_WRITE; - kvm_pfn_t pfn; int r; =20 r =3D follow_pfnmap_start(&args); @@ -2867,37 +2886,13 @@ static int hva_to_pfn_remapped(struct vm_area_struc= t *vma, } =20 if (write_fault && !args.writable) { - pfn =3D KVM_PFN_ERR_RO_FAULT; + *p_pfn =3D KVM_PFN_ERR_RO_FAULT; goto out; } =20 - if (kfp->map_writable) - *kfp->map_writable =3D args.writable; - pfn =3D args.pfn; - - /* - * Get a reference here because callers of *hva_to_pfn* and - * *gfn_to_pfn* ultimately call kvm_release_pfn_clean on the - * returned pfn. This is only needed if the VMA has VM_MIXEDMAP - * set, but the kvm_try_get_pfn/kvm_release_pfn_clean pair will - * simply do nothing for reserved pfns. - * - * Whoever called remap_pfn_range is also going to call e.g. - * unmap_mapping_range before the underlying pages are freed, - * causing a call to our MMU notifier. - * - * Certain IO or PFNMAP mappings can be backed with valid - * struct pages, but be allocated without refcounting e.g., - * tail pages of non-compound higher order allocations, which - * would then underflow the refcount when the caller does the - * required put_page. Don't allow those pages here. - */ - if (!kvm_try_get_pfn(pfn)) - r =3D -EFAULT; + *p_pfn =3D kvm_resolve_pfn(kfp, NULL, &args, args.writable); out: follow_pfnmap_end(&args); - *p_pfn =3D pfn; - return r; } =20 diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index d5a215958f06..d3ac1ba8ba66 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -35,6 +35,15 @@ struct kvm_follow_pfn { * Set to true if a writable mapping was obtained. */ bool *map_writable; + + /* + * Optional output. Set to a valid "struct page" if the returned pfn + * is for a refcounted or pinned struct page, NULL if the returned pfn + * has no struct page or if the struct page is not being refcounted + * (e.g. tail pages of non-compound higher order allocations from + * IO/PFNMAP mappings). + */ + struct page **refcounted_page; }; =20 kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0D6E21FA259 for ; Thu, 10 Oct 2024 18:25:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584753; cv=none; b=n4VRoB2TNzCIfDjaaZEcZ/QWslUx+aQKmM5REG+sY8GbJgWrLqlmWTijKcUxBhc3i47YiXctGJS5n8jWzHgYZtv8dMNbi5t54t/eQ19sIGMnZT/OAzaS0JTm8+SyvGptVsBomMWuFZBcJc3qV4q0f7ouPK/HmpNQS7fc0ze5Lrc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584753; c=relaxed/simple; bh=CmHVeWZ83J0EQfDqM5RvAa04rCEwL0d1DQJqfJodvCg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=g1tpmmO6DCucNtnSwdJJiWBKZPNFunZ6T3Nm1noOhd52aYmA3tXkWWW44TLwnJPLwdjkP3wDenCHMjUdO/EmcYZZwcGGVUVEarTz3HpJYUbPZ9oYBu+8EV4CvWRP7wiH8kuBAQQG3hyqcvV85PNagqtk9wxOhXEzg5S/z/AObnY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FXMkCkX5; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FXMkCkX5" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2e1346983c8so1277786a91.0 for ; Thu, 10 Oct 2024 11:25:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584751; x=1729189551; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=MotdKZcknVHCSnrsAh1XZemF3Qz2FYV2KdHfEEknv98=; b=FXMkCkX57wh+XHLq6n+XdE6kGmp0hkZZ6UA+5owP77BOfoNtvC7ytMt+Xm91GK2Ptt Z/XHN/Hk1/ZsHTw8wUK6ItSDN1PZ/Fd39gXJ0/20Uus/oeHrpT4bHO1ml/gJ2PjphReY RrNOOnUdwahhvvYNSbZQ/1IpMiASQC2EYHaf0+95rIjmy89mPbWSyHnf4XImQefy6klx RNtpXcGLTqbVqH6LtcwuGjG4TL0uJhOqs7pHAce0CgWIm6XbodXJ/ME1WZd0CQ9+NL1R U+EB7wNh59mYw7+giGo4f0SmCXTtVP1ct2vdXaUQ41Qoz11pFqCyoPvQon1ikHXhiajl 4aPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584751; x=1729189551; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=MotdKZcknVHCSnrsAh1XZemF3Qz2FYV2KdHfEEknv98=; b=drtX6e9VE/4UmcbmIZEPCajnpH42+r7VjovJeVZueLHyrJGuCOEo7FfU17vN1PqmAS ygE2ePufTe+jCSF03PnhMx8xqQLFO4jSdIvjPRR86wKY7MdqDaBwkEc2opPwu6rucUbr ELgLkS4iBE0S9wMONMamZsbprnmheVIFLemL0LNZfjxMpOejVw+9YYtlB/V+HLpm+Eak X8Ei7LvrvxSMBGTAYqrmFenaRa4uCgBsfOtdV+Zu9unwCJ92VdMK6qHMT9x3S9fHF3uy VMuJwatSnNLsPEr5UKZnSM9v9AUthEUtJ0Pl0Spzk+bACFv23JHxMKBqLekKAqjOxuyR Yypg== X-Forwarded-Encrypted: i=1; AJvYcCXNOtJ2zQaWVq+WkbJOohSLa1aBVgdLtnjwm541utuxJLIamOFXdsyWCLjxVGxw99gkBeiDmp0G8u1RALg=@vger.kernel.org X-Gm-Message-State: AOJu0YwJ8jdJtWr2vJFvyJ1QcwDpwXKfGHbXCkNCWd6O4eJAMihVoQHk 12EP063Cs6akv1yNv6YVn98QrGJaJmyC9IGk/ERky3A8JOj4zzUhoXD+fi55E1KSMTsfIM6vMXb pig== X-Google-Smtp-Source: AGHT+IHahb+vMbBUybe9JXpaiwYyoag4sz2flsLCX6vY5d7t+wo3V9TaPVQYV32tHeuijXZHqATezWyezMk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:d913:b0:2e2:a810:c3e4 with SMTP id 98e67ed59e1d1-2e2c81d4d19mr7321a91.4.1728584751203; Thu, 10 Oct 2024 11:25:51 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:30 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-29-seanjc@google.com> Subject: [PATCH v13 28/85] KVM: Move kvm_{set,release}_page_{clean,dirty}() helpers up in kvm_main.c From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Hoist the kvm_{set,release}_page_{clean,dirty}() APIs further up in kvm_main.c so that they can be used by the kvm_follow_pfn family of APIs. No functional change intended. Reviewed-by: Alex Benn=C3=A9e Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 82 ++++++++++++++++++++++----------------------- 1 file changed, 41 insertions(+), 41 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index e29f78ed6f48..6cdbd0516d58 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2746,6 +2746,47 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vc= pu *vcpu, gfn_t gfn, bool *w return gfn_to_hva_memslot_prot(slot, gfn, writable); } =20 +static bool kvm_is_ad_tracked_page(struct page *page) +{ + /* + * Per page-flags.h, pages tagged PG_reserved "should in general not be + * touched (e.g. set dirty) except by its owner". + */ + return !PageReserved(page); +} + +static void kvm_set_page_dirty(struct page *page) +{ + if (kvm_is_ad_tracked_page(page)) + SetPageDirty(page); +} + +static void kvm_set_page_accessed(struct page *page) +{ + if (kvm_is_ad_tracked_page(page)) + mark_page_accessed(page); +} + +void kvm_release_page_clean(struct page *page) +{ + if (!page) + return; + + kvm_set_page_accessed(page); + put_page(page); +} +EXPORT_SYMBOL_GPL(kvm_release_page_clean); + +void kvm_release_page_dirty(struct page *page) +{ + if (!page) + return; + + kvm_set_page_dirty(page); + kvm_release_page_clean(page); +} +EXPORT_SYMBOL_GPL(kvm_release_page_dirty); + static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *= page, struct follow_pfnmap_args *map, bool writable) { @@ -3105,37 +3146,6 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kv= m_host_map *map, bool dirty) } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); =20 -static bool kvm_is_ad_tracked_page(struct page *page) -{ - /* - * Per page-flags.h, pages tagged PG_reserved "should in general not be - * touched (e.g. set dirty) except by its owner". - */ - return !PageReserved(page); -} - -static void kvm_set_page_dirty(struct page *page) -{ - if (kvm_is_ad_tracked_page(page)) - SetPageDirty(page); -} - -static void kvm_set_page_accessed(struct page *page) -{ - if (kvm_is_ad_tracked_page(page)) - mark_page_accessed(page); -} - -void kvm_release_page_clean(struct page *page) -{ - if (!page) - return; - - kvm_set_page_accessed(page); - put_page(page); -} -EXPORT_SYMBOL_GPL(kvm_release_page_clean); - void kvm_release_pfn_clean(kvm_pfn_t pfn) { struct page *page; @@ -3151,16 +3161,6 @@ void kvm_release_pfn_clean(kvm_pfn_t pfn) } EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); =20 -void kvm_release_page_dirty(struct page *page) -{ - if (!page) - return; - - kvm_set_page_dirty(page); - kvm_release_page_clean(page); -} -EXPORT_SYMBOL_GPL(kvm_release_page_dirty); - void kvm_release_pfn_dirty(kvm_pfn_t pfn) { struct page *page; --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6AB841FAC50 for ; Thu, 10 Oct 2024 18:25:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584756; cv=none; b=cqo6sAQ+PwFkrw57+kAaX8MIy5T2JWw8YoRwURUZz68SMSoGDlH8rLi9okNELfJ7kEIKcaZvBJU5k2i6nqWpsFVcIAa1A//zifvmNORUp/x+PHwjptfTgiWaMNTaEkbSIJrIfP54obEVmTTpsHWj4a8YFv8A1jp3dQM6lcFFJvw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584756; c=relaxed/simple; bh=DZ/5QhAqV/AFQ+a9ItmswBEEvilydxqWBCs3ItnJewM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RNypRQUAsmfDmT6sQAf9TrApwMcj+FKF5DjIeykPwE342Tb3JZQdT/hOLXrO+PWMKManbbwTucuKPKrrx3Ms82I2NlH9TGguliLMzRo/B/N441ds+mDCxdaZCmzdm2jIQQl2NDsE6SM4Ft1IjWubSnRnGuZQnrwMz2IOQIfyIdQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=hmIwJ/Iy; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hmIwJ/Iy" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e02fff66a83so1965994276.0 for ; Thu, 10 Oct 2024 11:25:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584753; x=1729189553; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=nXVCtJlFVF227JmXan6PsTRYgw7//koA3Av9lMmMw70=; b=hmIwJ/IyhcXzHrpeXJmRadf+rJk2JLpf6qp/np19zFesdgrIj1iUPTmoJQNxJ6SI6X IHcMrrqOA4nE8IiNaHd2zwz6BvlPYPPf1ekepldTmgYUw+ak/bAI/v9hLzV0EYY6w0as Pd91JjhJ3PMfnyMtRXbJz4nddyfr9oLinVFCzp1mn4EqH4SlHOX9f3OICBNQavz6abuI pryFHHJbcxlbbdQzivVw5wyjGWIEhRw5l97zg8ZIES2+CB6VqneR/2wG6WO5fTubinUd HBmHOSTWDqWDcYOMtblw7eC8qj2WdaMLhZ9SkfWqev1nkC4dDfd9Onjs7JQevhMCO0rc 5ELQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584753; x=1729189553; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=nXVCtJlFVF227JmXan6PsTRYgw7//koA3Av9lMmMw70=; b=Rgv2nGsE+0mB+E8+V27PK621gJYH5iCAXHM457OBNcfQusmd29H9Pdsnu6aQbI4PsA in2MlR3O91Dy1aJOKWkj4jPRvSALJ72GiHUGazKMHzz2liEPLy64LmOMwahqEIJeBGS+ bjsF12kjesJHoDvyQzOQGOKKRM/0DRfqu4jBdjQydNhSJAtFpvrX6nmbWVZUBLCYDObr wdNT6QWu9DHBWrmQhcIPTP1ciuwaDg0rkDM6ZStuNEBIRHWXU7P1KgFAtn/9YLDu6WgX tCCH/wrQS55q73H/D7CcIpw+uLy8DggBwkbxKCR5Eyy5RYYiwipJzJ1PsSVUnxxGmqAh tk4w== X-Forwarded-Encrypted: i=1; AJvYcCX42wEQYq2w7bXL0gecq5zrLHFu/TcMqQj2KRRavEe9hkgfLMq+IzWZB+FUN/carp6B0Q8d481/8mQF8BE=@vger.kernel.org X-Gm-Message-State: AOJu0Yxue2x+Td0GZgNJFUoNGp8ZZgQq9iO57OohP6oE5TLaHeQk0t4Q nyoH0t/+ddREPAdqOTFq32FEKpMCRVk8EQf7VD0WNSobmt5x7MAxCzbWFAXqa6F/WmKMhAIrDKo oQg== X-Google-Smtp-Source: AGHT+IFqzRS1xOg2oJDncuDOdOtR7gITdwbfvBFuJ4xWEWgDQEplutdiZUKmbH0mIVak18Oaak6LM032atM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:b205:0:b0:e25:17cb:352e with SMTP id 3f1490d57ef6-e28fe43f3f5mr4095276.9.1728584753336; Thu, 10 Oct 2024 11:25:53 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:31 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-30-seanjc@google.com> Subject: [PATCH v13 29/85] KVM: pfncache: Precisely track refcounted pages From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Track refcounted struct page memory using kvm_follow_pfn.refcounted_page instead of relying on kvm_release_pfn_clean() to correctly detect that the pfn is associated with a struct page. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/pfncache.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 067daf9ad6ef..728d2c1b488a 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -159,11 +159,14 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_c= ache *gpc) kvm_pfn_t new_pfn =3D KVM_PFN_ERR_FAULT; void *new_khva =3D NULL; unsigned long mmu_seq; + struct page *page; + struct kvm_follow_pfn kfp =3D { .slot =3D gpc->memslot, .gfn =3D gpa_to_gfn(gpc->gpa), .flags =3D FOLL_WRITE, .hva =3D gpc->uhva, + .refcounted_page =3D &page, }; =20 lockdep_assert_held(&gpc->refresh_lock); @@ -198,7 +201,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cac= he *gpc) if (new_khva !=3D old_khva) gpc_unmap(new_pfn, new_khva); =20 - kvm_release_pfn_clean(new_pfn); + kvm_release_page_unused(page); =20 cond_resched(); } @@ -218,7 +221,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cac= he *gpc) new_khva =3D gpc_map(new_pfn); =20 if (!new_khva) { - kvm_release_pfn_clean(new_pfn); + kvm_release_page_unused(page); goto out_error; } =20 @@ -236,11 +239,11 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_c= ache *gpc) gpc->khva =3D new_khva + offset_in_page(gpc->uhva); =20 /* - * Put the reference to the _new_ pfn. The pfn is now tracked by the + * Put the reference to the _new_ page. The page is now tracked by the * cache and can be safely migrated, swapped, etc... as the cache will * invalidate any mappings in response to relevant mmu_notifier events. */ - kvm_release_pfn_clean(new_pfn); + kvm_release_page_clean(page); =20 return 0; =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5DBEC1FB3D8 for ; Thu, 10 Oct 2024 18:25:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584758; cv=none; b=lS3sxYMt4BQvHN79hCWWg6+deAiXHP38JzLLbtfypd2XfDgCxuhAgvmBbJqJpYJITK+msj2UFBbe7HoCpagsF1q63Uyx/idVMGvcbMID7aWoqzbE/ftw2zXmPQcfPPtJXWO1tk6YmpF7q3pazN6Wc5nbKW2AbMqV+CO6YrfGOSo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584758; c=relaxed/simple; bh=bnwD3uEfCD/YcddkLJu8d23/zYqKX46jf2vKJvYQlNE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fLpnJViU0yQced6zi03jcSDOCSwRPQ6DrjiSycslmYVVCVIEullpV//mlddLfcNGgJgbYzYBYu1Jr6kUAL3htxUJComNVdqEKHR9jcWGbmKyimm0xgm4ih/EFIJc2fdNMJBUzhtkWxPcBmPMVt7ooLHwe/tsHihY3gformHU0RY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=u+lh0lZQ; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="u+lh0lZQ" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e2d287f944so24078867b3.3 for ; Thu, 10 Oct 2024 11:25:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584755; x=1729189555; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=/prKt8Bu+DRISNBcov5NrkfgoAIYHMahITKzCWpSrCI=; b=u+lh0lZQ0iXDXtDscH8PiFZR0maNS7DQsBMYikW3/rmnfcBXpI7TGMudK3dfPgUTEB P0isjFKFnrvh+aV0DTRRLJCNrk0qEXoNyKTpSlURgiPJyby8IfIe3wTsXycMslYF84Uu LxbGHqs7w+ucwTevI1iLIdHaxVjE7F3ssW3jQgSMXBln1jgkY2wR5qIbf7ZdkBZ9CO4J LKRV/cxsUGAhvttyWa4tlVFNCym+L/JIU+85sWhn7wEMpOen81nC7NpKHxGCzaFpRxd6 w/zKDX4OeUl0XtF7v4HJT2epxE2xdrp+lv8Fag4R953vA0Hy5rN8/x7w1xKjZA+XIYl8 Vqew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584755; x=1729189555; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/prKt8Bu+DRISNBcov5NrkfgoAIYHMahITKzCWpSrCI=; b=g0Dv7eL8VxCIhAxqlX78UVFkupkFPP9IIQ08/j6EUtcEyksgARBJYYZyvgZidTUkck vrE1Tug6UDVFTCPfTukCyDvTlBQbsG6kc54NZL8B2HhRuGZsRbc2Zi1Tbt5h/MzSaBFr JZPxwAhYxZPTJTIkx/7kWrnTQUY/U05WAbtVsSq/oHHMkKr80GbMAXfGA+EHA6QHrc4o IZDjlSfcHIFv2qHZ9m9MuLOF/1c8020MtAKDkopsKzC/YwdA1nD06lI8rVyuFtGnmva4 e3K+gOt1oZEp41vwzdZeeHJBQaCPoOFpyRu3SKc5FHRvb0bpheOdmISUviFS2fWmEXVp BgAg== X-Forwarded-Encrypted: i=1; AJvYcCVHbKD/lvo1LF8oVEVUuPqNg5m6puoPZoDSQhmklTcXlyoLgWpLKRbB3LmY5J2bFU/BJ2IY6wobUY2Yxrg=@vger.kernel.org X-Gm-Message-State: AOJu0YynupGHWb9t5nB2VR9OhbjsOuTY7V4Rny9sdwVtpVOhoqmG7rth WS3zxAH1DeyNx1uizClrIFuCEmXZwJg5/+utfoggfzB4A1clMGQZpLL2hRpq6yKnWWB6q2TAXcX j9w== X-Google-Smtp-Source: AGHT+IHNZW39HKgOpo4XLx10sqcp5MldEDD35H2r/7z7SkgDatDPvbEuIFyvlgb9BtXWEl9gaR4EDWCFxGs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2093:b0:6d4:30f0:3a74 with SMTP id 00721157ae682-6e322171640mr1011347b3.6.1728584755409; Thu, 10 Oct 2024 11:25:55 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:32 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-31-seanjc@google.com> Subject: [PATCH v13 30/85] KVM: Migrate kvm_vcpu_map() to kvm_follow_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: David Stevens Migrate kvm_vcpu_map() to kvm_follow_pfn(), and have it track whether or not the map holds a refcounted struct page. Precisely tracking struct page references will eventually allow removing kvm_pfn_to_refcounted_page() and its various wrappers. Signed-off-by: David Stevens [sean: use a pointer instead of a boolean] Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 26 ++++++++++++++++---------- 2 files changed, 17 insertions(+), 11 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index e3c01cbbc41a..02ab3a657aa6 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -280,6 +280,7 @@ struct kvm_host_map { * can be used as guest memory but they are not managed by host * kernel). */ + struct page *refcounted_page; struct page *page; void *hva; kvm_pfn_t pfn; @@ -1238,7 +1239,6 @@ void kvm_release_pfn_dirty(kvm_pfn_t pfn); void kvm_set_pfn_dirty(kvm_pfn_t pfn); void kvm_set_pfn_accessed(kvm_pfn_t pfn); =20 -void kvm_release_pfn(kvm_pfn_t pfn, bool dirty); int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, int len); int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long l= en); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 6cdbd0516d58..b1c1b7e4f33a 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3093,21 +3093,21 @@ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) } EXPORT_SYMBOL_GPL(gfn_to_page); =20 -void kvm_release_pfn(kvm_pfn_t pfn, bool dirty) -{ - if (dirty) - kvm_release_pfn_dirty(pfn); - else - kvm_release_pfn_clean(pfn); -} - int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *ma= p) { + struct kvm_follow_pfn kfp =3D { + .slot =3D gfn_to_memslot(vcpu->kvm, gfn), + .gfn =3D gfn, + .flags =3D FOLL_WRITE, + .refcounted_page =3D &map->refcounted_page, + }; + + map->refcounted_page =3D NULL; map->page =3D NULL; map->hva =3D NULL; map->gfn =3D gfn; =20 - map->pfn =3D gfn_to_pfn(vcpu->kvm, gfn); + map->pfn =3D kvm_follow_pfn(&kfp); if (is_error_noslot_pfn(map->pfn)) return -EINVAL; =20 @@ -3139,10 +3139,16 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct k= vm_host_map *map, bool dirty) if (dirty) kvm_vcpu_mark_page_dirty(vcpu, map->gfn); =20 - kvm_release_pfn(map->pfn, dirty); + if (map->refcounted_page) { + if (dirty) + kvm_release_page_dirty(map->refcounted_page); + else + kvm_release_page_clean(map->refcounted_page); + } =20 map->hva =3D NULL; map->page =3D NULL; + map->refcounted_page =3D NULL; } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A8AA1FAC4F for ; Thu, 10 Oct 2024 18:25:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584760; cv=none; b=PpBU2OSzTEx2rDCU3w3EdlcaIriT7K6AExhkyBkosn71bdFtL5aBv2xyTGa+slUJxdKQuJBhbkDTAYUYo+YYlHx0xmXUKCee1CuGyWo7wgcEktmrk3VTsgRvqIzs9JAEENCWhGeAi3x03H8aBt8lswnuLkE3tqyYfqOE+d5VWIU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584760; c=relaxed/simple; bh=M2NCneg/Pye+UyLI2GGt5shIez0MQF8LN4Gs0ecBwhw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=og/z64Nk4pl85YEIAvD/o8JenBdtyy9UmElST7tZu4z7yQ/bVjLrQJQTdGnhh694eUlZktVBY0ww65HWRX0VlanUo3vPqGWafHHoYF+6IzgvLPnt8/bDLbsPrflNpkvlp/0bqC8YwOlJG/uWaX4Q2SfYSUF5HwQEhIUsgRWXlAI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CYNUenPL; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CYNUenPL" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e25cae769abso1655804276.0 for ; Thu, 10 Oct 2024 11:25:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584757; x=1729189557; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=dQaiAwtn89tr8uPQzQfay17fvGV3F3WPslUlHA+BgDY=; b=CYNUenPLiXIvtTyD74D8imBRAdC50F3hM1AGdNct9/duav3eSxT08JrxuH4uXDnr2p GlywDJgMcirPnzaX/MzQmoAche5Jq/wQ+s2jZ8bLV0HN01jab8FA4Xyx+wz08gu7Ei/1 aMZVlCmv09LzBHl5ILjwWuvSGgp907WUV3teWOwY2a5c3RZBVUymYlPJ+uSMbRo4gDHd BCHayDy3tx5zkYmvi0rcCgT1BlZperbNOxiXf9X2CrpcePe1UPYsgemnEtJcHtRrO8rT DTQAY2TnyFkSUZshaKMhikHuc//2hbWd69gYZuL5srQsImqM4AupFJHvu+dzaLsC46rc 9Ypw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584757; x=1729189557; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=dQaiAwtn89tr8uPQzQfay17fvGV3F3WPslUlHA+BgDY=; b=W5ZCQGA+sKCaoJDKykMFggODa6MqIWZQ0Gr6b6W6v01adEPZEzlohfNK3QtG6o4mNo arJYjg0ItuKPSGvebrYMsA3Jsd6/MjjK0Jgs5V9PcrO8YpnzALm23t45nYoIsdYyOs2b hnkD9qsWfXcwpz4ES6/fPKgd2KWGn3gvtPE6LHOUeOgHAiCmz2rO0H4bgsMVGdlgpyBb mKtSEtzE6N6NbMjZyp2AuIwzFDywRayTgU/pXZHkg6ZjuCAsXkvkVkfvBIzIqk6PeQ7Q NZpkyxFJXOrhPS8pcIw2MKkj4e2xzUgqgeaWpSzHeyOLynhHgdu0NoI9tfHvuTrAYz7l U6xw== X-Forwarded-Encrypted: i=1; AJvYcCUarTGoNbptDcs7/7BUVIgkP67RwHYWaRUdSDK6oXjWYrY5OcOlwly28Ru6+hL+7oGbC3SuaRSvw7EemWc=@vger.kernel.org X-Gm-Message-State: AOJu0YyN7zRaLCIyVMZtrefOmEttYKeRZtP42+RXVLMwHG2ynoG79lWS GDWspcoCT4rqo7ig4R6YukfnJuScJe7hliLqBqWrUWkhgWaC/ltSpR/AuTO0yfUUiHrtg4w2c0R fgg== X-Google-Smtp-Source: AGHT+IHY0yNiFmgeAXsfVKl7EbEBY2R3q1IXvjRBHtSPoxXhe2HbHq4twyB0Huv0R+/OHvP6V1ZW3nZzpzQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:80cb:0:b0:e20:2502:be14 with SMTP id 3f1490d57ef6-e28fe410672mr4860276.7.1728584757233; Thu, 10 Oct 2024 11:25:57 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:33 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-32-seanjc@google.com> Subject: [PATCH v13 31/85] KVM: Pin (as in FOLL_PIN) pages during kvm_vcpu_map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Pin, as in FOLL_PIN, pages when mapping them for direct access by KVM. As per Documentation/core-api/pin_user_pages.rst, writing to a page that was gotten via FOLL_GET is explicitly disallowed. Correct (uses FOLL_PIN calls): pin_user_pages() write to the data within the pages unpin_user_pages() INCORRECT (uses FOLL_GET calls): get_user_pages() write to the data within the pages put_page() Unfortunately, FOLL_PIN is a "private" flag, and so kvm_follow_pfn must use a one-off bool instead of being able to piggyback the "flags" field. Link: https://lwn.net/Articles/930667 Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 54 +++++++++++++++++++++++++++++----------- virt/kvm/kvm_mm.h | 7 ++++++ 3 files changed, 47 insertions(+), 16 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 02ab3a657aa6..8739b905d85b 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -280,7 +280,7 @@ struct kvm_host_map { * can be used as guest memory but they are not managed by host * kernel). */ - struct page *refcounted_page; + struct page *pinned_page; struct page *page; void *hva; kvm_pfn_t pfn; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b1c1b7e4f33a..40a59526d466 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2814,9 +2814,12 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_p= fn *kfp, struct page *page, */ if (map) { pfn =3D map->pfn; - page =3D kvm_pfn_to_refcounted_page(pfn); - if (page && !get_page_unless_zero(page)) - return KVM_PFN_ERR_FAULT; + + if (!kfp->pin) { + page =3D kvm_pfn_to_refcounted_page(pfn); + if (page && !get_page_unless_zero(page)) + return KVM_PFN_ERR_FAULT; + } } else { pfn =3D page_to_pfn(page); } @@ -2834,16 +2837,24 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_= pfn *kfp, struct page *page, static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { struct page *page; + bool r; =20 /* - * Fast pin a writable pfn only if it is a write fault request - * or the caller allows to map a writable pfn for a read fault - * request. + * Try the fast-only path when the caller wants to pin/get the page for + * writing. If the caller only wants to read the page, KVM must go + * down the full, slow path in order to avoid racing an operation that + * breaks Copy-on-Write (CoW), e.g. so that KVM doesn't end up pointing + * at the old, read-only page while mm/ points at a new, writable page. */ if (!((kfp->flags & FOLL_WRITE) || kfp->map_writable)) return false; =20 - if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page)) { + if (kfp->pin) + r =3D pin_user_pages_fast(kfp->hva, 1, FOLL_WRITE, &page) =3D=3D 1; + else + r =3D get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page); + + if (r) { *pfn =3D kvm_resolve_pfn(kfp, page, NULL, true); return true; } @@ -2872,10 +2883,21 @@ static int hva_to_pfn_slow(struct kvm_follow_pfn *k= fp, kvm_pfn_t *pfn) struct page *page, *wpage; int npages; =20 - npages =3D get_user_pages_unlocked(kfp->hva, 1, &page, flags); + if (kfp->pin) + npages =3D pin_user_pages_unlocked(kfp->hva, 1, &page, flags); + else + npages =3D get_user_pages_unlocked(kfp->hva, 1, &page, flags); if (npages !=3D 1) return npages; =20 + /* + * Pinning is mutually exclusive with opportunistically mapping a read + * fault as writable, as KVM should never pin pages when mapping memory + * into the guest (pinning is only for direct accesses from KVM). + */ + if (WARN_ON_ONCE(kfp->map_writable && kfp->pin)) + goto out; + /* map read fault as writable if possible */ if (!(flags & FOLL_WRITE) && kfp->map_writable && get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { @@ -2884,6 +2906,7 @@ static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp= , kvm_pfn_t *pfn) flags |=3D FOLL_WRITE; } =20 +out: *pfn =3D kvm_resolve_pfn(kfp, page, NULL, flags & FOLL_WRITE); return npages; } @@ -3099,10 +3122,11 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, = struct kvm_host_map *map) .slot =3D gfn_to_memslot(vcpu->kvm, gfn), .gfn =3D gfn, .flags =3D FOLL_WRITE, - .refcounted_page =3D &map->refcounted_page, + .refcounted_page =3D &map->pinned_page, + .pin =3D true, }; =20 - map->refcounted_page =3D NULL; + map->pinned_page =3D NULL; map->page =3D NULL; map->hva =3D NULL; map->gfn =3D gfn; @@ -3139,16 +3163,16 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct k= vm_host_map *map, bool dirty) if (dirty) kvm_vcpu_mark_page_dirty(vcpu, map->gfn); =20 - if (map->refcounted_page) { + if (map->pinned_page) { if (dirty) - kvm_release_page_dirty(map->refcounted_page); - else - kvm_release_page_clean(map->refcounted_page); + kvm_set_page_dirty(map->pinned_page); + kvm_set_page_accessed(map->pinned_page); + unpin_user_page(map->pinned_page); } =20 map->hva =3D NULL; map->page =3D NULL; - map->refcounted_page =3D NULL; + map->pinned_page =3D NULL; } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); =20 diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index d3ac1ba8ba66..acef3f5c582a 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -30,6 +30,13 @@ struct kvm_follow_pfn { /* FOLL_* flags modifying lookup behavior, e.g. FOLL_WRITE. */ unsigned int flags; =20 + /* + * Pin the page (effectively FOLL_PIN, which is an mm/ internal flag). + * The page *must* be pinned if KVM will write to the page via a kernel + * mapping, e.g. via kmap(), mremap(), etc. + */ + bool pin; + /* * If non-NULL, try to get a writable mapping even for a read fault. * Set to true if a writable mapping was obtained. --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E22791FCC45 for ; Thu, 10 Oct 2024 18:25:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584761; cv=none; b=D2Bm1iQuJPwrp/AadiRYGHJZizkUJSfUoc4HHHpkg8Tpsu4yXw5lUjWlffPu9Apkl0fAD30MuFMCAc1RHCcttsz8UhbWEFZzaaQ8bD0PbcS8NKc+3RT9k8tlNTmFeflHDIuqsco7U4+XB9QR/+HZ0EiMi7ehfz/x4yvaOtTTpKg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584761; c=relaxed/simple; bh=6AchJ9iF2qZs5ZSAOBrdf0urgGqdHLpa0Cd8lTPRDhM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=slUSN9EGvSi1eCLm47B+7jsOYQapCV2KcqZDbm6GJFpHgCvf6c3Lk0afoeAVTCU0ONdIiv/hQc+EUwKXmrs5vMm5/CMvYX7iipeYMMTugjdRdIemfdPJw5tbVYF9xtxbGUgeaCDYMXv+SpiwC+ix2nnCzQQcSJVkuerFzWBUgu8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=KeBY5py2; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="KeBY5py2" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7ea05b8ea21so1335566a12.2 for ; Thu, 10 Oct 2024 11:25:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584759; x=1729189559; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=VITkHk6wYCk30qmAXnWfpQxFdItqbxje0xl/TrLgm28=; b=KeBY5py261h3po2p7EcoPVpo67GiOj7+6dOP6EoAEriscYrzhTzO39NHb0NJsAW1jR n42KW4EwGgaUs3yx/oCZEuQvV/M1idASTdIADhBN6ON4GT5SGgjV1ooFQU62n4X7Qr84 /fRWijapZZzZVKePZxyFZegIWlOsafg0wDZM5BHf4vXa9CU5WL9dY+OB5HyQ609h9s7e VPCG5daQbQMIE3NYESeghcC9iMjOQZxPTOapKpidLBB73TVQIpNb1Ze+j7LMk9Bx3x2i vO15GDeuHOJlaco62A9bG0UiqB8QoTlfPHg1Bpr7sbnSv5dMGej5HVQzlTOoDtu38Q9N 6JtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584759; x=1729189559; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=VITkHk6wYCk30qmAXnWfpQxFdItqbxje0xl/TrLgm28=; b=PYJttwyfk5Y8yMlJ5uOe+0s/Upe7vkeLb75Eq6gDP5Iqkfzhdy15tVAv27JDAyUNzh TfPXDmGS+36me4RTevYXeURl4jE8R6miKkFa0VLbfS0mLpstIdCwPWfNlKIh5Z33d6L3 W6qXxsCKxwOwRQFGBkIwD/51OLW73rzOsER0PjBNoQuWpmLORntHtWT+yXc+DdYrC4O8 Nq6zFx5saHOtvNPW/gB/T2HEFGqhDsTCLxaPDVOvsBoQPxxf5s7L8dSZkvHiu+Xv73Qa hfs8bi2HdSZEreFTFxQHAdIf40g0Dkiw73cYv29CMC4hJU363RWhNAq2VpB/w75Lmiaj fMJQ== X-Forwarded-Encrypted: i=1; AJvYcCVtujReNM5k/O9VlTU6qEg19Ak+Dd8MdBw2b6nXJTK2wK4NbiYX9bbif6+ntjyhUACdIQ314+mUiEdVdF0=@vger.kernel.org X-Gm-Message-State: AOJu0YymVN/RrqdM2GgE6TdqGcMvfEPIyDEHj6t2VMbIaPnVj68p9vCJ QANvDnMSJZmQnHZ2WXCDVU8vhk7m+rLbfMmU+3I9jIIQON27u5WfemnfMdDGRxqvLu7dBspk7zc KKg== X-Google-Smtp-Source: AGHT+IEekecUM0XVWoBmNjTczqp1a2/ixTMw1fOAmBrSm/Dp9XgHLEkyd54mJG+SlDPSB0szYk/zKOq5dxU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a65:678f:0:b0:7e9:f98c:e9f7 with SMTP id 41be03b00d2f7-7ea5359ed14mr32a12.10.1728584759044; Thu, 10 Oct 2024 11:25:59 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:34 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-33-seanjc@google.com> Subject: [PATCH v13 32/85] KVM: nVMX: Mark vmcs12's APIC access page dirty when unmapping From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark the APIC access page as dirty when unmapping it from KVM. The fact that the page _shouldn't_ be written doesn't guarantee the page _won't_ be written. And while the contents are likely irrelevant, the values _are_ visible to the guest, i.e. dropping writes would be visible to the guest (though obviously highly unlikely to be problematic in practice). Marking the map dirty will allow specifying the write vs. read-only when *mapping* the memory, which in turn will allow creating read-only maps. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/vmx/nested.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 81865db18e12..ff83b56fe2fa 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -318,12 +318,7 @@ static void nested_put_vmcs12_pages(struct kvm_vcpu *v= cpu) { struct vcpu_vmx *vmx =3D to_vmx(vcpu); =20 - /* - * Unpin physical memory we referred to in the vmcs02. The APIC access - * page's backing page (yeah, confusing) shouldn't actually be accessed, - * and if it is written, the contents are irrelevant. - */ - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); + kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, true); kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); vmx->nested.pi_desc =3D NULL; --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC4711FBC91 for ; Thu, 10 Oct 2024 18:26:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584763; cv=none; b=JBIaPNeTA6egGlxstbliBry576fKAxgzYTsnpXCqBBN0V5QZizOkjq3GLL/6fT6tTG+u9yiLAQ/YHtQarjDExnLzUzUM28Qeit8wTiZvrFlV29rVgNhsQ7ccDTNqXJ21IvOaw1oqabIYh5ntAnuIiDTDr2o/Roi+Uod2heq59Ts= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584763; c=relaxed/simple; bh=mQXhN5iZpOZWnnd2N5GwBNgcRKXl5xUfnEr6O0WaUwk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=F4iCQ9tuPEJ02A/tHI0kttWW2SC1LisCg8njpssF6gfLXlEFZjvzg+16L6SCd7q5LEMJEXMK3oGWRbJdeqjeiUBs9LGG7YHSoHG1GrTjaYfJHJn8QTnjowDqY8+iYhxpTNV5FR/v4VzlwKdS/4Ryvxs6mO3F/P6f1PguCmuyHbI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fhtsmxz8; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fhtsmxz8" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-690404fd230so25174407b3.3 for ; Thu, 10 Oct 2024 11:26:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584761; x=1729189561; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=+DlXoUFYFzJpPJut8ySNZ/avzuJs4Hk+oSmN6VTWgSk=; b=fhtsmxz8vpfa2op10hhZgWbQg3a/Ux0j9mEjjKDNpT+6H68jzG7G30Lrt0DgwS2OqA VAQch49mqfwm5cA16yd/W6I1emiwxouDMXVSvKt1yo8/wuYywAj8WmU3ri9BLGwOXV2O ev+52ekxcamGE3PinuEE6axN2+AjKb8EBbMunUi5MpzIPNmR7Cd+feQewcjH6urh4T2Y L7YmQKAT/bzQUgg9L16aEUHrMjmkiyjytv61vSD0Smo/6KXR/dS0NqmYK+EDeZ81iIUB BpN0M+BzPxJAzpf2q5xvJT+CdoJM7Ovvc/CKmFvaOg7Bed7+EqAlOAxWv/v1tDiHH+OX ghxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584761; x=1729189561; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=+DlXoUFYFzJpPJut8ySNZ/avzuJs4Hk+oSmN6VTWgSk=; b=FvK5sTy/28CLxgIVealHd0/BXyoxh0x8MyaZwiT2YhFPu+Q6CtcnvH1exHHD9SmASJ RakRCT+qnLaeN5gRXMh0AKcN/asW6xpz0kfGGx4B+tNbgXJmbWooiBprcbONH9o3fCDj VNXiDwKjWwfER2ythhqsBWkdouHbNwmQCX1uCo4udI5klrcY61I4JlGI/Wlyhv1+T56q w+d5SwiqIKIHwwV3ceQ6EKPURFo6n1egiFfL7lm3QfCbVm5ZLf6aWVjP9j9qhyh0luDT BKHtYt8nv8YyNQ3zKA3+MIKgm3L/qF9Bll5ZWT4sd5TUVMtV8dhhln5eHINHBkH7K8by mGMw== X-Forwarded-Encrypted: i=1; AJvYcCX1vKQWWkTgp9zYBNA8bCdxycPxiQ/R6jI5F1UB31Y0Hk1S7QM4Gi2bxD4WwzJMTj71bvsYm2HPZzHV8wc=@vger.kernel.org X-Gm-Message-State: AOJu0Yy+KlhRFJdJaaN5TT+PRbSdbp5nNvnN8qv0+3ayznojBbIHM4t9 RBd/R6drGRexB5TR/K0nXqQeeHuwIvq9/84Q4hViLHLpTam9eL6hCfQAM72zdd0fql2zzZQpl8n btw== X-Google-Smtp-Source: AGHT+IHYBe/gylmYYB76Y4JbCGQqBxyU7NvKFk8CzbtQRqqOSH0kk5njqmBLFW7wxt+Okx9MKc9CHg/F7Og= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:4349:b0:6e2:1713:bdb5 with SMTP id 00721157ae682-6e32217cfd3mr118607b3.5.1728584760829; Thu, 10 Oct 2024 11:26:00 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:35 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-34-seanjc@google.com> Subject: [PATCH v13 33/85] KVM: Pass in write/dirty to kvm_vcpu_map(), not kvm_vcpu_unmap() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that all kvm_vcpu_{,un}map() users pass "true" for @dirty, have them pass "true" as a @writable param to kvm_vcpu_map(), and thus create a read-only mapping when possible. Note, creating read-only mappings can be theoretically slower, as they don't play nice with fast GUP due to the need to break CoW before mapping the underlying PFN. But practically speaking, creating a mapping isn't a super hot path, and getting a writable mapping for reading is weird and confusing. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/svm/nested.c | 4 ++-- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 8 ++++---- arch/x86/kvm/vmx/nested.c | 16 ++++++++-------- include/linux/kvm_host.h | 20 ++++++++++++++++++-- virt/kvm/kvm_main.c | 12 +++++++----- 6 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index d5314cb7dff4..9f9478bdecfc 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -922,7 +922,7 @@ int nested_svm_vmrun(struct kvm_vcpu *vcpu) nested_svm_vmexit(svm); =20 out: - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); =20 return ret; } @@ -1126,7 +1126,7 @@ int nested_svm_vmexit(struct vcpu_svm *svm) vmcb12->control.exit_int_info_err, KVM_ISA_SVM); =20 - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); =20 nested_svm_transition_tlb_flush(vcpu); =20 diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 0b851ef937f2..4557ff3804ae 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3468,7 +3468,7 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm) =20 sev_es_sync_to_ghcb(svm); =20 - kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map, true); + kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map); svm->sev_es.ghcb =3D NULL; } =20 diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 9df3e1e5ae81..c1e29307826b 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -2299,7 +2299,7 @@ static int vmload_vmsave_interception(struct kvm_vcpu= *vcpu, bool vmload) svm_copy_vmloadsave_state(vmcb12, svm->vmcb); } =20 - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); =20 return ret; } @@ -4714,7 +4714,7 @@ static int svm_enter_smm(struct kvm_vcpu *vcpu, union= kvm_smram *smram) svm_copy_vmrun_state(map_save.hva + 0x400, &svm->vmcb01.ptr->save); =20 - kvm_vcpu_unmap(vcpu, &map_save, true); + kvm_vcpu_unmap(vcpu, &map_save); return 0; } =20 @@ -4774,9 +4774,9 @@ static int svm_leave_smm(struct kvm_vcpu *vcpu, const= union kvm_smram *smram) svm->nested.nested_run_pending =3D 1; =20 unmap_save: - kvm_vcpu_unmap(vcpu, &map_save, true); + kvm_vcpu_unmap(vcpu, &map_save); unmap_map: - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); return ret; } =20 diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index ff83b56fe2fa..259fe445e695 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -231,7 +231,7 @@ static inline void nested_release_evmcs(struct kvm_vcpu= *vcpu) struct kvm_vcpu_hv *hv_vcpu =3D to_hv_vcpu(vcpu); struct vcpu_vmx *vmx =3D to_vmx(vcpu); =20 - kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map, true); + kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map); vmx->nested.hv_evmcs =3D NULL; vmx->nested.hv_evmcs_vmptr =3D EVMPTR_INVALID; =20 @@ -318,9 +318,9 @@ static void nested_put_vmcs12_pages(struct kvm_vcpu *vc= pu) { struct vcpu_vmx *vmx =3D to_vmx(vcpu); =20 - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); + kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map); + kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map); + kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map); vmx->nested.pi_desc =3D NULL; } =20 @@ -624,7 +624,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct= kvm_vcpu *vcpu, int msr; unsigned long *msr_bitmap_l1; unsigned long *msr_bitmap_l0 =3D vmx->nested.vmcs02.msr_bitmap; - struct kvm_host_map msr_bitmap_map; + struct kvm_host_map map; =20 /* Nothing to do if the MSR bitmap is not in use. */ if (!cpu_has_vmx_msr_bitmap() || @@ -647,10 +647,10 @@ static inline bool nested_vmx_prepare_msr_bitmap(stru= ct kvm_vcpu *vcpu, return true; } =20 - if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), &msr_bitmap_map)) + if (kvm_vcpu_map_readonly(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), &map)) return false; =20 - msr_bitmap_l1 =3D (unsigned long *)msr_bitmap_map.hva; + msr_bitmap_l1 =3D (unsigned long *)map.hva; =20 /* * To keep the control flow simple, pay eight 8-byte writes (sixteen @@ -714,7 +714,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct= kvm_vcpu *vcpu, nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, MSR_IA32_FLUSH_CMD, MSR_TYPE_W); =20 - kvm_vcpu_unmap(vcpu, &msr_bitmap_map, false); + kvm_vcpu_unmap(vcpu, &map); =20 vmx->nested.force_msr_bitmap_recalc =3D false; =20 diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 8739b905d85b..9263375d0362 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -285,6 +285,7 @@ struct kvm_host_map { void *hva; kvm_pfn_t pfn; kvm_pfn_t gfn; + bool writable; }; =20 /* @@ -1312,8 +1313,23 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn); struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn= _t gfn); kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); -int kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *ma= p); -void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool = dirty); + +int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *= map, + bool writable); +void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map); + +static inline int kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, + struct kvm_host_map *map) +{ + return __kvm_vcpu_map(vcpu, gpa, map, true); +} + +static inline int kvm_vcpu_map_readonly(struct kvm_vcpu *vcpu, gpa_t gpa, + struct kvm_host_map *map) +{ + return __kvm_vcpu_map(vcpu, gpa, map, false); +} + unsigned long kvm_vcpu_gfn_to_hva(struct kvm_vcpu *vcpu, gfn_t gfn); unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, b= ool *writable); int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data,= int offset, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 40a59526d466..080740f65061 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3116,7 +3116,8 @@ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) } EXPORT_SYMBOL_GPL(gfn_to_page); =20 -int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *ma= p) +int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *= map, + bool writable) { struct kvm_follow_pfn kfp =3D { .slot =3D gfn_to_memslot(vcpu->kvm, gfn), @@ -3130,6 +3131,7 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, st= ruct kvm_host_map *map) map->page =3D NULL; map->hva =3D NULL; map->gfn =3D gfn; + map->writable =3D writable; =20 map->pfn =3D kvm_follow_pfn(&kfp); if (is_error_noslot_pfn(map->pfn)) @@ -3146,9 +3148,9 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, st= ruct kvm_host_map *map) =20 return map->hva ? 0 : -EFAULT; } -EXPORT_SYMBOL_GPL(kvm_vcpu_map); +EXPORT_SYMBOL_GPL(__kvm_vcpu_map); =20 -void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool = dirty) +void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map) { if (!map->hva) return; @@ -3160,11 +3162,11 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct k= vm_host_map *map, bool dirty) memunmap(map->hva); #endif =20 - if (dirty) + if (map->writable) kvm_vcpu_mark_page_dirty(vcpu, map->gfn); =20 if (map->pinned_page) { - if (dirty) + if (map->writable) kvm_set_page_dirty(map->pinned_page); kvm_set_page_accessed(map->pinned_page); unpin_user_page(map->pinned_page); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F5941FEFC4 for ; Thu, 10 Oct 2024 18:26:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584766; cv=none; b=onOcQGuJfaZ9UXHR7cbSa2y/+NPwKENey+b9s2461o7OFJSK77jXtbF2LvNBXySSho7AnlsbsdcxQqxIhvssTc8mfoUL0bVv+Q32GuanYeU0yelQrmGiNd6Na8gmRNMC9gy0S/hKKSFnFcYhC2aeL6jGAjddSuuRc01sH9fYpWQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584766; c=relaxed/simple; bh=+qwur9UMvfPsT7xhW5+1hFXGI3F0T2CjnbFVAQFEZ9U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WqH6qKiJAFf5MWctb9YvE0igLP9Vz06ur6r7jH7lpgm0knMuNh4NieQegdmejwd1W1Q9BC1JfWphkm1ngHNwiFxHXaAIYUh7UI33tcM11Ckn0rL18ZIEsSu+IX7qYmZCcR+31Icf+aS0aCrVeB3LDv8qa2e4S7DHdKIbsO+ykKs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qwMYhkdO; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qwMYhkdO" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71df051770aso1193819b3a.2 for ; Thu, 10 Oct 2024 11:26:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584763; x=1729189563; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=w3u38MFKj4Hgu0NiviVwRjtFpk/ioMNBGBDQlt4DAgY=; b=qwMYhkdO+k0O3GAu86CC0Oovaqx2YjaPKBv6LeLzcOh3PtNiOaQrcv5HBilI8icD9g y0Ltij7dFk7sin4usHA+6Wv1hFiwFPtq5Ajocz2SMLSw7EHhQ24kqq+Bt+H2kyBdKaty qPo1gORXTlpg4cwuwI2/Xd/UCp8kWbToIyady8yUqr5BbwLt+Kkm56JwyHrVdgysxAdq eCA0ipk6bTZMsprIdAiZDgjFHxpqT0t6hWRsy+Feo3csNnBxmMja133Zk9XVBeicQl/q 2iXz1SFCo5UtodgUyE2QoPpXLNLLkpEupG73DPzHBSBkCdaePTQJNXxdK3uBHhhZG/wU Akeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584763; x=1729189563; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=w3u38MFKj4Hgu0NiviVwRjtFpk/ioMNBGBDQlt4DAgY=; b=uqXh5Lo9mOWD4CTlGRE3DhjFQt74dwz2M3Va21H6W7RCMeJwFNrdyEn2Mcbp88Chjn +uvEhF1WcoFjHxGm6fcezjm2ZDRLxmuiIRg/b6UbhtO6R8URkR4RxAwdt0HjdxjVZanX FKrQ0krMMYQ5IBIZiLdq/O0VL+YpBaiSEw/kg2yWuncDxkUWs7MDZB7p225Bk4/xCr/v fJ5tGjg1ni/VPBRCwUfwd1JqTW5AnSgUizJG8TGvNjq+tF4gMVFSGH42wIg7A9P0Hj2D BGzJbSzt2RnuDs1N34IFBZ3tmiopvTXmcSo3AMzncagiAooydRO9N9QbJcW2t95oRej9 Re8w== X-Forwarded-Encrypted: i=1; AJvYcCXOW4FTEygub3LDFCGTa+vWKsM2GYscIsjes4aK/5FZx7a6qmJGHJ3Vt6t29R4w/mIC+Aqm+gZHN6ACC2A=@vger.kernel.org X-Gm-Message-State: AOJu0YwEQVbRbkNuGkg0LSbZcav3Vb9Xp2dBma+Eddv7Hl1v6PzX1n+8 cfVzIcZG9nAoVtx/OlwYqMcNc2i+GzQfR8LiC41gCEhczqKL6SXZL7cXAglKQ01vyReM2jBehVW gHA== X-Google-Smtp-Source: AGHT+IHthU1bxxWE76UmRU/cmSB/2b+tJYVulTyd1SGmZGLYBHQFEbukUNmDJAlwNs2CFth+SfvGFADhDCI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2f51:b0:71e:dd:8f9b with SMTP id d2e1a72fcca58-71e1dbe8750mr6976b3a.5.1728584762866; Thu, 10 Oct 2024 11:26:02 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:36 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-35-seanjc@google.com> Subject: [PATCH v13 34/85] KVM: Get writable mapping for __kvm_vcpu_map() only when necessary From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When creating a memory map for read, don't request a writable pfn from the primary MMU. While creating read-only mappings can be theoretically slower, as they don't play nice with fast GUP due to the need to break CoW before mapping the underlying PFN, practically speaking, creating a mapping isn't a super hot path, and getting a writable mapping for reading is weird and confusing. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 080740f65061..b845e9252633 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3122,7 +3122,7 @@ int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, = struct kvm_host_map *map, struct kvm_follow_pfn kfp =3D { .slot =3D gfn_to_memslot(vcpu->kvm, gfn), .gfn =3D gfn, - .flags =3D FOLL_WRITE, + .flags =3D writable ? FOLL_WRITE : 0, .refcounted_page =3D &map->pinned_page, .pin =3D true, }; --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90F761FF7A8 for ; Thu, 10 Oct 2024 18:26:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584767; cv=none; b=TOel7F09Bx6CpY48I1CA0eXtVo4HqNl4SoFekPMUtEo1Z4zU9EpUE+nu+mQF4LL8sRLBpZF/2u+3kbQtH0/LOtSX5bRk8NcJdQf+hkgCZeWsBlM0XR8+dR+s19/HLLOvNqqzc9noUhFJn9C7NXJmKPvRZZmalDbR1ezjSPxAPN4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584767; c=relaxed/simple; bh=JluxJpntN2XzeRJ5Yz03kdoesvuYdAA2z8LW9UOaBnA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Pr2BZivmeEMdSSGsDKK2NhYa13gGcZEaAAbJgP6XymxVVtlaestVDpd4VLkLnM+AUnKo/HT0T9DQQcngViHoV5AZq2biBnBF8yQn7wBmCbMbnuDhmXbhDgMC5g9roZCeNKoInwH0+dHMNaDtuS9xdn5oH21Etl6wAgRw/F9XWsU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=VfKoGAUz; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="VfKoGAUz" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7ea37e8894bso1587461a12.0 for ; Thu, 10 Oct 2024 11:26:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584765; x=1729189565; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=ByBZxBThzycQ9+5cWv7LHsEIs8PvxVG+sWS1etQw10Y=; b=VfKoGAUzy1tyfifoK0lzuC5EO6vmRc5H8EipIe+ZLEuWAXzAG8pU2LiyJzlZVu0hi0 mOO9VygcpFe8zOSwUmfQQHakO15699SyqMVOWWSAiUH2fJJdgC8rguMny3tmzuNm6qma xqCi/swqvFmWY9n9yk/3JuaiFkFnRNBPBb225gAWzmoJNnJjRf6EzDz+lob/Swy2Zd8F 5WPg5QuuVmEjsJhFvUHJUB4SSv4y2afPWrVKR+wB8gSLJdQ+AEm1DKIqe37EnkjyN8cB 4U807FVw2rgqm/ygIdqMTxDCTj1f1B9aQDFvY0AHA0uFNQgR8DeiTSbrOcm3suHvH+5h 7yIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584765; x=1729189565; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=ByBZxBThzycQ9+5cWv7LHsEIs8PvxVG+sWS1etQw10Y=; b=uJEWsHC+sw8KmKbkTVQv0/L5V58gya61aed9rKZ5EKZ/hsu283bL0VRK3H/LtcVK9Z 5itQFBjopHojBZ6rMd+QSiQdRrkMmGK+ZBjwErItVbrPeo1CByLXvXWu6yPnx8zq0KgA i53Gfiao0UdpqvVdBjeuZjZGEd74YqZAh32tiB8yqWWRa/heOA6fNqNtHdjC6b1zD+Ja FKYPh2Ttnl7gaC8mHrNmkVraoDEjpA9rZ82oO3k67yVvEz+LfcJcr4pOOjAJaURgNIhT J+5ClHxGl3zivS0PIIz5404eWignra4DbRbzM4ZFUE5LzaouCEuJ4I4q1uj+o77p33uS AhAw== X-Forwarded-Encrypted: i=1; AJvYcCVG1lLlMKdlAR5sy6rpiaa02pSbBcpzpmBJjCdXmaHjJ3bsx2Hy4SXbwCZ9OJ7DMQvruL/0ineuA1I7/v8=@vger.kernel.org X-Gm-Message-State: AOJu0YzUfChca3LNU1zdQW3jYXaj3aE6CrmfI34kMecef1GY4ceVNOUA qEjV6RmEFaexEc6meymvY7P6XqSZgNwnFwT6hpqaHXCWEiqZZ2hFH09M+PbFvXPnt2s01pMY673 f+g== X-Google-Smtp-Source: AGHT+IE23QkhjcWr/3QASJ/S9+dapZUNy7Z4CCepJKqapzuUx8eBu6OhWGfM5laLrlsw47tRhKuFSYugMsQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:ff61:0:b0:7cb:c8c3:3811 with SMTP id 41be03b00d2f7-7ea5356a658mr52a12.5.1728584764791; Thu, 10 Oct 2024 11:26:04 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:37 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-36-seanjc@google.com> Subject: [PATCH v13 35/85] KVM: Disallow direct access (w/o mmu_notifier) to unpinned pfn by default From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add an off-by-default module param to control whether or not KVM is allowed to map memory that isn't pinned, i.e. that KVM can't guarantee won't be freed while it is mapped into KVM and/or the guest. Don't remove the functionality entirely, as there are use cases where mapping unpinned memory is safe (as defined by the platform owner), e.g. when memory is hidden from the kernel and managed by userspace, in which case userspace is already fully trusted to not muck with guest memory mappings. But for more typical setups, mapping unpinned memory is wildly unsafe, and unnecessary. The APIs are used exclusively by x86's nested virtualization support, and there is no known (or sane) use case for mapping PFN-mapped memory a KVM guest _and_ letting the guest use it for virtualization structures. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b845e9252633..6dcb4f0eed3e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -94,6 +94,13 @@ unsigned int halt_poll_ns_shrink =3D 2; module_param(halt_poll_ns_shrink, uint, 0644); EXPORT_SYMBOL_GPL(halt_poll_ns_shrink); =20 +/* + * Allow direct access (from KVM or the CPU) without MMU notifier protecti= on + * to unpinned pages. + */ +static bool allow_unsafe_mappings; +module_param(allow_unsafe_mappings, bool, 0444); + /* * Ordering of locks: * @@ -2811,6 +2818,9 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pf= n *kfp, struct page *page, * reference to such pages would cause KVM to prematurely free a page * it doesn't own (KVM gets and puts the one and only reference). * Don't allow those pages until the FIXME is resolved. + * + * Don't grab a reference for pins, callers that pin pages are required + * to check refcounted_page, i.e. must not blindly release the pfn. */ if (map) { pfn =3D map->pfn; @@ -2929,6 +2939,14 @@ static int hva_to_pfn_remapped(struct vm_area_struct= *vma, bool write_fault =3D kfp->flags & FOLL_WRITE; int r; =20 + /* + * Remapped memory cannot be pinned in any meaningful sense. Bail if + * the caller wants to pin the page, i.e. access the page outside of + * MMU notifier protection, and unsafe umappings are disallowed. + */ + if (kfp->pin && !allow_unsafe_mappings) + return -EINVAL; + r =3D follow_pfnmap_start(&args); if (r) { /* --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF879200103 for ; Thu, 10 Oct 2024 18:26:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584769; cv=none; b=EcfnBXIRML+7X8KaTFwI/ay93OWZsmTQ0IYmRoKVZYxz93lNgfuQEnc4hauv8rpO0vo+T9zT/+hNuvKqvKHx/j27McNZWil6SRvwF8DalwF3Sx98v4uL49jEDZ1E+soAl6ybZNLCjdgtCsXQGFBP6UJOSj4IH/+LJTflFZvNJtQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584769; c=relaxed/simple; bh=HP41e6SfsC4ul+oSDEyzHZ0hlt7iv8seYpnKNCv0jUQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=aBINEimqQXBwx/RPQFM+xOb1Zi0PeECYJgUyf42SbHBU+YVp3KSwPpz1U6bA5Fmr5Xt7XVCPmfJu80gxGI08jrCKE1fOpbbR0a29aeyCsFCpGhH0pcnqpnbwg0LJHNotEw10oG0Z/Mjy3bs0WzUTpvbXgwchSUVmHI5s4Xfnn6I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3iE+fjr+; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3iE+fjr+" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71e04c42fecso1289996b3a.0 for ; Thu, 10 Oct 2024 11:26:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584767; x=1729189567; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=n3l8C52m2RjpLTXcY6dq+UIBo01A1fTC4GBMzDz1kHI=; b=3iE+fjr+J+BhPA8v3iqwSMU7gQHtmXoLc2UoAeE8K7W7XowZPMedacJz0kDLjxUDYQ QBHxZX1NpRwXYSGnH0HL3vJtR2VFd/IfEGEWVjktv7EDSMjvcPUq75M3pmwLSNSY+vlM TTGcpYpnDBEcT6Yq4ZgXXOtSRMnft79vdJT2U4MoIAfvpL1GKwjhtjsMIhwYKH1sdUdg mA89AXe5f0SdxOYRU+ZmAG4KCc7jke6+CVqu1SVWGJg4hyIVrQYwExBFF/T3JmjQdnSL ThQ6E+6hJCDsmRcl4PwyBxiNMZtTBKOsC/MsBvMbt48BIiFEhaYo/ha2z7lutfruNTUk snKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584767; x=1729189567; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=n3l8C52m2RjpLTXcY6dq+UIBo01A1fTC4GBMzDz1kHI=; b=C1W+JZFDkHJ7oYkoQHiMCHwZrA7nb1vGszkxQ2KGhCsHLkUYnGfjbmRiBsjonapVgs 1vLw+Bs3gGLNfrJNW6/IlPt0vsB7plaTjoLqL8VUBtzRy/oU5DzQk9VUoWY1lrwWcn6D CjOWwnNB7OmKENDIuC0L5V/VH2wLHxiSgmmxm01b5qI9gqCR1p9xhItXqo5YBGJkuDWb hkH1ZbXbDAoO/TfIm7wHWLsje8e4BRFDX2l6gvGe8Mnf/Pa7XF7g9CPSr5vFg/wJ/bY0 qL9ErL5a1sZBDlpCeDkI2U66c5TKcWF3jGqD6prQO0dOPvI62BQDvKu+yuAAjUsH+Dxh 7Pvw== X-Forwarded-Encrypted: i=1; AJvYcCXmIYDTsVI+8gVNwgcTLW7WOGD+l8A4G7cz/KcMY+duyaUgaqMU/NLq/Y6HB+bDcafkZWHVxAi5Ukuu4kc=@vger.kernel.org X-Gm-Message-State: AOJu0YyORl/QTLC2UrzPE97vlZajvPC7s3SGI7GceiXRjqoQ5IvwDvfy 9gELHGGMxbpURBtOT3zwGsbLjiqYC7XHEpAUoZUDxKcgXwML3vUBH0vwS/GOryV0qwskDNG4jX8 WNw== X-Google-Smtp-Source: AGHT+IH5y4h19aQ8D3UFZNHEjFs0l+IkJ/F+GwDOdY5CLNZ6WKY+eP+zfpO4DiXt2PfVXCIgSRJOtCb+AeE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:6f44:b0:71e:268b:845e with SMTP id d2e1a72fcca58-71e26e53c16mr11733b3a.1.1728584766802; Thu, 10 Oct 2024 11:26:06 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:38 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-37-seanjc@google.com> Subject: [PATCH v13 36/85] KVM: x86: Don't fault-in APIC access page during initial allocation From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop the gfn_to_page() lookup when installing KVM's internal memslot for the APIC access page, as KVM doesn't need to immediately fault-in the page now that the page isn't pinned. In the extremely unlikely event the kernel can't allocate a 4KiB page, KVM can just as easily return -EFAULT on the future page fault. Suggested-by: Paolo Bonzini Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/lapic.c | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 20526e4d6c62..65412640cfc7 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2647,7 +2647,6 @@ void kvm_apic_update_apicv(struct kvm_vcpu *vcpu) =20 int kvm_alloc_apic_access_page(struct kvm *kvm) { - struct page *page; void __user *hva; int ret =3D 0; =20 @@ -2663,17 +2662,6 @@ int kvm_alloc_apic_access_page(struct kvm *kvm) goto out; } =20 - page =3D gfn_to_page(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); - if (!page) { - ret =3D -EFAULT; - goto out; - } - - /* - * Do not pin the page in memory, so that memory hot-unplug - * is able to migrate it. - */ - put_page(page); kvm->arch.apic_access_memslot_enabled =3D true; out: mutex_unlock(&kvm->slots_lock); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B45142010E9 for ; Thu, 10 Oct 2024 18:26:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584771; cv=none; b=R8hzqpR6LGFBcoofXSDDV5F/FOw4KrQ2PzH0ayKFYAMX0rpi/X+VkRZzqmqRb9J8mW7YIhxqVkzu+fnY/SF4Ct+shVYw5xVltnMV4ad38jf9sRTKJtLYxmLLfGVgekDdGkNQq05NITw0zG/P5EzMZMlee1RO67vyJn8e36w/lGs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584771; c=relaxed/simple; bh=FOrzmk+4dQuoAh6BJ51EHAqwvBR4fq2vz5+jTHFfYUI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WQhzcmCd0MH6CyXGKdAU//lNi0d9oXRmJfICnlS5fgoDcu4JxGeZupgfi7yf2V911Fg7/ZCKBifq915oROSbjDZKtBYI5U4RaP5mftz6O/CvHlG/BgdEyPh/NJXeIUWtf7/7UjGsDvqNscSczGoPncOOUWi8FVFvoz+WaSKV+kM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=n3ZB9Hl9; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="n3ZB9Hl9" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-20c8b0b0736so7708565ad.3 for ; Thu, 10 Oct 2024 11:26:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584769; x=1729189569; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=551DSOChXG5A+yDanUyc1gC1jxg9ryEf27npUXDW1CE=; b=n3ZB9Hl9GSMKmEx/lUk3TOxxvmnE72q9Nb6QJN/j4oQK/ZZl7fEE+aU1iTmEyHusGF MvskZF7JItYNQUwaksK1w6E2W6vRxUpqG/GQskA591YhhwipexkEwz3mVDnMI35S5jsv Y4WSCAtt0cfpUgxFf5UIV/1sPoj+lv6W94JgPBO2jl9ksD7m1Suxh1wAL4H+VTW+S7Pw k0Dmcxek3i15MXvGlTyKrDmdYFW9aHLcUrFd1RXD7zqVAp9dLiiN4ZKuzQ+JtjjDKIk+ B6ODMMAUkUqPGWTVvDQlBjpM7hat/NMris04mBqZ5fLmXqjpagd/DFoKuVvenG/nokos h98A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584769; x=1729189569; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=551DSOChXG5A+yDanUyc1gC1jxg9ryEf27npUXDW1CE=; b=ldPApwT8bBPtTUp28o03sDocGvaY0s3Op+Q839duj2kYdSz1Z5hzKtRzgKmDLc4ON4 HfeXH7P39rcRLD70agaqWm8CraCt/19Wvc2Daq9fjf0l6U6XYcx9JgHGVY+rbFmmqv6i xhX3Y3vrvKPSt7N/U+KGUsY96UMiWepfS/rCc0vDWp+NtIOG0BAtlG9vTCxs13uat1ck exMhZMnviYmThl1LIMjpIh9rps3gTFVoomK1P6xYFR0HTppgBUh9+ELQkmyW1YuRZhhA TbrcLa52hyHSSFE5ulEafVs/DMTNQzJm/msDaqFqdoWeH4tLi1GSGeL80Vje7wJqqUOP EHwA== X-Forwarded-Encrypted: i=1; AJvYcCU9Q8iwlM/1/9rY6W+wHqVVDcBibJRNFF5cA9a4ppbaYyHi5/wpFalCtf38SxAPmd+jYNVTpaXMgR/xICk=@vger.kernel.org X-Gm-Message-State: AOJu0YxbCDLKq6Z14CeWxcqOFsytHekBKZpBhvamjV17yNTYWOi8DOVf 09Ya3FdsHuaqZm44mb6Uq5F0C+iHSu8DXnqxOH9h9OKYqJIhfss8PT7RTk2LFy2ccon2YetlD1M o7w== X-Google-Smtp-Source: AGHT+IGKm7yq47izgr1FmkV+xWCQVhrzT+Qt86SXT8qqzZ7digIfsw6vfVxyVo72E4861BiCHpsXRIV5BzI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:e80f:b0:1fa:2ae7:cc6a with SMTP id d9443c01a7336-20c63722d7fmr648865ad.4.1728584768900; Thu, 10 Oct 2024 11:26:08 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:39 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-38-seanjc@google.com> Subject: [PATCH v13 37/85] KVM: x86/mmu: Add "mmu" prefix fault-in helpers to free up generic names From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Prefix x86's faultin_pfn helpers with "mmu" so that the mmu-less names can be used by common KVM for similar APIs. No functional change intended. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 19 ++++++++++--------- arch/x86/kvm/mmu/mmu_internal.h | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 28f2b842d6ca..e451e1b9a55a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4347,8 +4347,8 @@ static u8 kvm_max_private_mapping_level(struct kvm *k= vm, kvm_pfn_t pfn, return max_level; } =20 -static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, - struct kvm_page_fault *fault) +static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault) { int max_order, r; =20 @@ -4371,10 +4371,11 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu = *vcpu, return RET_PF_CONTINUE; } =20 -static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault = *fault) +static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault) { if (fault->is_private) - return kvm_faultin_pfn_private(vcpu, fault); + return kvm_mmu_faultin_pfn_private(vcpu, fault); =20 fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, fault->write, &fault->map_writable); @@ -4409,8 +4410,8 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault return RET_PF_CONTINUE; } =20 -static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *f= ault, - unsigned int access) +static int kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault, unsigned int access) { struct kvm_memory_slot *slot =3D fault->slot; int ret; @@ -4493,7 +4494,7 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, str= uct kvm_page_fault *fault, if (mmu_invalidate_retry_gfn_unsafe(vcpu->kvm, fault->mmu_seq, fault->gfn= )) return RET_PF_RETRY; =20 - ret =3D __kvm_faultin_pfn(vcpu, fault); + ret =3D __kvm_mmu_faultin_pfn(vcpu, fault); if (ret !=3D RET_PF_CONTINUE) return ret; =20 @@ -4570,7 +4571,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault if (r) return r; =20 - r =3D kvm_faultin_pfn(vcpu, fault, ACC_ALL); + r =3D kvm_mmu_faultin_pfn(vcpu, fault, ACC_ALL); if (r !=3D RET_PF_CONTINUE) return r; =20 @@ -4661,7 +4662,7 @@ static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vc= pu, if (r) return r; =20 - r =3D kvm_faultin_pfn(vcpu, fault, ACC_ALL); + r =3D kvm_mmu_faultin_pfn(vcpu, fault, ACC_ALL); if (r !=3D RET_PF_CONTINUE) return r; =20 diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 633aedec3c2e..59e600f6ff9d 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -235,7 +235,7 @@ struct kvm_page_fault { /* The memslot containing gfn. May be NULL. */ struct kvm_memory_slot *slot; =20 - /* Outputs of kvm_faultin_pfn. */ + /* Outputs of kvm_mmu_faultin_pfn(). */ unsigned long mmu_seq; kvm_pfn_t pfn; bool map_writable; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 143b7e9f26dc..9bd3d6f5db91 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -812,7 +812,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, str= uct kvm_page_fault *fault if (r) return r; =20 - r =3D kvm_faultin_pfn(vcpu, fault, walker.pte_access); + r =3D kvm_mmu_faultin_pfn(vcpu, fault, walker.pte_access); if (r !=3D RET_PF_CONTINUE) return r; =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 81D76201244 for ; Thu, 10 Oct 2024 18:26:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584774; cv=none; b=iMi63pUGzGAAHDxS9UCVhdJNWPSS+Jsu6nF1egtjPX5c44rEr1HwlyELXQ7lJOjPRqQynZ0VF8bfFcCpPXG0Ry+I0io2s8mXxxDNRxT5CG6krUAavGDWyUbvVf1JDL4UHB3gvLkejQnokdGh6bUkn+AEe2E0mai0SBxmyebatsw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584774; c=relaxed/simple; bh=t32hhiDzmz/KF55Q/3vtmKLh+T++3xGW8zC+Y7mhAxY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WeTrhex4v0u1gPanCcL15a9ihylm50Ht28eSfd/mKFg7V5zgQielucb0DlS53qbWAyeXu1496fs0HaUZqpdXyZHsyqmrnJIPrdf864zqfsjUm9mZwZW7jDQI8ESWq3opKb70vXufSqGIvU3Oo5IQ/RZZuioEsT8xNBviZjmgRFQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bMjYV7ju; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bMjYV7ju" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-70ac9630e3aso1195401a12.1 for ; Thu, 10 Oct 2024 11:26:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584772; x=1729189572; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=UQkfEcPm4BCGA2NCPxYbIcRTjXNaFQKnzNzKtIM0PSc=; b=bMjYV7juBeFlVMGvI7NvnOECGx9NNcc/zEtv4XL/auuP5Ka903KuEIjSTMzcC1Eugg j8rtVp7Wpeix1cw/WDIwsZIU0wTWT5U751Nxt1Ns0Xa2Fr7UkWo6hT9YrDpr7YpS6OTw LU+YXtX5CGLiB6ZjvIB4nH+NtxFny5aWZnAqymn5BW+gfWLaiXeYGy+NAT0NJ5UGu00k 4dsKzgGcgqUs6EjEUOnGq7lwY8emCm7AxNltyvDSO0iq23nJjbJoWCfuddHB0rSEfkzM JWAWaFN301LUh25d47GCwzrcPcxjw1ki4q6ZHEZ7nWT/H19p2Slj5JxKjRL74P3AOnEm sJ0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584772; x=1729189572; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=UQkfEcPm4BCGA2NCPxYbIcRTjXNaFQKnzNzKtIM0PSc=; b=dRV7KLNXus1xnebEPjh90vsuUjZawgotftTlRtlS6Vvka1AdNHt1isEpCLOMk5pA5N 9pHnPuRw1lfdRoO6yclh+jXfgVvxD7nUkBxRMaowyaW54jD0411nEhVsvMBFF5UO7LWZ ODlPXDx5YJQp6upPs5OL18fj9SVOQ0R6jR9IwGDbabEvKzo3OOSbGVp3mogbdDPXq9Sl sirthjIYH25TVVoc3FpxaBeATa5u55M3kd2Ek1+hn/BYVyWfK0t4VrjsxVOYNyKeB6ec 3PCHuMH0apm0A2fwCjjJyANhZQMZI57p+ARA9ytSDxsaa8PI7Ysfw9zqQKwBsrKeiRL9 akUg== X-Forwarded-Encrypted: i=1; AJvYcCXWXBQpbK0Lm9yncNXIVqb2whGKY58RwK7zZ8ThLmnOxVZctNcrGzUe/Rvg+3Ma5crGOdP+otBBOza3SHA=@vger.kernel.org X-Gm-Message-State: AOJu0YyS7xF5t3cTi5iIA9P8YjQauhRnIWRsYllTdvRNrtwaDuqnvcGf fqQg7rCxMnzFg8jcibJkif25RnQaAmgSrMI2hja1nFAb5BeyrCrO3Kq8iB4ocaLrCXXvorTbXs8 BVQ== X-Google-Smtp-Source: AGHT+IHq9hr54ORGaKAudrJMY0Hy9bRYjmUvRYz8zAm0RAj3x+bD9YD3f2uWGLgDblXeQYOnnP141duUJXA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:1d03:0:b0:7db:539:893c with SMTP id 41be03b00d2f7-7ea535a65c6mr17a12.9.1728584770873; Thu, 10 Oct 2024 11:26:10 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:40 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-39-seanjc@google.com> Subject: [PATCH v13 38/85] KVM: x86/mmu: Put direct prefetched pages via kvm_release_page_clean() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use kvm_release_page_clean() to put prefeteched pages instead of calling put_page() directly. This will allow de-duplicating the prefetch code between indirect and direct MMUs. Note, there's a small functional change as kvm_release_page_clean() marks the page/folio as accessed. While it's not strictly guaranteed that the guest will access the page, KVM won't intercept guest accesses, i.e. won't mark the page accessed if it _is_ accessed by the guest (unless A/D bits are disabled, but running without A/D bits is effectively limited to pre-HSW Intel CPUs). Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e451e1b9a55a..62924f95a398 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2965,7 +2965,7 @@ static int direct_pte_prefetch_many(struct kvm_vcpu *= vcpu, for (i =3D 0; i < ret; i++, gfn++, start++) { mmu_set_spte(vcpu, slot, start, access, gfn, page_to_pfn(pages[i]), NULL); - put_page(pages[i]); + kvm_release_page_clean(pages[i]); } =20 return 0; --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 561AD20125C for ; Thu, 10 Oct 2024 18:26:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584776; cv=none; b=Q8aGJsJvteJxf520ngApOiwJ+S0D1DWomVsBKJhB6k1wXQab17oyr2r0f6v2rPExPuhd6ZHQils6CD28aRyx1ghJHMi6q5bhLCgucMauU3TSEgZICNB5ozIsW4HbPwx7gCa/j44NKa7t+Pmez+dWpR6CBDyiIOEa/lRKgALVp4Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584776; c=relaxed/simple; bh=AMgsQWUhFHfCzgS24A9rVoaxkKJ7Z500dfTCiVjIUZQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oBRAKdSuNaSKVQXWxm8mwcYHYM53oaKzfzPWoMLI4g+A91NDvVb5oUFNLJ1Q4Dh/p491pSYlet2hrsvXHPlHFdo3A/d1KA9cnu6v5B5hg1XvB/NJJU8lAYcgvJ09Hz1TWVANrpFutrlQJ773yXleAOckbV0u+LF7hQFWT+jIFKU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uwFCrjqa; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uwFCrjqa" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-206da734c53so13139725ad.2 for ; Thu, 10 Oct 2024 11:26:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584774; x=1729189574; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=bLMllFEXz5NbCB79T5EDeXxXCOdT0E/lPN9LlBgR2FM=; b=uwFCrjqa7ghmvrDiWY1KS05snxhx368q6kuiBqFzKfDm/bDcUbsQJzABMEIxvC3s/D Slu5mrI9UMYuTQmhw0sA92WKhhiNVsMx9HKgola5QQUGUxYW6ooZJiq5CZ2pOxm9TWY3 Jpfh0H5DOQibZ6lLTjBLWEPcYgkVnXuwvNL/oqgFZ6aLSk/iPMXxJP8vFMUY4hwuYBQE OE1YgF94rIC52TMb4mM+yjZUo/IKJ+IQVB3ZvxV8RZ87he18StRjX489yhFk4pPyaGq+ 1nUyzh9EOKMTJ/MxSSmXIoFRbwC6Hq7zQw2bNbNqevOKWkDvKcgjZgEqXKxi0/s2g2wa tuXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584774; x=1729189574; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=bLMllFEXz5NbCB79T5EDeXxXCOdT0E/lPN9LlBgR2FM=; b=iiKF7bl3RH9Ez0SUFbVSPGZeNmnmF2xSEU/cwOR+cZuk1MKCbEGmw/SpSwu2UJdmrx n7skd0exZKqcEZhgEZzwMvDlQqS/r9zVkwMzdi3l5iOVa17MvrQb7LC7qyJpdxn+/ciZ np5Un32NTpTCS6SpgrcCXzOGan5xpxZC++J+/mVhlay/J9VvyMRo1QfCMrOehu4wnFC+ IZoEhB8jdH3yHvYCrVFreXUEULVlOIQ+S8MJIuPjr3mprDGxC5vTqDukPT7rFY9NXxKb B00uc/XGJ6GhdWVAkwcOvw6iIgmjeZkGdsA8Ntsx369+eJw63BhLpex6dO14ceAGkMgS gObQ== X-Forwarded-Encrypted: i=1; AJvYcCVB1PfH9NVFz44Bc5YYstgfsZe7gnINDVOP7zuaKQ4nl4TGwWXvMUwVhcMF81KboCkCwy5Rjt1GyhNlVgA=@vger.kernel.org X-Gm-Message-State: AOJu0YxTK4BadX2JId4UyeFcjAtNN839ZeOnFU4BXIh5swriplUZMg3P JtD4Fbm/4Ob1hHGz8SSRZNKPblcVbFkHh9YbPOZDUTSKC+LbUNF75SLtrW7OTXH5zwMZO3WqS0a wwQ== X-Google-Smtp-Source: AGHT+IHTeV3dFyfdACUyzit3PmgjtoD4THwZ77SaAB8Cv7KtK6sEvfc4IjBdPT+nDHGdhMcMXcKC7R1qaBw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:1c8:b0:20c:716c:5af with SMTP id d9443c01a7336-20c716c078dmr946605ad.3.1728584773496; Thu, 10 Oct 2024 11:26:13 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:41 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-40-seanjc@google.com> Subject: [PATCH v13 39/85] KVM: x86/mmu: Add common helper to handle prefetching SPTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Deduplicate the prefetching code for indirect and direct MMUs. The core logic is the same, the only difference is that indirect MMUs need to prefetch SPTEs one-at-a-time, as contiguous guest virtual addresses aren't guaranteed to yield contiguous guest physical addresses. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 40 +++++++++++++++++++++------------- arch/x86/kvm/mmu/paging_tmpl.h | 13 +---------- 2 files changed, 26 insertions(+), 27 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 62924f95a398..65d3a602eb2c 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2943,32 +2943,41 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, stru= ct kvm_memory_slot *slot, return ret; } =20 -static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu, - struct kvm_mmu_page *sp, - u64 *start, u64 *end) +static bool kvm_mmu_prefetch_sptes(struct kvm_vcpu *vcpu, gfn_t gfn, u64 *= sptep, + int nr_pages, unsigned int access) { struct page *pages[PTE_PREFETCH_NUM]; struct kvm_memory_slot *slot; - unsigned int access =3D sp->role.access; - int i, ret; - gfn_t gfn; + int i; + + if (WARN_ON_ONCE(nr_pages > PTE_PREFETCH_NUM)) + return false; =20 - gfn =3D kvm_mmu_page_get_gfn(sp, spte_index(start)); slot =3D gfn_to_memslot_dirty_bitmap(vcpu, gfn, access & ACC_WRITE_MASK); if (!slot) - return -1; + return false; =20 - ret =3D kvm_prefetch_pages(slot, gfn, pages, end - start); - if (ret <=3D 0) - return -1; + nr_pages =3D kvm_prefetch_pages(slot, gfn, pages, nr_pages); + if (nr_pages <=3D 0) + return false; =20 - for (i =3D 0; i < ret; i++, gfn++, start++) { - mmu_set_spte(vcpu, slot, start, access, gfn, + for (i =3D 0; i < nr_pages; i++, gfn++, sptep++) { + mmu_set_spte(vcpu, slot, sptep, access, gfn, page_to_pfn(pages[i]), NULL); kvm_release_page_clean(pages[i]); } =20 - return 0; + return true; +} + +static bool direct_pte_prefetch_many(struct kvm_vcpu *vcpu, + struct kvm_mmu_page *sp, + u64 *start, u64 *end) +{ + gfn_t gfn =3D kvm_mmu_page_get_gfn(sp, spte_index(start)); + unsigned int access =3D sp->role.access; + + return kvm_mmu_prefetch_sptes(vcpu, gfn, start, end - start, access); } =20 static void __direct_pte_prefetch(struct kvm_vcpu *vcpu, @@ -2986,8 +2995,9 @@ static void __direct_pte_prefetch(struct kvm_vcpu *vc= pu, if (is_shadow_present_pte(*spte) || spte =3D=3D sptep) { if (!start) continue; - if (direct_pte_prefetch_many(vcpu, sp, start, spte) < 0) + if (!direct_pte_prefetch_many(vcpu, sp, start, spte)) return; + start =3D NULL; } else if (!start) start =3D spte; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 9bd3d6f5db91..a476a5428017 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -533,9 +533,7 @@ static bool FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, u64 *spte, pt_element_t gpte) { - struct kvm_memory_slot *slot; unsigned pte_access; - struct page *page; gfn_t gfn; =20 if (FNAME(prefetch_invalid_gpte)(vcpu, sp, spte, gpte)) @@ -545,16 +543,7 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm= _mmu_page *sp, pte_access =3D sp->role.access & FNAME(gpte_access)(gpte); FNAME(protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte); =20 - slot =3D gfn_to_memslot_dirty_bitmap(vcpu, gfn, pte_access & ACC_WRITE_MA= SK); - if (!slot) - return false; - - if (kvm_prefetch_pages(slot, gfn, &page, 1) !=3D 1) - return false; - - mmu_set_spte(vcpu, slot, spte, pte_access, gfn, page_to_pfn(page), NULL); - kvm_release_page_clean(page); - return true; + return kvm_mmu_prefetch_sptes(vcpu, gfn, spte, 1, pte_access); } =20 static bool FNAME(gpte_changed)(struct kvm_vcpu *vcpu, --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71457202F6C for ; Thu, 10 Oct 2024 18:26:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584778; cv=none; b=IStNzfkzbiuI7j9g5MYdeSFoC1JrUTAZ+kOh1EM/TyNoL0aMQJMJynKVe0QYMhgeTVqGOKkpGWBSEieB3Ef7oPh1Zt/CUCg+GwDR0qBdmSpwVAk5/dGlhQbVaI4ymfXdPdX8ytE6M6jG1U+xQtirpQ/sqb5W9LEXMZL1GNAo0Fc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584778; c=relaxed/simple; bh=PQXxus6kr8nuokmnnM+Lbqq2T+wYMcW3c8L/2YrIVXU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=u7Vfij+3c9ZzJ6uFGV9C5oGn2hPDmzlqIrdjhLUdbJ9QirBEtaAe7J6SxZy0cKaEdX84xAcJamjKsNjr8CjSSgAvlO6YzWhgFd27hHVWLdiPDzJwu1bmol0m4bKfa7rBYRHbIasttUY4ym7Czg6mJ5d9GhghE/ZMmaG415wS3/8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=gciGGDh5; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gciGGDh5" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71dff575924so1543258b3a.1 for ; Thu, 10 Oct 2024 11:26:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584776; x=1729189576; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=GAJaAoo95YXsHy4joQlb4gblEl3cKLpcYsG4EH0XckI=; b=gciGGDh5HFpYWLuKNpe3Ev4O1droYOafPNHG8yohFPMnZzjhRji78ghkqI9VqZhT+B Juf6rcXU/vrIP8E/Z1uAY9snG3nfyrtMdkveyBqYvx2MzjD4hY+IMWPMKq+/pXqVwVGX ea0byuwg6VXoi/3PVusUXLHHDb//D85kFO9B+vfoit+AWNu+8Ifoj/K1gWHZ1ayEP7bM 5W/mtnUCiPZ18zjIWT0zKiLM5FJtZBfhithRFXzW4f9XDSnPiQ81VGioPPD2PpwuXPhl XqHt3I/ZMv4OFkktHewNKir892D4AGcVInKoLbwo/PslHwbVt4L20OtMF8zjtlhzbESm QwcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584776; x=1729189576; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=GAJaAoo95YXsHy4joQlb4gblEl3cKLpcYsG4EH0XckI=; b=HGmfj5zMbyQ6vUa6+zy6gFZ6E77O1tjKesLXhJJm8ppDCVOTnygWJJzAD2T1l6NboF Cnt409XQbOAmnKdU8h+L8q5WNTaTt/uhbwNL9fyJdPhW8htigF4dRoShxpOagptLfW7M fdU+Uk5Cc5rgpWt7jNh/JWGq36lNvmK/ABbTZgNlWYaAlRuJAoAIKy+mLFPBHb6aWxOO gjy7EkKp6ZMCxG+5uF0SeR78+fnFFIkzTXQZA3Fh0Mom8abHQJZnQKOZd7oBNIOa9Kxy FbALhTC95PXrzrKDIogF4BiGtHooLb6IH2BBZqiamSmBTGf0qUMDN9+/Gww4yWJHA6FY kdXg== X-Forwarded-Encrypted: i=1; AJvYcCWNQ4t/ODOnM2ZWfCLaxEuRhJWIPH90KbAwfoWPu5sOxYbEqc5hU6sxQoxDSoV0dH3wRLI3h+ZZQeFkFeo=@vger.kernel.org X-Gm-Message-State: AOJu0Yzlx69XYW17IHKWAAQBY229mFx5CZB8bORCtqVFBZGr8nvz8pmS 92HkjpsnGUN9rhlBLU7Zz14ZzxVTOhhKAM9M9PmbBOS/qHzZKSuPEYTczTnMik/OkubEhrJ2Bb6 BSA== X-Google-Smtp-Source: AGHT+IEXxr/4vsCtgUsUHHkctm5xGRqb6tsNipBtkL7vgMjj75AzVZauqGsQqnffIf336Hhi8e9e6vZrhMQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:9184:0:b0:71d:f7c9:8cb3 with SMTP id d2e1a72fcca58-71e1dbf1ab5mr7792b3a.5.1728584775458; Thu, 10 Oct 2024 11:26:15 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:42 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-41-seanjc@google.com> Subject: [PATCH v13 40/85] KVM: x86/mmu: Add helper to "finish" handling a guest page fault From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a helper to finish/complete the handling of a guest page, e.g. to mark the pages accessed and put any held references. In the near future, this will allow improving the logic without having to copy+paste changes into all page fault paths. And in the less near future, will allow sharing the "finish" API across all architectures. No functional change intended. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 12 +++++++++--- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 65d3a602eb2c..31a6ae41a6f4 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4357,6 +4357,12 @@ static u8 kvm_max_private_mapping_level(struct kvm *= kvm, kvm_pfn_t pfn, return max_level; } =20 +static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault, int r) +{ + kvm_release_pfn_clean(fault->pfn); +} + static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { @@ -4522,7 +4528,7 @@ static int kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, * mmu_lock is acquired. */ if (mmu_invalidate_retry_gfn_unsafe(vcpu->kvm, fault->mmu_seq, fault->gfn= )) { - kvm_release_pfn_clean(fault->pfn); + kvm_mmu_finish_page_fault(vcpu, fault, RET_PF_RETRY); return RET_PF_RETRY; } =20 @@ -4598,8 +4604,8 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, s= truct kvm_page_fault *fault r =3D direct_map(vcpu, fault); =20 out_unlock: + kvm_mmu_finish_page_fault(vcpu, fault, r); write_unlock(&vcpu->kvm->mmu_lock); - kvm_release_pfn_clean(fault->pfn); return r; } =20 @@ -4685,8 +4691,8 @@ static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vc= pu, r =3D kvm_tdp_mmu_map(vcpu, fault); =20 out_unlock: + kvm_mmu_finish_page_fault(vcpu, fault, r); read_unlock(&vcpu->kvm->mmu_lock); - kvm_release_pfn_clean(fault->pfn); return r; } #endif diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index a476a5428017..35d0c3f1a789 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -836,8 +836,8 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, str= uct kvm_page_fault *fault r =3D FNAME(fetch)(vcpu, fault, &walker); =20 out_unlock: + kvm_mmu_finish_page_fault(vcpu, fault, r); write_unlock(&vcpu->kvm->mmu_lock); - kvm_release_pfn_clean(fault->pfn); return r; } =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76CCD202F92 for ; Thu, 10 Oct 2024 18:26:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584780; cv=none; b=K/yfIQwWM3JVsaGxbRMbzfH9B1rWZrx95PG4FjaWmu1saNztFuSloGVsRxxQ1z6RpqaUqXkfRNqOMt0tgenUNmzgmQbO0bu3jc0Vtb8B7vhinA+7TmxeO0jGXFzQsqkJTm/Rb2A6oCaY/iRfLRa7IrN3BzTkKSBfRAb6jWqlA4M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584780; c=relaxed/simple; bh=wP8WMCipPwqkHPMKwQ03EbdALKWDvH1se8OSiGfXvd8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=JdupKfT9pr+PN5FMZdrO2+aQ1HXmj+WxN8Eyg9VW0ftugJFHkboWopyviMKcLHvulFYwjhuD5cVMdFbioTC+7NrzhWbWlA5saZBlqNKH86nN29YUcJWBIr3PnGB820fV2DI2nQ1+GaRfgSkZkvRtT5A+SHFvb2Ie4AYdPAl1YDo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=NPuFqHnM; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="NPuFqHnM" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-20b6144cc2aso12174605ad.0 for ; Thu, 10 Oct 2024 11:26:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584778; x=1729189578; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=sgK+YcjRpEQ+YhbfYv/o4akq/yPA2qNOXCMP41gc9EY=; b=NPuFqHnMcNqMW7akh4WWzd8zhX+rRXRzBI7zESBDBZBdwT0VEGK+uUaNUj3twhjoaK rcZEJ1pzRTQnu0rv1PaSI4+ObWewHv+EBTozp88TZjRIVplFldJLM+UkcY09Q1ua6RkR Dw9nQw0P48Ua8VAOW6e8QPalxUz5psROQe6Q4WAKd1T6oYcKBfrDyxTj1cLgmmtJXDMK f6TXqyy3Ji++fL8mOKm9O78UFQjHLpTNhmKisvw5USEpHdm6PhXanNcXX1Nq8NMsY8Gf nPwcM1Aql+6FmfWqD78E6aTWg5BuzTjuo501MWC6u3CV3+dNVtlksmD4K27Kqv/V+XkD gs3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584778; x=1729189578; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=sgK+YcjRpEQ+YhbfYv/o4akq/yPA2qNOXCMP41gc9EY=; b=Ya/yjYs/pOHCPRM84LXgfdIA5HSG1Lx9DxdJF0EhXrAW3p4kp5XkqV2Ffylg4nQNZf MBYhkzsU+ltjgikmlRxBh6+1tzCi31xUV1DVL5FQduxUpmiL0QXGkal7Go3pizOw0VRM hkUJxWu/RCIsoc2FpbLRd4EEY795XlTU+0+acfuk9TM3R4Zd1lnxKbuaaZMLSHM/30NI 9f6UNWLCKYvLjiS1HAKM/EV1sH0t5x3gHnHVVHPcdgp+uui1jV27hHjrjcTvQXcQKJ4s r5P0unYpNmtWMmQ1wQLfhB2okxbBP9GAHPlpdWkZbY0XsqbnyXEFbe5NwOAlOQQugwwo GE6w== X-Forwarded-Encrypted: i=1; AJvYcCW2PURUrgqyadyOcI76E5iRdBmOIGhGkL7emHoBTdGaNNB+J75D6J7UYP/M7FVGqm9bBz7vb4AVopUsmsA=@vger.kernel.org X-Gm-Message-State: AOJu0Yzy1LxUTEJ2RN/p+qpsZkv9se8B3lF+FJ7cF/88q1KsKgiBWMDt nnU58qqeYYrT6w7OjcumcJmAb3TWGGGGukxaUs3MoAGxbBYXkl4IQNLuR4cNL8gO6bmHI37A6ls xDA== X-Google-Smtp-Source: AGHT+IHgES02wxBmUBuw4MXImaMxOgBvLS6HzbS699NHinso/VI3SIlimd+hdhb5F/AgwxmTSyUtUzYKtBo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:ec85:b0:20b:7bfa:ac0f with SMTP id d9443c01a7336-20ca037f212mr485ad.1.1728584777475; Thu, 10 Oct 2024 11:26:17 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:43 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-42-seanjc@google.com> Subject: [PATCH v13 41/85] KVM: x86/mmu: Mark pages/folios dirty at the origin of make_spte() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the marking of folios dirty from make_spte() out to its callers, which have access to the _struct page_, not just the underlying pfn. Once all architectures follow suit, this will allow removing KVM's ugly hack where KVM elevates the refcount of VM_MIXEDMAP pfns that happen to be struct page memory. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 30 ++++++++++++++++++++++++++++-- arch/x86/kvm/mmu/paging_tmpl.h | 5 +++++ arch/x86/kvm/mmu/spte.c | 11 ----------- 3 files changed, 33 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 31a6ae41a6f4..f730870887dd 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2964,7 +2964,17 @@ static bool kvm_mmu_prefetch_sptes(struct kvm_vcpu *= vcpu, gfn_t gfn, u64 *sptep, for (i =3D 0; i < nr_pages; i++, gfn++, sptep++) { mmu_set_spte(vcpu, slot, sptep, access, gfn, page_to_pfn(pages[i]), NULL); - kvm_release_page_clean(pages[i]); + + /* + * KVM always prefetches writable pages from the primary MMU, + * and KVM can make its SPTE writable in the fast page handler, + * without notifying the primary MMU. Mark pages/folios dirty + * now to ensure file data is written back if it ends up being + * written by the guest. Because KVM's prefetching GUPs + * writable PTEs, the probability of unnecessary writeback is + * extremely low. + */ + kvm_release_page_dirty(pages[i]); } =20 return true; @@ -4360,7 +4370,23 @@ static u8 kvm_max_private_mapping_level(struct kvm *= kvm, kvm_pfn_t pfn, static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, int r) { - kvm_release_pfn_clean(fault->pfn); + lockdep_assert_once(lockdep_is_held(&vcpu->kvm->mmu_lock) || + r =3D=3D RET_PF_RETRY); + + /* + * If the page that KVM got from the *primary MMU* is writable, and KVM + * installed or reused a SPTE, mark the page/folio dirty. Note, this + * may mark a folio dirty even if KVM created a read-only SPTE, e.g. if + * the GFN is write-protected. Folios can't be safely marked dirty + * outside of mmu_lock as doing so could race with writeback on the + * folio. As a result, KVM can't mark folios dirty in the fast page + * fault handler, and so KVM must (somewhat) speculatively mark the + * folio dirty if KVM could locklessly make the SPTE writable. + */ + if (!fault->map_writable || r =3D=3D RET_PF_RETRY) + kvm_release_pfn_clean(fault->pfn); + else + kvm_release_pfn_dirty(fault->pfn); } =20 static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 35d0c3f1a789..f4711674c47b 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -954,6 +954,11 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, str= uct kvm_mmu_page *sp, int spte_to_pfn(spte), spte, true, true, host_writable, &spte); =20 + /* + * There is no need to mark the pfn dirty, as the new protections must + * be a subset of the old protections, i.e. synchronizing a SPTE cannot + * change the SPTE from read-only to writable. + */ return mmu_spte_update(sptep, spte); } =20 diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 8e8d6ee79c8b..f1a50a78badb 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -277,17 +277,6 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_p= age *sp, mark_page_dirty_in_slot(vcpu->kvm, slot, gfn); } =20 - /* - * If the page that KVM got from the primary MMU is writable, i.e. if - * it's host-writable, mark the page/folio dirty. As alluded to above, - * folios can't be safely marked dirty in the fast page fault handler, - * and so KVM must (somewhat) speculatively mark the folio dirty even - * though it isn't guaranteed to be written as KVM won't mark the folio - * dirty if/when the SPTE is made writable. - */ - if (host_writable) - kvm_set_pfn_dirty(pfn); - *new_spte =3D spte; return wrprot; } --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E9D3202F90 for ; Thu, 10 Oct 2024 18:26:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584781; cv=none; b=KvjFq8AaSdnmUtc73ut438tDHZItBZj3HPzR8fcNiRoo+R6NBrLgDFqQKiWu1ugpJHlrBBvOAWWxirx53fKA05ieIYOgV1Q90XKuIDkFxUVQN/h8BWja7JlcOHTNTM3ejzAcycjLRQpK5JgGMq0NnpQq9MAbivrny8Q9oN+D6iA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584781; c=relaxed/simple; bh=hSHnz2G1ntKemXewFOwXdDJXjhCTKhhhRrw1wSIGnaU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=tdECsv9TR+eIbSUP0quhFc4wNCipFoik5z5yETN21stBCoMVTnwkqQPlXRPGhyJNSOi2hKWjbolAtsFPbIQQ6DXhVLBEhoEIrMIRVVvkB+TzZh7fKqJWV9uNp4BIujLtRpK4VakJJ+iJPMc79Ny4NfRnGLW71RtofSRG6kb1bOc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=E53iqmq8; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="E53iqmq8" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71e1e989aa2so1655349b3a.3 for ; Thu, 10 Oct 2024 11:26:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584780; x=1729189580; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=8DwvYXI8CmE5CO0Ycvk+EgdgF52vTMi74FFh8TONwA8=; b=E53iqmq8I7HHufRa9Nfa4QpSUD3mPUf0HdiC+gr3+ZJVA0PtJRMGFCy2RlpuApodID MHkYZafORHG4Fo5pdlmIa2Zu49fY42Dn8Mwbg8BEIZ/Pa8OJQiqqaoX3cE0NBkDUycJi 8knjazonwqf9LvKUPFmBk2crmB0MGGb2WbPM1i7pjkVZjFgAzrwTCr7oBWE4eumlOw58 NDuf4DR/sTZJYm1oq12ic4ny/HL0dHq+Txgeh165RAyYFWZt+w+um9fwU7jF7aG4NAnz uIpRYBDmcMn403NAN6nAehV9jbOFJtSN+/wsSbKYsr5xfV60UWvUZH56iYMQo+BvtQRs hZkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584780; x=1729189580; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=8DwvYXI8CmE5CO0Ycvk+EgdgF52vTMi74FFh8TONwA8=; b=aIskNM9luHqe8fcq3BHxh66Y0vqbpDaJbdHKJnzGfTOi9+iN9cQXnwQBiNPDegJVGm spJ0KdMAloWyDd6rmDsmLdp1+vaIXDGTDDeoUtaf580CdPE+1QTRHjGSJBx+LR3eQZm3 Fl/PmIWWAJBWKSVZJp6U+/7/lwUWdiOeP81UYqE6FG4O7aIHlX0dryeKsa6BcDu+qCpF QttJnPC52ikRBE4yLAkw3Zq2N8ZTneCWlgelxBxHXOyu+MMdPRHmfNCwCcvW2DGulDfY 3UZ9I1WKcOkqpNYmtmHTBpVTGGbUdi0s7y+JQlHZMs47zowLBNZawqKPELIpWlG/PvVo WgZg== X-Forwarded-Encrypted: i=1; AJvYcCUBqWVfw7icTiw5tG5lwP9sNRLfvEjIBOK+Yf1qCjla2pLFadSHc3FyeNT5tGBGr+OxA+D4ZaniEkTkc2w=@vger.kernel.org X-Gm-Message-State: AOJu0YwrNEkHMdEn/chsB38B+HL/wx4eq6/+1DJFR92sqrO8R2KmJzlf i8Q6kxHMPPuHlC9WgDjWis9AcduPyuxSupiSUIF9dQteJ5flJMFqXipA6ryD3SmIKn3Iz9v1UYF EJg== X-Google-Smtp-Source: AGHT+IG3pPvsekcqYnotExeZVZUXEYlJLq+6e3a1eutYJX4gtVKcKQbhJKMNrxmpaRIGMBxtyW/kZ/ByL14= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:91a3:b0:71d:fb06:e79b with SMTP id d2e1a72fcca58-71e37c53abamr28b3a.0.1728584779496; Thu, 10 Oct 2024 11:26:19 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:44 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-43-seanjc@google.com> Subject: [PATCH v13 42/85] KVM: Move declarations of memslot accessors up in kvm_host.h From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move the memslot lookup helpers further up in kvm_host.h so that they can be used by inlined "to pfn" wrappers. No functional change intended. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- include/linux/kvm_host.h | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 9263375d0362..346bfef14e5a 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1168,6 +1168,10 @@ static inline bool kvm_memslot_iter_is_valid(struct = kvm_memslot_iter *iter, gfn_ kvm_memslot_iter_is_valid(iter, end); \ kvm_memslot_iter_next(iter)) =20 +struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn); +struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); +struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn= _t gfn); + /* * KVM_SET_USER_MEMORY_REGION ioctl allows the following operations: * - create a new memory slot @@ -1303,15 +1307,13 @@ int kvm_gfn_to_hva_cache_init(struct kvm *kvm, stru= ct gfn_to_hva_cache *ghc, }) =20 int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len); -struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn); bool kvm_is_visible_gfn(struct kvm *kvm, gfn_t gfn); bool kvm_vcpu_is_visible_gfn(struct kvm_vcpu *vcpu, gfn_t gfn); unsigned long kvm_host_page_size(struct kvm_vcpu *vcpu, gfn_t gfn); void mark_page_dirty_in_slot(struct kvm *kvm, const struct kvm_memory_slot= *memslot, gfn_t gfn); void mark_page_dirty(struct kvm *kvm, gfn_t gfn); =20 -struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); -struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn= _t gfn); + kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); =20 int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *= map, --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A9C7204086 for ; Thu, 10 Oct 2024 18:26:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584784; cv=none; b=boIIfIQnQOIM/bEqTucyM9vgFq0UK7wfLCZkvTbGdwFpdx+oxY+1TSrkBqM34CwQs1cEghT7jWnV6JCcxdw8Cc9boMuQj8oPlTzXXg6r0y43BgD5Hs57y36W3uCmES9x0PiB9kvByeRs5seXAxpzCdhIhHFFB2IUJwwSkrg4EQI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584784; c=relaxed/simple; bh=acp8IhS6KrAOGkkRIhpxPxZ31aINXkAYcQRqekRRiYM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=APjL5vScxqGjfRMwPg4XVpvbZu36gFvs5SePFrl93p3gg8Is4k3LJJZptGbMHiKg+UU12Hf2hx/C1dqHRqnrfhVrjW6b/EGYLbfRja8pAga7Z2jwkpZzk2l2v3VwURP2xP2TrQdfOrc3HXQXcCSidIxRAIFKZh7qUCxpDUFB43s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Xqt2Fs5n; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Xqt2Fs5n" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e29b4f8837so18944137b3.0 for ; Thu, 10 Oct 2024 11:26:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584782; x=1729189582; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=CHy2j9VNwFKd5G/Ubc3O4QkXvNoIm5ox3M6XZ4oswcY=; b=Xqt2Fs5nbw3a9rY6oVLT35/D0l+G/AD+Q49HX1QdXMfC2SfNz4O5O1dvtdhGzDWUDl JpqTDsMCsN6OxFww5+NH7ydMQnMFtfrmh34ZhDsUCf/uTkDGYmH6VlOYAcds86k0XnLu dP0P0MTgvn/Qf9H0M5XOt0K7URLrCKcwSGSrKXhaF7fEgRm8tzA0j54XzIeR4IkT7Ji5 AoZrpW3iiSMjwhmWTfZ64gNRi5aT38cY6iP8t9xd8k0HMzsVYBTUhBM1elfqfp5jqfzO ADF/8T0Dad0vzNIZJhArpIvd/mEd9y0HE3XU4hpM3jp77eHkpOjUClLqF4+0clJvSyS9 WkzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584782; x=1729189582; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=CHy2j9VNwFKd5G/Ubc3O4QkXvNoIm5ox3M6XZ4oswcY=; b=il+rzKFo6I1oklxehqWblzrHmn9hURUlR+YZOrbB1a8yzXrDJjcTVkdkQePf0eefVr 7apiqNbL9MfBhZ26uiH2DTFemtw5IoTIwa5Rmj+Tunh/pFzyRyNN84NPfxDRQk3Fzjr7 i8at3DKnnp0cKMnIsY7aLYftzWLD8x1jN98u2QHXSuFXFToTCU3day2au2pQsW7lf6ou jfSIHD+2K/GfLr1Z0AdZBNAeQ6qhz1x4upVsMdx9bAVjpf5N6gqYanJDwFt4iJlWOFs+ b+zOOHYmNp016RrOgReGp+G0gLv6yk1IaXpIGdbu+oKMvQhNXuC5kP/JmWVTY8Oqbx/N Wn2g== X-Forwarded-Encrypted: i=1; AJvYcCWJPkTlfW33XBKQN+OSSL7XG4IFmYImSLZayQfOtu16vDOBi5/wZ4L6bNQ0SoX8AbzNGDo4qEKpb5sHSEo=@vger.kernel.org X-Gm-Message-State: AOJu0Yw1BR8dKr0z8SNOg0k4+OrPhjE2yvucU3JwHkKo8cVPgbffoA31 XeAamjEixkJYCZvLhqj2/ikztUFmkFf+mBJJtrLaDx1y+/p89q62XDqizJUTXd8U6wyCg46vuKD xaw== X-Google-Smtp-Source: AGHT+IEBXjEpYcXfPYRm3Nf4G9ZN1ucR55aqjscHr2p9QPsSrcRZYI0185ucK64AXquh+1QVcruWQv+kNDM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:5605:b0:6be:523:af53 with SMTP id 00721157ae682-6e32f2f4e28mr919077b3.3.1728584781575; Thu, 10 Oct 2024 11:26:21 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:45 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-44-seanjc@google.com> Subject: [PATCH v13 43/85] KVM: Add kvm_faultin_pfn() to specifically service guest page faults From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a new dedicated API, kvm_faultin_pfn(), for servicing guest page faults, i.e. for getting pages/pfns that will be mapped into the guest via an mmu_notifier-protected KVM MMU. Keep struct kvm_follow_pfn buried in internal code, as having __kvm_faultin_pfn() take "out" params is actually cleaner for several architectures, e.g. it allows the caller to have its own "page fault" structure without having to marshal data to/from kvm_follow_pfn. Long term, common KVM would ideally provide a kvm_page_fault structure, a la x86's struct of the same name. But all architectures need to be converted to a common API before that can happen. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- include/linux/kvm_host.h | 12 ++++++++++++ virt/kvm/kvm_main.c | 22 ++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 346bfef14e5a..3b9afb40e935 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1231,6 +1231,18 @@ static inline void kvm_release_page_unused(struct pa= ge *page) void kvm_release_page_clean(struct page *page); void kvm_release_page_dirty(struct page *page); =20 +kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, + unsigned int foll, bool *writable, + struct page **refcounted_page); + +static inline kvm_pfn_t kvm_faultin_pfn(struct kvm_vcpu *vcpu, gfn_t gfn, + bool write, bool *writable, + struct page **refcounted_page) +{ + return __kvm_faultin_pfn(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn, + write ? FOLL_WRITE : 0, writable, refcounted_page); +} + kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 6dcb4f0eed3e..696d5e429b3e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3098,6 +3098,28 @@ kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu,= gfn_t gfn) } EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); =20 +kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, + unsigned int foll, bool *writable, + struct page **refcounted_page) +{ + struct kvm_follow_pfn kfp =3D { + .slot =3D slot, + .gfn =3D gfn, + .flags =3D foll, + .map_writable =3D writable, + .refcounted_page =3D refcounted_page, + }; + + if (WARN_ON_ONCE(!writable || !refcounted_page)) + return KVM_PFN_ERR_FAULT; + + *writable =3D false; + *refcounted_page =3D NULL; + + return kvm_follow_pfn(&kfp); +} +EXPORT_SYMBOL_GPL(__kvm_faultin_pfn); + int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, struct page **pages, int nr_pages) { --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 297D42038B7 for ; Thu, 10 Oct 2024 18:26:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584786; cv=none; b=RGS5S0zRFrBO4bzMLZsguTGV7HzBrNE2RzwseyOjMWBfwUqHm8dD8O5kgFv0jC1hs2eMWBPLcDF5SbhUme4S/FtgRJhOAif4FL+2H733q/zTNykTiC68XpOhArfttgd0i0ObN6m2nWy/IpTVKnbPcb7UmlhxoFeeWYmx3MW53GY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584786; c=relaxed/simple; bh=Hkx6OqV7mFxqog7S2nXIkbHpJQToxM9/kCtdODEngY8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=mU7mqpW7NJGu9cr6Cr+DwdRO/3bNH5PvUGucC2ofmc+2fpo2ovftisgIlahOAnW9adRYC4054oHBJUb5pz5gJb+JsDWCV8EfkahhsqsxsXvWDMFf7don78sRMM89dLTwTHhejKa3EF4SKdcDUnWbx/1I+nugSA83Kni2d+sdTdA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bQ8YWjsA; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bQ8YWjsA" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-6818fa37eecso1273911a12.1 for ; Thu, 10 Oct 2024 11:26:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584784; x=1729189584; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=O48RmE1hmLPs0qNEdPIh+k/rpCPu+e8A6GC4+tMXVMw=; b=bQ8YWjsAmSCtND2aP1+tnixLSReqcEMO5NgW5XaAIrBbL2LakVMBpTYBr79a6Ie4it yNo7Kq1xSA5MpJ+3LWpNyQ8FebCsfQePx6QL7MRRwa7HUCZ4EAISOfHABrJmf07r19dm vC7F1QJ2GzmgWC7lF+MFU+KbFzyvgv1xoyoqQgeGA7foloGcAJt5EuXt6aPoIbgmFg8d S+O2XvahdLb6BSZR7nlHR5Hzv2cmwMq+KpUVit9qdBBsIoaQJtbbi96Ot04RWV1vcYCh XiFcWmxVFgOdE1nBSgaaQ0SHKFiqMCAzj5V7S3qW2P8xyztxxmcbI6IW8VfCE4tAXTJs qhzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584784; x=1729189584; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=O48RmE1hmLPs0qNEdPIh+k/rpCPu+e8A6GC4+tMXVMw=; b=ZMg9Hy4S/wSbGR6A2xpszDsKu7sVPg9YLB1DrOr9BeexyGRPkH6QIn1nn6CWkMvFIP ZZPFvlyi8+/d4uFrGWnAoKD1x5pqgK3TKn9aPUf+QhpwA2JCzvMAAOuw+yOo4scvyVUh zuQ+7bRykYIO0BoAc9O9Hw8Fa1Ya6HnZ6dq/coMCmCjSz81j2Z7DxMsN6HNWRA2+0Ytk lRHS5wdDEYaqa0745lKOZlHTAfnPE3/KdePVebt7/zSDyHtLxusK+OlDdJssG0ik2Cwj uS8/XlIPh8oEtZiYSxP81QeNfhwKN8Gjnu4R8Q0nGpG0Rku4UOSzJV5OaVfJwrNzQjwK A45w== X-Forwarded-Encrypted: i=1; AJvYcCWoTWsyhvz/P1JlRIqToA+wVtA7VFlhXLjYRU/ynHhmmDBA5FIYv5XnbmRZqyp69+K4/C6Bza6GRKiee/4=@vger.kernel.org X-Gm-Message-State: AOJu0YwUGSc6h0I/XH8VdNJEEjVmXGCNTWUY8EH9xF2HJliiVit11/xR poG/vsqz+ZaSJPYzbLxhiG6UZBJI70zKWt9xMROGcNwvmkwiJ2H1AaQnWjTMOC7sjlveOBcHoE8 ryA== X-Google-Smtp-Source: AGHT+IGY3J0AFIsOlJbVDrqn6majLeEUOkcp3LAakwsbgSwDXT3SnAGgA9L20idMLdLGQvBeoRsHZDJxo50= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:2445:0:b0:75d:16f9:c075 with SMTP id 41be03b00d2f7-7ea5359cd2amr19a12.9.1728584783356; Thu, 10 Oct 2024 11:26:23 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:46 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-45-seanjc@google.com> Subject: [PATCH v13 44/85] KVM: x86/mmu: Convert page fault paths to kvm_faultin_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert KVM x86 to use the recently introduced __kvm_faultin_pfn(). Opportunstically capture the refcounted_page grabbed by KVM for use in future changes. No functional change intended. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 14 ++++++++++---- arch/x86/kvm/mmu/mmu_internal.h | 1 + 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f730870887dd..2e2076287aaf 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4416,11 +4416,14 @@ static int kvm_mmu_faultin_pfn_private(struct kvm_v= cpu *vcpu, static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { + unsigned int foll =3D fault->write ? FOLL_WRITE : 0; + if (fault->is_private) return kvm_mmu_faultin_pfn_private(vcpu, fault); =20 - fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, - fault->write, &fault->map_writable); + foll |=3D FOLL_NOWAIT; + fault->pfn =3D __kvm_faultin_pfn(fault->slot, fault->gfn, foll, + &fault->map_writable, &fault->refcounted_page); =20 /* * If resolving the page failed because I/O is needed to fault-in the @@ -4447,8 +4450,11 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vc= pu, * to wait for IO. Note, gup always bails if it is unable to quickly * get a page and a fatal signal, i.e. SIGKILL, is pending. */ - fault->pfn =3D __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, - fault->write, &fault->map_writable); + foll |=3D FOLL_INTERRUPTIBLE; + foll &=3D ~FOLL_NOWAIT; + fault->pfn =3D __kvm_faultin_pfn(fault->slot, fault->gfn, foll, + &fault->map_writable, &fault->refcounted_page); + return RET_PF_CONTINUE; } =20 diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_interna= l.h index 59e600f6ff9d..fabbea504a69 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -238,6 +238,7 @@ struct kvm_page_fault { /* Outputs of kvm_mmu_faultin_pfn(). */ unsigned long mmu_seq; kvm_pfn_t pfn; + struct page *refcounted_page; bool map_writable; =20 /* --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F784204945 for ; Thu, 10 Oct 2024 18:26:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584789; cv=none; b=jmgtgDwDOHhWRwfY72wfTIRN5e7zwLbV+fTVSnCLXPkJTZZsGYD/qJJYoo2t6kP122t30Hh61fvctCulZ6ZeGiEoEJkFlaidhlN5G/p26U/rbDNzc14he7nZFbjYpQ4lj6lIUvxfOHWM5nVi5GtnkueTE+rS1rorcu65693Nxjc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584789; c=relaxed/simple; bh=YHJlLBVBXana8wfsUYax0JzdySoal3WoOfu+2A9pl0c=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=qUI4vIQM46wtqJwT/Ru+CiCpjrbVjv1ivtpzkTWztkoYgIucQa8duWHwn4+VzPaV82Ctqu7oL6MlOZ5zjGvveghDM/tjS0HD7mJNMDjy7HInMM3NHzKLL2ezWfvi2sQx6YuBVWteUA0SnCGRnlmYFx5GltmvnPA3QheEqjnItxM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dzGkvQXL; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dzGkvQXL" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71e019ab268so1589212b3a.2 for ; Thu, 10 Oct 2024 11:26:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584787; x=1729189587; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=uRLwC7za7TRhyOe/CfUzhvpHD8RlodmrAsAEh9/RrLY=; b=dzGkvQXLwKnQ8kV49+00tqUsM+pb+EFzb+lv8vwXDt79FPnTmt4s8Md8nPZ0C2odAg BR9Oj/cN5OU7Z34cg4SiflaOOJ5uSAxjPR2L8QNFBM0aMzMUmmkaHfupKWrtFiM8kejf VixuAbaQpGtr0YhglX051laUICzExDICU+fAYRcbxNncGOKDgEvabPI3E8c8OvZZimsV 3a5isk8DF+25PgKctldAVdBofKibWUxqq0QyK7kk7WWlPWAEs6Px7f46IOU1Tcdum/We /OMbeEr7lpmmWWpGWaNU3NyZrQEJEuyYs8E+PUOurydDDM/bBLcqQeYJuELStICgdxYN kUHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584787; x=1729189587; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uRLwC7za7TRhyOe/CfUzhvpHD8RlodmrAsAEh9/RrLY=; b=CdnV3cBz0WRVIgIGFZx0M2ulLaACxJXt2KT3mKFoikCiMBGWDo+zSIHWfuCNg7UfiU CkP79BT3WiXkShf+qIZJGAqapNaiPZy6rEPL62nNPVSpEGpXDj8VOUsGrSQaa1SY/pPM NazoXNgSdS2Fyz2q2MDhvG/BpinP0VT6YadEYs68U+vjyb1XOSqRpY04ec0E7OhG75it PRPrsBfq2/NlIRKLw6Yv1gH0JcSJp+4cCPLJP8YMH59de5bF8UP5U/iMMAZFyhIPkNKZ BqXyDZbUyJ1DqhzG8AbUgZatk7z+Md6l8y5c6PSezbLacHAn3g19K84nkXxSBPtXDLO1 EnNQ== X-Forwarded-Encrypted: i=1; AJvYcCWhIhO/wHtMHr5p4/l+o55bs1yG1rtVt5x/6/Bca5JfO4BhCt2IXTHLSxspKGsOmTafVfkcwtGueagwIZ4=@vger.kernel.org X-Gm-Message-State: AOJu0YzZhg/D0QAXJf+dk8s1RYApZ8XZol1nTTPE+8xRwRPgBddf5oAq jsqGaLfdsCBH+M+OC7o8MBEZ32bE8Ir4J4sPMHii5PNNGJ6PKbQ+N3+2AQzd/9hofjtoCySSnis 7kQ== X-Google-Smtp-Source: AGHT+IHljZntO7iRsDkdAW1llbbIMc26iesy/7PCA9rtG6130gQpp6naB/6JGeYrnZXaXdo41CPgD6F3FKs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:9184:0:b0:71d:f7c9:8cb3 with SMTP id d2e1a72fcca58-71e1dbf1ab5mr7795b3a.5.1728584785442; Thu, 10 Oct 2024 11:26:25 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:47 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-46-seanjc@google.com> Subject: [PATCH v13 45/85] KVM: guest_memfd: Pass index, not gfn, to __kvm_gmem_get_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Refactor guest_memfd usage of __kvm_gmem_get_pfn() to pass the index into the guest_memfd file instead of the gfn, i.e. resolve the index based on the slot+gfn in the caller instead of in __kvm_gmem_get_pfn(). This will allow kvm_gmem_get_pfn() to retrieve and return the specific "struct page", which requires the index into the folio, without a redoing the index calculation multiple times (which isn't costly, just hard to follow). Opportunistically add a kvm_gmem_get_index() helper to make the copy+pasted code easier to understand. Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/guest_memfd.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 8f079a61a56d..8a878e57c5d4 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -302,6 +302,11 @@ static inline struct file *kvm_gmem_get_file(struct kv= m_memory_slot *slot) return get_file_active(&slot->gmem.file); } =20 +static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot *slot, gfn_t gfn) +{ + return gfn - slot->base_gfn + slot->gmem.pgoff; +} + static struct file_operations kvm_gmem_fops =3D { .open =3D generic_file_open, .release =3D kvm_gmem_release, @@ -551,12 +556,11 @@ void kvm_gmem_unbind(struct kvm_memory_slot *slot) } =20 /* Returns a locked folio on success. */ -static struct folio * -__kvm_gmem_get_pfn(struct file *file, struct kvm_memory_slot *slot, - gfn_t gfn, kvm_pfn_t *pfn, bool *is_prepared, - int *max_order) +static struct folio *__kvm_gmem_get_pfn(struct file *file, + struct kvm_memory_slot *slot, + pgoff_t index, kvm_pfn_t *pfn, + bool *is_prepared, int *max_order) { - pgoff_t index =3D gfn - slot->base_gfn + slot->gmem.pgoff; struct kvm_gmem *gmem =3D file->private_data; struct folio *folio; =20 @@ -592,6 +596,7 @@ __kvm_gmem_get_pfn(struct file *file, struct kvm_memory= _slot *slot, int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, kvm_pfn_t *pfn, int *max_order) { + pgoff_t index =3D kvm_gmem_get_index(slot, gfn); struct file *file =3D kvm_gmem_get_file(slot); struct folio *folio; bool is_prepared =3D false; @@ -600,7 +605,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory= _slot *slot, if (!file) return -EFAULT; =20 - folio =3D __kvm_gmem_get_pfn(file, slot, gfn, pfn, &is_prepared, max_orde= r); + folio =3D __kvm_gmem_get_pfn(file, slot, index, pfn, &is_prepared, max_or= der); if (IS_ERR(folio)) { r =3D PTR_ERR(folio); goto out; @@ -648,6 +653,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn= , void __user *src, long for (i =3D 0; i < npages; i +=3D (1 << max_order)) { struct folio *folio; gfn_t gfn =3D start_gfn + i; + pgoff_t index =3D kvm_gmem_get_index(slot, gfn); bool is_prepared =3D false; kvm_pfn_t pfn; =20 @@ -656,7 +662,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn= , void __user *src, long break; } =20 - folio =3D __kvm_gmem_get_pfn(file, slot, gfn, &pfn, &is_prepared, &max_o= rder); + folio =3D __kvm_gmem_get_pfn(file, slot, index, &pfn, &is_prepared, &max= _order); if (IS_ERR(folio)) { ret =3D PTR_ERR(folio); break; --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3CB7204F6A for ; Thu, 10 Oct 2024 18:26:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584791; cv=none; b=L2HCiD4YXwShvvdSCyS6B/ryGdGgV1ojyMnVCeEfYWEW51EfSfnHA1SCa6OechmP4CQo0/me8+u6BIozJu+oD010fVVWPZ7JQaComWXVkcse5Wr7mE0xv2ViyKlTWj8RWBlUlEZwgCarkj1bsI3EPSUFnMy7CwpG/RQh5jPBtfg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584791; c=relaxed/simple; bh=bZ2ermi+i+GlJV3rIdZ7irm2X1sw1usJ5ghB+vq7WnM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Jes9LIG15mftsdbWr/cNFjQCMZH1whZYhaX9AIePXzXPdSjmqILSmge5/dj0hqF+xG4BC4gwEja7H/FqMSQgUL2y4FUEqYQ1VbyOlkijBgHAue/javK96Ngf0VW8EI+Aw5GwcvPNEcaJD0yzXBGl71VXd2jNnb9kI7mcxUDciWM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=fpNaofRE; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fpNaofRE" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e28b624bfcso20396517b3.2 for ; Thu, 10 Oct 2024 11:26:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584789; x=1729189589; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=/DfuS3cqNKQbdjipc1jp6gNPynmr8H/oDbYVrbrij3Q=; b=fpNaofRERnQsVJWoNEKh/7Nowa8mnaYcBTqm4xvL96gZzqD0HTZnR8uv/7QUn44Mug vEiJev1UgG+mhZqqAgsDxOJvxhY9x3JDP2NQGvFiuEvxX24RgTLILDckgh/2qq6X3KtK 6Dc37IioCu1GchTrprfk2zvXU/JJTKtdXYop7F8ngmHmgB8Ds7A6py1mzGRTvBuh1e5v +tZ5qSEUmDv46Q7aijpywSEqaqKx6/E4e0UoBTUqX5sTdpf4Hd3zgayPbargNUbcalP/ ZD1NEvP7da5EwZ7MewAiUiFrQgpkfzQgUoU9SYS7aXy/Ax2P3Iv3RNqnAhJDoCKcHA5A WlSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584789; x=1729189589; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/DfuS3cqNKQbdjipc1jp6gNPynmr8H/oDbYVrbrij3Q=; b=XyAkGq2GkfkvNpOG03PLoL5QJOAf2f2R3OsbdOd38jDs/PnTrjSN+3Rk1NV3x2t6Cw 9TMCTMC7pRXAg8GVmYbopOQSLsSAP7G9+Cx1lkBv+rX/GboRzcLYPMLArIbJZWRqRsTL 3fp2/QQGyTW2TzCh3dHvwb9W1x4G+mfAROd/J+qksBhJzn/FSpMPPTuVwVH3VWhjXx7m rJKzODcnaRcGPIuh4Vkccz+IwfiAru+saLpBQnEi5nrH7MErrH2RoHGrzUIEqeDj9iSm sWsPfO4h/5SoAEAWQPmLWrVH8atNQlUd0f7AlX4iShYd3EAk2+P1qraigXldcH84uzq/ 2+LQ== X-Forwarded-Encrypted: i=1; AJvYcCW2YhEzfO8JOjHQYiZntet4z5wgFf9RU5FVhsme3daZwlwgAhCKUqS9CwYOR4BACfLG+fKmF6q4hj/HKEM=@vger.kernel.org X-Gm-Message-State: AOJu0Yz7nQ/Z+Wz43hZ7yvCJwM6LLV6MpzakQscjEPKZaRVzn7AizRqe MXlpOGDdayyC+nakWU+Bh3OW94HazAKAjn3dtSMxVehGLGLqzrVkc/h8U/2V25DHlh8dqQthbAw 5Qg== X-Google-Smtp-Source: AGHT+IHLQFn8WLOIHu4PHZ3K+TrLKBMF9JlmWl30kTXK43TD90b+TNbg3ZepRFQQdmJeel5bU/EhDzobpdQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:b191:0:b0:e1f:eaf1:2254 with SMTP id 3f1490d57ef6-e28fe41acaamr82536276.8.1728584788687; Thu, 10 Oct 2024 11:26:28 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:48 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-47-seanjc@google.com> Subject: [PATCH v13 46/85] KVM: guest_memfd: Provide "struct page" as output from kvm_gmem_get_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Provide the "struct page" associated with a guest_memfd pfn as an output from __kvm_gmem_get_pfn() so that KVM guest page fault handlers can directly put the page instead of having to rely on kvm_pfn_to_refcounted_page(). Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/svm/sev.c | 10 ++++++---- include/linux/kvm_host.h | 6 ++++-- virt/kvm/guest_memfd.c | 8 ++++++-- 4 files changed, 17 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 2e2076287aaf..a038cde74f0d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4400,7 +4400,7 @@ static int kvm_mmu_faultin_pfn_private(struct kvm_vcp= u *vcpu, } =20 r =3D kvm_gmem_get_pfn(vcpu->kvm, fault->slot, fault->gfn, &fault->pfn, - &max_order); + &fault->refcounted_page, &max_order); if (r) { kvm_mmu_prepare_memory_fault_exit(vcpu, fault); return r; diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 4557ff3804ae..c6c852485900 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3849,6 +3849,7 @@ static int __sev_snp_update_protected_guest_state(str= uct kvm_vcpu *vcpu) if (VALID_PAGE(svm->sev_es.snp_vmsa_gpa)) { gfn_t gfn =3D gpa_to_gfn(svm->sev_es.snp_vmsa_gpa); struct kvm_memory_slot *slot; + struct page *page; kvm_pfn_t pfn; =20 slot =3D gfn_to_memslot(vcpu->kvm, gfn); @@ -3859,7 +3860,7 @@ static int __sev_snp_update_protected_guest_state(str= uct kvm_vcpu *vcpu) * The new VMSA will be private memory guest memory, so * retrieve the PFN from the gmem backend. */ - if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, NULL)) + if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, &page, NULL)) return -EINVAL; =20 /* @@ -3888,7 +3889,7 @@ static int __sev_snp_update_protected_guest_state(str= uct kvm_vcpu *vcpu) * changes then care should be taken to ensure * svm->sev_es.vmsa is pinned through some other means. */ - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); } =20 /* @@ -4688,6 +4689,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_= t gpa, u64 error_code) struct kvm_memory_slot *slot; struct kvm *kvm =3D vcpu->kvm; int order, rmp_level, ret; + struct page *page; bool assigned; kvm_pfn_t pfn; gfn_t gfn; @@ -4714,7 +4716,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_= t gpa, u64 error_code) return; } =20 - ret =3D kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &order); + ret =3D kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &page, &order); if (ret) { pr_warn_ratelimited("SEV: Unexpected RMP fault, no backing page for priv= ate GPA 0x%llx\n", gpa); @@ -4772,7 +4774,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_= t gpa, u64 error_code) out: trace_kvm_rmp_fault(vcpu, gpa, pfn, error_code, rmp_level, ret); out_no_trace: - put_page(pfn_to_page(pfn)); + kvm_release_page_unused(page); } =20 static bool is_pfn_range_shared(kvm_pfn_t start, kvm_pfn_t end) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 3b9afb40e935..504483d35197 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2490,11 +2490,13 @@ static inline bool kvm_mem_is_private(struct kvm *k= vm, gfn_t gfn) =20 #ifdef CONFIG_KVM_PRIVATE_MEM int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, - gfn_t gfn, kvm_pfn_t *pfn, int *max_order); + gfn_t gfn, kvm_pfn_t *pfn, struct page **page, + int *max_order); #else static inline int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, - kvm_pfn_t *pfn, int *max_order) + kvm_pfn_t *pfn, struct page **page, + int *max_order) { KVM_BUG_ON(1, kvm); return -EIO; diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 8a878e57c5d4..47a9f68f7b24 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -594,7 +594,8 @@ static struct folio *__kvm_gmem_get_pfn(struct file *fi= le, } =20 int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, - gfn_t gfn, kvm_pfn_t *pfn, int *max_order) + gfn_t gfn, kvm_pfn_t *pfn, struct page **page, + int *max_order) { pgoff_t index =3D kvm_gmem_get_index(slot, gfn); struct file *file =3D kvm_gmem_get_file(slot); @@ -615,7 +616,10 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memor= y_slot *slot, r =3D kvm_gmem_prepare_folio(kvm, slot, gfn, folio); =20 folio_unlock(folio); - if (r < 0) + + if (!r) + *page =3D folio_file_page(folio, index); + else folio_put(folio); =20 out: --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91C01205E0F for ; Thu, 10 Oct 2024 18:26:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584794; cv=none; b=h3L8rX/KTQsn2Z6QhZGZ2IHO4aYtIZbR6cuIwy6OWmFl+Z7ee/l00Mk2PHjTSqU9Ev5A42sRvCVzI2xJJkXX7S7S3NI5QaBTFPzzl8VMYDSXwLtI7Hc/l5WGuQtLL0G2DLnSPE4n2H2ZS9YFxCy8uJXnwmBqMbxnnGp/D6WQtcA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584794; c=relaxed/simple; bh=P6yv0TC0koYnkFHtfd6W3SlzMlk4IripfIbi6+Lm/9A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sTRuALZsyMPbl7RmhPQUBnCQWL4TQ1EPnJJHBbJ9DmjY2TALMG58dtMQEbQpXgtM+bH0m2HnRuwOnfW0pQz29YNRwYYNq3foR0FohXdcfU24j3NKiQTYhtV2UuYtDOkjz44yRJOzQ4VB/gBV2wbPp6vUwjUtyDIxBGT7RRIfsE4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=a77PMqjI; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="a77PMqjI" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71dff575924so1543451b3a.1 for ; Thu, 10 Oct 2024 11:26:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584792; x=1729189592; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=l4aL9ye2ZxrCnqTpLqQlinT65BUul16gfmDzid4NN5Y=; b=a77PMqjIq3vLWks/ShFXHpZbDnuBVQmai91zT69BbUwBg3+sNEzUevImv4vYPqB3SZ om8MeiFZmfuFyqTQa0KjSKJA9hlghMqeoCasv0rEqK8+1XvrBNkUsubtWjj9vD7aGzDK DWHK1/6pg+wg4oYoW9aC+R4ALvw4SAmWD1iz5nlxSjYqP9yc0vS3FrHLDcpYL5IfkSPZ ZjeNHX7Lj0IluQY7kzhgYqhHrmn/TxGtn+2w11P0tE/6zbCPcnROX6y1R0rOcOpCorWd xxlujfXtkPqmsR8pGzTSMnqlSfuQ2+mvEbtk+8vSE9cw0rUnXeF2Yz7zxJeQSu3EBkzH +QFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584792; x=1729189592; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=l4aL9ye2ZxrCnqTpLqQlinT65BUul16gfmDzid4NN5Y=; b=l750SiO0j6X9Vr5rcGN36XTj3vfsJ94bFXOaiMEqm7RZOEpJp+QNveGTuEq3HNb8zz q/70e0CUnbiDxjecHwHhzJNxcm4zidvBHwC2q3NpmPhi3Kiz/8uS0Qg0oQHytumpz/qm GiJwlprfuOQ2Xn4LX9RwilT8TTmRe7bh+JwKKstCaIe1FPo0O6d7FNylCiKnUNcDPAKS aAscf/S8h3KcNHgajiSfCvNLX9xyv/XmoPLGBCOErMuKfwJtluC4r+cFkT3voYkahbHj y+ZthTvjBYh/UgzjkcWPoGG+8I3KhEfj1GbhRc5I/I40Nn0KYyV133+43E8v9ZFIyOLe gScQ== X-Forwarded-Encrypted: i=1; AJvYcCVKJdhN+3KqkLbbNfiIlEiZ46LRMafAyKoJNkFNv7iiBq4jhiE7wG0zt7VcuRrMiVkEPFeYVLfEZeDWroI=@vger.kernel.org X-Gm-Message-State: AOJu0YzqDxlgu885/nrWjx++3ABJV7tYmycYyDg+2KAgtQcKmnD4QbOx h58A1lm8aTgpDAYm4bTvzkzBC/WHEyspTWllVpK3SoONoqKTAP6embImBdqeocc4HS5t6cqfCR2 VVA== X-Google-Smtp-Source: AGHT+IFunsAHL0GYDEyxrwNnSExkBiXOv88aJkEgjVxw5ZDXOBsEr4Eha3ByEK4QIBQ4extfr7SnAz+Fl/s= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:9184:0:b0:71d:f7c9:8cb3 with SMTP id d2e1a72fcca58-71e1dbf1ab5mr7799b3a.5.1728584790633; Thu, 10 Oct 2024 11:26:30 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:49 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-48-seanjc@google.com> Subject: [PATCH v13 47/85] KVM: x86/mmu: Put refcounted pages instead of blindly releasing pfns From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that all x86 page fault paths precisely track refcounted pages, use Use kvm_page_fault.refcounted_page to put references to struct page memory when finishing page faults. This is a baby step towards eliminating kvm_pfn_to_refcounted_page(). Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a038cde74f0d..f9b7e3a7370f 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4373,6 +4373,9 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu= *vcpu, lockdep_assert_once(lockdep_is_held(&vcpu->kvm->mmu_lock) || r =3D=3D RET_PF_RETRY); =20 + if (!fault->refcounted_page) + return; + /* * If the page that KVM got from the *primary MMU* is writable, and KVM * installed or reused a SPTE, mark the page/folio dirty. Note, this @@ -4384,9 +4387,9 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu= *vcpu, * folio dirty if KVM could locklessly make the SPTE writable. */ if (!fault->map_writable || r =3D=3D RET_PF_RETRY) - kvm_release_pfn_clean(fault->pfn); + kvm_release_page_clean(fault->refcounted_page); else - kvm_release_pfn_dirty(fault->pfn); + kvm_release_page_dirty(fault->refcounted_page); } =20 static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08DDF206054 for ; Thu, 10 Oct 2024 18:26:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584796; cv=none; b=e1gu8oyHgkpHWlNsVULh8k3/x0Np8YyTbb2qqwHeyQaEx/PmPKGWL7Ub20lQhkbbz1c9kV6wxCTJlpOARN7Ws8HHU6yfbeJYjRmhh1t+smAReHgSaOa7npyrb3y648ge6cpkDWpnYARNcVu4fillfJWhKoEeS0GXVvpyYQJF014= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584796; c=relaxed/simple; bh=aOhxn0fpApYS19RFll+QfwBJLF7yF1QDDa2KZCpFRDw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RsU5S5YNwF12FoGxdndYHccUQBp09hiPMKSlFIb68BtFsjljlmg7wHnbUgMCz9w2UxE76IIyaabOot1Xez7U82VFB+D7nbuEvLP69I9gCYGI2138+sGP41wNeu0RgltqsO1ijHeArp6aTrN8mHWQ/nmFyn434yK9Waa9zi1gVqk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=0FbCESn6; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="0FbCESn6" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e2b049b64aso20314337b3.3 for ; Thu, 10 Oct 2024 11:26:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584794; x=1729189594; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=BN+hsHae4h8wzhzshUJRGKM30XxzO0F311sC9z4rUGc=; b=0FbCESn6n9wvwnedhP6g2o0aQ3uGz7LbaDMietMd23Zpw5H+WqY+ubJNtwyvC9nEJv FWG1G2wSjqX+D1kLYEKQ0FHw8fShSQ9I1S4nI3dc82vP/efq88a1zYT1Twk5N0zf9ScQ /53A6MT01CFUmrfBGbUUJJnvgQCVHLryQ3ee2m4qzskSNjfHdhLhZsMphQs5+zcS9Thx dQDoRI6PZz3brGVXYophhSUWOYJnUXXzqTn7D+S/653MDkAwi1Meo569GoTGnQl8jyTd gG9Yppau1gFkgT2RyuAm8WpGDImvPtucCOwbYKUzjX6gz+sxcdjTnHuWB3tqJHMrzQma e97A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584794; x=1729189594; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=BN+hsHae4h8wzhzshUJRGKM30XxzO0F311sC9z4rUGc=; b=J+IxqVH8yOr+AAGpIXoLbc7gJRFrbU22+1loM8hhulHgKugrVVlpY8d/J0J3dT+fpd DOwcEbKMxqcAuUrHccfhf7BsyMm0yrDqy5fU7Raqaww8Upt4tmSU8iCzMUf9G8HZhL8A 07dAmHjZ4LiVmnFzE0q6xAdf/457lP1ju0l9yCbIfQZzNZ3OLBqr6u44+c04VSV+Y1zY E760ReNhoo7qfNa/jN06BSZSrFyrelLa7C7gjcbIk5hhZK7AdBZ0F+a01EIKqleTgEtl O81SCQkjMq3dnV+9zXGlnHn0FITtCWlgIFb3pgr5BCOHJIK7TPbnodfGcCyLNroh4z8E VOsA== X-Forwarded-Encrypted: i=1; AJvYcCWV+g1YwPCAiInQJpT1t2FOZ/JBeTnajWWzmNKoLMynIwX6xq97GM6gej8NGXyS/xpPk/AB6dl1DP9h8l8=@vger.kernel.org X-Gm-Message-State: AOJu0Yx+ZED4XixRiaUxxtjCHinkHT+7i4cQ4TlauobVNJbfSKC77K1y CL+79AqJkoUi1fCPrpUsV+vO0X29G/uu493wxI1d9TZuPaTzSqpnnR99IxKjywMFLiGcSirog/w yow== X-Google-Smtp-Source: AGHT+IEjw5s9ln5Mmco0bHKHDYNNIWwRc71SIREcZZABOz/fcRYPRAU5+BeNEqASF/uZrX6+QTap8C7eoHE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2c01:b0:6e2:3e17:1838 with SMTP id 00721157ae682-6e3221407b0mr440167b3.1.1728584793806; Thu, 10 Oct 2024 11:26:33 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:50 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-49-seanjc@google.com> Subject: [PATCH v13 48/85] KVM: x86/mmu: Don't mark unused faultin pages as accessed From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When finishing guest page faults, don't mark pages as accessed if KVM is resuming the guest _without_ installing a mapping, i.e. if the page isn't being used. While it's possible that marking the page accessed could avoid minor thrashing due to reclaiming a page that the guest is about to access, it's far more likely that the gfn=3D>pfn mapping was was invalidated, e.g. due a memslot change, or because the corresponding VMA is being modified. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f9b7e3a7370f..e14b84d2f55b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4386,7 +4386,9 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu= *vcpu, * fault handler, and so KVM must (somewhat) speculatively mark the * folio dirty if KVM could locklessly make the SPTE writable. */ - if (!fault->map_writable || r =3D=3D RET_PF_RETRY) + if (r =3D=3D RET_PF_RETRY) + kvm_release_page_unused(fault->refcounted_page); + else if (!fault->map_writable) kvm_release_page_clean(fault->refcounted_page); else kvm_release_page_dirty(fault->refcounted_page); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1CFC320607F for ; Thu, 10 Oct 2024 18:26:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584798; cv=none; b=dI6EY/OtsvPnfHxpn+mFeTC8tstIgUxlygj7/ajOy1j1u9yg0ru1VCMGdbW7X7MxMta9A71m8LNGB70HfgiNRfvzsVsrxbu7hBnvFckCpfIi5eoY3HGknBjGBYg79iO8Wj4aZGPn2yQr7HJ8mls27pR8hxFa55eP5eqJfcRrXuA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584798; c=relaxed/simple; bh=7Japh8LNPjEnA4nandXvXnjfnzZjLu2hqYfo+wet/X8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GvHP/uRSbre5kyMZOu+0OE/9H3h35fms8iAxlzfujk41gcrkM4r/3cd3C2AIf2xYFbbTJ43Gh1ctH2IKNme9v3PTIJmTBQvfT0KI20cNDfSi78HziKYEsNxD2h2sTEOuL4By+dfGqnfyr1ZqxW2p6OrEU1V4/ss3J8ILRI/fQ1k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xxvwKwOg; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xxvwKwOg" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7b696999c65so863531a12.3 for ; Thu, 10 Oct 2024 11:26:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584796; x=1729189596; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Ndpfmsd07O3XHkbROvy9SWySHpmtrF0LUOG5EHCQjAE=; b=xxvwKwOgsGq9OHAeEg7ulS5M+xc0H7aGJvU84LLWDRvOqgU3TkZZQEPsrvINQkKuQh a0kSZ2m3MyVoMs8rT3NEB6ipyNHqf7wvINEndO31pndYJIj4JwdiGbpEIOldXIyZ4JUH m2Xwt+waP1WmBQ71gKCZlGDkYqz50bxSUom0dv824jufQ4rZn5keWyyQCE4jabUEcNHX 8IIDaN8klrGuagpek/bjhFComuRxJZEumPnJcNvLYqDKl0lJxlPsm74awlyYEz6E8Cx5 VMALXIP+FsaprgSWZvSmcNFCJoD0wV5yGOvsPvB1jxazYhU9yqzSqRKG2eEf1zu2lj+c BfPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584796; x=1729189596; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Ndpfmsd07O3XHkbROvy9SWySHpmtrF0LUOG5EHCQjAE=; b=DE8TAATQ5atyQRisQgpDORmVz2UAPbn+IUzqxg5jBttF137xjFgsQduAnTjAaaQKt3 4VbfjI8x601lEicRbGYivyJuKEyT5QyEOFwVEAtQTpzJJ+ZCrMTynT77xtXo4YSoXcSX Ux+f9YtqjdQxsQwimhlHhFd0ILBYT+RJDs5sEyVAzapLV/ditgPrzKARbHmojasoNEOk FNK5H5H/Yw1q2yjVpx2e4+EI10Am43jYwGlgQT21jCb1BoqziFmpf694sG224s/hBxQ4 OzU+6A1kfFXPxDanu8b/YUUDnGi7YrAW/yWuDwTrvb3FB1uLm3sxtat9FXa7/qW70FHM fK2g== X-Forwarded-Encrypted: i=1; AJvYcCVw3w7bUm1/hnIhWoPuhr1tF4t7jlby0WRWhDmQVf7wpk2rIpmiAvbkRKZ5LY4fuFRJq7C7f6V+z2iq0KA=@vger.kernel.org X-Gm-Message-State: AOJu0YySz6fosoqSlZkG5l6hx/sydIFNVxNVgl1JSJfPDN40tp5o+1Wv SG9bj5U6mqy571/FOLnUwxnTGS/sBDXRb6CmbbJycRi9JQPJSW+Sh/vNJ3xxeIwTm8TiA0TXT1e 2YQ== X-Google-Smtp-Source: AGHT+IGJc8hgBZq6lOovV4Dgly5ctIXEs7jDnJfEupYxVqninaLorgqTg5WNW0BPJZYnJnHdEtp0yWaf7UM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a65:450a:0:b0:717:a912:c302 with SMTP id 41be03b00d2f7-7ea53525e73mr49a12.1.1728584796119; Thu, 10 Oct 2024 11:26:36 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:51 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-50-seanjc@google.com> Subject: [PATCH v13 49/85] KVM: Move x86's API to release a faultin page to common KVM From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move KVM x86's helper that "finishes" the faultin process to common KVM so that the logic can be shared across all architectures. Note, not all architectures implement a fast page fault path, but the gist of the comment applies to all architectures. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 24 ++---------------------- include/linux/kvm_host.h | 26 ++++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 22 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e14b84d2f55b..5acdaf3b1007 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4370,28 +4370,8 @@ static u8 kvm_max_private_mapping_level(struct kvm *= kvm, kvm_pfn_t pfn, static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, int r) { - lockdep_assert_once(lockdep_is_held(&vcpu->kvm->mmu_lock) || - r =3D=3D RET_PF_RETRY); - - if (!fault->refcounted_page) - return; - - /* - * If the page that KVM got from the *primary MMU* is writable, and KVM - * installed or reused a SPTE, mark the page/folio dirty. Note, this - * may mark a folio dirty even if KVM created a read-only SPTE, e.g. if - * the GFN is write-protected. Folios can't be safely marked dirty - * outside of mmu_lock as doing so could race with writeback on the - * folio. As a result, KVM can't mark folios dirty in the fast page - * fault handler, and so KVM must (somewhat) speculatively mark the - * folio dirty if KVM could locklessly make the SPTE writable. - */ - if (r =3D=3D RET_PF_RETRY) - kvm_release_page_unused(fault->refcounted_page); - else if (!fault->map_writable) - kvm_release_page_clean(fault->refcounted_page); - else - kvm_release_page_dirty(fault->refcounted_page); + kvm_release_faultin_page(vcpu->kvm, fault->refcounted_page, + r =3D=3D RET_PF_RETRY, fault->map_writable); } =20 static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 504483d35197..9f7682ece4a1 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1231,6 +1231,32 @@ static inline void kvm_release_page_unused(struct pa= ge *page) void kvm_release_page_clean(struct page *page); void kvm_release_page_dirty(struct page *page); =20 +static inline void kvm_release_faultin_page(struct kvm *kvm, struct page *= page, + bool unused, bool dirty) +{ + lockdep_assert_once(lockdep_is_held(&kvm->mmu_lock) || unused); + + if (!page) + return; + + /* + * If the page that KVM got from the *primary MMU* is writable, and KVM + * installed or reused a SPTE, mark the page/folio dirty. Note, this + * may mark a folio dirty even if KVM created a read-only SPTE, e.g. if + * the GFN is write-protected. Folios can't be safely marked dirty + * outside of mmu_lock as doing so could race with writeback on the + * folio. As a result, KVM can't mark folios dirty in the fast page + * fault handler, and so KVM must (somewhat) speculatively mark the + * folio dirty if KVM could locklessly make the SPTE writable. + */ + if (unused) + kvm_release_page_unused(page); + else if (dirty) + kvm_release_page_dirty(page); + else + kvm_release_page_clean(page); +} + kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, unsigned int foll, bool *writable, struct page **refcounted_page); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49B18206E62 for ; Thu, 10 Oct 2024 18:26:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584801; cv=none; b=j7e06PoDQVovmuaJUmjA4aEVXDsSVB4SgxEBXO1BverB29fdNO/X9woBoEMFRG/zrjteo0LFzg7btQLPgfh8frUWVk3NZsg+r+7EGwbDGvqSHBKP3SAyX1pIiYuL1ccm/6vpuH6ZcDE0Gi3q+OaoHoRwjvdlFnycPguKcxI7g7o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584801; c=relaxed/simple; bh=saU0+rag3xXF2/z4L9d0pyohkblMwsyLsKfPnspMwVQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=aDpVRWd31IrG9cLUMqdZNu62Qdq3s8ClxiELrO0TN/z+o3mQZEFmBLowsF2X/dwKNFM9WBejWF2AxCeWrjKcg6BsQ2Cn8AARp7vPJmWITsNvq/T+6p/chddbTS06Z4MqNn/G5cydIVY8CGzy6yuoSc+psm3uASw++ZBpCG5Cjwk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=n7BtqPXO; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="n7BtqPXO" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2e2de6496easo621779a91.1 for ; Thu, 10 Oct 2024 11:26:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584800; x=1729189600; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=iay4RI/uPKTQE0By5qqA//dP+sG5w7++36T/RHxbCNM=; b=n7BtqPXOYdiWJ3X9kD0iG4C8w4FjCr57M/d0x7tEi5ZIPg/i+rqFcqjVa9Iu7dhTRm nS6nozj5Gx6r6aaboU/1PQ32h7s+9ziIKF4h+h1LWCrLRiUwW8rZY+NEeX29VkUMajfa kPp4Uu0Z6uHkKNXYVdvtCr+F14kxdarXnJ+OFvhdU/rGA6+vEjLsWSXDQnUXI1V6EUWB g/1qV+76T6dKYhGWKC3Y8+mwi7W/PFdRqfEjjmH1UO824LRmq6fMUl3qRoUJPp2+avqD GaAVk4tv/PFte/ebRcd7xjinGzyNHBzovubXDRuPTUDZ/OKUAHrM6Pk0WgOK4dH0HDWj 0krw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584800; x=1729189600; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=iay4RI/uPKTQE0By5qqA//dP+sG5w7++36T/RHxbCNM=; b=N6PtpfWHQwaUwomNF37CDY0XGYSnmksQeGMF/hV02Y2hh1u0E6bVmZo8E7d6S7pOa8 SzO05OGYK0bgYW4p5aszzDGv/FRvmqHIeF5nMOBeyr+TEqDgpWrbp+Ws7xLLkKrpuba5 FkrP48miImEPlPsWDeQuDwuJfWNUiG4ZPGkHbfs7APIRJKpqPa8u4aVvIPJY0mzu9H7T QB2ZfFHvXth/GAvWSoFXdjxzxIXx7unv7PJoo82wHzFrx+rvkvD57emqeG4gv4Vr1ZW0 6VRlw00p4J4B2BTwYs8QVuZG/ZqyUCKXBWQUfTBJvYid2qDnj1ZIspCjl1YrEqcAQGpx Hb/w== X-Forwarded-Encrypted: i=1; AJvYcCWByP01j1taFQ0O1xDFS4gpB850uS6iiry6fAMGUOtDcfu7hFtzntSW2Q+AHcFxxJxbiF3M7r2malMH04g=@vger.kernel.org X-Gm-Message-State: AOJu0Yz+rJM0EcouZmFel98Ds2+zzpOC5jijFnHsNQBe9cNFF+jCpd/J JRPIquFuvXn3holvD0CPPppLJsmcnS0bY3PrylmMXTqsJqgO1aoQfefq4ppb0gdRlMbeAgkb0++ Z3Q== X-Google-Smtp-Source: AGHT+IEd4uxjZrUyEQQM0MU+jkiJqh3LQboji5/ZXtrCgOTVgrRv0yMO5ln1NKE0y5PwnoqkXxjnsp8xFDc= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:fe86:b0:2da:872e:9ea4 with SMTP id 98e67ed59e1d1-2e2f0d7e9d7mr54a91.3.1728584798284; Thu, 10 Oct 2024 11:26:38 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:52 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-51-seanjc@google.com> Subject: [PATCH v13 50/85] KVM: VMX: Hold mmu_lock until page is released when updating APIC access page From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Hold mmu_lock across kvm_release_pfn_clean() when refreshing the APIC access page address to ensure that KVM doesn't mark a page/folio as accessed after it has been unmapped. Practically speaking marking a folio accesses is benign in this scenario, as KVM does hold a reference (it's really just marking folios dirty that is problematic), but there's no reason not to be paranoid (moving the APIC access page isn't a hot path), and no reason to be different from other mmu_notifier-protected flows in KVM. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/vmx/vmx.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 1a4438358c5e..851be0820e04 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6832,25 +6832,22 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu = *vcpu) return; =20 read_lock(&vcpu->kvm->mmu_lock); - if (mmu_invalidate_retry_gfn(kvm, mmu_seq, gfn)) { + if (mmu_invalidate_retry_gfn(kvm, mmu_seq, gfn)) kvm_make_request(KVM_REQ_APIC_PAGE_RELOAD, vcpu); - read_unlock(&vcpu->kvm->mmu_lock); - goto out; - } + else + vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); =20 - vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); - read_unlock(&vcpu->kvm->mmu_lock); - - /* - * No need for a manual TLB flush at this point, KVM has already done a - * flush if there were SPTEs pointing at the previous page. - */ -out: /* * Do not pin apic access page in memory, the MMU notifier * will call us again if it is migrated or swapped out. */ kvm_release_pfn_clean(pfn); + + /* + * No need for a manual TLB flush at this point, KVM has already done a + * flush if there were SPTEs pointing at the previous page. + */ + read_unlock(&vcpu->kvm->mmu_lock); } =20 void vmx_hwapic_isr_update(int max_isr) --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3CF45206E7E for ; Thu, 10 Oct 2024 18:26:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584803; cv=none; b=IHa3YGyFR75voscdX9uZw6ztOMohV9i/iNWGpI33jdUSYD0C8PYch25k4duZA0SMG+40BZmXSX9yQ+O/8a0I1QJwhMpACvXqMHZcbcJh7uS550ngmypqJJI09Xp9FVveGCdus4IJS7BEg/yfFf14eMScLeSxzkPmvufQ539WaUg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584803; c=relaxed/simple; bh=ux21RVGWhz9JRdIoM/Dq30UYNcfog+x0iowAjNZ/4XU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=VUqHEzDGNyVqkl81d5MUfKEQEtKFE8rK4Fpx9pHmrWoVX+P+KVEMVMLjI9PCF+b3/dWkaICWJyw2gAX4/1dah6Rdbwv50ta7yPYghs+63FNsBotOvU6Y8IMtNWGOd0sYBJTE8WDSahYSLF8DgExq4UCoABqGrmYAgfMBQY7OrYU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TkN0pgzM; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TkN0pgzM" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e2261adfdeso21207427b3.2 for ; Thu, 10 Oct 2024 11:26:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584801; x=1729189601; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=W6Zzl9AmyqedpEIwNpLXiwPcmrnHw6ZPa/9pyFpLcZ0=; b=TkN0pgzMcbtXcehOT3G5lQY8rkiUBvkBcxM6gFvQBdXPdSHR+Bdw7nrzPweN+yVkaN uTb/gUN2mQ2Vg1tKjaOAVMtsZSWMylN5yuKIXvVBtG2VEPvienlnVdJRXRAHwEu0kkyK 9Ofhzp0jp3Ck9eoskAcxSdzp6PspHPEsug9npp829r5JsnbCN9OEKkWW2XlQrAVK1qDz HR2epCDdOmr5wBKAPBrQzUwIntkDyltheKd/ormx9zR0906kw75/PioCQuOEzlpeTg8X zE3iJHzZ2VWDViZQG9tcL7+JnaOlpG6gu6+8GDdZFhQ//6LMtrKiMkaqGEETsqG0K2s3 zyjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584801; x=1729189601; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=W6Zzl9AmyqedpEIwNpLXiwPcmrnHw6ZPa/9pyFpLcZ0=; b=ZoIOQx3tDpJr6lXhK6DWoMNYouyy1MW2GiMj9vDc/ckb9OZJkJxY7WOUxfBOq2YnFs ytGxWwwcza3IWOWMla9meETrwcqe1jjJ/m8eu44AAd1DagWXRrMdNgDoi5Ey1YEwjnR4 x593nBqpOh9wgMzEXJEKz+6NDdplzEt1Pp2qj6xr6wlvFWaYmW9l6+QEEeoNIM3Mu6h2 FvziKv0bM4nReBsCZ2AE17HP8txhfcV3RzA6mMJVhDdvcNyuMgY4I6fhcXdZOXGEtUGS L33tVKlmD1oPn48IBUO40s5PofmJ2YebT0j1XxE3Hv95egJLtnlVIhp0dF5XHi9SGQ7X HzcQ== X-Forwarded-Encrypted: i=1; AJvYcCV1jSQk92Red4X0uKNqEK7MMZx4+2+Eljh/vwfrSLZNGpLH1D/YoUMrsnGIjhU1UHZ3aqUgd+Kx+yuexSg=@vger.kernel.org X-Gm-Message-State: AOJu0YwK8isiZdslHk0/P0hiEhrAnRcn3N2k+NQ8hU/vqSskEZ8ChnTc vzuKCGLs2VvsEDly2lq7mk5Dz574nd1A4kfEEohtY06oK1BZmuQ6eMUys8lqpKMH0PxGaXcKa+N LYw== X-Google-Smtp-Source: AGHT+IGFUTZueI+Si3awRWj8jA+YGlhXMEzUcm/2nm7iA7dHF8070ZlvI1MSF0nUJUhFBhXPPz9v/uVg5Fg= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2f08:b0:6e2:2c72:3abb with SMTP id 00721157ae682-6e3224db667mr816567b3.7.1728584801239; Thu, 10 Oct 2024 11:26:41 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:53 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-52-seanjc@google.com> Subject: [PATCH v13 51/85] KVM: VMX: Use __kvm_faultin_page() to get APIC access page/pfn From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use __kvm_faultin_page() get the APIC access page so that KVM can precisely release the refcounted page, i.e. to remove yet another user of kvm_pfn_to_refcounted_page(). While the path isn't handling a guest page fault, the semantics are effectively the same; KVM just happens to be mapping the pfn into a VMCS field instead of a secondary MMU. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/vmx/vmx.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 851be0820e04..44cc25dfebba 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6790,8 +6790,10 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *= vcpu) struct kvm *kvm =3D vcpu->kvm; struct kvm_memslots *slots =3D kvm_memslots(kvm); struct kvm_memory_slot *slot; + struct page *refcounted_page; unsigned long mmu_seq; kvm_pfn_t pfn; + bool writable; =20 /* Defer reload until vmcs01 is the current VMCS. */ if (is_guest_mode(vcpu)) { @@ -6827,7 +6829,7 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *v= cpu) * controls the APIC-access page memslot, and only deletes the memslot * if APICv is permanently inhibited, i.e. the memslot won't reappear. */ - pfn =3D gfn_to_pfn_memslot(slot, gfn); + pfn =3D __kvm_faultin_pfn(slot, gfn, FOLL_WRITE, &writable, &refcounted_p= age); if (is_error_noslot_pfn(pfn)) return; =20 @@ -6838,10 +6840,13 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu = *vcpu) vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); =20 /* - * Do not pin apic access page in memory, the MMU notifier - * will call us again if it is migrated or swapped out. + * Do not pin the APIC access page in memory so that it can be freely + * migrated, the MMU notifier will call us again if it is migrated or + * swapped out. KVM backs the memslot with anonymous memory, the pfn + * should always point at a refcounted page (if the pfn is valid). */ - kvm_release_pfn_clean(pfn); + if (!WARN_ON_ONCE(!refcounted_page)) + kvm_release_page_clean(refcounted_page); =20 /* * No need for a manual TLB flush at this point, KVM has already done a --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27C1C206E9C for ; Thu, 10 Oct 2024 18:26:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584805; cv=none; b=kyLnNysQBCH6cF/ZZB9LY/w9QTYtkDx6lkyln19/tafkHre0MpaZr4OUZMldcLpKPp8MqUpf8Et4sOjQ9skuafc4PkeTn/Scv62mkbcMX34zpeaBCyyAP6+sqUkHkC20RL7SkCwrsQK4HIkMUTdpgNBwl7VTm+AfRHJ5m0EHIL0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584805; c=relaxed/simple; bh=NP4IgcbBDVq5DZEeFtyiO8CdEZFJxxnULnDPMD13pKg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PPATC4KFtowOFz5A0n0CKFjXmInIZnfW1ZO7bRtqvzN7JSoWuB6RUBaKSZjjDZN/dz3UFouA2xlYIEcD1WH1isBXo6Ru6MQjzywJAzGO3CFlFpFYmQ6/eZeXKO2fgwbDgNgNq9/pnSW9c33iandvvAzgOBF7c4cO+YyNTauj8ig= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=g5hzo9jY; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="g5hzo9jY" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e29b4f8837so18952187b3.0 for ; Thu, 10 Oct 2024 11:26:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584803; x=1729189603; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=/fC7L3Fchvvuu9GyGKk9VEzuiyLOIeBtO2SWEaXDNWE=; b=g5hzo9jYuMsLef5elH7BXii/Rz3O5NNAzg3BaELLMkoxekYgKvv3GlwOx0MeUswR5S u7D79PlxCqfi2Eui27GQ/wf2PDK/FtSU9HQpxYIEp6FYcUNcZ4RxRkWKsOxypmmScggd bXGvbabN+3GVR3u3Z5evpvBvV7hTapUH6bSv1xuzsGrxP42UL0bRjk5kLAIDqUd2xwc1 HxptH7WG5BO6Qq5eztg3t9rrh5agOngJ9zqfaGOzOhRlW/rHlxLQl7mVwlclH/5jU39z SiwFtCrrtukpmLM9KZ9QtNIJl9cGgY/kEqCKhzMSjtywjCUbz1pRhU34CDuRasgyTV4R S5uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584803; x=1729189603; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=/fC7L3Fchvvuu9GyGKk9VEzuiyLOIeBtO2SWEaXDNWE=; b=aoqGgtYxqURuaGphanrFLOvasn14Gt+8xPGIkvthHgd83PARtV3AREJNkcaVKPyqO9 D59ph/Zm3aJq6EzM9jZ/z1++ymXGyVygjQ8JUk14DgcopYX+xMpIQunI40iW/4CDiOpt kuaBAiEEOTgbnUZYDa2RJ/Rc4+IHreIG5NsGMZOX0kSLMggnwPWNMg8g3RaBU75ZR5ZK j6TgaX/ixvEc9mpCQfFxoV0zLIXAbRcJ6wcGw+CcD7LfNtRpH9Y0ykh7KOEHR8n5U6WA 2Rb+mXiYsjlb9APl+z/s3x5qfQrkDUeJwEOmXQzzswNOFXKx5J/Ta0Snov3pBDmKq7NM XjDg== X-Forwarded-Encrypted: i=1; AJvYcCVUozWtw+fznTTddjtLNJRgE2CrIq1vA71Gk2VpEBFxNtX/NeXYRVWyFQQqjloClhU9qa5Tobkp4k4ytxA=@vger.kernel.org X-Gm-Message-State: AOJu0YwTtN8EzndGAc1pEl2cXdogJmusYHCUvbIldOXZ4oHA7X7jlsRm Cjr3OWlBXYKd1ORlERuT4YFCxKqNV6enSaCWcVTVY6PgVm27WTr8f3rLFFUMBn1x8ta6MjDCGGd g9w== X-Google-Smtp-Source: AGHT+IF0bQSeBV+k7XDKFBXYzNvC7N99ueMTlOEEH5KNk6JZHzu/00W1ndtLnT51eyBFoV94Bmz3yNyB+P4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:dd1:b0:6db:e464:addc with SMTP id 00721157ae682-6e344ce9308mr1337b3.4.1728584803335; Thu, 10 Oct 2024 11:26:43 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:54 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-53-seanjc@google.com> Subject: [PATCH v13 52/85] KVM: PPC: e500: Mark "struct page" dirty in kvmppc_e500_shadow_map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark the underlying page as dirty in kvmppc_e500_ref_setup()'s sole caller, kvmppc_e500_shadow_map(), which will allow converting e500 to __kvm_faultin_pfn() + kvm_release_faultin_page() without having to do a weird dance between ref_setup() and shadow_map(). Opportunistically drop the redundant kvm_set_pfn_accessed(), as shadow_map() puts the page via kvm_release_pfn_clean(). Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/powerpc/kvm/e500_mmu_host.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_h= ost.c index c664fdec75b1..5c2adfd19e12 100644 --- a/arch/powerpc/kvm/e500_mmu_host.c +++ b/arch/powerpc/kvm/e500_mmu_host.c @@ -242,7 +242,7 @@ static inline int tlbe_is_writable(struct kvm_book3e_20= 6_tlb_entry *tlbe) return tlbe->mas7_3 & (MAS3_SW|MAS3_UW); } =20 -static inline void kvmppc_e500_ref_setup(struct tlbe_ref *ref, +static inline bool kvmppc_e500_ref_setup(struct tlbe_ref *ref, struct kvm_book3e_206_tlb_entry *gtlbe, kvm_pfn_t pfn, unsigned int wimg) { @@ -252,11 +252,7 @@ static inline void kvmppc_e500_ref_setup(struct tlbe_r= ef *ref, /* Use guest supplied MAS2_G and MAS2_E */ ref->flags |=3D (gtlbe->mas2 & MAS2_ATTRIB_MASK) | wimg; =20 - /* Mark the page accessed */ - kvm_set_pfn_accessed(pfn); - - if (tlbe_is_writable(gtlbe)) - kvm_set_pfn_dirty(pfn); + return tlbe_is_writable(gtlbe); } =20 static inline void kvmppc_e500_ref_release(struct tlbe_ref *ref) @@ -337,6 +333,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_= vcpu_e500 *vcpu_e500, unsigned int wimg =3D 0; pgd_t *pgdir; unsigned long flags; + bool writable =3D false; =20 /* used to check for invalidations in progress */ mmu_seq =3D kvm->mmu_invalidate_seq; @@ -490,7 +487,9 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_= vcpu_e500 *vcpu_e500, goto out; } } - kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg); + writable =3D kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg); + if (writable) + kvm_set_pfn_dirty(pfn); =20 kvmppc_e500_setup_stlbe(&vcpu_e500->vcpu, gtlbe, tsize, ref, gvaddr, stlbe); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06AC7207208 for ; Thu, 10 Oct 2024 18:26:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584807; cv=none; b=EA4mJCkFaUCgRyPfsIBc1YjfE3gIqryeXCjpQH+ZWLmfnQiRCreCRy5tEd27GT1KR1RKybKdwc5+D7WvwTVuZqPK3jI54O+WwREHan2gKQ3cLSAc5IhkCLmJSHgqnKzbO2hKWLfQH4Xi22BEWeE4Q1SoCl5LJ1veHZqAH72iRTk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584807; c=relaxed/simple; bh=kbjwflG2kz8fn2XlGwl/ZUkp2MoQm+/BMH1MpANV1Tw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PoozIGhyILTkiuy8nnO48iY8oKzUalat0ee0Ld2gSQ2zm336JYakyRyRu4JkzCljzkohGV/mbdDjT7O0po8LtdB/ykf06DC6uezM1wUZlz5cocK3dyme5t3Hu44sDUJ2nq7DgPSiDjRbsbcheZfGUg2comGhcvEmjfZ9TCaruf8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lmtzsAHw; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lmtzsAHw" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e2eba0efc6so172834a91.3 for ; Thu, 10 Oct 2024 11:26:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584805; x=1729189605; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RAEBOjiR86B8MKio5svj+gL/GJb0VtWq7ImNxArS0H8=; b=lmtzsAHwQyNwRvm4YycGayWTaD3mMWa0KP6qhPQcjm6Xns70mqWPYJS1kLd7s22LTY /T7yWx+i0180z3+0c870icqFaPkxG/5jw4mabTqo8QyB0EcckP0wh91y7Mr5DVDEcAvZ hQO7SxtZXgjJ+5PNG6GYQ8lhMuxPsiaTv7bKN0qEUlt9rMOH5ix6Gd6u/08WvrX8BdhV sxBtnyqiiR8igp8kOk/PVonv0d4Scqtx+EB6UWy3fO105xlv1kZig9jJkbcTPFSX83n6 4g/v3dji6XZ61FVYO2EBHHk9ucMefuuWtbYJ3TSw4Isga9JWzpWIfkjpdmCzxAee0iT7 Q0eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584805; x=1729189605; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RAEBOjiR86B8MKio5svj+gL/GJb0VtWq7ImNxArS0H8=; b=PlbdTrMNWtinwYD6Cmds4Sbc8N6pD2+DAsedRpLnitGZzci1HM9RsrQmZwDUzgO0Fu T7NvbxRqOezv/Q8Fzmt4qfIWGSVf9R/yBvGN7YIj9KnD8XVCZaAGamv591Nh74DVpe8P h485egE3ZL3nVf1q/pfUKlbNwWC/JYQSA3QuV7/bK0BtcZo5tQjTHOYyu4m4JVLxWNGC IApirQmoA9CxkVq+fiaIW24SY0xgHGQD4bfUrJgQRRQkGclIKVgfGyhfp+V440kvBN2f AbVYqU02l2prz11c0p67sNQomrDbbzTaH8dDsbHZaLu+6574lLqQ8D3y4UE3hIThKFd3 uYrg== X-Forwarded-Encrypted: i=1; AJvYcCVNEtGbiJvAnig59+Olc5cIK8xoRrMqE9UwCijMJKdj+A+/+ebllO+kLd/sHbfvZh1VAVq58yWWeHBe2uk=@vger.kernel.org X-Gm-Message-State: AOJu0Yy9Veqst2F690Xku466SJp47j/GOXzPUHQt7g6Vy+dbREL5cmoK Jt9hXcnEij3VK0F5bIIDJEFg/21GkaHQugwZpKtqUxtd7eqKxVEQ3HLABJXWTNjhuQay2kXZ/nr QCA== X-Google-Smtp-Source: AGHT+IH3g+or1uGKI8zaqA7g6VjAId1uToioip8sNGZo/KyXggeZoODfUOK1q5z450FXHERnWYgwR7EwX2I= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:5107:b0:2e2:ca3e:10fe with SMTP id 98e67ed59e1d1-2e2f0f88a54mr24a91.8.1728584805241; Thu, 10 Oct 2024 11:26:45 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:55 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-54-seanjc@google.com> Subject: [PATCH v13 53/85] KVM: PPC: e500: Mark "struct page" pfn accessed before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark pages accessed before dropping mmu_lock when faulting in guest memory so that shadow_map() can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Marking pages accessed outside of mmu_lock is ok (not great, but safe), but marking pages _dirty_ outside of mmu_lock can make filesystems unhappy. Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/powerpc/kvm/e500_mmu_host.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_h= ost.c index 5c2adfd19e12..334dd96f8081 100644 --- a/arch/powerpc/kvm/e500_mmu_host.c +++ b/arch/powerpc/kvm/e500_mmu_host.c @@ -498,11 +498,9 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc= _vcpu_e500 *vcpu_e500, kvmppc_mmu_flush_icache(pfn); =20 out: - spin_unlock(&kvm->mmu_lock); - /* Drop refcount on page, so that mmu notifiers can clear it */ kvm_release_pfn_clean(pfn); - + spin_unlock(&kvm->mmu_lock); return ret; } =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 61FE2207A17 for ; Thu, 10 Oct 2024 18:26:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584810; cv=none; b=g9kjVeoAtLGOcPBaj1HxN3eNb//WmaDef1wAAswWsK8PIwx+j28oQr/f0CZ1XM2OnunOoBque6R0Py3/pgwi3ta4HjKtjfSN66fWKAPTkcZLlgtQ7Po0ME1TMez5sZ/S3hYkwFVCDMEyjkKlKwe7PCMICy0sbtuaMe0pqYlQUYk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584810; c=relaxed/simple; bh=ZWIlv2h6Cx1yrF7GGy3UqCKI1dfxdLpdeKJSlsj4QNk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Gnsa+35qelRudkHPnPBch6YqQ8pZjMF8Ww6MZH0V0BAEhHUzAQIAmM8AB0TGYVkys8fz/Kx6LAKu1nlcdfv/drJMOOR8jY36zwlFhymJt+zA+u6dbv2DkXZzSzr8LBeb8HGWmCrrTFPM8CBsl73gQrs1pwXX0sXN8+6UmDWoKKc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WnVYZ++e; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WnVYZ++e" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e290a9a294fso1855089276.2 for ; Thu, 10 Oct 2024 11:26:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584807; x=1729189607; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=nHP38EtPz8K1+ebGaPqvgAKvgaQP36cKGSrn8JpxRdw=; b=WnVYZ++eloUHmaNH+PMyPBVaTNNQV7PpyzwKfQNf8E0TWCFXkHdgRSE8m9QZdbFqgl LUO9GmnO14pJlNUz/a9YQDP4sRdsmqE0I77QJPEJQLiNtELH3T55kxgvZXX+ZS04dduF HOubAFdITv4Os+cf8tlrjsJ28Wj+yEbPvdLMEALhRVl3zs/sWfw9mSXKeFLyukSkfF0t s159ZU6SoLfXQ7k/TJ51qYCX6bhgSQvRTMspjJT5LynWuHOAFbYEj7AgiVCpYczdtFzC uNelGCXQnKo8zvzdO7U5pHqlDSTPwijrS7Od2lq6i7bZd6sgWjEtE+f9igLiaozAMB78 DGDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584807; x=1729189607; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=nHP38EtPz8K1+ebGaPqvgAKvgaQP36cKGSrn8JpxRdw=; b=H2DIXw2hfWVVXTGGMky1U40dkOZn6+FH+a0fLkyeCBOXUmxZ/6GFfZVAisIMAQQLUG qjfit9V26Rz+KrIX5m3lUyL6ZnH9yLZTEW87S+b2MuTBMwR5eHNAI4WHxGVZBxsn5B+k HcMza1/INwjp84xsJFHUiUOhTUUsBNUbniyr12UlO+ws/boF9TjL/sKuVLM/xCiblnv9 bs+Y269Eg8DcnUNahS47U03mZcL3VqRioeya9jhudVPScSNcLRj+PoE56FyD7trCJk6O gLdghw7LqmvHME8h7rJI9xvuv07wxaM7eh3kAnEnkjP/9u6oQpYy5Jaa8UfmRAAUYdWB 1hYQ== X-Forwarded-Encrypted: i=1; AJvYcCWTiCsS17H/PEhdK+ddzxDwOBqR4TIaclS9JcT2vfaLzK5elCNzr7Y8QVrb/QIEdWkMj2AjZPAqlM+VvQw=@vger.kernel.org X-Gm-Message-State: AOJu0YzTklyy/zRgrHvx/wEKBK9/1ER/dEd6qF+65M3NnfIeFkyvNsii v23PB3UpxFrzrP9G84lV/hONuZQNaRvQ2rrvJKzfQFDGK6stxFr1nYSJ3DLJxVT0eCz2n6vi2qT nJA== X-Google-Smtp-Source: AGHT+IHQvsj0FcPHwm2v4Ks73fb0XY1uO3QmFFCKoUyfp7OKRYByAKtc8uXQ4anZ8QaSR/IH5oavOTtruEI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:b86:0:b0:e0e:8b26:484e with SMTP id 3f1490d57ef6-e28fe516b5amr5188276.8.1728584807403; Thu, 10 Oct 2024 11:26:47 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:56 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-55-seanjc@google.com> Subject: [PATCH v13 54/85] KVM: PPC: e500: Use __kvm_faultin_pfn() to handle page faults From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert PPC e500 to use __kvm_faultin_pfn()+kvm_release_faultin_page(), and continue the inexorable march towards the demise of kvm_pfn_to_refcounted_page(). Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/powerpc/kvm/e500_mmu_host.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_h= ost.c index 334dd96f8081..e5a145b578a4 100644 --- a/arch/powerpc/kvm/e500_mmu_host.c +++ b/arch/powerpc/kvm/e500_mmu_host.c @@ -322,6 +322,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_= vcpu_e500 *vcpu_e500, { struct kvm_memory_slot *slot; unsigned long pfn =3D 0; /* silence GCC warning */ + struct page *page =3D NULL; unsigned long hva; int pfnmap =3D 0; int tsize =3D BOOK3E_PAGESZ_4K; @@ -443,7 +444,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_= vcpu_e500 *vcpu_e500, =20 if (likely(!pfnmap)) { tsize_pages =3D 1UL << (tsize + 10 - PAGE_SHIFT); - pfn =3D gfn_to_pfn_memslot(slot, gfn); + pfn =3D __kvm_faultin_pfn(slot, gfn, FOLL_WRITE, NULL, &page); if (is_error_noslot_pfn(pfn)) { if (printk_ratelimit()) pr_err("%s: real page not found for gfn %lx\n", @@ -488,8 +489,6 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_= vcpu_e500 *vcpu_e500, } } writable =3D kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg); - if (writable) - kvm_set_pfn_dirty(pfn); =20 kvmppc_e500_setup_stlbe(&vcpu_e500->vcpu, gtlbe, tsize, ref, gvaddr, stlbe); @@ -498,8 +497,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_= vcpu_e500 *vcpu_e500, kvmppc_mmu_flush_icache(pfn); =20 out: - /* Drop refcount on page, so that mmu notifiers can clear it */ - kvm_release_pfn_clean(pfn); + kvm_release_faultin_page(kvm, page, !!ret, writable); spin_unlock(&kvm->mmu_lock); return ret; } --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B01DD208200 for ; Thu, 10 Oct 2024 18:26:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584812; cv=none; b=Eua3LGfl1jCTw7tN+bZkhb8oH6gADm+8TpLrV136ZGThbnKdGlSKqMIGwY5mXmLCGl/IEfzG0TLzwwRd9YD3Z/s3j8wz8JKPqCpQp5N48sSDezqrGVMUEBgHAajgdp4NZfXMPKpwd49uY7oUm3VlUdI6ntYHL1yvLGVto028WvA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584812; c=relaxed/simple; bh=lNDyXtKPmFS6sOILzqWSFd9L+nyXpdsd6JebnDHI5l8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lNVa+UG+95SR8Yx146PwdYufAjPRfhWvR/1/FeX6ZAoCxQ221uZea//b9LdRuCFy9FlKcMRsQTPaOmTEsidyyhVJg1wASpN0OXUUHI4t67t5iaYn/zz2D2pV+1dUZVemauGsiGCOs16XFixP28rI0e7fV+SZswSkhFdrLPYRI5M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dg9gcvn1; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dg9gcvn1" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71e123d3d22so1429259b3a.3 for ; Thu, 10 Oct 2024 11:26:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584810; x=1729189610; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=K6YUqJX41UvLftiTjNafBx3/10X+6KBTIzQjXduU2xw=; b=dg9gcvn1EbsSMEFFWFfwFAC3/SO7ovIMxsDf+/JNqUPrNu+sX4BsLpvzKD94FQExHx ct/k5mewMmav7ktqHjn4oB9I4zkkRwXyn5w0tIoApPyDKIiQH7p6tHkpXFCPJTtIseQ3 VoNYFKhdpcHf9CbYy26zVGFEvyT2J19mE20vTiQrjL6CB9Vq2Zh4KkGD82MaCrPd6y9W UUtvrC92Pn2SaMErlsyPwQ0Db0yjSoKXkYH0TPgJINJmnOxNVVxCzUM3KVkk776W72n9 gwT6JZjKyO2gcBCYOp3KcN9F1c/eMnDgfxq3VKPhPILvMxpVO8cOkwe7VhMm+/YDQ2ay oHcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584810; x=1729189610; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=K6YUqJX41UvLftiTjNafBx3/10X+6KBTIzQjXduU2xw=; b=cErWaDBgHm17cQmraKeN9ggnahq19Z0D1Ltfuj37RnCe7M4Iohgf+ds24QNoLaaR/K kNVQFxKENsTno/Uqk7gKypQEqtvqlPkBgnHIbcd3zenH9LrGJl7OLN6tD7ZEroYX28ye N3b4TVWnGPYEkPEj1Br193HAp8S3VNccY3d87DGv466mt8Anbqp99R0k+WOSHNeKo+zZ aya4E+3+kL5m/JB6ndb0y1oE/W+TQT0rS7Da3DoTBqJbqu6vmq3C2y4ksf2YAm7d8yJS sjinkddcKOUFfjDJ7AlzrnsV/fvBh9RmO/V/GV1WiP8DMcCE05DwgFkjDC9nqB5kDf+N dGLw== X-Forwarded-Encrypted: i=1; AJvYcCW6FTZ2tdPrPjfgwSavOx0kJIOYNs4gc+SrvDVlOiGebQGX80ccN1ouU/7HtE1O/fQ1qJ3vQ5wdmRjEnWU=@vger.kernel.org X-Gm-Message-State: AOJu0YxvQ73w7zDgSRPbLSVpCtEjvrJAoKNHCBu4Dax5FaXN1zbCPkrD 2Y5TTCgU47NTC13DMYL0+NIGxYNANkWAu1t+XJenWBfuGm+OlpWCEg38FATBLK7y0MByPuIUlCe TYA== X-Google-Smtp-Source: AGHT+IEO1ECBC98fGvB+GRqYHAQaBdxJcqvY9DmP3QI4gZm0KdOyij1fLmFl0bPXgfvra8JJdQe8JNFxBHM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:9199:b0:71e:1e8:e337 with SMTP id d2e1a72fcca58-71e1dbe467fmr8496b3a.4.1728584809106; Thu, 10 Oct 2024 11:26:49 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:57 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-56-seanjc@google.com> Subject: [PATCH v13 55/85] KVM: arm64: Mark "struct page" pfns accessed/dirty before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark pages/folios accessed+dirty prior to dropping mmu_lock, as marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). While scary sounding, practically speaking the worst case scenario is that KVM would trigger this WARN in filemap_unaccount_folio(): /* * At this point folio must be either written or cleaned by * truncate. Dirty folio here signals a bug and loss of * unwritten data - on ordinary filesystems. * * But it's harmless on in-memory filesystems like tmpfs; and can * occur when a driver which did get_user_pages() sets page dirty * before putting it, while the inode is being finally evicted. * * Below fixes dirty accounting after removing the folio entirely * but leaves the dirty flag set: it has no effect for truncated * folio and anyway will be cleared before returning folio to * buddy allocator. */ if (WARN_ON_ONCE(folio_test_dirty(folio) && mapping_can_writeback(mapping))) folio_account_cleaned(folio, inode_to_wb(mapping->host)); KVM won't actually write memory because the stage-2 mappings are protected by the mmu_notifier, i.e. there is no risk of loss of data, even if the VM were backed by memory that needs writeback. See the link below for additional details. This will also allow converting arm64 to kvm_release_faultin_page(), which requires that mmu_lock be held (for the aforementioned reason). Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/arm64/kvm/mmu.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index dd221587fcca..ecc6c2b56c43 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1692,15 +1692,17 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, ph= ys_addr_t fault_ipa, } =20 out_unlock: + if (writable && !ret) + kvm_release_pfn_dirty(pfn); + else + kvm_release_pfn_clean(pfn); + read_unlock(&kvm->mmu_lock); =20 /* Mark the page dirty only if the fault is handled successfully */ - if (writable && !ret) { - kvm_set_pfn_dirty(pfn); + if (writable && !ret) mark_page_dirty_in_slot(kvm, memslot, gfn); - } =20 - kvm_release_pfn_clean(pfn); return ret !=3D -EAGAIN ? ret : 0; } =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 07B25208224 for ; Thu, 10 Oct 2024 18:26:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584814; cv=none; b=BYr/LDwoPOPtCVXTdCpudjXZAKBS8Pk13WKtkgZnLzXZIyYyg4QFAtQO2uyDIw+CJAsD4A2T/MXQxQPsb7qgxL63UCVwacQTT34W5IFdbKH/BVhpbVe3GxiW5lEKWl4Wop6BsObH8RHRdWmOCjGgKCWcTUzS0F11Q2RjdkxZ8+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584814; c=relaxed/simple; bh=XrFS4rtFa/FgrThsq3EMvRUnjE+/w+4Wbq1cV5vxoHg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=pssAHLKH5Gvd/aex1xoaB9ao0Y4QTy3KX7Gzl+3Xi2QeH3lPuLzSSVa7wQF1HhD8l8wNq0oAHIhWo/wdpWPloJ9cgjx3XN85ZKe/Rour6g7wX9dQC5nseFJqKQOrsv8FkXuvJKhXe00cXkDCgD02w+BQpD1QecYz4bS60NLXdKY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cMRKCJL2; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cMRKCJL2" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e2bac0c38aso1505570a91.2 for ; Thu, 10 Oct 2024 11:26:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584812; x=1729189612; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=L6S7ecEpHeP2pNYK74CcyfNjVxKIOVPI1u0giMU3AdU=; b=cMRKCJL2KTT4vd8DOQ2od82kSNUiso8aIqKq07Yvp3S6byddo0DPcg2b8Jr9ogLGAo BNTKkHdxlFw0l/CcZ5axwvEmJkR/6UB71Pb7jQUOYe4rhWAIBZzwBX4HH87G3XGLEtqO sSChKOxVtrZ3EpUVsVzWJqNdqZbLXzbA7bJvOxpoDZx9KYx9Xz26RYucJeLlSpK0us3L e20ClV/b8sQtSPdFHLtStwrnxtJUeQ5hHPWY0AHVXja2oHzcD5JA+7iFwMVByjbOQ2jD oC5gACr1JoJhr0hdDFfDr1ptb4PWqWzd1vvHuvzjVquTsBV+1uKxCjicP33d2mq+QoPY sXfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584812; x=1729189612; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=L6S7ecEpHeP2pNYK74CcyfNjVxKIOVPI1u0giMU3AdU=; b=D3TuFs6VwRk4fIbWQccqwPit5/rvkkltJDPtzcjIA7Wg8PRN5w0ZK1pLCsWDzslxnw F130pR/WdjfrWN+Ud82IiYRZlUJuAdMPC2San/njtgiNsVv2HBJamdInhgPCuiPC4PDF S7LLLwsRY49CxazhLWBu5sSE3Cp9isiFvKp+PLlGsGg+dNO9lHlKr/GWpPfgE+dICB+m bJWltyHfXpOo6kqI5RRCMTOa3avVh4JuHM+MrhGvbLyl+XL3v7+HxvOajIQd1ZWSFt3f pwItK4aG07eMEe79tncof0rU4ic1drHdbVgJXCS8fdoM+Mh2rwZbVgy4jdxPms7HQoj6 m3Wg== X-Forwarded-Encrypted: i=1; AJvYcCVh0zCQ8Ug2DIfCx1wHT3ZhAvIOpOZYFAeYzC/T08QV62oCKbWfPeDHCRLZy5CIwZwSveBepkJjfPuOKCs=@vger.kernel.org X-Gm-Message-State: AOJu0YwqCG4aoSENEUA7JPff1Ht6yTQ+l/yndjTZEpjzGTEcmyXb+7MA Z+NyGKTXWM4Qwap3QeZQK437VNM3vFB+UHJS3/QAtk5D5pv/0zjpCK8+Ncwt+cb4mukrvBHWgsB MCQ== X-Google-Smtp-Source: AGHT+IFNqFi+zU38KeaNCnXXdZA3xW66mqPsBP1LkkibZO//JxtU2HWKMaWXgt9tHH1pSDOFY+ZWY6Gk4GE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90b:4f4b:b0:2d8:bf47:947c with SMTP id 98e67ed59e1d1-2e2f0c456f8mr35a91.3.1728584812033; Thu, 10 Oct 2024 11:26:52 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:58 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-57-seanjc@google.com> Subject: [PATCH v13 56/85] KVM: arm64: Use __kvm_faultin_pfn() to handle memory aborts From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert arm64 to use __kvm_faultin_pfn()+kvm_release_faultin_page(). Three down, six to go. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/arm64/kvm/mmu.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index ecc6c2b56c43..4054356c9712 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1439,6 +1439,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys= _addr_t fault_ipa, long vma_pagesize, fault_granule; enum kvm_pgtable_prot prot =3D KVM_PGTABLE_PROT_R; struct kvm_pgtable *pgt; + struct page *page; =20 if (fault_is_perm) fault_granule =3D kvm_vcpu_trap_get_perm_fault_granule(vcpu); @@ -1560,7 +1561,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys= _addr_t fault_ipa, =20 /* * Read mmu_invalidate_seq so that KVM can detect if the results of - * vma_lookup() or __gfn_to_pfn_memslot() become stale prior to + * vma_lookup() or __kvm_faultin_pfn() become stale prior to * acquiring kvm->mmu_lock. * * Rely on mmap_read_unlock() for an implicit smp_rmb(), which pairs @@ -1569,8 +1570,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys= _addr_t fault_ipa, mmu_seq =3D vcpu->kvm->mmu_invalidate_seq; mmap_read_unlock(current->mm); =20 - pfn =3D __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - write_fault, &writable); + pfn =3D __kvm_faultin_pfn(memslot, gfn, write_fault ? FOLL_WRITE : 0, + &writable, &page); if (pfn =3D=3D KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); return 0; @@ -1583,7 +1584,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys= _addr_t fault_ipa, * If the page was identified as device early by looking at * the VMA flags, vma_pagesize is already representing the * largest quantity we can map. If instead it was mapped - * via gfn_to_pfn_prot(), vma_pagesize is set to PAGE_SIZE + * via __kvm_faultin_pfn(), vma_pagesize is set to PAGE_SIZE * and must not be upgraded. * * In both cases, we don't let transparent_hugepage_adjust() @@ -1692,11 +1693,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phy= s_addr_t fault_ipa, } =20 out_unlock: - if (writable && !ret) - kvm_release_pfn_dirty(pfn); - else - kvm_release_pfn_clean(pfn); - + kvm_release_faultin_page(kvm, page, !!ret, writable); read_unlock(&kvm->mmu_lock); =20 /* Mark the page dirty only if the fault is handled successfully */ --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 116A320822F for ; Thu, 10 Oct 2024 18:26:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584816; cv=none; b=bCTyKG3Jxq1KyX2LVkJXh5kE5z1g+XnwPCUyKvu1Tx9KjTALc7G/jJEyctVxmOPliECw6+KO7Ad2m3YdcI4ftMWMMsgCrEL7Hkq9Y3swScNL4x9kNOQLcYUjAZeuKsNUUlSdw7ldBV7X7AcY8NAWiLy0Llm5snQmzt9K7dmJWj0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584816; c=relaxed/simple; bh=tUWxw/XFi5Jjw4MLGoQx4yzkrGK2c3j8OG0Ux9/pXvk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=YSEZQ3TeJUwhlIk89XYp1FnJ+Yv7obOqEhsShzO1A1KA/I6kphGcxSLN1SySVHxfUzXCfM1UteHzPgshqNL4IhwM/PuvSausmVpF8gL2H690xbP8jv5KBrfiP9h34Mu5Z25BG3711/EKSqLKwMIYzOkjbwK5kx9DHUpXYzZ0Msw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=XJ5G74vA; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="XJ5G74vA" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e1fbe2a6b1so23290207b3.2 for ; Thu, 10 Oct 2024 11:26:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584814; x=1729189614; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9PLNR7J7/StxhsadOj2FvapRbIqIfrDfQ6nFmhEUvk8=; b=XJ5G74vA5iAj8/LHLvdWEOyxWm1sfVqNq+jfgoUcBvu61bKrqTg0Rh5GOJLcTJf+Aq TL7FokBfV0aZSwz/KOejArh1F3p2bP0Yd44s9m4WX/lwCfKaHYxqgq5J8q0UCN1NNYda BH05WWr+Tayqa620VrfEhx9mjQLJkXQDcrwZGDsqxDGFZyd04Umi094470nxsXBb69Ur klYor8P+lwqKm5AFy3yGOCs8lPPDqjyRxQmoDgihuPO+cs/d0BqSVSdu7FrlZ3DPLEyd IdLqIfUnIN/c1Gy68qnUPzDojMCUIhUQknpkbfu0TTJvkhlb2kRKb2OlFqBihneknzCP bOKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584814; x=1729189614; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9PLNR7J7/StxhsadOj2FvapRbIqIfrDfQ6nFmhEUvk8=; b=dT8JIpMmfgHcJyMvPVmeExCBs2MXoxelvJCHfpDjolL25U4hlH+ciBHJ/uQQQ+9LFe srzF1I5hyuOuzGlYhv1vJy6oHKcTAB5x0vKBtlHtaKDyzKbfGMrKnoP9ZsQMduRRqkDS sXl8iNTvKvlkadrQmC/katuIGaQnSXQttymbZyJFaUo7HAcsVZMyLtT/IcC7/NPllJql nfJhUgVeSCYRZn4vgxdLeO08vJni4qSDz28DL9JSJS66kI/esrk4QOiPSkkTQGwq6wy5 LWuGto9rY9qLaCxm8WSVxSMR8TUY0QAwhksZJV9AN7tcKosEgh/2hBDgmDST8EfM3KZj 0RUg== X-Forwarded-Encrypted: i=1; AJvYcCVfeoP+ExdN4oVSvavysQ0jH+O6X0nLo6/Cu1gT0J4hRyDu+F/fPkXCM+ctBSOHeOzz2yFDOXw5qM4/5OI=@vger.kernel.org X-Gm-Message-State: AOJu0YxfZPjR7/LV2d/gsXkpXDX6aSN6l/FpWpGAkcjwSvJVscOYDCOB pMjyhD2/O8Z+hgC6APQBFjvNvEkxr/iY9PBTy56lvlRULBERdKx5iDZw0mndPivEoSu8iQtx7+t nog== X-Google-Smtp-Source: AGHT+IFTz5/NgoL1lG2XSkf5pquU6CJl4M8l3x57mh4U7FbzzS84rW+oHIk+OXb87gKu92fZB4cgNqwOitE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:46c3:b0:6e3:21cf:a67f with SMTP id 00721157ae682-6e32242fb5emr1006557b3.7.1728584814138; Thu, 10 Oct 2024 11:26:54 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:23:59 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-58-seanjc@google.com> Subject: [PATCH v13 57/85] KVM: RISC-V: Mark "struct page" pfns dirty iff a stage-2 PTE is installed From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't mark pages dirty if KVM bails from the page fault handler without installing a stage-2 mapping, i.e. if the page is guaranteed to not be written by the guest. In addition to being a (very) minor fix, this paves the way for converting RISC-V to use kvm_release_faultin_page(). Reviewed-by: Andrew Jones Acked-by: Anup Patel Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/riscv/kvm/mmu.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index b63650f9b966..06aa5a0d056d 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -669,7 +669,6 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, goto out_unlock; =20 if (writable) { - kvm_set_pfn_dirty(hfn); mark_page_dirty(kvm, gfn); ret =3D gstage_map_page(kvm, pcache, gpa, hfn << PAGE_SHIFT, vma_pagesize, false, true); @@ -682,6 +681,9 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, kvm_err("Failed to map in G-stage\n"); =20 out_unlock: + if ((!ret || ret =3D=3D -EEXIST) && writable) + kvm_set_pfn_dirty(hfn); + spin_unlock(&kvm->mmu_lock); kvm_set_pfn_accessed(hfn); kvm_release_pfn_clean(hfn); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00F2820898E for ; Thu, 10 Oct 2024 18:26:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584818; cv=none; b=HYbZmY/jdEz/HQmv8Z0oigqiDKCtPCo1yDnd/9ziyWitdOxD0OMIKH7bbTpv+kNj6UXhmhGIaLofM9M1nvbMmGWBwA1glGCyMhO+NyL2LcjBpQdsV9FBcYX+qBP7js+oCvCy82hs9INHMx4wVRdC7P9Oiqvsr8rAi6k6S1Qahlo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584818; c=relaxed/simple; bh=XU8sCtvYc/Uw+Iuw1o18evVVUc/CvxyvropaE1RipJE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=LZIEbUQ1KYh4r0a11nHpZHWrjGkgZsIdVBxSb36/E9mJnx9XKpPuOo1OnPA/yf77lVdTDKLqy4t0toXPhBiQG1nG//H/8eRpSjZGUvIGS/apSsZe/HI1VJBshhomeM2FbiVjegFSKqJCGlB2sktct9pACBXwEv2RTAkGk4ZJt50= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=W+oxKjMU; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="W+oxKjMU" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e28eea1074dso1984130276.0 for ; Thu, 10 Oct 2024 11:26:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584816; x=1729189616; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=h7pIaRprT8i6iodquNsVL5vPZSNT9KsZLdIQsfz/EL8=; b=W+oxKjMUfyh+EqY/9j8uTawVGS+361nMyI2WTh0heIuHPPflaTW6hcERh//ZIPGINt hlypNXBr6qobPwsVdXmHX6BEqbZGa3FbwKmQ5Sd8OPggbYFVJFGhHU7MM+e8FbNNd40V 2sP5QAjfViExYotUYGsZOg6HLHcp9O01YQvah7hUFc9sAA6kvJYFAZeN1//luN4wfFDx PzSOdMsfcFKPsHiAosOR5+PBT3aMMCz3e61LQPme5dH6VbZADVeqVvCgX4TieT8OQL0u n2O7FIS+uh6/HFTeEZYPKIRdCh/EQxAPOAwyFhbsAZf74548oWBqEZcLrf0aGcNxzTXf 6CeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584816; x=1729189616; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=h7pIaRprT8i6iodquNsVL5vPZSNT9KsZLdIQsfz/EL8=; b=o7TTXV/S+CiaWqHvcMzde5SIE8O4Enof1JuMRrWqTuxWg1Hm66K5eyVa1/IBkaeEJW 1bpH2VUv0PAxYZdBuG8KoJ5/HVwnNiLYrkIqcUGBlqeWWC068tEGqwHpjuHE1zqmz21v d6O4EsSsJJ75HJdEY1Sqv12jt5h+jF0V9ZqCV1BCEqUsjta+rnAfV4ileFaOXeisOztn 1leQpfxDaLqWUFgPdm3R2f5Cucf4NI0VdnWrM3d/OdxElB83QcZerUerF09IQbACKEy9 Hi11S0O/GK3pNHPbHlxPQyay20DKB8tTSIL1fatxawkECneu6PZ7oKZds+AEqdilsFkt n29g== X-Forwarded-Encrypted: i=1; AJvYcCXMEtssU1yAPoJ2SO03s2qvJwPa7MC++WNa/X0DyFHoevNNreboJEpgpiIXPbzSXPiaQHcni5AGssP/CJk=@vger.kernel.org X-Gm-Message-State: AOJu0Yw+6s63+0xO+C7BlpsJNLYTAGNl6E+T13teJCGtz1k0ToMB37qT BO2bzWtCX/5tPz8XuMPt7VsN3J/qTdmEA50TPGEqN0HNv7zhWU2yKmcuxuo7xCWjFfkjBeS29Hr zPw== X-Google-Smtp-Source: AGHT+IER2ThMFqRrPlrH2YqE2Mm6yZZ3YtFM0VnOojG5BgkcM2HHrRnMjpl0ZvMfhTqewlD6nF2BR0rNKwA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:c45:0:b0:e0b:f69b:da30 with SMTP id 3f1490d57ef6-e28fe41d0e6mr89144276.9.1728584816073; Thu, 10 Oct 2024 11:26:56 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:00 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-59-seanjc@google.com> Subject: [PATCH v13 58/85] KVM: RISC-V: Mark "struct page" pfns accessed before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark pages accessed before dropping mmu_lock when faulting in guest memory so that RISC-V can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Marking pages accessed outside of mmu_lock is ok (not great, but safe), but marking pages _dirty_ outside of mmu_lock can make filesystems unhappy (see the link below). Do both under mmu_lock to minimize the chances of doing the wrong thing in the future. Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Reviewed-by: Andrew Jones Acked-by: Anup Patel Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/riscv/kvm/mmu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index 06aa5a0d056d..2e9aee518142 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -682,11 +682,11 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, =20 out_unlock: if ((!ret || ret =3D=3D -EEXIST) && writable) - kvm_set_pfn_dirty(hfn); + kvm_release_pfn_dirty(hfn); + else + kvm_release_pfn_clean(hfn); =20 spin_unlock(&kvm->mmu_lock); - kvm_set_pfn_accessed(hfn); - kvm_release_pfn_clean(hfn); return ret; } =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6FD2209675 for ; Thu, 10 Oct 2024 18:26:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584820; cv=none; b=Nk8U6amYeVOzZVlYE2lUfZA9MbVSvs/c1cg9d8v5pQSt/QmdOsIkvsGsgU7vkBDnYwK2Nzi0GGdMrtXYaT6oGVEq1rwVjSyNDNb7m6TqLLa2Au3Hi3lGVXwdaKveO0z7WmKNkuREx34+8vpavWQL9iyXu8Tbgb8bhKhrQ13Ndgs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584820; c=relaxed/simple; bh=h8Ty3DwV1uJmCoppjwTeaFXY32gukNMrofdCY+cnYhk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=t1Odvhv7V3WbwTIy1+1XLEE8+CK8b6rEmqVxDrGKjn4oHbUF3OQwfU7UdEak6XUBhq77tUzTtaySTtH/Cv5CASNL+VCCsPcQtckHgc0y1LU4MmpZWtUTA8J/DNbUfdYoPlTnrsagnImqeT5gEoeKyd36cV2anGUoU931Ma3HW4U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JNfpTpqi; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JNfpTpqi" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-70ac9630e3aso1196103a12.1 for ; Thu, 10 Oct 2024 11:26:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584818; x=1729189618; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OPPvMTYiUGb6pAXW02Q14XiUF2+KvArosM2sRK6VjAA=; b=JNfpTpqiBYBKFLYDya+q/+VqUsSM16GJP14qVcVJBPKXZq4oW86J7IKlLAOiNob6Px IEuSrspqaxeP+LwqTSlCTpUpi94WLf5XXZW/QoJCLOugk9sV3+x2J4yYTdzkt3tvnLQx nE5oE/+OqR1Yuki9kQowd4XCBCzHiNhnKLTalavt1s+dTpNB4LgxIrPIUAPsx+IFSoQe Dxl8Q0gJ0H3By5DV9zN8gz91BDltC3iw79l9V7QEzDzNavm6Mkl165TAaQtIf5TNC1uq y5ayftOuatk9cCD2wXS5Rbnu24zhxUq7JElq0/cz+LsUk+p71mBe6JOzZybeoA2rg/hT EvgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584818; x=1729189618; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OPPvMTYiUGb6pAXW02Q14XiUF2+KvArosM2sRK6VjAA=; b=fYYCqGIBA7ZAbVqC2PI1bW8GAJDI7kmHlb9Mx4f/IcliZe8rp3gvRdFOVzSxjBynxe NFknrA4Wbf75m9uATgFIRkl9aLVQC5cqSq4Xeyi3HpLlaD1p78OWVHXyXLcYc2mQJhD1 ApNPv6FkTgtapRhGn8uwHHUZhiDqm2XbMT3PHYvkbhb37oWkmvB9CV+RrA8zj463OCQ8 MqbdtF3+dxPAQEPUPM3F5D/1JZwKaENg2jYRVkS7i/irezkxGvye+Uk7r1kiY7qaAlHa n8lEXUcU7k1Caep1CP4V/LTZLwrFhr0Xv6PGC1pEBZhiLzyqaO65oQd9++us9GTCapTm VKJA== X-Forwarded-Encrypted: i=1; AJvYcCUd6zuST/H0D4Mv2u9Ud9yosGx7mCxjnAdJu73/isgeW7Tj6f7bgH924bgAaAjwxa9NiUtm0BpBwUNJMCY=@vger.kernel.org X-Gm-Message-State: AOJu0YywqSwJpjWT4EnInIhO005df0j2fa30/GuBwYBhRcvgiuayWgGc 0JVbsUgOKrsPUfgB6wWydHeWDQnhweHQftI4SBk+p8WBVsxD/JleBnh4dp6ShQ/hnxY48COaOmq b2g== X-Google-Smtp-Source: AGHT+IHVZkXcNH8eDb3GIQubykYaXKWHdrpGe/zHTLQHIjPMwQpAN0ra/QncFunLfkL/hgYZuPxk0qgLkH8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:2343:b0:20c:747d:ec15 with SMTP id d9443c01a7336-20c747df618mr290175ad.1.1728584817934; Thu, 10 Oct 2024 11:26:57 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:01 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-60-seanjc@google.com> Subject: [PATCH v13 59/85] KVM: RISC-V: Use kvm_faultin_pfn() when mapping pfns into the guest From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert RISC-V to __kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Opportunisticaly fix a s/priort/prior typo in the related comment. Reviewed-by: Andrew Jones Acked-by: Anup Patel Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/riscv/kvm/mmu.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index 2e9aee518142..e11ad1b616f3 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -601,6 +601,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, bool logging =3D (memslot->dirty_bitmap && !(memslot->flags & KVM_MEM_READONLY)) ? true : false; unsigned long vma_pagesize, mmu_seq; + struct page *page; =20 /* We need minimum second+third level pages */ ret =3D kvm_mmu_topup_memory_cache(pcache, gstage_pgd_levels); @@ -631,7 +632,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, =20 /* * Read mmu_invalidate_seq so that KVM can detect if the results of - * vma_lookup() or gfn_to_pfn_prot() become stale priort to acquiring + * vma_lookup() or __kvm_faultin_pfn() become stale prior to acquiring * kvm->mmu_lock. * * Rely on mmap_read_unlock() for an implicit smp_rmb(), which pairs @@ -647,7 +648,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, return -EFAULT; } =20 - hfn =3D gfn_to_pfn_prot(kvm, gfn, is_write, &writable); + hfn =3D kvm_faultin_pfn(vcpu, gfn, is_write, &writable, &page); if (hfn =3D=3D KVM_PFN_ERR_HWPOISON) { send_sig_mceerr(BUS_MCEERR_AR, (void __user *)hva, vma_pageshift, current); @@ -681,11 +682,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, kvm_err("Failed to map in G-stage\n"); =20 out_unlock: - if ((!ret || ret =3D=3D -EEXIST) && writable) - kvm_release_pfn_dirty(hfn); - else - kvm_release_pfn_clean(hfn); - + kvm_release_faultin_page(kvm, page, ret && ret !=3D -EEXIST, writable); spin_unlock(&kvm->mmu_lock); return ret; } --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C13A720967B for ; Thu, 10 Oct 2024 18:27:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584822; cv=none; b=KM0fpITGdGyQYb66CMzyF2CbK+rpTT+X0VFzWy68JbkzK8HMd6t8jEz7jVCyj5anw8u0PbhrGL708hblbnhTKuYGCvQDhji5y/ma7YOpMji2NLc9eMcOk+BMzfMJK4yEPbOUOfhHNsbAUaE8mQzWx70LUnQ5vSCvjxvc3oEelfI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584822; c=relaxed/simple; bh=dTkbtmFokyvOqUKSDyYRNtMSjRIKbeJnigSrLVN5f5s=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=TNkw4ukEQjNlXWze98zX2nAhmSxmm1U0VJ44KyCJ5kbB6xHW8qz4PFwpioZwUcz+H9oghoBF9UInIGYC/zsyA6rszZCvHsX4NQFkeYtFRoVZq5RSRD0nDaX+NquY0OAjDTHcS4B+UPU9SKQN2D9NNhFHu/aDxhtrsxExnD8ue3A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=E4/Nn2nf; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="E4/Nn2nf" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e32e8436adso19843957b3.0 for ; Thu, 10 Oct 2024 11:27:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584820; x=1729189620; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=SD6mqWUZeYJtl6GnT7U0LdjoV3gPIs/AdF7iNpavyUo=; b=E4/Nn2nftJfFfMfz7XGdS1N64U1kTObHmK9jXjNfOx0RjrtgUP73dR5xAlk3+GMtGB VyKZy9lVNuNg6pbjDWdFmORwCovT3DjZBTB+WZ837kHeHvK2326FSbxxs8lJBmA+cqfJ AzLzFEr8HhjDktjfMFXgEDl8CY1fwEV6eCqYsTZqVWyTx1h236L085DUqUUmeS6vrvHB GSntK4yQbkZEuyozHwXiNZd4wO+6Z3lbwa+JRbDcIw4FDtwcrBqi9nXBbIffNp9qA3U0 zaS4plFaV8cXO7/+pyiIv0T2C+lmVigBUlVA+qtqRH+dtAzYEjWbWCUNPXbPKT5e2PWb 3B2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584820; x=1729189620; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SD6mqWUZeYJtl6GnT7U0LdjoV3gPIs/AdF7iNpavyUo=; b=kjCSBtU/iL3ar2yYtm03UlJ6GMFwcgkEDVwPYA5EDD+lo1IYEeG6HRsZq3+xIAQqVD zk2HnatsHBWRb5enrs4CrmBKxB6j8szEiPrdKL3Wg1q0qVX3KpLrbFVjDSboA9nqcRtu ZPsrPkcKm8oZ5xHJuJs7o1ETVmujDd4+yulRgnBSbOz/kRp35d3SQ+B8EowqFFDHjYgL ZmWVipd+3YfqJuJ5Ax7iv0ePaJOqi7Oa5msnS6LezJsm67BqTZBNA+PQexSVhkM3Yjys rYFn4qvrmRqWVcJtHUXVnZ397fMRz7vFwVbr7skFKJ3wkTjf3yxFCY5TjbopcjzW+2or aqnw== X-Forwarded-Encrypted: i=1; AJvYcCWxvGtXaQeBRajscP9z4eySGRP519PtiNwHFncc74KDtvrJMeEZd2oW2dT9B6g4MblVrZ/f1OpEPf7+saM=@vger.kernel.org X-Gm-Message-State: AOJu0YwTs7KtZuzjOVWkfjY4nAn0cgJASPiNcX1KbK2ZqukMps6CxYzU OO0/zIz6SAlDQ07IKKU1WUp2SKhjN/z3jfiOPVmfHPzp8HriAKb82jzwJ33i6Lw9+e1tH9oB0Vq ujA== X-Google-Smtp-Source: AGHT+IGRQvxFZI4ySDHPZ8aV1dJi8vCYkQEt4dLiT8ndc3AvxflLIddldJBCv79vdNFr7/DaBU5iasgDdOk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:ce11:0:b0:e28:fb96:d0f4 with SMTP id 3f1490d57ef6-e28fe33505emr40781276.2.1728584819828; Thu, 10 Oct 2024 11:26:59 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:02 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-61-seanjc@google.com> Subject: [PATCH v13 60/85] KVM: PPC: Use __kvm_faultin_pfn() to handle page faults on Book3s HV From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace Book3s HV's homebrewed fault-in logic with __kvm_faultin_pfn(), which functionally does pretty much the exact same thing. Note, when the code was written, KVM indeed didn't do fast GUP without "!atomic && !async", but that has long since changed (KVM tries fast GUP for all writable mappings). Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/powerpc/kvm/book3s_64_mmu_hv.c | 25 ++++--------------------- 1 file changed, 4 insertions(+), 21 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_= 64_mmu_hv.c index 2f1d58984b41..f305395cf26e 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -603,27 +603,10 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu, write_ok =3D writing; hva =3D gfn_to_hva_memslot(memslot, gfn); =20 - /* - * Do a fast check first, since __gfn_to_pfn_memslot doesn't - * do it with !atomic && !async, which is how we call it. - * We always ask for write permission since the common case - * is that the page is writable. - */ - if (get_user_page_fast_only(hva, FOLL_WRITE, &page)) { - write_ok =3D true; - } else { - /* Call KVM generic code to do the slow-path check */ - pfn =3D __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, &write_ok); - if (is_error_noslot_pfn(pfn)) - return -EFAULT; - page =3D NULL; - if (pfn_valid(pfn)) { - page =3D pfn_to_page(pfn); - if (PageReserved(page)) - page =3D NULL; - } - } + pfn =3D __kvm_faultin_pfn(memslot, gfn, writing ? FOLL_WRITE : 0, + &write_ok, &page); + if (is_error_noslot_pfn(pfn)) + return -EFAULT; =20 /* * Read the PTE from the process' radix tree and use that --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3E52209F55 for ; Thu, 10 Oct 2024 18:27:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584824; cv=none; b=da56Sc9CpjQhAaq2jQtZ5g+AdpFkc/F7/gj0hIdtgdS1LwTD59WsYL2OTRtVTzIQk+JmkTemUNqVYsymvFTJ15Ex/snv3vExVGRrWHGfm2dD25Ut2jlnq4+u6a700IcOKfCde9Ik4EY6ZvjPiNCRsZxVmfbYBMv4NeU0fQR3muY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584824; c=relaxed/simple; bh=fHLEjIjpsClQjytVJZuAJLLMS5Q01wz8isn/MOvrB+A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=eAEw2cGJ6qmCAvJEPCvmPtGilQ+k8fiMALFJeNSQSb6sav5lHjtWLfotv3zPaumUXbNholBjh5bqoJQ+YmzfwsNSPklNwY4Up3hoQC43kjCRmYcx0vNgT2TwUsCvucMnWitNfQBwWMIqWepjY091jp+W5X0Y83RrC0uBdu3WRXw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=STJqucsL; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="STJqucsL" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e290b8b69f8so2139792276.2 for ; Thu, 10 Oct 2024 11:27:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584822; x=1729189622; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=YGpJ+GsXH4oG8p6vIrNjfshF8svSSAFECmaA34wp4no=; b=STJqucsL1kdpv+t2aaRSW8Dsd/k9W8Xwx0z/3KChOk2f7SqOVYc8Fpu9FgNXp5fiLv U2zVArrxm+zFzt6S1ZnESa2Fakw84zykbu9yrPyt4txY5HGwhVwXFA7Q0M1ej5kOsfTF cOdyNhQsGU94JfmwxOT9eZyARvp1cSidtXQIDBSmdrHrJCdXoEDgKL9knbEsHRoX8+Q7 x+Vw3mSWTwWjREDYZH57XAFRL/3RValbk71y72Jep2YJ1nJzbIRF8212ST78cHKuXiCE RyAFsegZWGTdL8tf6s6VHlUs9C4I+kaJpsg8j16L/FE68+XPw0gkSDLsBwjx5xn1Kv6y vMIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584822; x=1729189622; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YGpJ+GsXH4oG8p6vIrNjfshF8svSSAFECmaA34wp4no=; b=fXOGZvPGG1WK5TzHhYkEyBF69uEcbJAxBV4yBYn5u7DcMnNzuJNJH7MkyjK6OTRqRE ll0vJMNZUeLtbwJWQDliw3npZnNpv8oEecnSJ5vRizE6eZZzcODUEUGTw6Q7vVhkim5i 1YF4NKTxalw4IxDLEraZJ5GYKO5fu5AneoewW9TjwS0Z/W95BprBeynQCDr0f2+DClDi M/tA7oHw8iJpC1ClhWZVPWr7eO/TRT9QXdrKOl6Xvy2Ys6HpwZesMKGfdFfO5ajhmtSS pe5piYKzEP25Amub4QAUp9m2w0G6Z/oFHTJZlFdvdiBnthhDoguJtU9jRF4AdI9PGRG6 iiIg== X-Forwarded-Encrypted: i=1; AJvYcCWM/ii0Nq53JqbL+jqAaAVgy4cVVhv1FHpJl268gWQ7vTsiJ8RXK2OyNDyA1VWVj+pVke7KOOQ8fPHAn74=@vger.kernel.org X-Gm-Message-State: AOJu0YxLy5A/ZwNeRxmETxhQkZ/R6d8xEcMG2vvbNmgYB6B2H4ULtbBW g/1ConXN4fT2Uihx+uXg7Ntx0Hde98aJhnJccJaKjo3wsyOABwcUewkHPqQbCVjWaNdw2BK51nJ 2yg== X-Google-Smtp-Source: AGHT+IFyzgxXPfejJoY0SZCQ3T+Vo8hzb1HJlB5l94FFDfZID39IeT9MHN/ZgR4beqIvxQ/gO8qC3c4Qafk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6902:4ac:b0:e28:fdfc:b788 with SMTP id 3f1490d57ef6-e28fe4426b8mr4979276.9.1728584821558; Thu, 10 Oct 2024 11:27:01 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:03 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-62-seanjc@google.com> Subject: [PATCH v13 61/85] KVM: PPC: Use __kvm_faultin_pfn() to handle page faults on Book3s Radix From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace Book3s Radix's homebrewed (read: copy+pasted) fault-in logic with __kvm_faultin_pfn(), which functionally does pretty much the exact same thing. Note, when the code was written, KVM indeed didn't do fast GUP without "!atomic && !async", but that has long since changed (KVM tries fast GUP for all writable mappings). Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/powerpc/kvm/book3s_64_mmu_radix.c | 29 +++++--------------------- 1 file changed, 5 insertions(+), 24 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book= 3s_64_mmu_radix.c index 8304b6f8fe45..14891d0a3b73 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -829,40 +829,21 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *v= cpu, unsigned long mmu_seq; unsigned long hva, gfn =3D gpa >> PAGE_SHIFT; bool upgrade_write =3D false; - bool *upgrade_p =3D &upgrade_write; pte_t pte, *ptep; unsigned int shift, level; int ret; bool large_enable; + kvm_pfn_t pfn; =20 /* used to check for invalidations in progress */ mmu_seq =3D kvm->mmu_invalidate_seq; smp_rmb(); =20 - /* - * Do a fast check first, since __gfn_to_pfn_memslot doesn't - * do it with !atomic && !async, which is how we call it. - * We always ask for write permission since the common case - * is that the page is writable. - */ hva =3D gfn_to_hva_memslot(memslot, gfn); - if (!kvm_ro && get_user_page_fast_only(hva, FOLL_WRITE, &page)) { - upgrade_write =3D true; - } else { - unsigned long pfn; - - /* Call KVM generic code to do the slow-path check */ - pfn =3D __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, upgrade_p); - if (is_error_noslot_pfn(pfn)) - return -EFAULT; - page =3D NULL; - if (pfn_valid(pfn)) { - page =3D pfn_to_page(pfn); - if (PageReserved(page)) - page =3D NULL; - } - } + pfn =3D __kvm_faultin_pfn(memslot, gfn, writing ? FOLL_WRITE : 0, + &upgrade_write, &page); + if (is_error_noslot_pfn(pfn)) + return -EFAULT; =20 /* * Read the PTE from the process' radix tree and use that --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75CD520A5DA for ; Thu, 10 Oct 2024 18:27:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584825; cv=none; b=TdTPzSQ/vLl+pOE0ZwPwzKylCDMzZU011exeOOOyaMnyw7J4q6c8JW+Ww3c7Y1tdWjyWQppspbYiQ5IjBXikbRX6CPleBtYLo29cOGDBxN90EE+aAHD/Nv24mRTxgCSgZFYF+/eTQjbI64b8pK2cYpBv9UTwMIcT6AmBDCmcvgU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584825; c=relaxed/simple; bh=lnamoAhiaB2k3D/cv0luJANVeCRwGvcQfaabdaB/y4g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=F2U+uFv5g1dkR6yQS6W7rkasMJizqPxJvGscx0dLuKc9ZjD8q/+Wpwpgk8EWJYKbiGmTO8bfi3BIPwoPPgiNkGQ+srswKNf6d94p15XPlCi6w4F7kELTe3jCyoSG8OCYgNJBolK609aL/E16v6O28TRQkv5xMm8wdxzjGcpW0mU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bjTwwhuo; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bjTwwhuo" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71e1ed58e9bso1479113b3a.0 for ; Thu, 10 Oct 2024 11:27:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584824; x=1729189624; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=8JV5lSZeo1By1ou6uwDGqoH8kmaYenoeIxSHz9El/d0=; b=bjTwwhuoN0SpvNaeySCVuMnmyKMqY2y7VhEvpQdH8Tydsmp9oc3ImEzNIB7x2f58Vc LW75/zBKZ8Dtw2BE5RKmttfdjQIE7EcP0Et20wml0hzKLI3e8x/5FaIzoiMPPwoGie9t 02kIjtfFUWb/70njpoTggzuy3qcK/ZCuTPOg9Uy3fR3Pn5LLKSSXdxVywqZg7ZuphLmL dJqWYhncEpAB9sXQgmaXSJ7FaOaCiNrjHLf7jQsWXNhvERuNF0jIbJ84cEpV6XNBHYNv Z4H1YTKXoiAb/B1jipjTL2HYW90e2uWFVEIFwneMeDCmQziwMbpeaLSFC2sEZceI1UsL NTaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584824; x=1729189624; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8JV5lSZeo1By1ou6uwDGqoH8kmaYenoeIxSHz9El/d0=; b=OBdkDZxoXBLTF4YHsnMTRHd+tpTkj6rBZY+JiJBl0kkSOFKrkPncBy12CT6QZPJnDA ET/Zg/tboPKqZq9FtyAIKH8gR+7FgF+ZZ08bTRNGivzqodpGPD3T6KdZnydE66Bt1czN x/80Y1VEJaiLRdOm5lxi5vT0iVJh9lE0B9BtwtCiCvMgpz0DKclejHc1F5dCnq1nGr9N 3aIdoEc0vTw4f0YhL4lbgXTwKyY/RQpSgwDUyvJa43W3yK2BoTQ/+b/5ux+UHyWaQZxq OBDD9n8Jaj4VijJ9FB3qqFWF1BWcFJY0hosOVmbHhDh1QncNgWGdtGJwnHtdCgNBmebU AeqA== X-Forwarded-Encrypted: i=1; AJvYcCWZCkf3Frj/JPDqN2uGmo4WR2vPxuhnd4En8JQ4KRISaOArT25gLHWe9H7Tm/asG8GxCN3tWE+D895KX0U=@vger.kernel.org X-Gm-Message-State: AOJu0YxaJWeNsjJ7GTljyVdV2Spj5+cAN3m3EbOuvQGdIAn1nS1xGinw R5YknR9idyn2hPHHVlY/9Uad0RaH/aPOY1VIptsibGpHyxNnTX0Jen0vUL+BlAeLuzZXhG8543P SbQ== X-Google-Smtp-Source: AGHT+IFev0i08WnhWXOHYVCiDa8hDSrBgilu3q+XI+BXP7ns5qqyHbYBrSKz+aJVjoCmfpAuiTEXvCWN4KQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2daa:b0:71e:401:6580 with SMTP id d2e1a72fcca58-71e1dbf2587mr27202b3a.6.1728584823552; Thu, 10 Oct 2024 11:27:03 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:04 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-63-seanjc@google.com> Subject: [PATCH v13 62/85] KVM: PPC: Drop unused @kvm_ro param from kvmppc_book3s_instantiate_page() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop @kvm_ro from kvmppc_book3s_instantiate_page() as it is now only written, and never read. No functional change intended. Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/powerpc/include/asm/kvm_book3s.h | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 6 ++---- arch/powerpc/kvm/book3s_hv_nested.c | 4 +--- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/a= sm/kvm_book3s.h index 10618622d7ef..3d289dbe3982 100644 --- a/arch/powerpc/include/asm/kvm_book3s.h +++ b/arch/powerpc/include/asm/kvm_book3s.h @@ -203,7 +203,7 @@ extern bool kvmppc_hv_handle_set_rc(struct kvm *kvm, bo= ol nested, extern int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long gpa, struct kvm_memory_slot *memslot, - bool writing, bool kvm_ro, + bool writing, pte_t *inserted_pte, unsigned int *levelp); extern int kvmppc_init_vm_radix(struct kvm *kvm); extern void kvmppc_free_radix(struct kvm *kvm); diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book= 3s_64_mmu_radix.c index 14891d0a3b73..b3e6e73d6a08 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -821,7 +821,7 @@ bool kvmppc_hv_handle_set_rc(struct kvm *kvm, bool nest= ed, bool writing, int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long gpa, struct kvm_memory_slot *memslot, - bool writing, bool kvm_ro, + bool writing, pte_t *inserted_pte, unsigned int *levelp) { struct kvm *kvm =3D vcpu->kvm; @@ -931,7 +931,6 @@ int kvmppc_book3s_radix_page_fault(struct kvm_vcpu *vcp= u, struct kvm_memory_slot *memslot; long ret; bool writing =3D !!(dsisr & DSISR_ISSTORE); - bool kvm_ro =3D false; =20 /* Check for unusual errors */ if (dsisr & DSISR_UNSUPP_MMU) { @@ -984,7 +983,6 @@ int kvmppc_book3s_radix_page_fault(struct kvm_vcpu *vcp= u, ea, DSISR_ISSTORE | DSISR_PROTFAULT); return RESUME_GUEST; } - kvm_ro =3D true; } =20 /* Failed to set the reference/change bits */ @@ -1002,7 +1000,7 @@ int kvmppc_book3s_radix_page_fault(struct kvm_vcpu *v= cpu, =20 /* Try to insert a pte */ ret =3D kvmppc_book3s_instantiate_page(vcpu, gpa, memslot, writing, - kvm_ro, NULL, NULL); + NULL, NULL); =20 if (ret =3D=3D 0 || ret =3D=3D -EAGAIN) ret =3D RESUME_GUEST; diff --git a/arch/powerpc/kvm/book3s_hv_nested.c b/arch/powerpc/kvm/book3s_= hv_nested.c index 05f5220960c6..771173509617 100644 --- a/arch/powerpc/kvm/book3s_hv_nested.c +++ b/arch/powerpc/kvm/book3s_hv_nested.c @@ -1527,7 +1527,6 @@ static long int __kvmhv_nested_page_fault(struct kvm_= vcpu *vcpu, unsigned long n_gpa, gpa, gfn, perm =3D 0UL; unsigned int shift, l1_shift, level; bool writing =3D !!(dsisr & DSISR_ISSTORE); - bool kvm_ro =3D false; long int ret; =20 if (!gp->l1_gr_to_hr) { @@ -1607,7 +1606,6 @@ static long int __kvmhv_nested_page_fault(struct kvm_= vcpu *vcpu, ea, DSISR_ISSTORE | DSISR_PROTFAULT); return RESUME_GUEST; } - kvm_ro =3D true; } =20 /* 2. Find the host pte for this L1 guest real address */ @@ -1629,7 +1627,7 @@ static long int __kvmhv_nested_page_fault(struct kvm_= vcpu *vcpu, if (!pte_present(pte) || (writing && !(pte_val(pte) & _PAGE_WRITE))) { /* No suitable pte found -> try to insert a mapping */ ret =3D kvmppc_book3s_instantiate_page(vcpu, gpa, memslot, - writing, kvm_ro, &pte, &level); + writing, &pte, &level); if (ret =3D=3D -EAGAIN) return RESUME_GUEST; else if (ret) --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65EC81CEEA1 for ; Thu, 10 Oct 2024 18:27:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584829; cv=none; b=l84IiBULjoQldKWhEmEytQLVBlbCesmSPUHQty/pu/QHAKXOIU/me6j5mDxUtXIHc9BgVheo8FESzdxyapqzrmj68m8/05eqDzubkz3SVyJCldWs8tweCwpYUXRvATV4M68LLiwkRlfAH3Rf3U87gC5xeVzcAsT/lb3QWATdeVA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584829; c=relaxed/simple; bh=PLrYiBlw82beEM6tLfodLLTPBglt9VZ+TVoJtuJPFlg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oE4lrELYdAHPWNa3FF9fYOD6pG3eEjiQq4NCGEz8S5g+Mt/vKHzbpSmjpg4NaRnLaWrx3+DcdUd6jZLPhDWRKbwr/UW+TOiVwXFG0/TBuG8tfO+/E76trQQ5xxPbzrvYG9Sr46KjtLzsThNv8PULbRYoaBXgJKfps2jnQxnzQlo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ooDlC/pc; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ooDlC/pc" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7e9f9a927a5so1044403a12.2 for ; Thu, 10 Oct 2024 11:27:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584826; x=1729189626; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=I+2gp5vLlKRJWMtv7YF+IvP9h9YjJXEpCIK4AMigCis=; b=ooDlC/pcgIiD9tsIsdI/XR8hhjsKnl/ubOYfWwqTcVLjG/9/GB7swjVJDrSqoLoWOw 7vkXEltPbHGSo6eNQSnfEV9KrL1bmyzVtt77RGsOKB4v9yqQdHB78U1BuZ/hIDzG2FZx cpC29Q8dE35bdHWwfpE6KKYut07InE+/YOzDqpcW4mYYCpTnffL1vL41QmzJeXlv5JSV Vy6UxPe1LGdKUb6imMlvnN/L7ACFiPOGAu0qVfgVxbFsbFAK29wlRNmnvUlIrLTo62bY 1kZmPlVjpXSazX1NzmqGpjSByYskWop3ZgGxIZFVYyv/jRa6XYdaRoFScRBxFJa6oghl JhXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584826; x=1729189626; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=I+2gp5vLlKRJWMtv7YF+IvP9h9YjJXEpCIK4AMigCis=; b=wJ6BMWggK+RrZ14HapqEy+JjYeGJKkEFgUMillhr7alttgF5KHVA1NSaFE1N/PpaOA og5kzC3IoSNVbcVVW00dcivMLYR2bf066b9QsCj+0bZ3oR0W3PY8iylPObjh8u0jq8wm 56XPOe4PTfmAMymQPmsvbAczrPYCHfJenXNlT812tjFyQM0CppT3V2tHpT5I+fFkLaXK W+8fV+KjUXt6DKya5i4QUw2bYjgqbIoTOJNh+mu7m5lVAt8jykr+f9bn+Wd1M4q4Tb4X wCidJDrJv461aFiWNn2LKHvtx4yNz8y8gJ2f7H3UnhezdJghSmhZSsDiBeXh3dsRBCd/ Gl1w== X-Forwarded-Encrypted: i=1; AJvYcCXen0XiX2pguQZ1uqLf9IJ3chtL7qlzCBqFJ3INbFBnL6qY/aJjDHddver6yc6QVK8VU7wUFRj2fepMcg4=@vger.kernel.org X-Gm-Message-State: AOJu0YxDJBXL1lb6hXZg83LqVF248DjRCVY0iH0LDjmLxi3WYhoqjTJj uuPKLIf+geMq4ywtpLA8DdFCZ3UXiDIRcJYsbCHFUjpW0rvBt9CWMcppjdlFC+ZIkpEOq2c2kr1 W4A== X-Google-Smtp-Source: AGHT+IEBzxfSdV5URAWtipQSxApJSdWqJAcjcFibIBQs32g4G2eVPMYjVJgo5zFwBPuTEySHq4oumcdrCOE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:e546:0:b0:684:6543:719 with SMTP id 41be03b00d2f7-7ea535307afmr40a12.4.1728584825522; Thu, 10 Oct 2024 11:27:05 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:05 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-64-seanjc@google.com> Subject: [PATCH v13 63/85] KVM: PPC: Book3S: Mark "struct page" pfns dirty/accessed after installing PTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark pages/folios dirty/accessed after installing a PTE, and more specifically after acquiring mmu_lock and checking for an mmu_notifier invalidation. Marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). See the link below for details. This will also allow converting Book3S to kvm_release_faultin_page(), which requires that mmu_lock be held (for the aforementioned reason). Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/powerpc/kvm/book3s_64_mmu_host.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3= s_64_mmu_host.c index bc6a381b5346..d0e4f7bbdc3d 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_host.c +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c @@ -121,13 +121,10 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct= kvmppc_pte *orig_pte, =20 vpn =3D hpt_vpn(orig_pte->eaddr, map->host_vsid, MMU_SEGSIZE_256M); =20 - kvm_set_pfn_accessed(pfn); if (!orig_pte->may_write || !writable) rflags |=3D PP_RXRX; - else { + else mark_page_dirty(vcpu->kvm, gfn); - kvm_set_pfn_dirty(pfn); - } =20 if (!orig_pte->may_execute) rflags |=3D HPTE_R_N; @@ -202,8 +199,11 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct = kvmppc_pte *orig_pte, } =20 out_unlock: + if (!orig_pte->may_write || !writable) + kvm_release_pfn_clean(pfn); + else + kvm_release_pfn_dirty(pfn); spin_unlock(&kvm->mmu_lock); - kvm_release_pfn_clean(pfn); if (cpte) kvmppc_mmu_hpte_cache_free(cpte); =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8EFCE20ADFF for ; Thu, 10 Oct 2024 18:27:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584830; cv=none; b=qpPqIrCg7IMOB99AH/EMw8cjOZ1vHNiuGAIMSy9nXaKbRj/yET424E6eOpIkyHl6XPnj+L8W5YT1sr3Q6e9d9YEXLtVe9jZ+srWydqPXQFMGe9YjaNZS0cFWHFrYcrtWZ4/QkGV3GrlwAP+AywBWL36ojVzf4q6Bx/r/vAJIRpY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584830; c=relaxed/simple; bh=2hWIRT2fMV5KiR0O+VosZUqCujtj2BkFwotH6/Eh4cU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sSyq6jZxmPCESjYWutrINhSsqx4Lq9ZDFCSLqIQfk0RUUMcTxOwjjBxcHpIvJIatVcqqD/FbB05fJlL/rEX0HK1vNJyfTzIBYwNnDClcle+kj+7ccZKBi86GCWdxl8OM/Z3E47I6ZOdCDm35F8RU39gaZ8SsB1JuViqHYcPrV/8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=G6z4vMP9; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="G6z4vMP9" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e291426b528so900313276.3 for ; Thu, 10 Oct 2024 11:27:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584827; x=1729189627; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=dJnHBDyCwUFZKgDYTVV2s9CsyA2Ll45yxWv/Qp8Ijpw=; b=G6z4vMP9tzYY14hXAhxfo/HfE8ULyiyaSY7AT4hs+6XX7YyXXgEe6O37wq9BBRAEHY JjYKYtqD1iWPnNOP0QUJOqkipQau8FTv+SOCs4bSJZ/+xsRiTlTk8hAtWgVqN39tY9wb 5wuQAD6Me87ITDD+d5kwWjudoFu47bo9mH4wNF57jSfLISkmDJFq+8MNbu+tNL3vkKn2 v2ApXZjUUxYtS4slwS0/LY1XeiQ/zBTeCGV9GgWF21giZfBEjJ4CFOW4/P1yJbjy43hn 4G0JRBGtzbJl4nPCqsics+FV23cU7ErCcRDF50f3YIrP8/H28BTAqVBAOIwwOx4dDcxN NaNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584827; x=1729189627; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dJnHBDyCwUFZKgDYTVV2s9CsyA2Ll45yxWv/Qp8Ijpw=; b=S4mn5KuK1O0qKpBRZqa3ngz/hGF7DfhHSQftYVzSrI65FaToR9a2HvRRqgYFIs16MB tPkONo5IS+EutVdAMm/VaQp+nazm94F1+CVkQVJyBvaLONx8wzpGL0THvSDJcFH5d83B K9AGlkwPO6aETM+LebnNO9C/hgjG92dI9UhMV3nVyyvBGzoN1IDCEZzNdGmuLNG/290z Mk4QngWvYR6YyLTx9NXCyh0h68JG8BvZtiAl9bjYy/uvyMzxptt2WRSt+LnejHVhGMAp d8U9rAvlYMo95CcB9ICaXTQcIPhc5yjt70DrVzwNaymLU6HWHNAN40fcgPN/wvo0bIkK z/Dg== X-Forwarded-Encrypted: i=1; AJvYcCX7yT/6kTc3yyZJMD8hhLLuYWkf7QGILpYG1uSuDUjsJxwv1eg6TyEt/S8ZPsfkwEoRhNcTPxYoCtALw/k=@vger.kernel.org X-Gm-Message-State: AOJu0YybD16IQDTqRMVjVuELPfU+5y4brZ5FeLV9FX+1XG+z//CqHajL KHoE5sdQUxtJD2+OfPhAq3Yp7HsD/1IQuGHzDE1SQiM6hyO7yBlHSntQ20T13ldBhRIsnFjHIPX TLg== X-Google-Smtp-Source: AGHT+IE3wQG+AmAWHFVOEkWtTHcZkQX/AbMzk25RszCXWWBD0mfEZbS2aMxfU1imj+f8RQ2cZBD1mpHhlQw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:a07:0:b0:e28:ef8f:7423 with SMTP id 3f1490d57ef6-e28fe355de0mr48920276.4.1728584827639; Thu, 10 Oct 2024 11:27:07 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:06 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-65-seanjc@google.com> Subject: [PATCH v13 64/85] KVM: PPC: Use kvm_faultin_pfn() to handle page faults on Book3s PR From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert Book3S PR to __kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/powerpc/include/asm/kvm_book3s.h | 2 +- arch/powerpc/kvm/book3s.c | 7 ++++--- arch/powerpc/kvm/book3s_32_mmu_host.c | 7 ++++--- arch/powerpc/kvm/book3s_64_mmu_host.c | 10 +++++----- 4 files changed, 14 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/a= sm/kvm_book3s.h index 3d289dbe3982..e1ff291ba891 100644 --- a/arch/powerpc/include/asm/kvm_book3s.h +++ b/arch/powerpc/include/asm/kvm_book3s.h @@ -235,7 +235,7 @@ extern void kvmppc_set_bat(struct kvm_vcpu *vcpu, struc= t kvmppc_bat *bat, extern void kvmppc_giveup_ext(struct kvm_vcpu *vcpu, ulong msr); extern int kvmppc_emulate_paired_single(struct kvm_vcpu *vcpu); extern kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gpa_t gpa, - bool writing, bool *writable); + bool writing, bool *writable, struct page **page); extern void kvmppc_add_revmap_chain(struct kvm *kvm, struct revmap_entry *= rev, unsigned long *rmap, long pte_index, int realmode); extern void kvmppc_update_dirty_map(const struct kvm_memory_slot *memslot, diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c index ff6c38373957..d79c5d1098c0 100644 --- a/arch/powerpc/kvm/book3s.c +++ b/arch/powerpc/kvm/book3s.c @@ -422,7 +422,7 @@ int kvmppc_core_prepare_to_enter(struct kvm_vcpu *vcpu) EXPORT_SYMBOL_GPL(kvmppc_core_prepare_to_enter); =20 kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gpa_t gpa, bool writing, - bool *writable) + bool *writable, struct page **page) { ulong mp_pa =3D vcpu->arch.magic_page_pa & KVM_PAM; gfn_t gfn =3D gpa >> PAGE_SHIFT; @@ -437,13 +437,14 @@ kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gp= a_t gpa, bool writing, kvm_pfn_t pfn; =20 pfn =3D (kvm_pfn_t)virt_to_phys((void*)shared_page) >> PAGE_SHIFT; - get_page(pfn_to_page(pfn)); + *page =3D pfn_to_page(pfn); + get_page(*page); if (writable) *writable =3D true; return pfn; } =20 - return gfn_to_pfn_prot(vcpu->kvm, gfn, writing, writable); + return kvm_faultin_pfn(vcpu, gfn, writing, writable, page); } EXPORT_SYMBOL_GPL(kvmppc_gpa_to_pfn); =20 diff --git a/arch/powerpc/kvm/book3s_32_mmu_host.c b/arch/powerpc/kvm/book3= s_32_mmu_host.c index 4b3a8d80cfa3..5b7212edbb13 100644 --- a/arch/powerpc/kvm/book3s_32_mmu_host.c +++ b/arch/powerpc/kvm/book3s_32_mmu_host.c @@ -130,6 +130,7 @@ extern char etext[]; int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, bool iswrite) { + struct page *page; kvm_pfn_t hpaddr; u64 vpn; u64 vsid; @@ -145,7 +146,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct k= vmppc_pte *orig_pte, bool writable; =20 /* Get host physical address for gpa */ - hpaddr =3D kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable); + hpaddr =3D kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable, &= page); if (is_error_noslot_pfn(hpaddr)) { printk(KERN_INFO "Couldn't get guest page for gpa %lx!\n", orig_pte->raddr); @@ -232,7 +233,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct k= vmppc_pte *orig_pte, =20 pte =3D kvmppc_mmu_hpte_cache_next(vcpu); if (!pte) { - kvm_release_pfn_clean(hpaddr >> PAGE_SHIFT); + kvm_release_page_unused(page); r =3D -EAGAIN; goto out; } @@ -250,7 +251,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct k= vmppc_pte *orig_pte, =20 kvmppc_mmu_hpte_cache_map(vcpu, pte); =20 - kvm_release_pfn_clean(hpaddr >> PAGE_SHIFT); + kvm_release_page_clean(page); out: return r; } diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3= s_64_mmu_host.c index d0e4f7bbdc3d..be20aee6fd7d 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_host.c +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c @@ -88,13 +88,14 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct k= vmppc_pte *orig_pte, struct hpte_cache *cpte; unsigned long gfn =3D orig_pte->raddr >> PAGE_SHIFT; unsigned long pfn; + struct page *page; =20 /* used to check for invalidations in progress */ mmu_seq =3D kvm->mmu_invalidate_seq; smp_rmb(); =20 /* Get host physical address for gpa */ - pfn =3D kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable); + pfn =3D kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable, &pag= e); if (is_error_noslot_pfn(pfn)) { printk(KERN_INFO "Couldn't get guest page for gpa %lx!\n", orig_pte->raddr); @@ -199,10 +200,9 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct = kvmppc_pte *orig_pte, } =20 out_unlock: - if (!orig_pte->may_write || !writable) - kvm_release_pfn_clean(pfn); - else - kvm_release_pfn_dirty(pfn); + /* FIXME: Don't unconditionally pass unused=3Dfalse. */ + kvm_release_faultin_page(kvm, page, false, + orig_pte->may_write && writable); spin_unlock(&kvm->mmu_lock); if (cpte) kvmppc_mmu_hpte_cache_free(cpte); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D9C620B1FC for ; Thu, 10 Oct 2024 18:27:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584831; cv=none; b=pNFuwkhhgT+ICneA4/ZAy32ga/1doh4yH+YIRMPYcamcA7L5TwVwU5TJyerGEPT/QzpZBk4prYAwNV58G8CpRyF3MO+QbSNVda8oDCGe9fVaTO0vPgwioZZI824LxWSA6bfs8XUObN8nGWS+JRNcvtwH3b5xGzgeiOOaokkPJ88= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584831; c=relaxed/simple; bh=ffJ+Y8PQp+slQQo0S2lzOgDI6zSQ/cN8IdVD7ZeuwkM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=WkwvcP2eNh7t3B3wVFxHm91r2l/yg5xQZc3mrWjzjHrj7OvJc1q63kSi0eEJxUWCZEOeI7UKZKbKpi6U1Z94iwe36uQwXa5u4Befnwm2ePFbjk8hfmAijsflZZHd479IujQJrJi7uDhXTRSdZ9EbR3aaIYii208v2lHP5zspU2M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SNuiX9KG; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SNuiX9KG" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7ea0d46d1f0so1514632a12.2 for ; Thu, 10 Oct 2024 11:27:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584829; x=1729189629; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=0kIT1cNFK/xt59MDB/Io88ATgyMybLemEwY+DfOgUM8=; b=SNuiX9KGV9gd57zGcX+khtxMQ/WZyfcI8IOEqbUSxhtcMIlWJDhUZzOYw3O/Jtp5tp 1NWu+kxo5Hi64N0d8Vocbim1DEDIN7pKYEMAT7CmUIRPux5M5gBhtwooiZa036V75h+k Ychu0Yrt+CY1DTpdXIkHOsKY0bGLGDYVFrglD82TfIlMDlvD77LbhWwQE2/GuTcCLoye BlO64+wQ9ljMZU5XEby944HMtp4M+bmKjEJdM7giaFb7OEfLYmaZJu6Xf8a4cSqRhdfW SsOXG0o6p9nmOEvzhsK039Qzqg4ApQUtgbICzcD04kyCCbohbJE5YCYQwAAz1ROtSzRt QapA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584829; x=1729189629; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0kIT1cNFK/xt59MDB/Io88ATgyMybLemEwY+DfOgUM8=; b=ju83vDWpZxHVX9PFP35mSAs9QxcllP3jtb5FpXD2m3dESVkXG6qTyumF2RK4OhYpn0 oBqdluUO89vQuzo07N0yiBNX2H2a9DiukAYH9pHRlC+dfl8X7oZ2aBRd4di+vrxCmr7f 5sjJSMWxjdVV/RO7/Ac9JLUdYwZ2BANyablRnGWAVOuvH2djYoZO+4QHyKFrMhPoZoeM kZ8b445cO3Kq8IlKDNscJE4YKZjvItfje+doUVL/O/coVzROyYG1Q7j9/I/G+6DcPprY aqXLywHBArLkeZlPryp8kce9DuGZquGyPeqMFm5l7deSGl9V3d+J47AOgObKMASFoUOe Pm+A== X-Forwarded-Encrypted: i=1; AJvYcCXK9a0YCzWHoBcBXz8wwKmGup0sbHAyHyOL033K79Sq0AZZ/p2kWFSMdRPGC18AvL8R/0LPRKvlaKM1qC4=@vger.kernel.org X-Gm-Message-State: AOJu0YyjO261T3zDq4F0rIvEd/J/PBJKONUVnNlKfPzPut/E/06CKiE9 LvikAt0VxE+9rMtYUe9DB/9Ejuu+KN83hYZ/3EBtBEEQfycnwJxMnKWTDnDNaTXWSFFGjK6w8Sp KTg== X-Google-Smtp-Source: AGHT+IEVjHGN/ZoxQo+Z413816spxD1Vig97FmVkGQ57WYZx6k/7CY2I8TFVs2SPVm+Oq4UkGx7TgXf3+Fw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:192:b0:20b:bd8d:427a with SMTP id d9443c01a7336-20c6377ab7bmr1062205ad.5.1728584829203; Thu, 10 Oct 2024 11:27:09 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:07 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-66-seanjc@google.com> Subject: [PATCH v13 65/85] KVM: LoongArch: Mark "struct page" pfns dirty only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark pages/folios dirty only the slow page fault path, i.e. only when mmu_lock is held and the operation is mmu_notifier-protected, as marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). See the link below for details. Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Reviewed-by: Bibo Mao Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/loongarch/kvm/mmu.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 28681dfb4b85..cc2a5f289b14 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -608,13 +608,13 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, u= nsigned long gpa, bool writ if (kvm_pte_young(changed)) kvm_set_pfn_accessed(pfn); =20 - if (kvm_pte_dirty(changed)) { - mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); - } if (page) put_page(page); } + + if (kvm_pte_dirty(changed)) + mark_page_dirty(kvm, gfn); + return ret; out: spin_unlock(&kvm->mmu_lock); @@ -915,12 +915,14 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsign= ed long gpa, bool write) else ++kvm->stat.pages; kvm_set_pte(ptep, new_pte); - spin_unlock(&kvm->mmu_lock); =20 - if (prot_bits & _PAGE_DIRTY) { - mark_page_dirty_in_slot(kvm, memslot, gfn); + if (writeable) kvm_set_pfn_dirty(pfn); - } + + spin_unlock(&kvm->mmu_lock); + + if (prot_bits & _PAGE_DIRTY) + mark_page_dirty_in_slot(kvm, memslot, gfn); =20 kvm_release_pfn_clean(pfn); out: --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 45D3920C469 for ; Thu, 10 Oct 2024 18:27:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584834; cv=none; b=e8NO7DOoGB7g4UEGRF0bIdCqGQB1eKReJVkjEsDSL1DKFQ+MRaUmwP33uFuIDiOlQmJDcqPsRS2nBw5BHWHs3mxUi59+GAtyl6CgEyK8hWIfRDqloTbq6eYkP6zJPwRPizAF2S9rAocLD6i7iK3HLd/nvff6cv/y4wolfIeDxSQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584834; c=relaxed/simple; bh=Gq0ZEmsgjgD9l+p+SR2hvcEYSP8z96GfO+zPwvkDX5Y=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hWGaUeAcW4WqYi3A1QNYq8NOdLJohopRrX7qv29TwgbRq3oLVGmBYAicMgzi+BnnYuNxOlSK0vsaLYZElfwZHXMT2xow/Ed03laIq33MacTo+Gfdq6RyajL4EXl8NQsoxunhhs1JEaCx2JoJMG7BCvXyXLk1B3n6gl8JCWTpVEg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=EfkfFSQF; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="EfkfFSQF" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e02b5792baaso1677287276.2 for ; Thu, 10 Oct 2024 11:27:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584831; x=1729189631; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=2XctyNuj5rmp2glWM4MaDBrVp36o24kp0nhTjUz4HGM=; b=EfkfFSQF3X+VfzLOUd+NMMcl3GFvR0O6RLsG8kW9/3abGsJmogSRIrtQpV+OWWRYNJ GPPNPPUNAXbsIX23X5G7HUZQZMe1WBRNFQm0FfhGz6XCl6mJGiQ4ZSYhxhO7CPQx+8rt Vkr+5ZJ+ns3A7/GgnaSxX6V/Azj+OACWU++fMoUbLZtZNL2ustwBKTJDWDypFdll23fy MW3rG+4rwyG1DZ5JU3s8eppFPrGqE5zTUyzexpo9UP6Hmnrdezl7ABJ1sGb0IFTAI+zz SNg9Wa77JEnsbl4/wjMJHG/BXG4AFPmPEmio8T5u6AeNKeqBwqQn2ivs9Vc9+kwrlQHL nxTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584831; x=1729189631; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2XctyNuj5rmp2glWM4MaDBrVp36o24kp0nhTjUz4HGM=; b=JZO3VOlw0astBGy1mfBCdcN6T+7eI3nXpEbfbvaG0ZMKxbj3E9g2LazwTRRsoxdq6T NR+3TPcxmM/2XbNOyzUJzVnkURqOeYBaSlA3p/Wij7KNjdJAFgrk/Lv+7cdBB0HzzrKw yiJxPgc8UNRQFaTz1UXD3LdhyVYE43RcfJW3fk3skwRcTLF8+5Pn7Jif7So3AjwF5zjb IyN+wBmb8O0f3RVYPq/07NmTs70GeQgjAGX25LU/EVP4qJMbFDanhfU1YxidnGXMWlF7 BDQsANy/+n6VBNqr0IiB5pL3ZXM/JMHVxS6UHD874xNO5D6FEtkMqw4GvGtt68SNXqdt kazQ== X-Forwarded-Encrypted: i=1; AJvYcCUEa8xqKywMpnK0YMi/CcHcL/G5AK9qSmsBDsgfEdMcGCUqaY/P9+oXilgKD1op8WmrWvB+iZo9aKdRBng=@vger.kernel.org X-Gm-Message-State: AOJu0YyBxN84NIN1TL9Oe+zMFsRJBVbaWAWpSqyVxhTL1i9C+e/znzSx EFQh8TZH07lSXO5B/u4k/eUz8bR6e9ZuhqdaDA7GH4eZNSDVrwnCnjAf1GvnYOgl/aEItkmrNto u/A== X-Google-Smtp-Source: AGHT+IHdtHq1a5g5QmW1TlBMuTKUOZ7R4d6aOVSlzi7r2JK/pfzCqaKRixPsJw6iRhPElP6sWQ73LfqmPps= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:d353:0:b0:e28:e74f:4cb with SMTP id 3f1490d57ef6-e28fe0df614mr96115276.0.1728584831124; Thu, 10 Oct 2024 11:27:11 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:08 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-67-seanjc@google.com> Subject: [PATCH v13 66/85] KVM: LoongArch: Mark "struct page" pfns accessed only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark pages accessed only in the slow path, before dropping mmu_lock when faulting in guest memory so that LoongArch can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Reviewed-by: Bibo Mao Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/loongarch/kvm/mmu.c | 20 ++------------------ 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index cc2a5f289b14..ed43504c5c7e 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -552,12 +552,10 @@ bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn= _range *range) static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, boo= l write) { int ret =3D 0; - kvm_pfn_t pfn =3D 0; kvm_pte_t *ptep, changed, new; gfn_t gfn =3D gpa >> PAGE_SHIFT; struct kvm *kvm =3D vcpu->kvm; struct kvm_memory_slot *slot; - struct page *page; =20 spin_lock(&kvm->mmu_lock); =20 @@ -570,8 +568,6 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, uns= igned long gpa, bool writ =20 /* Track access to pages marked old */ new =3D kvm_pte_mkyoung(*ptep); - /* call kvm_set_pfn_accessed() after unlock */ - if (write && !kvm_pte_dirty(new)) { if (!kvm_pte_write(new)) { ret =3D -EFAULT; @@ -595,23 +591,11 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, u= nsigned long gpa, bool writ } =20 changed =3D new ^ (*ptep); - if (changed) { + if (changed) kvm_set_pte(ptep, new); - pfn =3D kvm_pte_pfn(new); - page =3D kvm_pfn_to_refcounted_page(pfn); - if (page) - get_page(page); - } + spin_unlock(&kvm->mmu_lock); =20 - if (changed) { - if (kvm_pte_young(changed)) - kvm_set_pfn_accessed(pfn); - - if (page) - put_page(page); - } - if (kvm_pte_dirty(changed)) mark_page_dirty(kvm, gfn); =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2159220C491 for ; Thu, 10 Oct 2024 18:27:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584836; cv=none; b=WOsw/thOYEHeh4eRfwNqSjV1Nc4BvDjyAYEAwYdT/4lwXTrQRb0i5PLdA+9GuACaAZZCs6GM/0aIIGhR9OhKemgVifeVvGYw1T5qTKnVEvufi7JcbcV9Q2tX+Jtt6BpS3aIFjCqRxmCCzzyndUFH5Zz3TBf2ihQAIWTr4UGG/h4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584836; c=relaxed/simple; bh=+TpPAbVVI1obWW7OdvHKpXsCMwy9+BV5v8TfW2u9vbU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HbZda94MjBlmb7PRTXTjg3CDNy7XcJ+9KBZ0Dxmv4X7SP+ed8Fv6Ea33bg38KIfKzlG+H5mKyTL49kCeqX3+Vh9MH5wjA8KH9UNwOQABCUJs070SvCD+rB0R+NLkAOu8ypboTkH5/4yG/z+tskcf6Z1EWVHq0SaawytfF1yuvfY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=GNF92OQx; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GNF92OQx" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2e148d4550bso1323673a91.1 for ; Thu, 10 Oct 2024 11:27:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584833; x=1729189633; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=3FFUjrr9Bcuid6sLVD5OEsWVTb8VG25P9bGjkNYbAzk=; b=GNF92OQxF0tq2Sy3NXgpLFxQxC22Wl9tw03n7HzL4JrVAJCkh+Hov4DvKeVAkbLbz6 guCgfWZA8nqZBp6XZ9pJ4Hawa9uVtgV3UErDEF6u0+OBm51ovt6v1pRCPWJe2lNSocFb 9IG5h2poJKL8NVnbhGa7Q5llq0RI6iyjaleF9SNYfF/o/fNFc2aKi+UykoVstbZ8OxwU d1wLhivyUM4mbES9K28ITC9CcdOVPnfsFw6ARgKHJiVEHnES4HjeBBPrQzXn/hChp5jb ncyckAUOmfR9C543A9BJWxeDUS9424yzgfnC3W2p+8MxlZym6ly+BGplBaNU9k30CJj/ w62Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584833; x=1729189633; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3FFUjrr9Bcuid6sLVD5OEsWVTb8VG25P9bGjkNYbAzk=; b=vZXQ0HdwCMusqHz+4b0YwYKp8qh1anODiVkIPAKUxiRE7ZHhud7oi5MvPhnQPJtH/B jl3p/OKFCojdb06vOyCMeLN3SfiQPsrx2SEqkgGl/otLS84bFkgvgz3ecAPiEvkiHt72 BWl5yA4eZ1997AfZHNWhOxV7g0ghRLkgRgaEn2nPRKMQJhjCdAGrwteOGH89yma0V236 xC6SPpzZsdvY05P9VaLSVGK0bazOwTIGsNS+t8gQb9Hk6NjDiJ5BCYIwEKHNljoYM2AP LiI+Wr542uH3UyJzdMK5xx3dQrD1J6j2tE3tWTOgnUN3SBvS4icVEv78ZVKrB6rGWbai 9vGw== X-Forwarded-Encrypted: i=1; AJvYcCWvaRjg9f9QFW7S1VMVuiw/YERpvEAq7sztt/ZsaIutYp8tVjCU2PVY2kZ8IpdrmYCpoE40ktw3QvEbgkU=@vger.kernel.org X-Gm-Message-State: AOJu0YyTdYQUNhTPCl3SVpF07ev6dk9IU+uOGBZytYVNg3OnGqCzOEUD uWojL0RuUbdIkkpanNpTpdkABcK+Vg/gQZlli3232sGb79+/3Fyj51taOTSOkx0ci7SmmnTeZem kZA== X-Google-Smtp-Source: AGHT+IFpwqxWKAWfrUvKF1EatvnBv5g5CrzcpgT/ZTuiTODJnzoCoYK053S1MFwF0WyL9wI3LVhDYDAiv74= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90b:46d4:b0:2e2:e148:3d37 with SMTP id 98e67ed59e1d1-2e2f09f1a56mr94a91.2.1728584833102; Thu, 10 Oct 2024 11:27:13 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:09 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-68-seanjc@google.com> Subject: [PATCH v13 67/85] KVM: LoongArch: Mark "struct page" pfn accessed before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark pages accessed before dropping mmu_lock when faulting in guest memory so that LoongArch can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Reviewed-by: Bibo Mao Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/loongarch/kvm/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index ed43504c5c7e..7066cafcce64 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -902,13 +902,13 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsign= ed long gpa, bool write) =20 if (writeable) kvm_set_pfn_dirty(pfn); + kvm_release_pfn_clean(pfn); =20 spin_unlock(&kvm->mmu_lock); =20 if (prot_bits & _PAGE_DIRTY) mark_page_dirty_in_slot(kvm, memslot, gfn); =20 - kvm_release_pfn_clean(pfn); out: srcu_read_unlock(&kvm->srcu, srcu_idx); return err; --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5283220CCEC for ; Thu, 10 Oct 2024 18:27:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584837; cv=none; b=TvHlK14wBd8cJJoCMUWoikQgY0mL7u6NHm7Y0Px4LdDHFB8YRJ/OISLqPCAdrrBfmPWBQkXTZ7GW+d4XzKAazhtckRNpQoh46uuZuAFiBDXA9s3m+uEZ+e1VG/HW1zliOJ9epxEc0i1a4XX8uPAj5rBq6AkUyn/SjMianTyoehs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584837; c=relaxed/simple; bh=HhUA1PeKZEb/51LjtENBtdXbtQZ0o3j1vGZCLDD8mMg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=snPkYl0m5hrEi84c19blsgBdSI+Bkxckij27J0+ylbUNmF7FiMaR9QCgLIMCu1DO0Cy8OdgtlCEuH2qclNAXy5Z6veOZAUgfM1Bg4B2ddPyxAcBuK8ZCM11jMhUTTtLdCYkiBYGLPmv+Kl3xr6cqiCsXaKKaiGhQp68K1bG2v7Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ZfysK4Eb; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ZfysK4Eb" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71e019ab268so1589833b3a.2 for ; Thu, 10 Oct 2024 11:27:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584835; x=1729189635; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9LTMkDJZxM2S5Qh4t0DfHllWvhtFq1rJ9EqXHTYHy/A=; b=ZfysK4EbyW9GP/xSuAfW3U3GxCRMKGJ1uQ0l+1a9fb8m3NTbOSK3zUlXUKjOSQ40T5 ULvpVNM76l5WMWoFdvGpGrBdcaqid0kWMM1s0kBGxkAqf4kwZrEANqHFgqecmt8eHYgx Ik8OK2Vg31lI7rB/rwW7MTUKastSn4QH1xeXn/WK8/jnpqBbiN9Zv5K+jntkQWm890PW KjLL2HC/A6kUxWwarJKMFJacXMwAT0kofIlFki+OA82WubYYtrJHX/t4g5yKgnDO+Ptq 4dA1ZW7S4NyO5r2DrnFx44K0GxT/uIbV+E5cT8JHXE51CWa5Cvp/710I6os174/V43Yl h/rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584835; x=1729189635; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9LTMkDJZxM2S5Qh4t0DfHllWvhtFq1rJ9EqXHTYHy/A=; b=SxYjG6KIJz0OfInqvaCnnQLdDFcQnZa+C9luAghnP8wbkzWKS2eDYQFJenYIeajw2L vE/TJkW4mhX6yFxoyqCU5935hcWmavD79dRiDeO/7gyJI8+sefNzNFCcWojSBfApgrEd 4gdDYyE/1oecZby+RuIQQNnRdoUwD3qVvrhQcZ6WnAGRcqf7RCZ6GcS8LMlrP8NP+AQS HfJYHwOUN7TS+eEHUnaRytL6ur19nRxmxeyKxSK8M3n7h0RrklVYSze6RS7ysc0ta/VQ djJdLI58Ivevh1dd4mKU2DGJb48LhfXfesZbdEhrmi423Ui9M257r1nm98XKuf01QSsU 6isQ== X-Forwarded-Encrypted: i=1; AJvYcCU/dpGnjbTveb3QzQa/lyHBxxFwPPTcS8OKPG+qPs+Irhh/nmPoroX29070VsJ2rOOOl1OE5v/ehq7X/qY=@vger.kernel.org X-Gm-Message-State: AOJu0Yy69dk6BA0Q2ULcnPbqzuhl1JxZAwOVSMIugiL2WReoEy6Xs6xR V0S9K/npammsfg+PZJTkLut8mEUPmYqNk8cFZB6T/dceKUaRdn0s5Zlp38xQNtGvEenW940EX5Z BMQ== X-Google-Smtp-Source: AGHT+IExt7obORK9hFSNzpsBdJKygVAVx/dPvrUuCl2bsyTFt9B1GPfq0Q1aHG/lNRl/yht5gsZcWFEXy1g= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:8614:b0:70d:fba:c51c with SMTP id d2e1a72fcca58-71e1dbd032amr6898b3a.3.1728584835380; Thu, 10 Oct 2024 11:27:15 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:10 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-69-seanjc@google.com> Subject: [PATCH v13 68/85] KVM: LoongArch: Use kvm_faultin_pfn() to map pfns into the guest From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert LoongArch to kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/loongarch/kvm/mmu.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 7066cafcce64..4d203294767c 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -780,6 +780,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned= long gpa, bool write) struct kvm *kvm =3D vcpu->kvm; struct kvm_memory_slot *memslot; struct kvm_mmu_memory_cache *memcache =3D &vcpu->arch.mmu_page_cache; + struct page *page; =20 /* Try the fast path to handle old / clean pages */ srcu_idx =3D srcu_read_lock(&kvm->srcu); @@ -807,7 +808,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned= long gpa, bool write) mmu_seq =3D kvm->mmu_invalidate_seq; /* * Ensure the read of mmu_invalidate_seq isn't reordered with PTE reads in - * gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't + * kvm_faultin_pfn() (which calls get_user_pages()), so that we don't * risk the page we get a reference to getting unmapped before we have a * chance to grab the mmu_lock without mmu_invalidate_retry() noticing. * @@ -819,7 +820,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned= long gpa, bool write) smp_rmb(); =20 /* Slow path - ask KVM core whether we can access this GPA */ - pfn =3D gfn_to_pfn_prot(kvm, gfn, write, &writeable); + pfn =3D kvm_faultin_pfn(vcpu, gfn, write, &writeable, &page); if (is_error_noslot_pfn(pfn)) { err =3D -EFAULT; goto out; @@ -831,10 +832,10 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsign= ed long gpa, bool write) /* * This can happen when mappings are changed asynchronously, but * also synchronously if a COW is triggered by - * gfn_to_pfn_prot(). + * kvm_faultin_pfn(). */ spin_unlock(&kvm->mmu_lock); - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); if (retry_no > 100) { retry_no =3D 0; schedule(); @@ -900,10 +901,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigne= d long gpa, bool write) ++kvm->stat.pages; kvm_set_pte(ptep, new_pte); =20 - if (writeable) - kvm_set_pfn_dirty(pfn); - kvm_release_pfn_clean(pfn); - + kvm_release_faultin_page(kvm, page, false, writeable); spin_unlock(&kvm->mmu_lock); =20 if (prot_bits & _PAGE_DIRTY) --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D015210186 for ; Thu, 10 Oct 2024 18:27:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584840; cv=none; b=hrDjyuH1eu4ULzI28xA2xIXIVZ7s038du40y7DUhO1HkoKsJJOEKyJ/fzY4H6LF89pLcEwxLmuzCS/cZl74jvcly51N52wu62x+sET9WkxN52ci/A6dU4CqG5fFj7AmvOU2Pp17l2dBFzOZStPNW0tTepAW5ErYbtnQLIR+BDv8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584840; c=relaxed/simple; bh=qdVWvBqKE29lnYElhxBDh75UWFXRti117FkDAjCdYI0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=e9QPt+HrgCSC6Hm5gZd4aBHkzjfvxeqicotGxrC8l17VQ8i8i60y8Mgi46hqH4P3y+/NpiGlCZjJpCvQQthHIDtyLGY9CGFsJiUptLLLQYng20ct+Sy7P1brmigak3H0GScK4ooR2QLU6l4LJiMAgxiJVUsoMmqY2hK/HLHig/U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WEliYGH/; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WEliYGH/" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e0b8fa94718so1790325276.0 for ; Thu, 10 Oct 2024 11:27:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584837; x=1729189637; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Sag0fj8OwucY2qRogRu5ELvjWktNcy71jZ/+qgEdHaE=; b=WEliYGH/EkhyOCm54H+/wesyw/aZ7v+0pWnBeiqQzyPlQ2Yj0TH4WvG+UvuO5i/tJP qqFT63mFFsNK8nCKRd2rdRSnBi6xiWs4I9YPkwQ/ORC/XHuM6kP3lGibolilYQnn8KkG 04jph3IKCWStNMMrOeDQO9d+9jis+VCzO92vxkbcdm50eGUrzmGw34nU9RjiIPyyw0Uh euFYKjXV0mCOLJBqh+xh9tHaXM4uh2RwdSp6qf7nlg/wQlMv1OOpW1BTqmQYcv2QbZgI qpCY90RHVgLxl5134r1g7+kqZbPFrbMCMGJiJssp7lQN4g6nR6yUrSuWdNXtIvq6MagM 3JOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584837; x=1729189637; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Sag0fj8OwucY2qRogRu5ELvjWktNcy71jZ/+qgEdHaE=; b=Fx0HD7OrZusILDvnLFBP5kbsGQeUltTVcqa8VvH5MjKhol4U1eyzdHuVxqpG9fZD5w KAMDgOoYsch+YEs05AehSe+7xG6Zg27PAHNOusjiwAOJdxtr43DsNWHrvr9ItPWAwTPk BeM4KRejrcZWkIbPxDTay/WN/JDmV14Q3bwMo6W6zKZvMLcUzeRDCAtU23TY878UdG1Q GFnqeoWIqfbIJHHqsXR0T8nWgcHPLE4nsDOEEoE09ppyRlFapD0JvquNC5+ig5TLW2sW e07PK4yrEE4AOhj7aE8C6nDaHensQatuq6YLiixO75iEP1pgpIEYM3lAEsdmTiA4pGzI gknQ== X-Forwarded-Encrypted: i=1; AJvYcCWvDjMIWa4BGrLCPYih1LiiuZEqdFhA8u1Ykj9GududUAyNEYXcjB+QSsrsxOBM1usgts8dgzXX3b1v3NE=@vger.kernel.org X-Gm-Message-State: AOJu0Yz44eaWSZNCfkPCesMirtJtWtwCyIHssyNDP4CWKB5CNJ9t0vjS dJrYj8qfMLBN7PYMf5phDQXLiv3dbNKIFe3jFUiq0lKye267jdFcqfFhlUTO5jHeS5xtR87begO HMw== X-Google-Smtp-Source: AGHT+IEjJriLMGICq9yty21Fx3Z3FQyTG0XCg9lS5qhc+MvTVoRASG8drmw0OxSoWZyAtrQmM3pJDo8jQYk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:a4c:0:b0:e28:f454:7de5 with SMTP id 3f1490d57ef6-e28fe52740fmr75826276.6.1728584837458; Thu, 10 Oct 2024 11:27:17 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:11 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-70-seanjc@google.com> Subject: [PATCH v13 69/85] KVM: MIPS: Mark "struct page" pfns dirty only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark pages/folios dirty only the slow page fault path, i.e. only when mmu_lock is held and the operation is mmu_notifier-protected, as marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). See the link below for details. Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/mips/kvm/mmu.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index c17157e700c0..4da9ce4eb54d 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -514,7 +514,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcp= u, unsigned long gpa, set_pte(ptep, pte_mkdirty(*ptep)); pfn =3D pte_pfn(*ptep); mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); } =20 if (out_entry) @@ -628,7 +627,6 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, uns= igned long gpa, if (write_fault) { prot_bits |=3D __WRITEABLE; mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); } } entry =3D pfn_pte(pfn, __pgprot(prot_bits)); @@ -642,6 +640,9 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, uns= igned long gpa, if (out_buddy) *out_buddy =3D *ptep_buddy(ptep); =20 + if (writeable) + kvm_set_pfn_dirty(pfn); + spin_unlock(&kvm->mmu_lock); kvm_release_pfn_clean(pfn); kvm_set_pfn_accessed(pfn); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19E68210C23 for ; Thu, 10 Oct 2024 18:27:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584843; cv=none; b=SgR9B5NvvoUyCg0AYai8+OIzbE1NqjXpcl9KtyWMRzfHdlRxt0QhEAn5naAR6ArV7yi8DNowAOH78W48lU+BJJefizqpJ8IR+f56UMDXhY58NHXQS8sDXyEAorr22o2SlAo1NgSbhTJus+bCTa+V2WBw/qLKsm8fFMA5eWHaNjY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584843; c=relaxed/simple; bh=TVe3szAAU/4RHYDireJVG9Y8gE1GRTl7yNXWTsweF6A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ozn7fe+vSrxi3aI5HYUtBNJmzArnTRmS8JbbUeowwcIddUAo//ha21p/ZbOXpiPsq7FPWE4Jw+qAZ9AkjM+3A+DgDJJ7V8j+IfZ8eunoEwFGV5FDlTJpb/kpDrlpDQTfZVKqj1AoVdw0SsSDsuOtHM9y215VWHAK+x2qr1OczgA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WEyF0Eo2; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WEyF0Eo2" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6886cd07673so27462127b3.3 for ; Thu, 10 Oct 2024 11:27:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584841; x=1729189641; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=KbunjaDKwfDzESX7bCyZnVXI4Jj60WAsli6cgjlK44o=; b=WEyF0Eo2XsJNPDFQCmldw4bvVns8foXPz6GhGq8QjDu8xJDPsgU2gDeg4tMkoGp7DG qpJiXq9ko9g1ZdHZfqLtcRYsvBPZ4zAKl/xQNiiKA25w/AQ8ybJ9Au1awx4vkn8oB+xt YHlrCluf/X2gZpoaDcL86BqHQoGLa8wJKT5byh7tuDXy8KhdxSaUh1uaieJMsHHoehat haDEfLKrjSNVFiwwnRXkhtFTP9jsOA1Q8XsG63RusobtgUY5tamyUlCGShrinJ6gi0cZ iOQEEssjZFClbHoFhivOJECpgHTXb0THhmZzFAYSyJ7PX3K8HOO54iWYJUfENuwbZtqd kRSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584841; x=1729189641; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KbunjaDKwfDzESX7bCyZnVXI4Jj60WAsli6cgjlK44o=; b=rRLl7WqTcPtbsE6lt8t+cuwDcpeITeZ+BOFNQjZOw/Sz+B9umQrk4BfCHuzOpcBHnU CXNKwXzkQJhWGG6zcLmy/UNiUrB4Vs3mZ2NqzpuhjsYs5NklfXnPn5FDpfenp4HttZ1A 8dofbUS2CPVv5VhymkgzUjHQzB0gXm3mfHFn6MasnFYHYaaTe7JMosKo7NNBLZBHyUlt ISwRpddTSXWB5CSA6jCDElQP25+4QKU5Nd9XxR3MArHvbmjA7RsE9VyL97TJ86JdE6rb emSqjO2/tU9dSuXtz/kOKUc+m0xr4qhJF4gwAf+CQJwsDr0GhwPUp8j8oh4zc36rOQND w/cQ== X-Forwarded-Encrypted: i=1; AJvYcCW2nCT7ma9uVP7wL2o4+VgIFEM8KcrnU/nFuJ1vOJ8F64Ywbdp4dN8cV3rNRjgD9OF7BuE6WobHyma021E=@vger.kernel.org X-Gm-Message-State: AOJu0YyZtbHg9CFK63yjaDq03+vum/q+m1R2e/D267gl6tPNN6ThHm6d ld6r4iZaKw7zJfx6gCsycU2YgUr0puD9FoiTE2YCXgn/j/WuSTbWUBTfKEBKZdoFpMtnJAOU+mk klw== X-Google-Smtp-Source: AGHT+IEbamZE/KJDJ4kw63fZFZN0PThnG54xKd82AQ09AwFkx3KIodEPvEqsDBHZyT6DCHkY/GuTD4px7aQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:6c05:0:b0:e24:c3eb:ad03 with SMTP id 3f1490d57ef6-e28fe540170mr1289276.10.1728584839383; Thu, 10 Oct 2024 11:27:19 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:12 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-71-seanjc@google.com> Subject: [PATCH v13 70/85] KVM: MIPS: Mark "struct page" pfns accessed only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark pages accessed only in the slow page fault path in order to remove an unnecessary user of kvm_pfn_to_refcounted_page(). Marking pages accessed in the primary MMU during KVM page fault handling isn't harmful, but it's largely pointless and likely a waste of a cycles since the primary MMU will call into KVM via mmu_notifiers when aging pages. I.e. KVM participates in a "pull" model, so there's no need to also "push" updates. Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/mips/kvm/mmu.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index 4da9ce4eb54d..f1e4b618ec6d 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -484,8 +484,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcp= u, unsigned long gpa, struct kvm *kvm =3D vcpu->kvm; gfn_t gfn =3D gpa >> PAGE_SHIFT; pte_t *ptep; - kvm_pfn_t pfn =3D 0; /* silence bogus GCC warning */ - bool pfn_valid =3D false; int ret =3D 0; =20 spin_lock(&kvm->mmu_lock); @@ -498,12 +496,9 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vc= pu, unsigned long gpa, } =20 /* Track access to pages marked old */ - if (!pte_young(*ptep)) { + if (!pte_young(*ptep)) set_pte(ptep, pte_mkyoung(*ptep)); - pfn =3D pte_pfn(*ptep); - pfn_valid =3D true; - /* call kvm_set_pfn_accessed() after unlock */ - } + if (write_fault && !pte_dirty(*ptep)) { if (!pte_write(*ptep)) { ret =3D -EFAULT; @@ -512,7 +507,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcp= u, unsigned long gpa, =20 /* Track dirtying of writeable pages */ set_pte(ptep, pte_mkdirty(*ptep)); - pfn =3D pte_pfn(*ptep); mark_page_dirty(kvm, gfn); } =20 @@ -523,8 +517,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcp= u, unsigned long gpa, =20 out: spin_unlock(&kvm->mmu_lock); - if (pfn_valid) - kvm_set_pfn_accessed(pfn); return ret; } =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41F73210C2E for ; Thu, 10 Oct 2024 18:27:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584844; cv=none; b=rLC0cTEcJl8DCRMklZLZZrb9RVgqB681r8V/D+4n91vnT2AtESzRTqm9qgFC6bWnWLULQr+PMmilVXsmkG/kDNH8O6FMgMr+NWsr0+geUvSPsT1rlmlLLud73bIRjEuLWwtu7DpsItW/vbpt0hon6TlSIWQQFomXYiU8LMN8PYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584844; c=relaxed/simple; bh=Q0WfZXD0AwljsCbiw/O0WdULhXfIX1X/By/5X/lP1LI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=mQFdC3J/BOE+L/9wgsUTDvRN8TKy5gug4zOrQZ8b585mfEw5g4Bwr6yuxgDRZuya3P+5UZFFpWNECeZCWKg6eCKMX4nP+JkVatHBeJK3GX669XlhrqNzXz6Mz/FQ4EQgE1OUdtdQ7ISnrgsVfepfxJIK5sdp7WZjp+IIFfbe8lQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3qpUkcRq; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3qpUkcRq" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e2916c47e9fso298656276.1 for ; Thu, 10 Oct 2024 11:27:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584842; x=1729189642; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OZ/c8QPJoSxQXGCWT5aWp1nMy1t7EEzE7+N8fqq+BFY=; b=3qpUkcRqOzUfCZcj+BZdHa8RALWFF7OtRiu+VQ3N3Sp/jKQU8vYZDxOavhvz5GzCPs 4wMtC27QPkYI4ZZHJTIhsyVCBkQHNDXjn80rQVg5sN8wuj71+RLQ2UFJPOQltkMKo9b0 qmkHu3LJr2xQyjv0+A4khnwPAAKx6T34tSkbj9Gi9Mhug6a0LgiqKq6ggRDbj1I13x3M Y3Gr8fNaFSD6n8ayAIf+4tHcbqgdwwrqOAUwKRd8eJ/bjgCPP4eKndQloqNzhdukKHBr xWjD7W7h8Vs/8C2ceSX6nGlC3uH/sYu6EzXd1UTRGjZgT5NEBX+d23twphqB7dwJBHG7 TZqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584842; x=1729189642; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OZ/c8QPJoSxQXGCWT5aWp1nMy1t7EEzE7+N8fqq+BFY=; b=O7l2qwbnTxC/ePAUPpogfnoR8vCJ5+eMtEucZap+6AcJ/zWCEUSWuGfO2Z7YKwSh5V tRN6MmWLfMBGBWg7QctruN4IM39zsCVdh8CJkAh1eK95T2SVU2nk4olOuxoLVPa86YOQ w6zctLZ64Os9BiN4arNJJyDHd0a2jgNY34LtpdvdYqBj59S5iUfmLHbQVpSD7pOK+nqp oVcjwDX86/fME6/adEOukwU4my7ixvEh41+aopdmr96aaJOzNWNTa9ybPLUVpSC+8U9j EZ0YEGRX6NfmO7i+9MY6f/uxU+P/34/s+jy0fDtjDfPRZxbx433DZ4ABEEAR+W/Kelcy 81zQ== X-Forwarded-Encrypted: i=1; AJvYcCVu4rPm44F8AAiobpr+I1KlPPW0hN3ne9vImyByb8vkf2DewX8p/7a6UBFhPr+jAtOxmQRmn8g5XEEmPLg=@vger.kernel.org X-Gm-Message-State: AOJu0Yx4pEmSDVS/Q1L9rhjGaCk34mO+jtDCH7/ZqCUgS+6VxJTeUSOG dleCqiROfLf0gvcUNZPCtlecLM3aZ7+on3wSV1xW195Xp2kWgWprQAUgjRVAZWepW+4gDfxMngS 6QQ== X-Google-Smtp-Source: AGHT+IHhD+Vk6TwFIS5v8fA24SNPRWU4vCcQM+1g+zVw6L+6pfrMpIdoNti+PehtHCeQDGmk8Jps/9lQ5Ek= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:d30a:0:b0:e28:fe07:9cc1 with SMTP id 3f1490d57ef6-e28fe4a3a81mr68085276.3.1728584842321; Thu, 10 Oct 2024 11:27:22 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:13 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-72-seanjc@google.com> Subject: [PATCH v13 71/85] KVM: MIPS: Mark "struct page" pfns accessed prior to dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Mark pages accessed before dropping mmu_lock when faulting in guest memory so that MIPS can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/mips/kvm/mmu.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index f1e4b618ec6d..69463ab24d97 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -634,10 +634,9 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, un= signed long gpa, =20 if (writeable) kvm_set_pfn_dirty(pfn); - - spin_unlock(&kvm->mmu_lock); kvm_release_pfn_clean(pfn); - kvm_set_pfn_accessed(pfn); + + spin_unlock(&kvm->mmu_lock); out: srcu_read_unlock(&kvm->srcu, srcu_idx); return err; --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7428F212635 for ; Thu, 10 Oct 2024 18:27:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584847; cv=none; b=e2DXEp68ho6lwgLiEOLKWCNtXpnFSmKmAQfqSDLfZPeh676Mc4mWMMxNfLgc6/7QJkSTS5Jsy3pMA7uTj5UB6xrkd3vr8c4r7gca3qbaCcJW+OFVeoMNhVXCdyNYSM4+tl1Zh0G+nnRCp/f6mfp5n0tTXkCKPNyERmzxqibu034= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584847; c=relaxed/simple; bh=8dGNSdkZyds89DzrI//tIWXaU4bIHqe2OXPOvN+ci/o=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=tZrzNKZmQbfgO+/8Zh6f3DZ6PmqsttLWw6tGTmtfKJB9My7iPkSAD5JHWiU8Q8xOGF3wlabNy1E1csZPzWhI5dlps2HjEOAzZvYCpJ6Pkij9h9ablrUYkeHVhII4sHikY+6LZ6GDB9DlGu7S7nyu+v/M+jbnc+M0ghXo9Aa4Nlc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JPv9zkzp; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JPv9zkzp" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e1159159528so3623549276.1 for ; Thu, 10 Oct 2024 11:27:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584844; x=1729189644; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=ojyPU0tzKIRRbswoqL6JjpEgqRgu1FFXpokE5rqa8U0=; b=JPv9zkzpTer/tRVsCcO0xpoVowSI9sq8yAveFD2GfRunYb5X0FtcPWNADF5sqlUd/3 iuZOBdxa/cq2pQY/AtFf67RWj9iN68fjShGVpAPx4/qOHMoD0+4MDx5ng50F99dpeqZz D7tj1u3tjV11ltIQLbM+e6UJ2OtxC2znpSGhxBoavgsmxRK8vxqwx1Wv6ERJeKC1sxzZ warJrQCxdXVL7hXsRhLLyu6K6XZyuCXcbnBSMP58XjLa6moCXuL2/gm/dZ9g29lFRGQD oz4+v/DmwPgsIZnAE2dQyOCO8flkAv8ygdRPm7vy8SmXxLTWo0sUXu9l6nOoTHE0oXK6 D4gA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584844; x=1729189644; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ojyPU0tzKIRRbswoqL6JjpEgqRgu1FFXpokE5rqa8U0=; b=RXvMTi4QX7hv27EseY4Oa+cjpBp/ZsDoCVc66GVwd6/N8NUzonSi/1EbQtValUn3uz 3NwL2LH1N7ShQKLs0cc+gx+xs7uNuVELMJZzNG1siDCOHF/CMGKteeRkrxB5nEgWupZ5 cSOldJ+N+tNitKujIswgJJPE/szenL9GX66lkcgQ/MGZ03oxj+mvUxJHDwbPlR+gEi8x n0knzizOdty/0z7rY7E+MbC2h0rm19Aq1YZ1GMG/nCModwkrKvfyjvyfPIMFPcpD6PVE uUi3J/XY89qf7qXoDsa+7Z//5W3dImfujsxdadZuH6OdKle2n+ViQC0cx20Xv+dFRqJX P47Q== X-Forwarded-Encrypted: i=1; AJvYcCXAm1XRyrAX4GD6UP+7KXFoQY7+Ef532Kosj1PFAEM2jqsbDGgP1lcSMFq/0lG9XG+jirRh5g/JugLJj+E=@vger.kernel.org X-Gm-Message-State: AOJu0YwVM9FbfuXsOGX5jgVeKwKK8X/6Po5UjI5Bhby88kCPxAFQT4vW uTDd7pFBi8cfD4BYVXLGxbg9E10/KG5TpUdZJ32CXqeB2Fp/ht2Y2X1wbA8l5tVQW6h1Fno/zYK VcA== X-Google-Smtp-Source: AGHT+IHEyKrdGSoNbMWcRNiKhgC/jPgVf/TjauaPCrq7yqhpXyuJQagleroMC0w40O7lZsOojHLr7EwrZqI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:6805:0:b0:e20:cfc2:a326 with SMTP id 3f1490d57ef6-e29184333b9mr26276.6.1728584844184; Thu, 10 Oct 2024 11:27:24 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:14 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-73-seanjc@google.com> Subject: [PATCH v13 72/85] KVM: MIPS: Use kvm_faultin_pfn() to map pfns into the guest From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert MIPS to kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/mips/kvm/mmu.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index 69463ab24d97..d2c3b6b41f18 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -557,6 +557,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, uns= igned long gpa, bool writeable; unsigned long prot_bits; unsigned long mmu_seq; + struct page *page; =20 /* Try the fast path to handle old / clean pages */ srcu_idx =3D srcu_read_lock(&kvm->srcu); @@ -578,7 +579,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, uns= igned long gpa, mmu_seq =3D kvm->mmu_invalidate_seq; /* * Ensure the read of mmu_invalidate_seq isn't reordered with PTE reads - * in gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't + * in kvm_faultin_pfn() (which calls get_user_pages()), so that we don't * risk the page we get a reference to getting unmapped before we have a * chance to grab the mmu_lock without mmu_invalidate_retry() noticing. * @@ -590,7 +591,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, uns= igned long gpa, smp_rmb(); =20 /* Slow path - ask KVM core whether we can access this GPA */ - pfn =3D gfn_to_pfn_prot(kvm, gfn, write_fault, &writeable); + pfn =3D kvm_faultin_pfn(vcpu, gfn, write_fault, &writeable, &page); if (is_error_noslot_pfn(pfn)) { err =3D -EFAULT; goto out; @@ -602,10 +603,10 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, u= nsigned long gpa, /* * This can happen when mappings are changed asynchronously, but * also synchronously if a COW is triggered by - * gfn_to_pfn_prot(). + * kvm_faultin_pfn(). */ spin_unlock(&kvm->mmu_lock); - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); goto retry; } =20 @@ -632,10 +633,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, un= signed long gpa, if (out_buddy) *out_buddy =3D *ptep_buddy(ptep); =20 - if (writeable) - kvm_set_pfn_dirty(pfn); - kvm_release_pfn_clean(pfn); - + kvm_release_faultin_page(kvm, page, false, writeable); spin_unlock(&kvm->mmu_lock); out: srcu_read_unlock(&kvm->srcu, srcu_idx); --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD47E21263A for ; Thu, 10 Oct 2024 18:27:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584850; cv=none; b=srxyr0byeDEyfFbCi0PrrySwhlcjh6Mg/2uL12cTXc13bTsLxtqLKu9aGX3FuAyTkKzHFQHKz0J4S93xacGDr9taAP2TdlMv2UnNHSH8wwRJtCNmU6+7ftu4jWjgzRn78hgZqYgo+KjNFHuLji4iit5LQM05NYSgSlcO7yygF84= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584850; c=relaxed/simple; bh=hdvy6TBStdDo47jCImbsHkCH8/fTSGPuTNA399ptBEY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OFLFupt/j1io6JF+rgrlFaef8OmOg4fhnfXNyODRfr8oFQLv30+evLYwRTnSwHrUfGB3Va5X0whYD0sdzE0W7QRpzDE2b9krh29BR8o7DG6PkFliBBZ7LBqsD8fHFM2g3mjf4/igjLGf/RxlXZ2i7BBUl6vK8SjXrnfO0SXKJmA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=DuDHV6yB; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="DuDHV6yB" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7e6af43d0c5so1109715a12.3 for ; Thu, 10 Oct 2024 11:27:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584848; x=1729189648; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=t4DEWxyHdS8FH8bmPlxV35CavnLWtw4MDWneo+GIfaw=; b=DuDHV6yBR6R4IF/Tk4tYF2+2idGq/vVgsnDPd/mRNzisey/lK2BmHd3cUo9vBvOANi zkoGepVCuNujuKVT0SvZmDE0XvYHerpurUIIJMAeWc+b57hH47kC0QJ5wHtQEMfJHmPh PpaE4aHKRT1h6TbKpgXWXqnsnto2OGNq0fH2Ws2KY8/MbS7jjFY3wbSKnu7MIzTUrs1G et9W4d1rTvNIgRtfcukOUMCi85rcuo5kTXGS1TQ9auZ4CcSXEEnywcZI9uI5KSy/FVdZ YPQRO6b5YAteoGfO1rbMctMWpre/nlO8BdIZjHegGDTKR2Jshft+Hp1edSWuOGnxTxrC JmMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584848; x=1729189648; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=t4DEWxyHdS8FH8bmPlxV35CavnLWtw4MDWneo+GIfaw=; b=aDqPikrK8p/J414BarPEOPFVzeaDzG7YuVWMp1CaKbWZizpCTZaM3wghzChsseuPF0 ALpCMCG91d3hHmHiQY9jwpmYpJdnz9RGjNEvIcX32bpz7RHNNlW3Gn3mpOjNH8m8i4bx N691VH7nYsd9GjDYUlLkScgT6igS77YFGA56s1wp6CHXd4NgR4QseZPXqrn9mKgWH92R lyY7Gnp4baUCd4tdF+dfNdTqbVcT2MKQPDe32YYuPFm6Cf1p20d7fjTGwA5uOoPoRKPs +4LQQcnPZEEm+b+i7ZYpvC0XPoZa+qD+ey0YVwpOKH/4rjo31Wccbby0SLBjNB5ku+hE c8IA== X-Forwarded-Encrypted: i=1; AJvYcCXfds0ODhVNhJNrrblzMLwEOYAIbFQx4gY9SgvbByBJQejLEsxXIfyPiURAgvIhG9dmcWRVcEdcRyDaFXQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzLJ45nTpBEyFJvDgqm8CFjKoYn4frdAvmmuYVD4WpDNJJyfo8w j+k1IiMqauWq2w5bmi5c2HhQupX92DYBxPmd6k7qMSI9kG+kOVRMgusjEiRZbxqbt4UmxByQ40+ c8w== X-Google-Smtp-Source: AGHT+IHU1NaHt28BJAfE9+Eihx21gIRaHWD3wJUzNDiwOfeaS4l1n7K6hdxyKTp49hAFCJqVbhS/fDDFwRQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:2545:0:b0:7e6:b4dd:fc0d with SMTP id 41be03b00d2f7-7ea535b0baemr20a12.7.1728584846518; Thu, 10 Oct 2024 11:27:26 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:15 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-74-seanjc@google.com> Subject: [PATCH v13 73/85] KVM: PPC: Remove extra get_page() to fix page refcount leak From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't manually do get_page() when patching dcbz, as gfn_to_page() gifts the caller a reference. I.e. doing get_page() will leak the page due to not putting all references. Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/powerpc/kvm/book3s_pr.c | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index d7721297b9b6..cd7ab6d85090 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -652,7 +652,6 @@ static void kvmppc_patch_dcbz(struct kvm_vcpu *vcpu, st= ruct kvmppc_pte *pte) hpage_offset &=3D ~0xFFFULL; hpage_offset /=3D 4; =20 - get_page(hpage); page =3D kmap_atomic(hpage); =20 /* patch dcbz into reserved instruction, so we trap */ --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E247E216434 for ; Thu, 10 Oct 2024 18:27:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584852; cv=none; b=W7RgQuwhPrACHXAXnzRBiADINxjP537/EyxHqaKn4Vja2yJcuRbj4F/5TpBFho9eXse6AGYLU8PSbGq+bYdrFE+VJCjQgKGbyyDYtQMegmATXp1axSK9efTvM0ZOykq+jzs0kEZw1R7EQbwCxa3ink9fsjotUKHV4Rxtx1xEyek= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584852; c=relaxed/simple; bh=69Upu5+wnipQK1zG8Arq0ad1gowf7mjUzNjaillb+fY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=iS6DicRl8CkE2NsrcSeA5QpTV82/0cKeUwV8zpMIXwtr/HJHN5CRCdo1QYSPk5llUtPZW/14F7+pSKFjHLUH302tYlqKkyU04XtkahEzHhBWuDS+yzc+c5BPS3ijunEpo0nuxPjqaIh/Ms7fQH80gR68tMnzAzYC2KUpptBbkA0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TYCOxp1c; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TYCOxp1c" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e32e920cf6so23672837b3.2 for ; Thu, 10 Oct 2024 11:27:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584850; x=1729189650; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=DOlbrv8SiXT4cHz79KGvl4RMUSE/a2Kzzqa4ZWuWNYk=; b=TYCOxp1cKgcq2S0piBT9YJkDFLpkMk1shhIozeymloTyKk4ZTR+gIawk/AuNbFbDQ4 J8TeUkJfKYxbM2AtDWZ02S3f2cAN7kjr7P+ogU4gcZwc5SJ2dw7hcKjGSiKZnDuWFVXD 0yacIImKBYGg/kOVQBDjgOI19vyelBez+dQ9CKKVnqCCN8KpSEniZAa2O+TAIbBYuk/8 u/EdPzuUpUAPgVZP64a88b75lAGbcbHuAV/J8cWjiNeHeXpsNY+rv2VipBNaNvpPRk0N 02Tdr0mvAbTRtwDHZcGpoGEUwZ3USAPeQZ3Kyha9LYFJVBqjMm+LcYDMRXO6L/g2oyJe dDcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584850; x=1729189650; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DOlbrv8SiXT4cHz79KGvl4RMUSE/a2Kzzqa4ZWuWNYk=; b=NcRITrrfxivpJmkGdZW+oHhasBsqXXye/awyUmr9se1xSmxcVgvD8MbFoUql53cEkc v+gdmB9wdfcxeLcrVuTCZ8W7io+HgkGkgmdWJc3Ewz6x6ipu9BNcm+eo7w7oSd/Zkgb8 xMMlAVHx2pmPuomv8HMhlN26njHVyfZsLpHBSjo+Zbn2mBsnCvXSmvPS1AqWYSQl0IXf d0u1airos5zGHOw+xYJrllYt6Ya/LxBtciwsOAeNnCq/DRcSYIj/OdkTEyA+eyqY2sOH CBfY/n59j9agY8flMZJ7zgKUOpnZDV5dWHiZINFFouY/Rt2edpPLqDH0ju/GFaJH+n0P djkg== X-Forwarded-Encrypted: i=1; AJvYcCVNn4Bd4XxXMmOcucNPuR4jYoBzEjJWc/O8lXtsfyj/6pwinFoJpfQ+1kQ1TQ27m6x0IlKgzYdwey0K//Q=@vger.kernel.org X-Gm-Message-State: AOJu0Yxpd2qQiCl/QhM9/AdY8CbJBsWaaJkecvtEQIm9WgQcycV5O45G je21l6duNKDz6+1mPE9wVGa3z/jWD57rKTbhnrLAhvRzm8xNBtdc5VkUIJULWMlwOAItLSvtZPl uTQ== X-Google-Smtp-Source: AGHT+IFYivPtvbNeJBK5QJ/MFgPKbd4XQ4GDAEMeTnfD7MuZnOh5ZwxKPEpX2RNQ0xKTZSwXcPzTQuagswA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a0d:f043:0:b0:6db:c6eb:bae9 with SMTP id 00721157ae682-6e3220dbc31mr798327b3.2.1728584849896; Thu, 10 Oct 2024 11:27:29 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:16 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-75-seanjc@google.com> Subject: [PATCH v13 74/85] KVM: PPC: Use kvm_vcpu_map() to map guest memory to patch dcbz instructions From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use kvm_vcpu_map() when patching dcbz in guest memory, as a regular GUP isn't technically sufficient when writing to data in the target pages. As per Documentation/core-api/pin_user_pages.rst: Correct (uses FOLL_PIN calls): pin_user_pages() write to the data within the pages unpin_user_pages() INCORRECT (uses FOLL_GET calls): get_user_pages() write to the data within the pages put_page() As a happy bonus, using kvm_vcpu_{,un}map() takes care of creating a mapping and marking the page dirty. Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/powerpc/kvm/book3s_pr.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index cd7ab6d85090..83bcdc80ce51 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -639,28 +639,27 @@ static void kvmppc_set_pvr_pr(struct kvm_vcpu *vcpu, = u32 pvr) */ static void kvmppc_patch_dcbz(struct kvm_vcpu *vcpu, struct kvmppc_pte *pt= e) { - struct page *hpage; + struct kvm_host_map map; u64 hpage_offset; u32 *page; - int i; + int i, r; =20 - hpage =3D gfn_to_page(vcpu->kvm, pte->raddr >> PAGE_SHIFT); - if (!hpage) + r =3D kvm_vcpu_map(vcpu, pte->raddr >> PAGE_SHIFT, &map); + if (r) return; =20 hpage_offset =3D pte->raddr & ~PAGE_MASK; hpage_offset &=3D ~0xFFFULL; hpage_offset /=3D 4; =20 - page =3D kmap_atomic(hpage); + page =3D map.hva; =20 /* patch dcbz into reserved instruction, so we trap */ for (i=3Dhpage_offset; i < hpage_offset + (HW_PAGE_SIZE / 4); i++) if ((be32_to_cpu(page[i]) & 0xff0007ff) =3D=3D INS_DCBZ) page[i] &=3D cpu_to_be32(0xfffffff7); =20 - kunmap_atomic(page); - put_page(hpage); + kvm_vcpu_unmap(vcpu, &map); } =20 static bool kvmppc_visible_gpa(struct kvm_vcpu *vcpu, gpa_t gpa) --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3D8B621645C for ; Thu, 10 Oct 2024 18:27:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584853; cv=none; b=b0pYlCmbiZ2j3poNVmc26cKgnaZX8l5PC0kdFy5esvMIU5ok2zvCRS/jwvp7HGvIBe3RK1oKuzjAfqFCe72qMl5lK30TyN1+jFOygdsTonnhucqrQnmnv0s9MpSOtF5w2pGx4xEkK/dElxF4Y+cEztzBUkMATYpaTDVofg7rhzo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584853; c=relaxed/simple; bh=6VFC6ZQ3q9XjUbMWhh1JwxOVsy5FNZVXQFKK9OKSZCE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=raM/Wy5zWfZPTi/8+lTWw+kYLGtT+usVoxdh4KoRyaNvu+OUQgYdDi56rbYRiHQpTwhmka+mEqi7D8j7cKiXYqLT1oMmK8dyjOzUpIlLOPcRZNtuQX8rP5b5YJrjdGNNgFN28Dhrxi7lYPHyzYQarT0+XC5fRxto4sEh3lDsZdE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=kkhtwVA3; arc=none smtp.client-ip=209.85.210.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="kkhtwVA3" Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-71e019ab268so1590140b3a.2 for ; Thu, 10 Oct 2024 11:27:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584852; x=1729189652; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=fZpnWf51XalE5JTpUzeqj9xBy85lPuBTlHLH/3crpzM=; b=kkhtwVA371S5ZHAeHS4k3XRilYdB/vGzQgltlsRLi0pp7zBjZNHvtNTHMuvtzL7CAF 6lNYOD3BDWvBSZ9L0rmgwIlueXDyhW/RA5LQ0k9HKGrI7aQrISXXF0UVGv1X7QGYXfmX nQAjpUZF9ow0QYSRFnzZERqDxBCGJILWB6bbaaxW+qU5NHSMCV90Jig9BhxHgjAoV1za TIvRad+uoWgS/qyZB0Ga3oI/+RuGbJNOcXT3YNU2DEx7phcpFbVb8mMgls/S1lDHVstd CsKALT5sXHk1JsYmc6mHfOyTLTtTo+ZkddLgoeZS9J57GTp1PUXVbgGiI+DyyF4UdFku wG3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584852; x=1729189652; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=fZpnWf51XalE5JTpUzeqj9xBy85lPuBTlHLH/3crpzM=; b=NZmBfE1cHTjzMktwi53Zx9iLjfnfReDIWKID5fmSZr5MivWsIEFaaIPf9/deEhlj+h dqA6nnRqTgLQk/sKhH9A8uYjWKgwQSnnCy1KOx6O682Ya1Pnmi1MYnSJ/d/G5CWS/LDU gjc8uP0K37i+FPUKiCPx5W6UGXZFVe8PKIvD4Dc3vjWlVPhn1GYF3T83vcffecFC93w1 HCy3hzQkG/FoNF6UuP8rJTd/OfntMVy5d2tV9TImcLb2VECewmZmWYSulSdiZ2Ci7HZO l7ZOfwY/uIrRJea81FYJl3LZzbX0bgaP3rKudRtrNyE0xrA5EW3cKtsjb9wY7kPLH54i f/tw== X-Forwarded-Encrypted: i=1; AJvYcCVzrH4YDhy2qeB81S+sgFmmUPnQzTpgLbmY+BdO84fx6hnSoHK+93P9l84PyTybDhFPEdV2iZTL4O27NH8=@vger.kernel.org X-Gm-Message-State: AOJu0Yw5pSptV67WOyHMTC3KIW89GeKQlY7KAOjwQfD+o+mTdYijIYQq u0rf7gcFu2zx4Chlky17T61ejehjapAs3mNE/MW8qrSr2e+hD+6owc99DMFyZLzMRNDUjV13ijr sYw== X-Google-Smtp-Source: AGHT+IHVUYgAS8Gi1OfHaumBjOVs/68U//Xc3NGlK2wEye9l2w1SuCdDQ/WQfsMbBzqgGqsCESzcy3AUxv4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:93b4:0:b0:71d:f64d:712c with SMTP id d2e1a72fcca58-71e1db65a16mr7365b3a.1.1728584851459; Thu, 10 Oct 2024 11:27:31 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:17 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-76-seanjc@google.com> Subject: [PATCH v13 75/85] KVM: Convert gfn_to_page() to use kvm_follow_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert gfn_to_page() to the new kvm_follow_pfn() internal API, which will eventually allow removing gfn_to_pfn() and kvm_pfn_to_refcounted_page(). Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 696d5e429b3e..1782242a4800 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3145,14 +3145,16 @@ EXPORT_SYMBOL_GPL(kvm_prefetch_pages); */ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) { - kvm_pfn_t pfn; + struct page *refcounted_page =3D NULL; + struct kvm_follow_pfn kfp =3D { + .slot =3D gfn_to_memslot(kvm, gfn), + .gfn =3D gfn, + .flags =3D FOLL_WRITE, + .refcounted_page =3D &refcounted_page, + }; =20 - pfn =3D gfn_to_pfn(kvm, gfn); - - if (is_error_noslot_pfn(pfn)) - return NULL; - - return kvm_pfn_to_refcounted_page(pfn); + (void)kvm_follow_pfn(&kfp); + return refcounted_page; } EXPORT_SYMBOL_GPL(gfn_to_page); =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 800892185BB for ; Thu, 10 Oct 2024 18:27:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584857; cv=none; b=AWNAjgTauhubmdW4JzI3nudi7lcDI4kC3KdHHlZVn+MiO7TtRx8EmQHiLfQL4A0uu4GQ5OzSGtYMfKiX1BV74Tf6RCJFK6s7I7qQOzdPgebiBCY5P3M7XgUHjEy3xDBwqNCx+pfRgWdk6DTMuVQEME6elvUjuvWx+l83jTd1b1g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584857; c=relaxed/simple; bh=h3dpe04J9rq+dGEj5vV4pEUoAEcWlYwK7aXFrp+Hc5M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=m4O7MosMVtYR9J51Jtdw6+J0FHM8FWiDzMbeyEHj8YsFnWR3lWC/lTYy8aR28L35DRG4cOEan2t8doTG6Xr01LI6zYIFoPQaqrl+fLPE30xAQsnY3/MG09fHNO4Wloy4YzxbyNsyAucdQ/2xlRfPyn8Vuoq6h6Kpp8YQmaP2iXc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qpMT9mNV; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qpMT9mNV" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e21dcc7044so22862057b3.1 for ; Thu, 10 Oct 2024 11:27:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584854; x=1729189654; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=fQZoDf0NIFdWK0Ory3IxzF5D2dmoMJWboRGGHV/nWyI=; b=qpMT9mNVMlM3JYWe1f9UU6REj2+oc4kj/yK50HSO4pjHutnOBh7JNWul7tfhz3o4gh +vruI0VZ+Fq6kyfhQqfEBqSUimUY7mr4tcemk0pyokvcHPV3ClhMsBLLWdPlziIY/gZr qbn/UhT5Hr+z4AUeczlu7097wCvqY9n+PJbB4r96PtwCFdRgaEAiAjYU0lZuAW3qFAjq oJfimXxrRFyYcyH8hc0hpxmWiRZ23C/Pk0qrQf5Tx+NbiOGiHhn+vpApay6F/DbuQk85 z2HtIRTfoM/9gXAmhP050j35DEG0wsNyQRzI2Y5vZAgVPQ0yrJEQ+eN7pVysywTjL1vy 1DuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584854; x=1729189654; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=fQZoDf0NIFdWK0Ory3IxzF5D2dmoMJWboRGGHV/nWyI=; b=Twf29IWBzADZSVJIDbTyNQ0TYz4aMCpXl+llgyl+VTEhWIfLp0RB61AVvHzqhk/Udp FMqxEHguVuRIDvYZWbna/cuXKnTqjq/a0KBH5Zz5L1yghziNlpErcOsyxMox6ek1FnMD Q6NSt6ey0JZNDmhOFpSMhTJJ0dYwbiNPYWZQd1TpKRScZeNVfPuGlcRk9JDuIQHv3ZTo ctf9P1Znb8DqD6eIDRYaLqLKXDpqGqudfnM5CPUYirk8PE345m6ALjLvQA7Q2vnTV7Jw XO7UVqeJYzv/7awEy9/VkPFcZpAhVCurCElEFxMkbJ5PVBdXsxFO35RAmHKQWqRkHU3/ MJcQ== X-Forwarded-Encrypted: i=1; AJvYcCU00XD9goY/yKI9hr4eQmjz8khp5Yo8HJ1l7Ay+TITxdY32x1jxBrUEIeg7RNjTnXzM4wRIvWYsTcCAFfw=@vger.kernel.org X-Gm-Message-State: AOJu0YwLaJPR2JNVbJNBUz1wMoUdEL+jGJWHr1Z+uHWcXjG17YN0ZDhR LbFj0zPA2QKDUs202lUhj3i0Gm3QDfHuCeenYK7+aGHHfOV4D9Lpkz6cSG7zYTtjkLGH8derBsE 5yA== X-Google-Smtp-Source: AGHT+IGkMjnJcd2oWHVLgC5xfQJb4mHLQjnUvb6XgW9ZV8VH9lzrMuhcRfl3e0TBTVqXdJBO9V/Z2ttoh2Y= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:4091:b0:64a:e220:bfb5 with SMTP id 00721157ae682-6e32213fbacmr192077b3.1.1728584853563; Thu, 10 Oct 2024 11:27:33 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:18 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-77-seanjc@google.com> Subject: [PATCH v13 76/85] KVM: Add support for read-only usage of gfn_to_page() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rework gfn_to_page() to support read-only accesses so that it can be used by arm64 to get MTE tags out of guest memory. Opportunistically rewrite the comment to be even more stern about using gfn_to_page(), as there are very few scenarios where requiring a struct page is actually the right thing to do (though there are such scenarios). Add a FIXME to call out that KVM probably should be pinning pages, not just getting pages. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- include/linux/kvm_host.h | 7 ++++++- virt/kvm/kvm_main.c | 15 ++++++++------- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 9f7682ece4a1..af928b59b2ab 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1213,7 +1213,12 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm, int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, struct page **pages, int nr_pages); =20 -struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn); +struct page *__gfn_to_page(struct kvm *kvm, gfn_t gfn, bool write); +static inline struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) +{ + return __gfn_to_page(kvm, gfn, true); +} + unsigned long gfn_to_hva(struct kvm *kvm, gfn_t gfn); unsigned long gfn_to_hva_prot(struct kvm *kvm, gfn_t gfn, bool *writable); unsigned long gfn_to_hva_memslot(struct kvm_memory_slot *slot, gfn_t gfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 1782242a4800..8f8b2cd01189 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3138,25 +3138,26 @@ int kvm_prefetch_pages(struct kvm_memory_slot *slot= , gfn_t gfn, EXPORT_SYMBOL_GPL(kvm_prefetch_pages); =20 /* - * Do not use this helper unless you are absolutely certain the gfn _must_= be - * backed by 'struct page'. A valid example is if the backing memslot is - * controlled by KVM. Note, if the returned page is valid, it's refcount = has - * been elevated by gfn_to_pfn(). + * Don't use this API unless you are absolutely, positively certain that K= VM + * needs to get a struct page, e.g. to pin the page for firmware DMA. + * + * FIXME: Users of this API likely need to FOLL_PIN the page, not just ele= vate + * its refcount. */ -struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) +struct page *__gfn_to_page(struct kvm *kvm, gfn_t gfn, bool write) { struct page *refcounted_page =3D NULL; struct kvm_follow_pfn kfp =3D { .slot =3D gfn_to_memslot(kvm, gfn), .gfn =3D gfn, - .flags =3D FOLL_WRITE, + .flags =3D write ? FOLL_WRITE : 0, .refcounted_page =3D &refcounted_page, }; =20 (void)kvm_follow_pfn(&kfp); return refcounted_page; } -EXPORT_SYMBOL_GPL(gfn_to_page); +EXPORT_SYMBOL_GPL(__gfn_to_page); =20 int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *= map, bool writable) --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AA2A218D91 for ; Thu, 10 Oct 2024 18:27:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584858; cv=none; b=PbfVTLs+YEqkNtTt2xIrsXhQtSUAmYQBfr8s/hGKXQXyRrMdFKX1AmrYANk9FyER3Dls8DA2Xg0bHmPvBjLkIx4uweKwnWtGFZPCFFFCnGGkF1IW0YKzVhPji52OVbW7xN0T615fOToaoiXXTmpKWhfm4ftmqxUf05zaI0j4RYo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584858; c=relaxed/simple; bh=FErwCaq8NdcoMJPhpWM/50ZCDjLg7qNPTjMbO8/YuvM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ar/O6jh/n94QUQ8i/aDniA3vka/mMRStZiyzPcaxcDcssRNiyjoRerlxgS+zMqqL3kkfRN0jT9BUzOh7emfnwwOiaKO41/H1ZQrJTwD/YSqAPr6GjOduEVZMypcKkKx1LN1w3e3eGbIGM5snTTIl1pNc/R1xI2r16npd9q63YeM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=yE9BBZab; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="yE9BBZab" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7db9418552fso1114277a12.0 for ; Thu, 10 Oct 2024 11:27:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584856; x=1729189656; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=04eHUk6AClz9cRMQcjlWjE46ZR1uXceY2ogFB00LGDM=; b=yE9BBZabvoeIqzV8CVJHKuzxLfL2NYkYVtIdFaPw/BUSfPofvy1yEt9sV7egKYCr6h ECdzuEmt/3ij7vj98XqRqiNePbTZy3r1+KnoPq0Y9I3SWy9nfiSrfM3+HN5Tt1cuMsMe P5BqP8mEprGZtOyicY4QMcSjOoZuBJSucYO3xKIILjywU8h/g9JRWiGzdULPCY/h7RrW gs/lx0IgmUkEs0f46fCv0eA4nTg5+x1s07DF9AqCC/d91Tc45agqIYRUIW48yJgViOz+ 6RSp5Afamp+TyeZzsa3QwnWcskOkhHtKwtjtk2jBDyPhGpbSYoTMcCOFE9DIiSVIltMb h9yA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584856; x=1729189656; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=04eHUk6AClz9cRMQcjlWjE46ZR1uXceY2ogFB00LGDM=; b=FYF3wgFookw2rD4+CR7Z8p3YOy8ruPkT4TlUeG5zyEmJ1YyJDITGhFuWSeYp7oJnyr 7neTF/aNDl1S7mU+Zx3QKEjWt/SVZvq6LZER/OjCygGG4vqcSSmzO2nsDFcrU893foOa v6aLbsIqFIZsENyxmWEPbTkzKzeX/+sS4AL5I8WDK8Sr1adGdlrqKTwdprGbTV2JVnlc ZSOK49lONEbqePiKLn2o63+0RDpFJs3CXmtdT5gortG1Kg1r7SUcz04E2zbBymrcYE9X qiljXaQefsXjq6F2Z4PMFcZSb2C3X9yA7vb8Cxwh85bzKUdvLpbvyE101TiPjnVMZ9WX VYQg== X-Forwarded-Encrypted: i=1; AJvYcCVChv5SptX+rXkjWu/HcZgzhM9MYOmjfldg7MU+hhmbiHl7mnLetEw1PNW0XAY+zQ8j8mJHoWRwL1/MTbc=@vger.kernel.org X-Gm-Message-State: AOJu0YzRv8PW5pzS6+eskpIEhYoBwAdG6pvZdTY1HF676di6D6SVPMdv zvqSzv038Mxcb9G5H/XTNZnc32VrYz/0DV4j7taj7oe1JwlOY53wtun2GcpQf3fO/Oi3Nu/xXPl hzg== X-Google-Smtp-Source: AGHT+IHGiwNd3UbnZUzPre1tk98dNYoM10q7gFGG4lq0PldsJuDm1rxMHI+g8Yc3UsVnmkzWnOurUMFmqIQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:41c4:b0:20c:8f78:67c9 with SMTP id d9443c01a7336-20c8f786946mr26385ad.5.1728584855618; Thu, 10 Oct 2024 11:27:35 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:19 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-78-seanjc@google.com> Subject: [PATCH v13 77/85] KVM: arm64: Use __gfn_to_page() when copying MTE tags to/from userspace From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use __gfn_to_page() instead when copying MTE tags between guest and userspace. This will eventually allow removing gfn_to_pfn_prot(), gfn_to_pfn(), kvm_pfn_to_refcounted_page(), and related APIs. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/arm64/kvm/guest.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 962f985977c2..4cd7ffa76794 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -1051,20 +1051,18 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, } =20 while (length > 0) { - kvm_pfn_t pfn =3D gfn_to_pfn_prot(kvm, gfn, write, NULL); + struct page *page =3D __gfn_to_page(kvm, gfn, write); void *maddr; unsigned long num_tags; - struct page *page; =20 - if (is_error_noslot_pfn(pfn)) { - ret =3D -EFAULT; - goto out; - } - - page =3D pfn_to_online_page(pfn); if (!page) { + ret =3D -EFAULT; + goto out; + } + + if (!pfn_to_online_page(page_to_pfn(page))) { /* Reject ZONE_DEVICE memory */ - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); ret =3D -EFAULT; goto out; } @@ -1078,7 +1076,7 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, /* No tags in memory, so write zeros */ num_tags =3D MTE_GRANULES_PER_PAGE - clear_user(tags, MTE_GRANULES_PER_PAGE); - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); } else { /* * Only locking to serialise with a concurrent @@ -1093,8 +1091,7 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, if (num_tags !=3D MTE_GRANULES_PER_PAGE) mte_clear_page_tags(maddr); set_page_mte_tagged(page); - - kvm_release_pfn_dirty(pfn); + kvm_release_page_dirty(page); } =20 if (num_tags !=3D MTE_GRANULES_PER_PAGE) { --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9063A2194A1 for ; Thu, 10 Oct 2024 18:27:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584860; cv=none; b=o41kfGmdRbb1Laan5K5alv1I+KTLay62zuxOInn9WWFRY3x4wEYHmE9CukD1SfoTCjkM5wPcVGKlv/SNnNfeJYleq9sg6HfEihQ4z3P830TLyEe3pL9QR+eSGM/ycegJT8Ck8q/aA5eE9mPAzNsHcnn0zRoYwrXqIDHL8cvkGYA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584860; c=relaxed/simple; bh=WU2bYG9iOpd2mUwv0Kp3QHVdfdxFDeft8vY+L5Knm2Q=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uGqJmrcRUknU1k66/+3iopImUChGAK5vtYCYl1aFqkr4z6SwRy1QwIIaliUwn0h851YSAHPHQR6/2gf0Qigi0m9TipuBCkgPiqmO0VltnfAsfYTRvT5ch0vQQKuDMFVtj66SBorVi/BE49YeZRHSvWL6cQiN4xBYDPlFw/ChIOw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=rVwSkyMU; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rVwSkyMU" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-778702b9f8fso1040812a12.1 for ; Thu, 10 Oct 2024 11:27:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584858; x=1729189658; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=43Dd+lWRPu/QQcFLGDLhCyyiH7uAqu5vbmltTcj5okg=; b=rVwSkyMURh+jnY5mhdfmEcQccS8IDp53PkuungLNC+QuxTGyFwJnXYuV4KpMgc91FZ bFvRWLz6gCAuL8I4pFdUmjoij8nlXt+RkgsWY/Ra7wXd/1Wau2HIby3umw7iSRAB2EmN ut+5qUUcerlx01e/0fA02Swap/VPizSrvEggtnZUOJ3WrTusH7oqshsyTmbh3f+dI20n uyRVKZBCTZawUdXzMhX2Ok6XvgFbIG0mVs+37d+uWqG7tQt6n6PSxhsbtHnD6bpftV5j rLI2JXT2gaMRXtSRAfpKWNB6WrSgK1uzbqCfQX+rH4CoQXpbwXxDDS5Cl51S6vCpTwBp n5Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584858; x=1729189658; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=43Dd+lWRPu/QQcFLGDLhCyyiH7uAqu5vbmltTcj5okg=; b=DacWp473OxRUFlxQ1X4Gb58OmUv5M/ihDOxsQ/v7x+3hxfxGbky/i/eSrRREDQ4h3t FMa3q5yC7Lp5lAgFRO9j0g7gNSB4j9/j6h1n+9z4IdAKrJCYqcsHunKMhfHwKWoAnvcd mwlUOPad6eIAXAQGQZGeHpXDXIzE2qFRJG3fFrjiLwo74HCAZzYNkPEjmtDG67KN8F+h /TZEbZk/gjF2BBnkAWkj3fwLKwlSmhAPR7GApNRbjnCfL3bmfWrdx4DRNa1ehh/aoVUB fD3tKUM884XyPxhlLRChgQ0uilVmpzDvX5Xx8r5Z/9WhSVgdLvIpx59kysGMu3Ci58CE IE/w== X-Forwarded-Encrypted: i=1; AJvYcCWtax0jVvWNJo0rFyUFlzk6YzYxei3osxx1Wv/2erOxoHp5edT1GaoPtlsfQfngJgNerzTwB0CEl5lWLFw=@vger.kernel.org X-Gm-Message-State: AOJu0YyqtkLkMA3ezzdFPbK+hWvtqOWnSpCBsMWrnUB7EmVcesdS3OW+ txWAqVkyivlbNLY5ShlXEbXcj3egTdjAz75067U/eHAJ7H3fTv9LIdbamXdiuH6b64UVZALbCe0 Lbw== X-Google-Smtp-Source: AGHT+IGjs5y0lK7jvR8/zbNWJNDHfDWRUWSPa98vwOnU2bZ6sNQJgoH/6j7HDiYxyC+lHu033MhghQMujcY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:b208:0:b0:7e6:c3f2:24c2 with SMTP id 41be03b00d2f7-7ea535dce6cmr30a12.6.1728584857741; Thu, 10 Oct 2024 11:27:37 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:20 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-79-seanjc@google.com> Subject: [PATCH v13 78/85] KVM: PPC: Explicitly require struct page memory for Ultravisor sharing From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Explicitly require "struct page" memory when sharing memory between guest and host via an Ultravisor. Given the number of pfn_to_page() calls in the code, it's safe to assume that KVM already requires that the pfn returned by gfn_to_pfn() is backed by struct page, i.e. this is likely a bug fix, not a reduction in KVM capabilities. Switching to gfn_to_page() will eventually allow removing gfn_to_pfn() and kvm_pfn_to_refcounted_page(). Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/powerpc/kvm/book3s_hv_uvmem.c | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_uvmem.c b/arch/powerpc/kvm/book3s_h= v_uvmem.c index 92f33115144b..3a6592a31a10 100644 --- a/arch/powerpc/kvm/book3s_hv_uvmem.c +++ b/arch/powerpc/kvm/book3s_hv_uvmem.c @@ -879,9 +879,8 @@ static unsigned long kvmppc_share_page(struct kvm *kvm,= unsigned long gpa, { =20 int ret =3D H_PARAMETER; - struct page *uvmem_page; + struct page *page, *uvmem_page; struct kvmppc_uvmem_page_pvt *pvt; - unsigned long pfn; unsigned long gfn =3D gpa >> page_shift; int srcu_idx; unsigned long uvmem_pfn; @@ -901,8 +900,8 @@ static unsigned long kvmppc_share_page(struct kvm *kvm,= unsigned long gpa, =20 retry: mutex_unlock(&kvm->arch.uvmem_lock); - pfn =3D gfn_to_pfn(kvm, gfn); - if (is_error_noslot_pfn(pfn)) + page =3D gfn_to_page(kvm, gfn); + if (!page) goto out; =20 mutex_lock(&kvm->arch.uvmem_lock); @@ -911,16 +910,16 @@ static unsigned long kvmppc_share_page(struct kvm *kv= m, unsigned long gpa, pvt =3D uvmem_page->zone_device_data; pvt->skip_page_out =3D true; pvt->remove_gfn =3D false; /* it continues to be a valid GFN */ - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); goto retry; } =20 - if (!uv_page_in(kvm->arch.lpid, pfn << page_shift, gpa, 0, + if (!uv_page_in(kvm->arch.lpid, page_to_pfn(page) << page_shift, gpa, 0, page_shift)) { kvmppc_gfn_shared(gfn, kvm); ret =3D H_SUCCESS; } - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); mutex_unlock(&kvm->arch.uvmem_lock); out: srcu_read_unlock(&kvm->srcu, srcu_idx); @@ -1083,21 +1082,21 @@ kvmppc_h_svm_page_out(struct kvm *kvm, unsigned lon= g gpa, =20 int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn) { - unsigned long pfn; + struct page *page; int ret =3D U_SUCCESS; =20 - pfn =3D gfn_to_pfn(kvm, gfn); - if (is_error_noslot_pfn(pfn)) + page =3D gfn_to_page(kvm, gfn); + if (!page) return -EFAULT; =20 mutex_lock(&kvm->arch.uvmem_lock); if (kvmppc_gfn_is_uvmem_pfn(gfn, kvm, NULL)) goto out; =20 - ret =3D uv_page_in(kvm->arch.lpid, pfn << PAGE_SHIFT, gfn << PAGE_SHIFT, - 0, PAGE_SHIFT); + ret =3D uv_page_in(kvm->arch.lpid, page_to_pfn(page) << PAGE_SHIFT, + gfn << PAGE_SHIFT, 0, PAGE_SHIFT); out: - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); mutex_unlock(&kvm->arch.uvmem_lock); return (ret =3D=3D U_SUCCESS) ? RESUME_GUEST : -EFAULT; } --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 289D1219CAB for ; Thu, 10 Oct 2024 18:27:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584862; cv=none; b=cyeuYxzynZF8Wr2KlZYHL07MGESfmUq3qHWdOHiKrywFBnzqn0Yjnpuy3Bl74VwDMN2YjAmkXdB+WyrU07gHTY3++fGDzSbJyiJTZY8BW74Sfcx8yC76yT0iyRxf+f+UTw3hodeAKgFF8Wf0gc0VO9kXmv36SqM76AhbepBbT7c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584862; c=relaxed/simple; bh=5AMkvnsvbpt8lQB2/GwTipD8a+sP1HpfiWdIDM6I+Qg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=k0+uSYkcjIWBXBN9SBuQjErd8riRu8NAz4wwUuIM633rGPFLuT8VkSrSMvdlXAiH+le3u5wwtH1FcqBRK1sdoCbuSpjizws/XksJ30e4AtuaixOdjRhRnmO1EhlYYrJTZsc1C+9P2kH7RxLwiR3Qlcv3f76WJylKei7fVMB54OQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=N38MzbV0; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="N38MzbV0" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e30fe2a9feso16561847b3.3 for ; Thu, 10 Oct 2024 11:27:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584860; x=1729189660; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Q1MnEbvSXNK1DBYHKiiPeGTlNBdM3PqCgrnaMRmX38Y=; b=N38MzbV0Opg6YS04YGXSDcFzpDbMb+69BxLeF5I/m5HWLMmyQi0t6ZpoTakUKsxG+Y x+62Qe6FkM5MXIYUqzR7ZuT5vHKJWIzNuNGguoPNHzQzRheqSS6wmdrvwE+E+IZ22h0E s9iPV/7n2nG5SP0RkA+0+K0w1YN5FwNO8br3GqOTRvzCCC4t0JdSc920HAAUmDRsQA+Y PwuIw/p9M4ndYcmO76PWt2AyRJmaA5G/ms6keQLf2kLMT2JU3Pns4jgtRATr7ImtkXkA jP4B02S0wLHqIbb3wVDpdw/3eyHLn7HPlSOgctNgsdNUTRNyJWcnfufIx/x/sAAhixZ3 RTkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584860; x=1729189660; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Q1MnEbvSXNK1DBYHKiiPeGTlNBdM3PqCgrnaMRmX38Y=; b=WapaQuDdDqUycyT9N1knry+7cOMXaDQDOZmrMhh8wFp80YcGdYXMtOvMqCNspfig8A 0nsi6kokyOdEdWrDq5mOCQ2eCdfgooXUxQ7O1nup4aieS1oesVuU0BIcIL9iuvL7T+wz G/LsZLooRwdf2d0RVBRl/IVf8G1ABeNTWbuiY47QM2lMK0CIcCoddYJQpBTZvw5JI45K D57eiJQCLPkmclCpo8AU+40qIX8RTzcFCcQWHpYhNty3vbfyQkB4/9OFIwGVwmQoDQ0R 2UUHwkLwr9o3wH5jVlibX2RpvEmy665WemJT7N5QmcZZo7xS5mDn6YnWUy5dNXOuO/C+ rYRA== X-Forwarded-Encrypted: i=1; AJvYcCXYW6JTfucaOMfQez16+Q6rkYkplaleuutNllBnpls9bz3XVrLON29kNjqqVhjAtXGDqq/YNZUX8b1bKyI=@vger.kernel.org X-Gm-Message-State: AOJu0YxeMeQBK8/9EClKE+S/Uiy5JsMS8/Z+vNkDTpuI20P+TX37j+s6 mvhCkTzUsZbDplMtJtxHWTX8PpGYWu2Da6mzRvfPDCP4+YLGMUyc7i3soqYopH2vk2+2+MnOH4R Diw== X-Google-Smtp-Source: AGHT+IGvHgo281GUU728l2kpqfRNdoQ8M4AtWxtMdwK+Sbd/Nmo7aGxeBXrCmxWS+HWzNnhFIgaVqhxkocM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:f20e:0:b0:e11:5da7:337 with SMTP id 3f1490d57ef6-e28fe3505c2mr129032276.3.1728584859805; Thu, 10 Oct 2024 11:27:39 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:21 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-80-seanjc@google.com> Subject: [PATCH v13 79/85] KVM: Drop gfn_to_pfn() APIs now that all users are gone From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop gfn_to_pfn() and all its variants now that all users are gone. No functional change intended. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- include/linux/kvm_host.h | 11 -------- virt/kvm/kvm_main.c | 59 ---------------------------------------- 2 files changed, 70 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index af928b59b2ab..4a1eaa40a215 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1274,14 +1274,6 @@ static inline kvm_pfn_t kvm_faultin_pfn(struct kvm_v= cpu *vcpu, gfn_t gfn, write ? FOLL_WRITE : 0, writable, refcounted_page); } =20 -kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn); -kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, - bool *writable); -kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn= ); -kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t g= fn, - bool interruptible, bool no_wait, - bool write_fault, bool *writable); - void kvm_release_pfn_clean(kvm_pfn_t pfn); void kvm_release_pfn_dirty(kvm_pfn_t pfn); void kvm_set_pfn_dirty(kvm_pfn_t pfn); @@ -1356,9 +1348,6 @@ unsigned long kvm_host_page_size(struct kvm_vcpu *vcp= u, gfn_t gfn); void mark_page_dirty_in_slot(struct kvm *kvm, const struct kvm_memory_slot= *memslot, gfn_t gfn); void mark_page_dirty(struct kvm *kvm, gfn_t gfn); =20 - -kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); - int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *= map, bool writable); void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 8f8b2cd01189..b2c8d429442d 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3039,65 +3039,6 @@ static kvm_pfn_t kvm_follow_pfn(struct kvm_follow_pf= n *kfp) return hva_to_pfn(kfp); } =20 -kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t g= fn, - bool interruptible, bool no_wait, - bool write_fault, bool *writable) -{ - struct kvm_follow_pfn kfp =3D { - .slot =3D slot, - .gfn =3D gfn, - .map_writable =3D writable, - }; - - if (write_fault) - kfp.flags |=3D FOLL_WRITE; - if (no_wait) - kfp.flags |=3D FOLL_NOWAIT; - if (interruptible) - kfp.flags |=3D FOLL_INTERRUPTIBLE; - - return kvm_follow_pfn(&kfp); -} -EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); - -kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, - bool *writable) -{ - struct kvm_follow_pfn kfp =3D { - .slot =3D gfn_to_memslot(kvm, gfn), - .gfn =3D gfn, - .flags =3D write_fault ? FOLL_WRITE : 0, - .map_writable =3D writable, - }; - - return kvm_follow_pfn(&kfp); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); - -kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) -{ - struct kvm_follow_pfn kfp =3D { - .slot =3D slot, - .gfn =3D gfn, - .flags =3D FOLL_WRITE, - }; - - return kvm_follow_pfn(&kfp); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); - -kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn) -{ - return gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn); - -kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn) -{ - return gfn_to_pfn_memslot(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn); -} -EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); - kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, unsigned int foll, bool *writable, struct page **refcounted_page) --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCEF121A6E2 for ; Thu, 10 Oct 2024 18:27:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584864; cv=none; b=DRa7/F1KPPK09mIhjIWebMF15i1u/RZUSvqa5BL/ZSwrNRl+dvtf86YVO5MXaAGlaAR8xa1Pm87huOh4u2nBCOeXZqA8Mza1u2KAlMeE3EfiY7H2IsYUNxXyrur1ONahiYihYzEIac1nhEtwRa9tixkAh0pEf+Mi/TIeMLVXM3o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584864; c=relaxed/simple; bh=Xp30Q5aCXuVbCuroW5D/p68jGCOUTPuU5mMuuixoyAE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=puIlmGdGz8TejKviMl8c/ZAlzHZhOZ/dFXaf8XeQZOaHf/KbY8v10wXbWTW/1iMsyRk1PXWcnwzV6VlRg/x/slbt8p4NhsEDjFqIhPg4Iuxw2OqtRPH3bdR9YAyNrHhdJIazuDZFQFWVBRRgD+XAz6Fusfwb+enzvH30tA7+Fbg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TC4r7pK0; arc=none smtp.client-ip=209.85.210.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TC4r7pK0" Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-71e1e989aa2so1656601b3a.3 for ; Thu, 10 Oct 2024 11:27:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584862; x=1729189662; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=A6YyPrl/aVt4zvDZFXUjzF/Zy9evtaTkH7tjbHPMI5E=; b=TC4r7pK0g3B0mqcPz8ZXxRS8Rw8BfJUEjGW4S/zxeQ0UkyLNagAVzGBjTNdiDb2SpF nBffEhUIrykBpiBbRrDSiRgwT5seeVZBDRzQyYetSIdUzr6J3plmmVwFPQ6RY8jHlo2a /bzTNyfYCg7FOcuV8L8aRSCfF0OxPZH0J98kU8yMIJP4RIJQCk+K5i6C+TXRFdGZ6eYI vpEeOfusRR2SHfTSTJqQIXxoxJLlil/pU/N0igi2UlS4qu3p4+hgRBxyMsiC7c3vwh79 jWUX53zTsO2dJgzKEQNNHmt+hOKbbgUbybt44zCMmnLOdc8l2Lu5JrNAilAaQXl0kghC 1L/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584862; x=1729189662; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=A6YyPrl/aVt4zvDZFXUjzF/Zy9evtaTkH7tjbHPMI5E=; b=BHPGgOFZkZjNDMc9bh8q7/h8NmBjIIF5KuFu6Xid5i41G+DmpYAIJdA11ME/RK8cXE oMsOS9lgIDR8mFWCpEvCB9hZKW9cQGyvwWolh7mRZpDx9iNiXtSnHb8QchhefPaeihX5 PSmj92COSylUUxOZ4kqw/uHuBFVRh2EbdHcvtZLiuSjnbuDHK+G3eXIkOyUMF6i4u2ql i6Op0mqullLjaTOB3SvtYC13xRvGxaJn4gaecbp2QSADeYTW4y8cHBR5tWcxmHhBhjKx v0DFc8pJDMuSIwX4LxOQU+dYQJiMPKCesVq3HnHHMziWDxzUhsWfr/Smdf1hAB9sMXb+ X/Yw== X-Forwarded-Encrypted: i=1; AJvYcCWSxouxKmPBFF6Ww2PmfKjXx8tLadtguDCroNl1aeIjsXJuw/Dko7ETxX18+pxmAz2J34uV2Ji8KmrjLX4=@vger.kernel.org X-Gm-Message-State: AOJu0YxDR4HgC4LMCTK/yHnl8lsuw33k7bqNc7O4HnjmE4M6Or3MiZ9J i4p4747mxily0WMR8LEG3d/UclYXxt/jUbfNkTk2DICPaEaFDk3VLapxePNbVuDTPG6YY/AJK51 2Lg== X-Google-Smtp-Source: AGHT+IFRVsKTyvnlBUrQtNNg8zvYT0OU1qw5Av8a7Gq/P6vdrCxit4uLl8sEcLrSKXXTWA/vla3myJTtPpY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:9443:b0:71d:ff10:7c4 with SMTP id d2e1a72fcca58-71e1dbd1d0fmr15196b3a.4.1728584861919; Thu, 10 Oct 2024 11:27:41 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:22 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-81-seanjc@google.com> Subject: [PATCH v13 80/85] KVM: s390: Use kvm_release_page_dirty() to unpin "struct page" memory From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use kvm_release_page_dirty() when unpinning guest pages, as the pfn was retrieved via pin_guest_page(), i.e. is guaranteed to be backed by struct page memory. This will allow dropping kvm_release_pfn_dirty() and friends. Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/s390/kvm/vsie.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/s390/kvm/vsie.c b/arch/s390/kvm/vsie.c index 763a070f5955..e1fdf83879cf 100644 --- a/arch/s390/kvm/vsie.c +++ b/arch/s390/kvm/vsie.c @@ -670,7 +670,7 @@ static int pin_guest_page(struct kvm *kvm, gpa_t gpa, h= pa_t *hpa) /* Unpins a page previously pinned via pin_guest_page, marking it as dirty= . */ static void unpin_guest_page(struct kvm *kvm, gpa_t gpa, hpa_t hpa) { - kvm_release_pfn_dirty(hpa >> PAGE_SHIFT); + kvm_release_page_dirty(pfn_to_page(hpa >> PAGE_SHIFT)); /* mark the page always as dirty for migration */ mark_page_dirty(kvm, gpa_to_gfn(gpa)); } --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D0B8021A711 for ; Thu, 10 Oct 2024 18:27:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584866; cv=none; b=mK5ykx/4dtKZ5YtbXWe2yyarXYXj8yonNoww0K3C3g3jUbcbz0dLvCajfUb/sHgS0g4YiGAeDCTDsYvXbK9p2P8Zu+6S4s8RS4fFzQCR6P4a28rrB2g07WyqDoKaISLwhmKVqSlRwI1FD5f3KwcGl2FhPTswrRwXSFio6qICv8Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584866; c=relaxed/simple; bh=Ioj1P42KohQLhw3WPnqB6EBTWlPlGqFgp6J41ckJwc0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XYrLuRaIfRql+ZJGSxakxuuBb6y0NOkxnD+4QCTzVfuMAdFlGy9X8gjpz/cUghy/OGI2nqkhA4ZFko+UsdKPdSBHQ2H06T7Bi9bFHIV73r5DEENqvWUSMqbIAn6nncrMerFLnJD5SkYku2qj2NdCthWsbFzmL71nh12m/sh44t8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4fg+oFS9; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4fg+oFS9" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e02b5792baaso1678027276.2 for ; Thu, 10 Oct 2024 11:27:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584864; x=1729189664; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=7IfrRBwnhP/ARkGggvkJ//bNWbTHtvwsCnjZUdYSoz0=; b=4fg+oFS9HQ6bkDg2/cCK4u+yUaY2HdLlUAKKyU3XiR5S4aFSzAjEWkOGXTGjmr5ePX vGkI99yd3NqFEQL5q+JkKGytg2VixnQYq4PGFlA34nG7QkyPIj6PJ0ZlrIWRk8AoR/X0 dosEXhI0H3WZr2UmtQAb75cyDhtq3eS64wMYjKmvudbvRsnzH7xd9K0hwQjEJYZEEdS5 S5ioD1i4hPY0tOXwvT+EQOL0urSQ1tchIC5V6FTIv7IWcFTlAadalixlcVZHj01gXoru OYG5pbvbDCYSnXyeBnRIHws5VAd7XO3tCVjhRsUtK87ywGe0CzqKMCGctDzPKDvNGGTs 7TYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584864; x=1729189664; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=7IfrRBwnhP/ARkGggvkJ//bNWbTHtvwsCnjZUdYSoz0=; b=L0RaevvzCKXgPk4tPPv5rqIDG5rqippCsJIoCfLitwCZoBACv9iH5wYcCoXwrUhVx3 fLU+Wwi47pogawNfweVWmvpI/c87wTkweMrM6HBERTUcESUZU/xqqC6Ax9bIshDTgq5d tS3+x3lXH888oYEXX5csNKeTEoZnJzQTd9MtJ0CUJaTflwRBWv66u+W31ngLEru0d9c8 OSr7FFm4R5Jc2/qLV5Bo/ZoIiwq3QdjZ7CVc39aOX/eBkgczBjJecHPcCLGQBNas2VX3 V3yLSaT//TFo2bSYX6JOXKLGUWD94PMh4B/k5BswtITiLnnVsJcehq5YiuUtHmZyUmkv Xxeg== X-Forwarded-Encrypted: i=1; AJvYcCXl1c76AoTEEzt2aEzKzwwWzmhwvyX+6xBLps9zWq5tTE9YswbIOQ6bpePDhMu37ksC7EYIhGMMpGFsUqA=@vger.kernel.org X-Gm-Message-State: AOJu0Yx1ZtF/nCFl0uIwNna71SzbNj+Ge+WN2MbWqf6izv0K7QnVrTta DmdqB2+g7c74Y70Ss0MJizQ1QHEtz06BRo5xoyjkN/4a/OWWeoyVFCgmUQUpa67ssazBlU/fhkN 12w== X-Google-Smtp-Source: AGHT+IEHmTL+CF0lwT1L0/srW8AumAgUZ3z+xXpT6s6F1mY7sAC4x3PdYh5uuuYsI/CKxaNz7ipzx99kqDg= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:ed0b:0:b0:e17:8e73:866c with SMTP id 3f1490d57ef6-e28fe421ddfmr136730276.10.1728584863963; Thu, 10 Oct 2024 11:27:43 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:23 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-82-seanjc@google.com> Subject: [PATCH v13 81/85] KVM: Make kvm_follow_pfn.refcounted_page a required field From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the legacy gfn_to_pfn() APIs are gone, and all callers of hva_to_pfn() pass in a refcounted_page pointer, make it a required field to ensure all future usage in KVM plays nice. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- virt/kvm/kvm_main.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b2c8d429442d..a483da96f4be 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2834,8 +2834,7 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pf= n *kfp, struct page *page, pfn =3D page_to_pfn(page); } =20 - if (kfp->refcounted_page) - *kfp->refcounted_page =3D page; + *kfp->refcounted_page =3D page; =20 return pfn; } @@ -2986,6 +2985,9 @@ kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp) =20 might_sleep(); =20 + if (WARN_ON_ONCE(!kfp->refcounted_page)) + return KVM_PFN_ERR_FAULT; + if (hva_to_pfn_fast(kfp, &pfn)) return pfn; =20 --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 62BD221B455 for ; Thu, 10 Oct 2024 18:27:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584869; cv=none; b=DoPjOZSumicjAIvSRpkagNqyqtAVCtfqKRavLyBhRjLkPvg8+FXe9o0PArrGPx5+PlqxUg5cgcZquLfhqQu3oN95czw8FlFkydmJY3ERY0ED9lWV5zgCZwOcHZQQZz0oY8dlwwD3pmYegkk7AQKaHb8YUl/AWDuRgDWunHpCcFU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584869; c=relaxed/simple; bh=PnrPzaCfKEgiP1btOoqWv8SGAKP2OnxCXZvE+imIcuc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Vt9kouPJy7S6NYYDPehzF7xGccgoPmEyu75gaL2LXgF71vj0PjpMxJQnWe/mUh43Ht6h0Q3yci4OGIFBXhFbEPkQbiu92UvVTUNR7+ehfVX7tl57sm2IbEg3Zk4SOEqtGbQ4FocWl8f9RCHRnKIvU8JGCn/C7Z4uaKM8c7b+nps= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=AchV74I+; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="AchV74I+" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e2e5e376fcso26578417b3.2 for ; Thu, 10 Oct 2024 11:27:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584866; x=1729189666; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=l0R/snR1lBsefu+Gd9v26W9gBjpskR/lINZXMlcZUSs=; b=AchV74I+JHnLrzKw1CnNCcRL41wNv//gHQc4NBTbfpjxRDy9PpJE1B/SqMD9myx1R0 mVHR+P8bSOu3/Oz7lQ1hgUtN9RY36X8gOczJcSEzTD6A4XAuLIdAiocO1EWVeSzm/4NK iNG3yhnElAt6f+kb1qQgRYpbE90dKeTPD0ImYwKFpZwH+xttYCi4J3DbFBX+y8+g6m/J qSa67DktV6zF5OIY/4SFr42+hJi/UbFrb3E+c81xKI47QPmXP/2WartrnYvjKaDYHUd5 jRzI+lw3Ax3tYbLNUtXYTLdiL9GbDm/+hYmfVoG+Cx077QveQVOlUtcb0lxUp1GhD1cL Ji/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584866; x=1729189666; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=l0R/snR1lBsefu+Gd9v26W9gBjpskR/lINZXMlcZUSs=; b=NLy5BHLpCBe41c2QRehRseQPsJIhc+KKYoGdlUjiC02Lu9qwUIcQE0D3jzzC6QLVSd oLYVs2bDXV/zNnkrURahy+2bwPn2VSq2EMVzkbPBANgA+cGPhcF3W6mk5ozqrncRT3Q3 boWnHruEhxS9Ifmj0EtMJNzTRs1WwAXZQTCh+wnIdtx1xANYZYL4BwFDzkqCY8hvcvAW 3hk1BjIoaSGZCuxztgeMoXMCU2t9xUf7E9lguWn1Wb93jO2YDsufJSIlZvgdB+DcjVvX /eB83+rwJ9sYvxpbOVVncY01Kb8+FgXHrVwY/Qq0Osk3zFS3vFAmv64fuY2RDsa7TiO5 CPEA== X-Forwarded-Encrypted: i=1; AJvYcCUJpTKhZLm40vuthqEI9T/zPl5E6qw9xFl+xZcmm78gUSGl7TKagAX+T9VJ84tOvdWStHfe/GqQidxGF/o=@vger.kernel.org X-Gm-Message-State: AOJu0Yx25FbjdlSRFGVVN7hYIJD8G5CmKeLGgvQbN4vNGqZVPpApiLZq dtl6wrePGDXpynDYAHDUmbQe4OhWoXp5j0PmHzINV9lKHswKNdOk8onueV2WGbgjPdNkWTtTaKE ApA== X-Google-Smtp-Source: AGHT+IE49fRlqLzz+ddXUJ4fW719WQNIEaF+exc+KxAZEodOOT6YKCh59uZdX+xzyPwnVHJ15Mb6QwFZ+jU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:470f:b0:6e3:39e5:f0e8 with SMTP id 00721157ae682-6e339e5f461mr119227b3.6.1728584866570; Thu, 10 Oct 2024 11:27:46 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:24 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-83-seanjc@google.com> Subject: [PATCH v13 82/85] KVM: x86/mmu: Don't mark "struct page" accessed when zapping SPTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't mark pages/folios as accessed in the primary MMU when zapping SPTEs, as doing so relies on kvm_pfn_to_refcounted_page(), and generally speaking is unnecessary and wasteful. KVM participates in page aging via mmu_notifiers, so there's no need to push "accessed" updates to the primary MMU. And if KVM zaps a SPTe in response to an mmu_notifier, marking it accessed _after_ the primary MMU has decided to zap the page is likely to go unnoticed, i.e. odds are good that, if the page is being zapped for reclaim, the page will be swapped out regardless of whether or not KVM marks the page accessed. Dropping x86's use of kvm_set_pfn_accessed() also paves the way for removing kvm_pfn_to_refcounted_page() and all its users. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/x86/kvm/mmu/mmu.c | 17 ----------------- arch/x86/kvm/mmu/tdp_mmu.c | 3 --- 2 files changed, 20 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5acdaf3b1007..55eeca931e23 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -559,10 +559,8 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte) */ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep) { - kvm_pfn_t pfn; u64 old_spte =3D *sptep; int level =3D sptep_to_sp(sptep)->role.level; - struct page *page; =20 if (!is_shadow_present_pte(old_spte) || !spte_has_volatile_bits(old_spte)) @@ -574,21 +572,6 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, = u64 *sptep) return old_spte; =20 kvm_update_page_stats(kvm, level, -1); - - pfn =3D spte_to_pfn(old_spte); - - /* - * KVM doesn't hold a reference to any pages mapped into the guest, and - * instead uses the mmu_notifier to ensure that KVM unmaps any pages - * before they are reclaimed. Sanity check that, if the pfn is backed - * by a refcounted page, the refcount is elevated. - */ - page =3D kvm_pfn_to_refcounted_page(pfn); - WARN_ON_ONCE(page && !page_count(page)); - - if (is_accessed_spte(old_spte)) - kvm_set_pfn_accessed(pfn); - return old_spte; } =20 diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 8aa0d7a7602b..91caa73a905b 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -861,9 +861,6 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct k= vm_mmu_page *root, =20 tdp_mmu_iter_set_spte(kvm, &iter, SHADOW_NONPRESENT_VALUE); =20 - if (is_accessed_spte(iter.old_spte)) - kvm_set_pfn_accessed(spte_to_pfn(iter.old_spte)); - /* * Zappings SPTEs in invalid roots doesn't require a TLB flush, * see kvm_tdp_mmu_zap_invalidated_roots() for details. --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F089621BAFA for ; Thu, 10 Oct 2024 18:27:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584870; cv=none; b=rjTzRdwsyFCp+S/loQlYAU66rUPOFLtRzWq4PeIf6evlm16DUe1nnc5YZjlftkbxQzLnuC7sjj5MQ8k/+SeNn7R0d8zZS8DEhv6JhGE3UOcFcaTdgapYd6I87wfZ8rLtDAUFIACi5B6/2ILQCht7jrwMSZVheuwhHCJmqOG0AI0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584870; c=relaxed/simple; bh=4ce0ST1uTErGaf3Fu8uQPVTe9dXk8k7UaRtdo8iN0BA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Qt5chIigUlgJ3widx395+swIXD/+80KFdBGvORaw21sM3aN+FtMgVY/Ai5EecBxbz4LZaH4VzX2ci3OYDXPOBjWMGM27ghYIkE7muwziNtgptSwbo7tjuTPs7/7V4Efa6Wh7EcRoUWPEFvyC2FcIfvl4w6V73qogKplxqVKLJV0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LR0bZlhg; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LR0bZlhg" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-20c8a637b77so13075015ad.2 for ; Thu, 10 Oct 2024 11:27:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584868; x=1729189668; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=qYRk8t9w2U8aQ/ID4AWjouVrA1tdB4hWvcNmB/sB3HQ=; b=LR0bZlhgBqqlK/AxVHUnYDgdPVeVfg62ritjsHW4icLlp2VXJMdBEQpqQ2DjaOB4Vo BMk5u0CdNiJU47xxEKEBNDUg1VIzyKQFA0lEenO//nWQFutF6qqNsjqYWpd0KkluabCG scoWgGmVwMXCV2bJcO3DxVbQdljLpb0S7K+9zHipw5iopVLZ5O/Et9tquTm7UfUH/R3W Tzhi8J9rL8rJ6Zw+R2IdaOJPRXvC3k33WFcoS6zAsYYyRLTyXGYrquN21MhPyyOeIlNt k76iPpUznp7HqqlLd6wYwP+ZU1DkIHHgVTgrkh+gxlQpcl0jRX8rnB7ytNb8WRs+Wq5s 9Kuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584868; x=1729189668; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=qYRk8t9w2U8aQ/ID4AWjouVrA1tdB4hWvcNmB/sB3HQ=; b=IPM15wGNU4Yf1G+hJlZ5GKukRUhWR4R5NztbwxsBg68JNMNQdsWGN7xvmMG4mifFBE gM0/fYJ+2InpeEnUAox7jR/6QgbE0H8+n9z39rUXIAOpGalBmlGWbA7o08ox2fifwfFD z8NWLKFolnNIyprRi5kpNwy82HbGx5QcRmn4TH79/IvSYv05jcOPE/pXis6IVUMuqJMS eCtiZeLnDZkdbUx71Jf/yNQRyVM851glv1Glo/EKkWbHt8IKpFKgUDWz5NR5g7c0sgd7 GRq3TDcMtHSIGownqsfit+2O5pstoOf4LInKJ6buGqa0E6q05/X67P7xXNMu+N1pXRsy HjIA== X-Forwarded-Encrypted: i=1; AJvYcCW9gp3dMwPk9TCjb85Vw0pxyT86/lRqg8lxNpBdUcJBeTrj2YlrkcMEFQKEUJeEUWrpa42aGfuZKmXJu9I=@vger.kernel.org X-Gm-Message-State: AOJu0YxQ9GtYcxvTeYoXJXQgz3SXkc04YM5cci3x+5vhxEPahC0+NRpZ z10A0oy+VclnTbRuc/F7mXgSpZyjYki+D4p/awuxdptqnrfO19KYh0GX2WyPLzsTcxxZjGvC07l EKg== X-Google-Smtp-Source: AGHT+IHfwq/leOSDpGNmElHsEevN2NLhktzxL5WWYsNV7s51yqV4fIBatX1NQsHzbv29AfLNcoyCuLVeClM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:e745:b0:20c:857b:5dcb with SMTP id d9443c01a7336-20c857b62ffmr639275ad.4.1728584868183; Thu, 10 Oct 2024 11:27:48 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:25 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-84-seanjc@google.com> Subject: [PATCH v13 83/85] KVM: arm64: Don't mark "struct page" accessed when making SPTE young From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't mark pages/folios as accessed in the primary MMU when making a SPTE young in KVM's secondary MMU, as doing so relies on kvm_pfn_to_refcounted_page(), and generally speaking is unnecessary and wasteful. KVM participates in page aging via mmu_notifiers, so there's no need to push "accessed" updates to the primary MMU. Dropping use of kvm_set_pfn_accessed() also paves the way for removing kvm_pfn_to_refcounted_page() and all its users. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- arch/arm64/include/asm/kvm_pgtable.h | 4 +--- arch/arm64/kvm/hyp/pgtable.c | 7 ++----- arch/arm64/kvm/mmu.c | 6 +----- 3 files changed, 4 insertions(+), 13 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/= kvm_pgtable.h index 03f4c3d7839c..aab04097b505 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -674,10 +674,8 @@ int kvm_pgtable_stage2_wrprotect(struct kvm_pgtable *p= gt, u64 addr, u64 size); * * If there is a valid, leaf page-table entry used to translate @addr, then * set the access flag in that entry. - * - * Return: The old page-table entry prior to setting the flag, 0 on failur= e. */ -kvm_pte_t kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr); +void kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr); =20 /** * kvm_pgtable_stage2_test_clear_young() - Test and optionally clear the a= ccess diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index b11bcebac908..40bd55966540 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1245,19 +1245,16 @@ int kvm_pgtable_stage2_wrprotect(struct kvm_pgtable= *pgt, u64 addr, u64 size) NULL, NULL, 0); } =20 -kvm_pte_t kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr) +void kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr) { - kvm_pte_t pte =3D 0; int ret; =20 ret =3D stage2_update_leaf_attrs(pgt, addr, 1, KVM_PTE_LEAF_ATTR_LO_S2_AF= , 0, - &pte, NULL, + NULL, NULL, KVM_PGTABLE_WALK_HANDLE_FAULT | KVM_PGTABLE_WALK_SHARED); if (!ret) dsb(ishst); - - return pte; } =20 struct stage2_age_data { diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 4054356c9712..e2ae9005e333 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1706,18 +1706,14 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, ph= ys_addr_t fault_ipa, /* Resolve the access fault by making the page young again. */ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_i= pa) { - kvm_pte_t pte; struct kvm_s2_mmu *mmu; =20 trace_kvm_access_fault(fault_ipa); =20 read_lock(&vcpu->kvm->mmu_lock); mmu =3D vcpu->arch.hw_mmu; - pte =3D kvm_pgtable_stage2_mkyoung(mmu->pgt, fault_ipa); + kvm_pgtable_stage2_mkyoung(mmu->pgt, fault_ipa); read_unlock(&vcpu->kvm->mmu_lock); - - if (kvm_pte_valid(pte)) - kvm_set_pfn_accessed(kvm_pte_to_pfn(pte)); } =20 /** --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1088F1E7C12 for ; Thu, 10 Oct 2024 18:27:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584872; cv=none; b=MqoAkyd6dkMX1x0PxP1VBIG1Vf6bRoMJeOpgjby5anmypfDNjPLpF643NDfp5oX2aThkxFOw/SdASpHsKzBmCYmqBGziJzQE0LTpnlidm3kljAhG1dQ0EPSmCpzgNXV9n1OssqRIPoJGcWfgyczUFS5VaXViyNPArUOeJ1iU8CI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584872; c=relaxed/simple; bh=9yDh4yVl+Y5ECwKMlXFEDQ/uzUa/qYgKcVRj3mqEjek=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=KnZHH8JiEXv2ODXclSpi/Je/879cpQf1AVuFkiZk1aobzNLzL4bKl1gRXQJHVtNEz+T0UWhny9o4kI+46fMlEwt0rszd2rM+Mb246EEkVVRi9Lyyy2w0+UvJwZfFOwl+s4J8FO0QTFF+UYV8/BWzfqJT3lB2/GnXogFnHhh6J+Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=1AdnNrnI; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="1AdnNrnI" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6e2e4874925so21273747b3.1 for ; Thu, 10 Oct 2024 11:27:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584870; x=1729189670; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=BeNSr3VxopvO4QfIOfKviYSPGRm8vVX1K3YR0sLhejo=; b=1AdnNrnIXLgNymQtK7DilLGthF1JsfkTOKsD8zYlozlgDyVnSyLY6rQs1ch4aUN54v fTlIRF6YDP2svUdnyjp88hSQcfOn6wvrAuXYoJPHVBwm+zBiaq88Wsg3ie8JPbOZVm6v +FdIhr7f29Y4owHrhAD9S/zkPO3/FI5JCSjOek0MkvkxKw6zlH8yk7tESmKbM59fwOxY truIpSu4JHQroLJXqzMSmcI2c+TlvXrk0XyyyXlQcN9BQPbIRRXz6PSG0liD/fpQxsWp 5Y2cwNpf+u5nZPuU2t9NEaeYfk9Jawe11Dvm4yVWD7vMf94BZ/9HtAVH1x+u2rygpxWQ 2Xsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584870; x=1729189670; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=BeNSr3VxopvO4QfIOfKviYSPGRm8vVX1K3YR0sLhejo=; b=dm2Giuv5yfWmVf1tmL9J3qEu1JOT+B0T1y43OYDX8/MEolzL8m/RSULYFMwOYi6mI5 n5mViH89WiA9jGyE4VRKou2qxlbwQu9oVRoBy7oQg0DHyjESAT160BZ/NMw9ilJtVom9 0/ZvzoebGT0qYxUjBFDEH2GKHEobO0Pg2jD2SdporqP16cal/R7gHTPrSRWfqqKtc+0K fOevE/vFFA6fYe5dlZzQ9zqFM5MbDanMw94j7iPHfMBINiEnaansfJGef8Rzmd+EIrXy gScHgwOfYKOj2e/FbadmfZTZ+sf9a+8u4Co0zpkhqoy8p1i/0lUBemPQ+TNOEruWCnh8 ojLA== X-Forwarded-Encrypted: i=1; AJvYcCWOjbuWmchX/v94pHt3HNmP4gB2q7l7FLd8w7hi/euP5U4DeIiDl2ZS6PJiVktXNNE9YnfQZpgxwujvXQs=@vger.kernel.org X-Gm-Message-State: AOJu0Yxk5vWZUrr6JH/sM857FNOzZ0G5zmc6V5+orDppYlfkPF+ZiGNJ FZdMtxpjoyA7J0HPvGMk0mzQ3EvevKwXvZ+5SN0+Xd+eyoSm75SiJUqeUtKHIRAdaX9DH2NV2HR Onw== X-Google-Smtp-Source: AGHT+IHhmEpfdjhYs/g96WpbaMl3fao5EtfU192MAf3yyGbJNoRiDL4cTPbNYR1fq37SA+x5liddjIzSKiU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2e86:b0:6dd:bcce:7cd4 with SMTP id 00721157ae682-6e3221404ddmr516647b3.2.1728584870148; Thu, 10 Oct 2024 11:27:50 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:26 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-85-seanjc@google.com> Subject: [PATCH v13 84/85] KVM: Drop APIs that manipulate "struct page" via pfns From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove all kvm_{release,set}_pfn_*() APIs now that all users are gone. No functional change intended. Reviewed-by: Alex Benn=C3=A9e Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- include/linux/kvm_host.h | 5 ---- virt/kvm/kvm_main.c | 55 ---------------------------------------- 2 files changed, 60 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4a1eaa40a215..d045f8310a48 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1274,11 +1274,6 @@ static inline kvm_pfn_t kvm_faultin_pfn(struct kvm_v= cpu *vcpu, gfn_t gfn, write ? FOLL_WRITE : 0, writable, refcounted_page); } =20 -void kvm_release_pfn_clean(kvm_pfn_t pfn); -void kvm_release_pfn_dirty(kvm_pfn_t pfn); -void kvm_set_pfn_dirty(kvm_pfn_t pfn); -void kvm_set_pfn_accessed(kvm_pfn_t pfn); - int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, int len); int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long l= en); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index a483da96f4be..396ca14f18f3 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3164,61 +3164,6 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kv= m_host_map *map) } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); =20 -void kvm_release_pfn_clean(kvm_pfn_t pfn) -{ - struct page *page; - - if (is_error_noslot_pfn(pfn)) - return; - - page =3D kvm_pfn_to_refcounted_page(pfn); - if (!page) - return; - - kvm_release_page_clean(page); -} -EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); - -void kvm_release_pfn_dirty(kvm_pfn_t pfn) -{ - struct page *page; - - if (is_error_noslot_pfn(pfn)) - return; - - page =3D kvm_pfn_to_refcounted_page(pfn); - if (!page) - return; - - kvm_release_page_dirty(page); -} -EXPORT_SYMBOL_GPL(kvm_release_pfn_dirty); - -/* - * Note, checking for an error/noslot pfn is the caller's responsibility w= hen - * directly marking a page dirty/accessed. Unlike the "release" helpers, = the - * "set" helpers are not to be used when the pfn might point at garbage. - */ -void kvm_set_pfn_dirty(kvm_pfn_t pfn) -{ - if (WARN_ON(is_error_noslot_pfn(pfn))) - return; - - if (pfn_valid(pfn)) - kvm_set_page_dirty(pfn_to_page(pfn)); -} -EXPORT_SYMBOL_GPL(kvm_set_pfn_dirty); - -void kvm_set_pfn_accessed(kvm_pfn_t pfn) -{ - if (WARN_ON(is_error_noslot_pfn(pfn))) - return; - - if (pfn_valid(pfn)) - kvm_set_page_accessed(pfn_to_page(pfn)); -} -EXPORT_SYMBOL_GPL(kvm_set_pfn_accessed); - static int next_segment(unsigned long len, int offset) { if (len > PAGE_SIZE - offset) --=20 2.47.0.rc1.288.g06298d1525-goog From nobody Wed Nov 27 08:49:16 2024 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FDEF21D16B for ; Thu, 10 Oct 2024 18:27:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584875; cv=none; b=bmGLS6oTMhAvrwQ8Rd+C/yNGkqdQbSldkCwu16/VhPN4CY5p4A13H+VginaVHgzhkDZvhsKg9ilgvppeHUjG6oJ6s3DfEYfXKUZhzjB5TrzdX0aj/0mPDlV8atEJWNa/JoGB7NSdVY7DbKJRyekOL41lrsIKJFbDwxwhlzz+34E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728584875; c=relaxed/simple; bh=aNI4I+LkKsxzC1YKs2vCoD7b+5doZynW6Blg4vQwwvc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=dALTs2r53hC50+rZA5mcKfNJq6WtN4ApU7rv/sibouZhFYPoUtCXsO5iJ2Up6V1g/FObuPq5+qSXVVTHKuzfJH+CcUBhgHg3LUwZcY8XAD7k3SbxfjDA4PJZcZSvnL+a6Fd02WTKcaReEMb+a54uYuIWTkf4JcHm+TwrPjVygbM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xyu1Nn/r; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xyu1Nn/r" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7d4f9974c64so900247a12.1 for ; Thu, 10 Oct 2024 11:27:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584873; x=1729189673; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=nwXbeHTlA0ocP8s6EMi2R3V8WkxZqKgY3fo4QQ/WBXk=; b=xyu1Nn/rp+FyjGRlvaySHpdLv16ogLwofUZlPSL88LrsPtUsdHxPdI4aMAZWpZ+LCb G47QycBP2achTmuGd1EXNbbjK4pjlf7ihMkj5vlGqDxw6OCVDaNJvwxpCB4vy6cQt+eC bPNyYrqL1IWM+djQgloaSwHlTl/m9PsGRY+UQiNbQLuX8csAdsfFUDyRbBcmemNyquwH u56PAc5JRwdiVO7tt7yODdJN/pEha8mrjjtisATQgI2/uYcwTl1QzP6LKGp+KqXDTb7X W3csPJXsIxIBK/YbHvH2KWvDbUJR0U0jHjWARSRlv5XaOYevRMa1tcC3LSfpUG2LIezf k0Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584873; x=1729189673; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=nwXbeHTlA0ocP8s6EMi2R3V8WkxZqKgY3fo4QQ/WBXk=; b=HT4jn8zWKsiFfkJSW4yoVXuXZ+q1piF/qdkpSnzcgbBAvbV89XyNsRrJT5LihHCwBJ jfj/Q0vdtT53ieUlx7PEVX97VeD1tZ8rgqkF4L6H4s/YO3AEAGOdgzI+ep8Dwt6h/vIv FZXwNspMVNwv9t/iLCijJmdFimSxcFvij9sDgisbdTqUTQnN/E5oh31ZsPSYtrMV2ZHQ CdyjBlj7TCcbpT1Ts8mOzoqmTBGc8DcKXzoor3n1LwNsPVuVzB3NDvCxg8ygkQjdBcYf rlfoHK9TJnBJkAZS0iYFNOi5v/9Vt4LwWLwZ91YA49GTsr+w6lrxBbF0Sn2WemOQGpc7 qDtg== X-Forwarded-Encrypted: i=1; AJvYcCXxX7HTCwnnQ0hzMnbvTekHfo2emZneotBu464SJOewnM6ttBSq+IUzyTQ1G4PvijiHynOPOe21BPThaQ0=@vger.kernel.org X-Gm-Message-State: AOJu0Yw4Id7aEiM7giZPlYZBvzLUcGSfWP618EEBMSSG9hA9pTLqKhoU iScbIPjT5l8YE69bf+YDMNx7HES8wY8SOgJvHe7ILCivcB0b5I0qWffw7CbYBVd8q47fhM5SCi+ zXg== X-Google-Smtp-Source: AGHT+IHZp/jcJ6VF/U4r8Ds8cBqCm7oOG+WyWj8b167iLxGpwwfpynwEnbVm0Ou2T4AqluEw0PW/YyS/yKE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:e715:0:b0:7d5:d088:57d4 with SMTP id 41be03b00d2f7-7ea535b6b4fmr26a12.9.1728584872016; Thu, 10 Oct 2024 11:27:52 -0700 (PDT) Reply-To: Sean Christopherson Date: Thu, 10 Oct 2024 11:24:27 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-86-seanjc@google.com> Subject: [PATCH v13 85/85] KVM: Don't grab reference on VM_MIXEDMAP pfns that have a "struct page" From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, "=?UTF-8?q?Alex=20Benn=C3=A9e?=" , Yan Zhao , David Matlack , David Stevens , Andrew Jones Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that KVM no longer relies on an ugly heuristic to find its struct page references, i.e. now that KVM can't get false positives on VM_MIXEDMAP pfns, remove KVM's hack to elevate the refcount for pfns that happen to have a valid struct page. In addition to removing a long-standing wart in KVM, this allows KVM to map non-refcounted struct page memory into the guest, e.g. for exposing GPU TTM buffers to KVM guests. Tested-by: Alex Benn=C3=A9e Signed-off-by: Sean Christopherson Tested-by: Dmitry Osipenko --- include/linux/kvm_host.h | 3 -- virt/kvm/kvm_main.c | 75 ++-------------------------------------- 2 files changed, 2 insertions(+), 76 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index d045f8310a48..02f0206fd2dc 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1730,9 +1730,6 @@ void kvm_arch_sync_events(struct kvm *kvm); =20 int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu); =20 -struct page *kvm_pfn_to_refcounted_page(kvm_pfn_t pfn); -bool kvm_is_zone_device_page(struct page *page); - struct kvm_irq_ack_notifier { struct hlist_node link; unsigned gsi; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 396ca14f18f3..b1b10dc408a0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -160,52 +160,6 @@ __weak void kvm_arch_guest_memory_reclaimed(struct kvm= *kvm) { } =20 -bool kvm_is_zone_device_page(struct page *page) -{ - /* - * The metadata used by is_zone_device_page() to determine whether or - * not a page is ZONE_DEVICE is guaranteed to be valid if and only if - * the device has been pinned, e.g. by get_user_pages(). WARN if the - * page_count() is zero to help detect bad usage of this helper. - */ - if (WARN_ON_ONCE(!page_count(page))) - return false; - - return is_zone_device_page(page); -} - -/* - * Returns a 'struct page' if the pfn is "valid" and backed by a refcounted - * page, NULL otherwise. Note, the list of refcounted PG_reserved page ty= pes - * is likely incomplete, it has been compiled purely through people wantin= g to - * back guest with a certain type of memory and encountering issues. - */ -struct page *kvm_pfn_to_refcounted_page(kvm_pfn_t pfn) -{ - struct page *page; - - if (!pfn_valid(pfn)) - return NULL; - - page =3D pfn_to_page(pfn); - if (!PageReserved(page)) - return page; - - /* The ZERO_PAGE(s) is marked PG_reserved, but is refcounted. */ - if (is_zero_pfn(pfn)) - return page; - - /* - * ZONE_DEVICE pages currently set PG_reserved, but from a refcounting - * perspective they are "normal" pages, albeit with slightly different - * usage rules. - */ - if (kvm_is_zone_device_page(page)) - return page; - - return NULL; -} - /* * Switches to specified vcpu, until a matching vcpu_put() */ @@ -2804,35 +2758,10 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_= pfn *kfp, struct page *page, if (kfp->map_writable) *kfp->map_writable =3D writable; =20 - /* - * FIXME: Remove this once KVM no longer blindly calls put_page() on - * every pfn that points at a struct page. - * - * Get a reference for follow_pte() pfns if they happen to point at a - * struct page, as KVM will ultimately call kvm_release_pfn_clean() on - * the returned pfn, i.e. KVM expects to have a reference. - * - * Certain IO or PFNMAP mappings can be backed with valid struct pages, - * but be allocated without refcounting, e.g. tail pages of - * non-compound higher order allocations. Grabbing and putting a - * reference to such pages would cause KVM to prematurely free a page - * it doesn't own (KVM gets and puts the one and only reference). - * Don't allow those pages until the FIXME is resolved. - * - * Don't grab a reference for pins, callers that pin pages are required - * to check refcounted_page, i.e. must not blindly release the pfn. - */ - if (map) { + if (map) pfn =3D map->pfn; - - if (!kfp->pin) { - page =3D kvm_pfn_to_refcounted_page(pfn); - if (page && !get_page_unless_zero(page)) - return KVM_PFN_ERR_FAULT; - } - } else { + else pfn =3D page_to_pfn(page); - } =20 *kfp->refcounted_page =3D page; =20 --=20 2.47.0.rc1.288.g06298d1525-goog