From nobody Fri Feb 21 23:32:32 2025
Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com
 [209.85.128.51])
	(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
	(No client certificate requested)
	by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1AD02179BF
	for <linux-kernel@vger.kernel.org>; Mon, 27 Jan 2025 09:41:50 +0000 (UTC)
Authentication-Results: smtp.subspace.kernel.org;
 arc=none smtp.client-ip=209.85.128.51
ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116;
	t=1737970913; cv=none;
 b=otLpg9kL7JaataTK15QmZ/1oK9S8XbA3EiSw7gBMNXaO/qN858L5heDon8D4NdDXfCzSDqi1XmRncCBvDjn+2rpOnWYC3mdyTiGnHCRGH0JVb+iRgOorsi6GoTG1QlDzhdhbFRyC2eFwCN9NSZKeUZUGZy5I74Ct+tEnZU9CBe0=
ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org;
	s=arc-20240116; t=1737970913; c=relaxed/simple;
	bh=wnlhWBIrY7S3vay9E0tvvsZWstynmDSXSh9cQxqW//k=;
	h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References:
	 MIME-Version;
 b=NrjpJuuxYkQMdElPaLbg5bPX1PDuvjI28aE2rPKQkD7vqsRRnh0tb9Sr1BnyWO3veRGoVzg357PZQPh6eGErUjOBj4g/ieCtPFjUe39GGQvg/QGbZusWtdD9+Z5PgRU4ynpUQybwN1Rlp/pbzzKEUBHo/3v3drkALM5JKxLK/bw=
ARC-Authentication-Results: i=1; smtp.subspace.kernel.org;
 dmarc=none (p=none dis=none) header.from=rivosinc.com;
 spf=pass smtp.mailfrom=rivosinc.com;
 dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com
 header.i=@rivosinc-com.20230601.gappssmtp.com header.b=H4GKkIxp;
 arc=none smtp.client-ip=209.85.128.51
Authentication-Results: smtp.subspace.kernel.org;
 dmarc=none (p=none dis=none) header.from=rivosinc.com
Authentication-Results: smtp.subspace.kernel.org;
 spf=pass smtp.mailfrom=rivosinc.com
Authentication-Results: smtp.subspace.kernel.org;
	dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com
 header.i=@rivosinc-com.20230601.gappssmtp.com header.b="H4GKkIxp"
Received: by mail-wm1-f51.google.com with SMTP id
 5b1f17b1804b1-436341f575fso45374625e9.1
        for <linux-kernel@vger.kernel.org>;
 Mon, 27 Jan 2025 01:41:50 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1737970909;
 x=1738575709; darn=vger.kernel.org;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:from:to:cc:subject:date
         :message-id:reply-to;
        bh=/Ib7BO4DbRcQ5HoSRv1ee1R7qySu3xzgQBO2hzqpTQY=;
        b=H4GKkIxpmPKG2ey2Am62MWynZzumtGlLu32IHRTg3hlqvbUJJZiRE+eURsOVSiKAEk
         JIZ2f0ZSBfWAnAtzpGs74bc4XsOSklNQdaV7jj/XV1gcjzK51j6w2o65fbhK2VQZrkGN
         g04Gg/Exo3uAZcDSAboexzELKe7v0iBsT+mD6efx82HRFRlAEAqmrhVC8zV3QYI5gSrY
         uPyIGhX5cNNx4eenjQHg3bL9fC2DYGgRVmdF3MQjE513A7/AvOQjJPRsVt3pqS0MGpg1
         HtS+Ip+5JVeuVw4U9a8ncD7Xf//Ish7UcHIVMX4gC8zDZfm4YctjYBtZGspkUNoBocMe
         nNzg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20230601; t=1737970909; x=1738575709;
        h=content-transfer-encoding:mime-version:references:in-reply-to
         :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
         :subject:date:message-id:reply-to;
        bh=/Ib7BO4DbRcQ5HoSRv1ee1R7qySu3xzgQBO2hzqpTQY=;
        b=rHQ0ejyPpzcsZ59PWlWe+cmYRGZbjQTho8fOQ2RBYZQYgGsYJVBAT+JsRtnW3iriC3
         Rl3AKkM3/PXe2q8vVVGXrldSKEbt922VbEqxFiCNQLubTpU8duFvaOR/7fhUZPIx2zW7
         mFomOW5MWRq4+PyD7mSYUAf1cFJuiL1R/QGXCH+/JmwXbmXi675XYbsOWVp0rjcE5bNf
         2+VG3K+dgPLdHFqvXCvr9uJyWedcpzzBQ1mM5/OR5NxbFthAlrh6caIw3HQgrwFRYSma
         bo4C4GY6ejbhP7WFYFmkfNXbvrsvWzAbDLE+Fx2erB2+N1mMDYKFjnaWduV32iaj6xV6
         P8xg==
X-Forwarded-Encrypted: i=1;
 AJvYcCXZLghNMv2Y0q/ctt7oHAFc5kJ1yDMxPH5JhSstj0LSrTblnqS07vlYrfdKnhQ2OecnQlGPz+fTshkqik4=@vger.kernel.org
X-Gm-Message-State: AOJu0Yw7aADkSRtb4wWW2Fuj+dQBPbSHi+P7z9RinbdxObNrtMsNvFE6
	qwozrqlCknZ9HKlYc978s7TqYh88btgEKNjENxqTMDRnrW4sJU4xTS4VSOYDpVBtByJik12WwzD
	f
X-Gm-Gg: ASbGncuIRrffkeTw5mjga7OaReXWott5GZI/zlVQ8nBKKi0LTgKh90TMnAnZJKmJ9bp
	3ZsEKAWLBMnY4yFQKSo86FUGt/7xQnYfLkOLYJ4G5NcFnWYbAVGYcV7eimPTp282YLo9ew/cm8H
	2g9qMYbr11r/p7/PuIw7Dyz/QgasZJ2tGZl/eU95Qu/jBG7X4old2zeq+MzeQ54LH/k0iUwHE41
	w0l2PE2nztRHiWtCXldjSCXa7t2H9ptg2kFLcf83iNPoP7vfy8dITRCtCRIyo7TRdFtG8UEMtUi
	4EwLG7VRpaMCqHPnuQm0LWSmbakDru0=
X-Google-Smtp-Source: 
 AGHT+IE0wvAkHX9Lz2nK7eyxGx0auM1eq+y4AS6IYBZCccECBQROh0UBY1MLjeLntJwSdARX6LIQ6w==
X-Received: by 2002:a5d:64a3:0:b0:385:f909:eb2c with SMTP id
 ffacd0b85a97d-38bf57a77a2mr48324115f8f.38.1737970909255;
        Mon, 27 Jan 2025 01:41:49 -0800 (PST)
Received: from alex-rivos.ba.rivosinc.com
 ([2001:861:3382:ef90:3e22:3e78:ce5a:32c3])
        by smtp.gmail.com with ESMTPSA id
 ffacd0b85a97d-38c2a1c42fcsm10707051f8f.96.2025.01.27.01.41.48
        (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
        Mon, 27 Jan 2025 01:41:49 -0800 (PST)
From: Alexandre Ghiti <alexghiti@rivosinc.com>
To: Catalin Marinas <catalin.marinas@arm.com>,
	Will Deacon <will@kernel.org>,
	Ryan Roberts <ryan.roberts@arm.com>,
	Mark Rutland <mark.rutland@arm.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Andrew Morton <akpm@linux-foundation.org>,
	linux-arm-kernel@lists.infradead.org,
	linux-kernel@vger.kernel.org,
	linux-riscv@lists.infradead.org,
	linux-mm@kvack.org
Cc: Alexandre Ghiti <alexghiti@rivosinc.com>
Subject: [PATCH v4 6/9] mm: Use common huge_ptep_get_and_clear() function for
 riscv/arm64
Date: Mon, 27 Jan 2025 10:35:27 +0100
Message-Id: <20250127093530.19548-7-alexghiti@rivosinc.com>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20250127093530.19548-1-alexghiti@rivosinc.com>
References: <20250127093530.19548-1-alexghiti@rivosinc.com>
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="utf-8"

After some adjustments, both architectures have the same implementation
so move it to the generic code.

Note that get_clear_contig() function is duplicated in the generic and
the arm64 code because it is still used by some arm64 functions that
will, in the next commits, be moved to the generic code. Once all have
been moved, the arm64 version will be removed.

Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
---
 arch/arm64/include/asm/hugetlb.h |  3 --
 arch/arm64/include/asm/pgtable.h | 15 ++++++++--
 arch/arm64/mm/hugetlbpage.c      | 19 ++-----------
 arch/riscv/include/asm/hugetlb.h |  4 ---
 arch/riscv/include/asm/pgtable.h |  4 ++-
 arch/riscv/mm/hugetlbpage.c      | 23 ++++-----------
 include/linux/hugetlb_contpte.h  |  4 +++
 mm/hugetlb_contpte.c             | 48 ++++++++++++++++++++++++++++++--
 8 files changed, 72 insertions(+), 48 deletions(-)

diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/huge=
tlb.h
index e4acaedea149..5c605a0a2017 100644
--- a/arch/arm64/include/asm/hugetlb.h
+++ b/arch/arm64/include/asm/hugetlb.h
@@ -39,9 +39,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift,=
 vm_flags_t flags);
 extern int huge_ptep_set_access_flags(struct vm_area_struct *vma,
 				      unsigned long addr, pte_t *ptep,
 				      pte_t pte, int dirty);
-#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR
-extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
-				     unsigned long addr, pte_t *ptep);
 #define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT
 extern void huge_ptep_set_wrprotect(struct mm_struct *mm,
 				    unsigned long addr, pte_t *ptep);
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta=
ble.h
index cebbfcfb0e53..c339b568ac51 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -1831,12 +1831,23 @@ static inline void clear_young_dirty_ptes(struct vm=
_area_struct *vma,
=20
 #endif /* CONFIG_ARM64_CONTPTE */
=20
-static inline int arch_contpte_get_num_contig(pte_t *ptep,
-					      unsigned long size,
+extern int find_num_contig(struct mm_struct *mm, unsigned long addr,
+			   pte_t *ptep, size_t *pgsize);
+
+static inline int arch_contpte_get_num_contig(struct mm_struct *mm,
+					      unsigned long addr,
+					      pte_t *ptep, unsigned long size,
 					      size_t *pgsize)
 {
 	int contig_ptes =3D 0;
=20
+	/*
+	 * If the size is not passed, we need to go through the page table to
+	 * find out the number of contiguous ptes.
+	 */
+	if (size =3D=3D 0)
+		return find_num_contig(mm, addr, ptep, pgsize);
+
 	if (pgsize)
 		*pgsize =3D size;
=20
diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
index 541358f50b64..0b7a53fee55d 100644
--- a/arch/arm64/mm/hugetlbpage.c
+++ b/arch/arm64/mm/hugetlbpage.c
@@ -79,8 +79,8 @@ bool arch_hugetlb_migration_supported(struct hstate *h)
 }
 #endif
=20
-static int find_num_contig(struct mm_struct *mm, unsigned long addr,
-			   pte_t *ptep, size_t *pgsize)
+int find_num_contig(struct mm_struct *mm, unsigned long addr,
+		    pte_t *ptep, size_t *pgsize)
 {
 	pgd_t *pgdp =3D pgd_offset(mm, addr);
 	p4d_t *p4dp;
@@ -277,21 +277,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shi=
ft, vm_flags_t flags)
 	return entry;
 }
=20
-pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
-			      unsigned long addr, pte_t *ptep)
-{
-	int ncontig;
-	size_t pgsize;
-	pte_t orig_pte =3D __ptep_get(ptep);
-
-	if (!pte_cont(orig_pte))
-		return __ptep_get_and_clear(mm, addr, ptep);
-
-	ncontig =3D find_num_contig(mm, addr, ptep, &pgsize);
-
-	return get_clear_contig(mm, addr, ptep, pgsize, ncontig);
-}
-
 /*
  * huge_ptep_set_access_flags will update access flags (dirty, accesssed)
  * and write permission.
diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/huge=
tlb.h
index ca9930cdf2e6..0fbb6b19df79 100644
--- a/arch/riscv/include/asm/hugetlb.h
+++ b/arch/riscv/include/asm/hugetlb.h
@@ -20,10 +20,6 @@ bool arch_hugetlb_migration_supported(struct hstate *h);
 #endif
=20
 #ifdef CONFIG_RISCV_ISA_SVNAPOT
-#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR
-pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
-			      unsigned long addr, pte_t *ptep);
-
 #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH
 pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
 			    unsigned long addr, pte_t *ptep);
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgta=
ble.h
index 08b24c0a579b..705d666e014d 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -590,7 +590,9 @@ static inline void __set_ptes(struct mm_struct *mm, uns=
igned long addr,
 }
=20
 #ifdef CONFIG_RISCV_ISA_SVNAPOT
-static inline int arch_contpte_get_num_contig(pte_t *ptep, unsigned long s=
ize,
+static inline int arch_contpte_get_num_contig(struct mm_struct *mm,
+					      unsigned long addr,
+					      pte_t *ptep, unsigned long size,
 					      size_t *pgsize)
 {
 	unsigned long hugepage_shift;
diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c
index e2093e7266a5..b44023336fd9 100644
--- a/arch/riscv/mm/hugetlbpage.c
+++ b/arch/riscv/mm/hugetlbpage.c
@@ -186,7 +186,8 @@ int huge_ptep_set_access_flags(struct vm_area_struct *v=
ma,
 	if (!pte_napot(pte))
 		return ptep_set_access_flags(vma, addr, ptep, pte, dirty);
=20
-	pte_num =3D arch_contpte_get_num_contig(ptep, 0, NULL);
+	pte_num =3D arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL);
+
 	orig_pte =3D get_clear_contig_flush(mm, addr, ptep, pte_num);
=20
 	if (pte_dirty(orig_pte))
@@ -200,21 +201,6 @@ int huge_ptep_set_access_flags(struct vm_area_struct *=
vma,
 	return true;
 }
=20
-pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
-			      unsigned long addr,
-			      pte_t *ptep)
-{
-	pte_t orig_pte =3D ptep_get(ptep);
-	int pte_num;
-
-	if (!pte_napot(orig_pte))
-		return ptep_get_and_clear(mm, addr, ptep);
-
-	pte_num =3D arch_contpte_get_num_contig(ptep, 0, NULL);
-
-	return get_clear_contig(mm, addr, ptep, pte_num);
-}
-
 void huge_ptep_set_wrprotect(struct mm_struct *mm,
 			     unsigned long addr,
 			     pte_t *ptep)
@@ -228,7 +214,8 @@ void huge_ptep_set_wrprotect(struct mm_struct *mm,
 		return;
 	}
=20
-	pte_num =3D arch_contpte_get_num_contig(ptep, 0, NULL);
+	pte_num =3D arch_contpte_get_num_contig(mm, addr, ptep, 0, NULL);
+
 	orig_pte =3D get_clear_contig_flush(mm, addr, ptep, pte_num);
=20
 	orig_pte =3D pte_wrprotect(orig_pte);
@@ -246,7 +233,7 @@ pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
 	if (!pte_napot(pte))
 		return ptep_clear_flush(vma, addr, ptep);
=20
-	pte_num =3D arch_contpte_get_num_contig(ptep, 0, NULL);
+	pte_num =3D arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, NULL);
=20
 	return get_clear_contig_flush(vma->vm_mm, addr, ptep, pte_num);
 }
diff --git a/include/linux/hugetlb_contpte.h b/include/linux/hugetlb_contpt=
e.h
index d9892a047b2b..20d3a3e14e14 100644
--- a/include/linux/hugetlb_contpte.h
+++ b/include/linux/hugetlb_contpte.h
@@ -18,4 +18,8 @@ extern void set_huge_pte_at(struct mm_struct *mm,
 extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr,
 			   pte_t *ptep, unsigned long sz);
=20
+#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR
+extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
+				     unsigned long addr, pte_t *ptep);
+
 #endif /* _LINUX_HUGETLB_CONTPTE_H */
diff --git a/mm/hugetlb_contpte.c b/mm/hugetlb_contpte.c
index c76d6b3d0121..0c86c6f77c29 100644
--- a/mm/hugetlb_contpte.c
+++ b/mm/hugetlb_contpte.c
@@ -22,6 +22,7 @@
  *   - huge_ptep_get()
  *   - set_huge_pte_at()
  *   - huge_pte_clear()
+ *   - huge_ptep_get_and_clear()
  */
=20
 pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep)
@@ -32,7 +33,7 @@ pte_t huge_ptep_get(struct mm_struct *mm, unsigned long a=
ddr, pte_t *ptep)
 	if (!pte_present(orig_pte) || !pte_cont(orig_pte))
 		return orig_pte;
=20
-	ncontig =3D arch_contpte_get_num_contig(ptep,
+	ncontig =3D arch_contpte_get_num_contig(mm, addr, ptep,
 					      page_size(pte_page(orig_pte)),
 					      NULL);
=20
@@ -86,7 +87,7 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long =
addr,
 	int i;
 	int ncontig;
=20
-	ncontig =3D arch_contpte_get_num_contig(ptep, sz, &pgsize);
+	ncontig =3D arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize);
=20
 	if (!pte_present(pte)) {
 		for (i =3D 0; i < ncontig; i++, ptep++, addr +=3D pgsize)
@@ -110,8 +111,49 @@ void huge_pte_clear(struct mm_struct *mm, unsigned lon=
g addr,
 	int i, ncontig;
 	size_t pgsize;
=20
-	ncontig =3D arch_contpte_get_num_contig(ptep, sz, &pgsize);
+	ncontig =3D arch_contpte_get_num_contig(mm, addr, ptep, sz, &pgsize);
=20
 	for (i =3D 0; i < ncontig; i++, addr +=3D pgsize, ptep++)
 		__pte_clear(mm, addr, ptep);
 }
+
+static pte_t get_clear_contig(struct mm_struct *mm,
+			      unsigned long addr,
+			      pte_t *ptep,
+			      unsigned long pgsize,
+			      unsigned long ncontig)
+{
+	pte_t orig_pte =3D __ptep_get(ptep);
+	unsigned long i;
+
+	for (i =3D 0; i < ncontig; i++, addr +=3D pgsize, ptep++) {
+		pte_t pte =3D __ptep_get_and_clear(mm, addr, ptep);
+
+		/*
+		 * If HW_AFDBM (arm64) or Svadu (riscv) is enabled, then the HW
+		 * could turn on the dirty or accessed bit for any page in the
+		 * set, so check them all.
+		 */
+		if (pte_dirty(pte))
+			orig_pte =3D pte_mkdirty(orig_pte);
+
+		if (pte_young(pte))
+			orig_pte =3D pte_mkyoung(orig_pte);
+	}
+	return orig_pte;
+}
+
+pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
+			      unsigned long addr, pte_t *ptep)
+{
+	int ncontig;
+	size_t pgsize;
+	pte_t orig_pte =3D __ptep_get(ptep);
+
+	if (!pte_cont(orig_pte))
+		return __ptep_get_and_clear(mm, addr, ptep);
+
+	ncontig =3D arch_contpte_get_num_contig(mm, addr, ptep, 0, &pgsize);
+
+	return get_clear_contig(mm, addr, ptep, pgsize, ncontig);
+}
--=20
2.39.2