From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1440664B3 for ; Mon, 29 Jan 2024 12:47:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532429; cv=none; b=RFmrKjDnWautvmQ9ybt1AkpNEHCA1CwgrgyemdoyIN7cKL5RXBXbRVB+VTej9vPyQLT4Rm+Xmj2+yMt4p5IzGDRMsLxOCW4FUtFzyHkTn5o6z9R1yZYh4/EmUMeobutIR2xw/ZlWcNtYZjBc9chFWfgVkvqS5UsPv43BJV21XRI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532429; c=relaxed/simple; bh=3K2Kvg31VimzS2TrRWFqKmxfZiiT3WStE+pi7Sb07/M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=n5xyLr72AIyjH6L5iIXz5c3CTGvqrHZl0F57uXE1WWHYGVfi9nUpk9oMil56UcsIup6gviZ7ocqbk4Eg5MvskjFHb6mz+9vIrzYYsl2SCmFfaYOR8rYIKg7RSjY1tZPdy+742A9FP8L4dYI6gWMzU0GgeGDo1lnGMBmJTn10DCA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=X9eZ37T6; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="X9eZ37T6" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3xazwhF5EUmMqWzaAq7c+f2E0xGFlCS1HyOOY3hDGx8=; b=X9eZ37T6ZptrpLNqE8WpWFciKnIJazgtIcQ6ImCl1h2HUSzHWdTg8xSu8NxrDCWvl+i+i6 FLyGS4FkwjddEEaY0yf3WRXU2WsmszwvwrVICyJFU0lf7UfEnYPbBjzWkmf/UxIgpiyYuO 5YcVAux6aPpdxsKwhaXDIzfExVAbaNQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-583-MiBqcbXQPtSpFj1e-vh6bg-1; Mon, 29 Jan 2024 07:47:03 -0500 X-MC-Unique: MiBqcbXQPtSpFj1e-vh6bg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AFCBF867943; Mon, 29 Jan 2024 12:47:02 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 118438B; Mon, 29 Jan 2024 12:46:56 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 01/15] arm64/mm: Make set_ptes() robust when OAs cross 48-bit boundary Date: Mon, 29 Jan 2024 13:46:35 +0100 Message-ID: <20240129124649.189745-2-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" From: Ryan Roberts Since the high bits [51:48] of an OA are not stored contiguously in the PTE, there is a theoretical bug in set_ptes(), which just adds PAGE_SIZE to the pte to get the pte with the next pfn. This works until the pfn crosses the 48-bit boundary, at which point we overflow into the upper attributes. Of course one could argue (and Matthew Wilcox has :) that we will never see a folio cross this boundary because we only allow naturally aligned power-of-2 allocation, so this would require a half-petabyte folio. So its only a theoretical bug. But its better that the code is robust regardless. I've implemented pte_next_pfn() as part of the fix, which is an opt-in core-mm interface. So that is now available to the core-mm, which will be needed shortly to support forthcoming fork()-batching optimizations. Link: https://lkml.kernel.org/r/20240125173534.1659317-1-ryan.roberts@arm.c= om Fixes: 4a169d61c2ed ("arm64: implement the new page table range API") Closes: https://lore.kernel.org/linux-mm/fdaeb9a5-d890-499a-92c8-d171df43ad= 01@arm.com/ Signed-off-by: Ryan Roberts Reviewed-by: Catalin Marinas Reviewed-by: David Hildenbrand Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- arch/arm64/include/asm/pgtable.h | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index b50270107e2f..9428801c1040 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -341,6 +341,22 @@ static inline void __sync_cache_and_tags(pte_t pte, un= signed int nr_pages) mte_sync_tags(pte, nr_pages); } =20 +/* + * Select all bits except the pfn + */ +static inline pgprot_t pte_pgprot(pte_t pte) +{ + unsigned long pfn =3D pte_pfn(pte); + + return __pgprot(pte_val(pfn_pte(pfn, __pgprot(0))) ^ pte_val(pte)); +} + +#define pte_next_pfn pte_next_pfn +static inline pte_t pte_next_pfn(pte_t pte) +{ + return pfn_pte(pte_pfn(pte) + 1, pte_pgprot(pte)); +} + static inline void set_ptes(struct mm_struct *mm, unsigned long __always_unused addr, pte_t *ptep, pte_t pte, unsigned int nr) @@ -354,7 +370,7 @@ static inline void set_ptes(struct mm_struct *mm, if (--nr =3D=3D 0) break; ptep++; - pte_val(pte) +=3D PAGE_SIZE; + pte =3D pte_next_pfn(pte); } } #define set_ptes set_ptes @@ -433,16 +449,6 @@ static inline pte_t pte_swp_clear_exclusive(pte_t pte) return clear_pte_bit(pte, __pgprot(PTE_SWP_EXCLUSIVE)); } =20 -/* - * Select all bits except the pfn - */ -static inline pgprot_t pte_pgprot(pte_t pte) -{ - unsigned long pfn =3D pte_pfn(pte); - - return __pgprot(pte_val(pfn_pte(pfn, __pgprot(0))) ^ pte_val(pte)); -} - #ifdef CONFIG_NUMA_BALANCING /* * See the comment in include/linux/pgtable.h --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DDD3633FF for ; Mon, 29 Jan 2024 12:47:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532438; cv=none; b=uv2LH1u+l2qC3ciMwNqYw0AqqQ20xNTFjko64T3Jt2WI2rjaL5Vzmqj0Jb2JV+qQFN492njnYqnfbthaK5eKnwI9qvqUChqDyfGUhJEBtam/2GgW54qhk8NRya6Tn+UAlntkC0h2j9+WIapHQz+F1WgBXjkjBnDLfqeDUjf+WsQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532438; c=relaxed/simple; bh=SRdmEsHVUm76nbPdHvaKN6K9iGsE/o5rw+NBn1BcWW8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZHUznAjv8K3B5A3zC7PeVQvP9a6wusH/dh6N9jE1O5V4o0Pm1AgpO1Lxtv0UO2Z/G+vQHyydlfSYbxBHLAvS5xzq2ZkgbTqihPGsLXnECgQNAPLadDdX66ZtKu6puJE2QFMqUB/zpW5mQ2xZvenlAe4sXhGXJjZDsFujpQ3B0dg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=iGL/bjIG; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iGL/bjIG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532435; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dZBuztrATUgWrwPaWrq4xl5RtiXYg83ssHnQD4Nso+0=; b=iGL/bjIGoBE7bi+kkA6wSNBkJXAGXlKDyhEncvnnnNOe+Oy8xPIJzfPFhqLtvWYZYI9P7a gW8Mo84c1faWIQY42KoF8PRKlo1Td3Qbx3seK949nlaPxnXU3i2L4xo8P9N6jcnPUoaFfz COcOfZlogR67m4TT2gqsR2hd7bcm6AU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-662-iEa3Fx0fMcCsFoiq-is6nw-1; Mon, 29 Jan 2024 07:47:10 -0500 X-MC-Unique: iEa3Fx0fMcCsFoiq-is6nw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D2C1F185A783; Mon, 29 Jan 2024 12:47:08 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1B5E5107BD; Mon, 29 Jan 2024 12:47:02 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 02/15] arm/pgtable: define PFN_PTE_SHIFT Date: Mon, 29 Jan 2024 13:46:36 +0100 Message-ID: <20240129124649.189745-3-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" We want to make use of pte_next_pfn() outside of set_ptes(). Let's simply define PFN_PTE_SHIFT, required by pte_next_pfn(). Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- arch/arm/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h index d657b84b6bf7..be91e376df79 100644 --- a/arch/arm/include/asm/pgtable.h +++ b/arch/arm/include/asm/pgtable.h @@ -209,6 +209,8 @@ static inline void __sync_icache_dcache(pte_t pteval) extern void __sync_icache_dcache(pte_t pteval); #endif =20 +#define PFN_PTE_SHIFT PAGE_SHIFT + void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pteval, unsigned int nr); #define set_ptes set_ptes --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A88866B21 for ; Mon, 29 Jan 2024 12:47:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532442; cv=none; b=FFOabWMmziLTw/gRC8+c7p8WpAsRvpPESH/JaC3Dd3BJLS1LcHfh+ul//xlW71wYG1C/OX83L6UsnM5kOOOl+zeCUysMtbj81UVG0TZTl9DVZLu5wOawqbe5qVyE8A9LjyZddME09GwZIJJbqMx8gGuy7sspisAHDVK9rj5Fdjg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532442; c=relaxed/simple; bh=6COO11Mp7lYQvHXO3tVMQ0CFOwDREGlorbZAOmcfcQw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RdXDDdBQ2RRGuCOr8J5dEauAwkgTngT3HeBEgfUjVPcGe/dbmHaAF1u4RrT5q/MBosHqzLf9TG7pBGem7AjwXiGzmmmEO+W6pWbr6XdO21x5bEsS9PqHxhn6dtd/MJIz0IO5Jf3wcB9/69EIQBzqZZDwApshoIgX1n3pq9/Ce+0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=gCt3GAF6; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gCt3GAF6" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532440; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1kG4t4M9Qk6cle/o9WwWAIalbkRSmv8fU7P3A1XWjZk=; b=gCt3GAF6icP6RJfc0SRXT1Xp7vqpeMlcgfywiOmU06jTokKMzQdU5xtlxU3BLInMENP1be n7Kdry8bezALcTnapuR9nGqJiPm+G0GmKXbxvlw8x27hbYbdP94GsuAfI/v8AtKorMzOSh VdSblFMlnX4Gnxl1rZXbLQuInyF6bPo= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-669-B9qoHUmjOAyZ8jvL8Xcr8w-1; Mon, 29 Jan 2024 07:47:16 -0500 X-MC-Unique: B9qoHUmjOAyZ8jvL8Xcr8w-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 33D0A3CBD51E; Mon, 29 Jan 2024 12:47:15 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3EC37111E6; Mon, 29 Jan 2024 12:47:09 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 03/15] nios2/pgtable: define PFN_PTE_SHIFT Date: Mon, 29 Jan 2024 13:46:37 +0100 Message-ID: <20240129124649.189745-4-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" We want to make use of pte_next_pfn() outside of set_ptes(). Let's simply define PFN_PTE_SHIFT, required by pte_next_pfn(). Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- arch/nios2/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/nios2/include/asm/pgtable.h b/arch/nios2/include/asm/pgta= ble.h index 5144506dfa69..d052dfcbe8d3 100644 --- a/arch/nios2/include/asm/pgtable.h +++ b/arch/nios2/include/asm/pgtable.h @@ -178,6 +178,8 @@ static inline void set_pte(pte_t *ptep, pte_t pteval) *ptep =3D pteval; } =20 +#define PFN_PTE_SHIFT 0 + static inline void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned int nr) { --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1256C63500 for ; Mon, 29 Jan 2024 12:47:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532450; cv=none; b=uzo+EWyrI5CQo7qlXCnaN1QYbKbmM1qbvf3WzgPGWpUOfG6+oI98n2PX/AlfT5P+UQGmOW5SKTwEihr04Gu1KSrgUVifZa/u1pjaxy+MnZBZaR/Sgk3X3QDQ7cz9zxve9LCofhuEx5BcNZZjCUDcV38KywX1xeEVoawpIT7UVP0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532450; c=relaxed/simple; bh=XUpdBnSB2SD7HxdopzJWgdZ5il3P2SEb9BD2BkpP+UE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DPFT8pTcawH97UGJiODtFdJivuSF5QGBeZpqbJr5Zhbj50ZDAOkcix02eBvKLNCr3HexzO+40KvPWDLyztY+6xWbiXy4p7OgyjnnU95r/zrXayGtpqmM79ti2xxBLngTzl1j9YQaDXmHxH021zNJCrTu0TX0nDE9NnF2pNNQv5M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=B/7MsjCe; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="B/7MsjCe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532448; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L0fTElPXKbhRcKZPS8pXPd43BafZhSoj1KxvgrbHzPI=; b=B/7MsjCensDAWyR4imcgMAg9gIs+ROjS5MOjNdf+aprzTrLkRgJjWE9l5i9cWv1UTXljq3 HDbwUrP9QE8xR8HZawDgCwzYRhqc31U9BObaB/tKaUqOrtSYhf5Vig0h9fK6AM0mMBmSa3 hk49rzYFPVjW+5N9OyZEVmS8PzFQaWs= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-176-gZS2zg1RPJqdlEPYFP4sAA-1; Mon, 29 Jan 2024 07:47:22 -0500 X-MC-Unique: gZS2zg1RPJqdlEPYFP4sAA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0C24D3CBD50A; Mon, 29 Jan 2024 12:47:21 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 850D28B; Mon, 29 Jan 2024 12:47:15 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 04/15] powerpc/pgtable: define PFN_PTE_SHIFT Date: Mon, 29 Jan 2024 13:46:38 +0100 Message-ID: <20240129124649.189745-5-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" We want to make use of pte_next_pfn() outside of set_ptes(). Let's simply define PFN_PTE_SHIFT, required by pte_next_pfn(). Reviewed-by: Christophe Leroy Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- arch/powerpc/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/= pgtable.h index 9224f23065ff..7a1ba8889aea 100644 --- a/arch/powerpc/include/asm/pgtable.h +++ b/arch/powerpc/include/asm/pgtable.h @@ -41,6 +41,8 @@ struct mm_struct; =20 #ifndef __ASSEMBLY__ =20 +#define PFN_PTE_SHIFT PTE_RPN_SHIFT + void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned int nr); #define set_ptes set_ptes --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F27863500 for ; Mon, 29 Jan 2024 12:47:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532455; cv=none; b=e/lD7+OL5VorlQGl8xkJsH7hi+5JPNtlfChB3mtvlUCSjYWbk3u32j2e2NQfypaDw7AJ99bZUJs74pC4kYflk9O6QFjDzVbz3LFOj3efp1mI8gGzzxI/fwcvQttXwAJdxvIvTW2FgRUcCw7N5VbDHYcChjSb1HF+Ar2+UsX93ls= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532455; c=relaxed/simple; bh=IY5FrFcWDNKL1OGj460vClP2gN7wCnsLeRSuqWL8xaY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=l534tHc66RFri8tI6ZQ433uJHxTQqiioqQioTj8XEk864ayK/m/qtch14NvXYhYoVVIcScqGCSMt1tCdMETsvepDjihypcf/fXAbrJHtz7OBvkqCOUSl/xIF4e14nRqzxZGDJL//BcFYAFnQj9qS8i8va8pllWrrcvHmGIVZpj0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AUlAxZPf; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AUlAxZPf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532453; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TR23YVq6t+vxDxzPS6q2xEOscQxVIS4+3klgCmYFiGs=; b=AUlAxZPf/abaQjKWW8udyU2DXs/Z4UFP5GbJFFG14ylASSA2xvGosx2de21MXUo7Wrjq9D /xJiL3IScHGj6OX3RnOfPU9gF5RXKFAXPlmM/gFBdjZmBGHA4mR/aOgX4CKnc61tFZ5AVU i/DOWu3yGF3o1sIArfuyZhAak+bvwDQ= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-317-aMm9C5FkN5Gfoo5yJn1Edw-1; Mon, 29 Jan 2024 07:47:28 -0500 X-MC-Unique: aMm9C5FkN5Gfoo5yJn1Edw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CB8C329AC02A; Mon, 29 Jan 2024 12:47:26 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5D62B107BD; Mon, 29 Jan 2024 12:47:21 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org, Alexandre Ghiti Subject: [PATCH v3 05/15] riscv/pgtable: define PFN_PTE_SHIFT Date: Mon, 29 Jan 2024 13:46:39 +0100 Message-ID: <20240129124649.189745-6-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" We want to make use of pte_next_pfn() outside of set_ptes(). Let's simply define PFN_PTE_SHIFT, required by pte_next_pfn(). Reviewed-by: Alexandre Ghiti Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- arch/riscv/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgta= ble.h index 0c94260b5d0c..add5cd30ab34 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -523,6 +523,8 @@ static inline void __set_pte_at(pte_t *ptep, pte_t ptev= al) set_pte(ptep, pteval); } =20 +#define PFN_PTE_SHIFT _PAGE_PFN_SHIFT + static inline void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pteval, unsigned int nr) { --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4ADF963500 for ; Mon, 29 Jan 2024 12:47:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532460; cv=none; b=cqqbzFdeBTPq2jUWlpNE6rtWfjmU/+dskxz0gTl4xa+kujrxmaPK6eiqipfc6lIGf/p4N1FresxpiIdu9rYektsomvZ09YuH+jhYUm5/3Dz6qczxJnS+Higxh46F7gjF03NjQg1mzz6tHwW123MG2Ubj4KALMIK2d2vJ2dvf8q0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532460; c=relaxed/simple; bh=IaFQMczO4hkkUgLS4AhA2SVxrlGS1xTOsrJGXYuJFIE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gtQSX4yfpkvxxfCsEIx2nogseQBTLcXxpd77MhjFjgi+6ZmS1+rwRwS4640OB/rCthHkhJUtw/4qwpINWxuP+EEWtIrGYUTbtzGV3JmdPQwSCEjeaZLC/D3/4DWRX2GWJODxCwvVS5D7y8SSO994JbHOpx4+kQvLAcoeI1qIjRA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=TmaNglse; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="TmaNglse" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532457; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q4tnrwU9nMLq/XRgt5SnLvg5SSqAxaQfTq+KC4Tr5Rw=; b=TmaNglseSx373K6jvLJcNeByjoiACIczb2yqndCRGWPBeqbvHdhQyqvr/jmDi5NiIbsSpb xn0Xwe7lw7eHqu98RDm2U72Oqh6YsXSgxTjSWGLt9OG95WzrtXJzaOjJs8FjQ3WmRimHJH iQHnWErluT0NcqSaze9zLQBBNGn0qjU= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-626-0qzEMwiFPS-I-NbcZuBhSg-1; Mon, 29 Jan 2024 07:47:34 -0500 X-MC-Unique: 0qzEMwiFPS-I-NbcZuBhSg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 208913814E8C; Mon, 29 Jan 2024 12:47:33 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1C9928B; Mon, 29 Jan 2024 12:47:26 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 06/15] s390/pgtable: define PFN_PTE_SHIFT Date: Mon, 29 Jan 2024 13:46:40 +0100 Message-ID: <20240129124649.189745-7-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" We want to make use of pte_next_pfn() outside of set_ptes(). Let's simply define PFN_PTE_SHIFT, required by pte_next_pfn(). Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- arch/s390/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtabl= e.h index 1299b56e43f6..4b91e65c85d9 100644 --- a/arch/s390/include/asm/pgtable.h +++ b/arch/s390/include/asm/pgtable.h @@ -1316,6 +1316,8 @@ pgprot_t pgprot_writecombine(pgprot_t prot); #define pgprot_writethrough pgprot_writethrough pgprot_t pgprot_writethrough(pgprot_t prot); =20 +#define PFN_PTE_SHIFT PAGE_SHIFT + /* * Set multiple PTEs to consecutive pages with a single call. All PTEs * are within the same folio, PMD and VMA. --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB01F63500 for ; Mon, 29 Jan 2024 12:47:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532469; cv=none; b=SQoyMxGgSJk1TJzso82MqQougFSjd9Ma2SCUCWOf81uC2E1zdRUJXferoi5cqIsF6CaVb7vkNdo8om+iCIE2eIee9qQ2xLHM92KJSGYm1P6lSL5gXeDAe5959uLSetDrHKi/j0+8L1d+rK6sfPpiPyHErLZNaQco4BF/dYA35VU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532469; c=relaxed/simple; bh=CdXM7auQEwBWbHk51llBEYb7b12MNO5uP/hZV5bh74w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iiIXR6Tmecm02GY4C9mUS4fMlyDjAVKyfNvJm2H2OUiAS3zBsCYBX1pgfzLyAVdIMGS+a62c2jsOHJjxqntkXRUbrxKyD2nh95PHtKVK2QAlPIt6IerMMyWKFbuqY7Ucx6YXdvPuXd3s2CxxoTjPW3mFaiikuVuw0jOPV2bP754= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=LvXuUCDJ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LvXuUCDJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532466; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8viUIwfAqrpZYGrKXKN1ylfsdDoFL9veEN2EDkxcPF4=; b=LvXuUCDJbOCG7QjnWTXI/qsfgT7GBB/gl0rcBgvPhB2hTiiHkXprECEnOLW2lrVgoNoS27 +KDtS1bBl7hV2gnI3f7vubeul2Y9tVN0N87gjaVgYb0x5pp8GHgCD6dvHACTB2z7bidU73 ksl+QHfmjYIiR+4235TEKFU9gkYBCNI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-513-FOOOGCS9Nq6C0wiBz3AC0A-1; Mon, 29 Jan 2024 07:47:40 -0500 X-MC-Unique: FOOOGCS9Nq6C0wiBz3AC0A-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ED2DB86F122; Mon, 29 Jan 2024 12:47:38 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 802EB8B; Mon, 29 Jan 2024 12:47:33 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 07/15] sparc/pgtable: define PFN_PTE_SHIFT Date: Mon, 29 Jan 2024 13:46:41 +0100 Message-ID: <20240129124649.189745-8-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" We want to make use of pte_next_pfn() outside of set_ptes(). Let's simply define PFN_PTE_SHIFT, required by pte_next_pfn(). Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- arch/sparc/include/asm/pgtable_64.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/p= gtable_64.h index a8c871b7d786..652af9d63fa2 100644 --- a/arch/sparc/include/asm/pgtable_64.h +++ b/arch/sparc/include/asm/pgtable_64.h @@ -929,6 +929,8 @@ static inline void __set_pte_at(struct mm_struct *mm, u= nsigned long addr, maybe_tlb_batch_add(mm, addr, ptep, orig, fullmm, PAGE_SHIFT); } =20 +#define PFN_PTE_SHIFT PAGE_SHIFT + static inline void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned int nr) { --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4664F64CCA for ; Mon, 29 Jan 2024 12:47:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532472; cv=none; b=RAhHIeQC7eAOD5h3c3pT7nhJcZCDqR0Ua+Yt38/EFV6IqdEusCvRh63yFE4Y7OpSfaVOwTB1bi7TPFgP4/KE0ELCpnuw6HMvFSD6ODCO2Le1xXPE3L8PbhOPt/BPIuXmmYBqSAo3321N3tvhkHMt5vU+qZmSn6wXCB9z9o+EkZ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532472; c=relaxed/simple; bh=NAbv74JbKmrqdh93Ibq9r1hHLRt7cxZLtU35wjHXMGI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FRbrZztvvvLfH81RSjqTiUc+tosSBwFiqDm5a1fCyuPBDt2YE0gIL0fnVZu54i2TEN5LvDPDtI44epwBdD7RsD4c9OyeiT3aqbqxbhN+kZSfdZGpsBBEavt+Z9cDMbuQHFqgnfd5yOFa+hs9FJGD9qgn9fyi3uwqRJF5ww8A1GI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=buX1YKBb; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="buX1YKBb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532470; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dXK02yTQR3iX9+2hOAsyvxeT8orGle3067Vuz1QJ0iE=; b=buX1YKBbCsioGm/KjK94h95ReJl75nIjKEJbJoS1FEuAFXRO2wy4poXWGMfc06K24Nwnkj yi1YiUSW29kWwqj4HDZB185GErAmv3sU1n45+kYKx8nVeqj2zG8xuKDSia3ch1qszNscrY LsC5FZmB5Um3EOqhK2clsmrTX/VkV5w= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-512-Yd-jPcF4M0WLV77ep4J5DA-1; Mon, 29 Jan 2024 07:47:46 -0500 X-MC-Unique: Yd-jPcF4M0WLV77ep4J5DA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C956E3CBD507; Mon, 29 Jan 2024 12:47:44 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 58D62111E8; Mon, 29 Jan 2024 12:47:39 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 08/15] mm/pgtable: make pte_next_pfn() independent of set_ptes() Date: Mon, 29 Jan 2024 13:46:42 +0100 Message-ID: <20240129124649.189745-9-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" Let's provide pte_next_pfn(), independently of set_ptes(). This allows for using the generic pte_next_pfn() version in some arch-specific set_ptes() implementations, and prepares for reusing pte_next_pfn() in other context. Reviewed-by: Christophe Leroy Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- include/linux/pgtable.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index f6d0e3513948..351cd9dc7194 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -212,7 +212,6 @@ static inline int pmd_dirty(pmd_t pmd) #define arch_flush_lazy_mmu_mode() do {} while (0) #endif =20 -#ifndef set_ptes =20 #ifndef pte_next_pfn static inline pte_t pte_next_pfn(pte_t pte) @@ -221,6 +220,7 @@ static inline pte_t pte_next_pfn(pte_t pte) } #endif =20 +#ifndef set_ptes /** * set_ptes - Map consecutive pages to a contiguous range of addresses. * @mm: Address space to map the pages into. --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DAB6C64CD8 for ; Mon, 29 Jan 2024 12:47:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532481; cv=none; b=bJNvzxg7+Qrd8jCHaweeRrZTBHHOEnzHcKKl/Vfvq8ryzCQEG+PFZzgLBHeuKTlnvmMa0E9q7Nb4lIczbBR/4eoQfmfEL9JODmiKutXepJDMkKY9c1WcyyNs36gECnQajlBglsX2fGV8tOAwDBZ1dnoQYcPnKWGJRzkVM9XNPXU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532481; c=relaxed/simple; bh=nuChnFMgPS9fPt8xp1x2/jToewLj0lu1rRdDXCEB68E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fufrSoxkFSjd17I7aWvXZZnGFv30ZhPU92R8BBfn5DwE/YfqEWsLWLmBdajH60x7cNJ3lrZFNSQ/VwRlLBT57Cg1GBrnx2EvLONZyXlu1ESxzSq2fmOkzLa3o8q6TEFaUC3VusQEg14aSu7kVTh8SaJT7S03Q1U0AvJV69lPGIo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=GYbE5ZOm; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="GYbE5ZOm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532479; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fKY0nJ8fYJnC2xkC+wk3vcUZKJJkEzEBPQxWgaxQbuA=; b=GYbE5ZOm3q31Ko41k8XMSbSg3uDvGADtxvtCJXlJEdcr12BdA3v6e3PctV3tjWeR7P5FYc 4jfSRVnOpo4kCGrtyEQkAcod2n8Df8VF5Umbtd3WGq97FVCAUPBj0FkuXz8jEfhZgo9I0+ LpMLHtGukTaVwCS1v6Pb+rDl3BgpJ0U= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-518-L0LUIbsCOk-VxGFEn38f-A-1; Mon, 29 Jan 2024 07:47:52 -0500 X-MC-Unique: L0LUIbsCOk-VxGFEn38f-A-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4AC391C04325; Mon, 29 Jan 2024 12:47:51 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 304AB8B; Mon, 29 Jan 2024 12:47:45 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 09/15] arm/mm: use pte_next_pfn() in set_ptes() Date: Mon, 29 Jan 2024 13:46:43 +0100 Message-ID: <20240129124649.189745-10-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" Let's use our handy helper now that it's available on all archs. Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- arch/arm/mm/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c index 674ed71573a8..c24e29c0b9a4 100644 --- a/arch/arm/mm/mmu.c +++ b/arch/arm/mm/mmu.c @@ -1814,6 +1814,6 @@ void set_ptes(struct mm_struct *mm, unsigned long add= r, if (--nr =3D=3D 0) break; ptep++; - pte_val(pteval) +=3D PAGE_SIZE; + pteval =3D pte_next_pfn(pteval); } } --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8DFEB679E0 for ; Mon, 29 Jan 2024 12:48:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532484; cv=none; b=H/nPzHbDoL29isbVY0nGHNMwT33/73xxocqNywbG3vMP9c5HlVQA/6tb2018yQA9Q3bQU5w4APw4DpOkxlXk95AcgizWKViWMt2xQiv0+ot9L3AjGIwYCLZ4L7qYVeO05tV6TYDAtdglc/tbFMq+V/arQvQQsEJs7+LhQaP8tuY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532484; c=relaxed/simple; bh=lsGdSPP9yjmN8xBq+o4rxdKZkBy0rfcqymp4kG2/GB8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Dsxz3S8YEtA3e+Jg4jWTBSyDczBcaUJp0HSWXkkYStFjzkM6d6Yaqd/QpNH0MPH0Lht1aBZJAF0CeyuyX07t2gEfdsKZuB/o83L4TQZLJh8TlhLwVkN2YDe+FbEQbVIevEc8CTKZRz/lUxaTl4EJ15M5nDv/4ZfGoZ+nhgkpvBw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=NByIchO8; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NByIchO8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532481; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y3Tbdl4mZ+WM96lxB3iHDtUIqesuPBdfScoz5at5AOg=; b=NByIchO8KE5MLtJDmZfH8OGs8LhDuR5XlJfDrE+D+ZFF7kvZjZg3oGzpFYv8JkhaYoDr0K 35qCMawuqCqv/98fmQ7lS/x2hcnYcRhOwCZBrrodxUL3Iqfj2IZG5Eqrdp29pTn1gg8ySC 9ahUEBPIg2NjP1z0Cy9vOtKDoL1JuE8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-380--x5t9X58Pm--BZadaKlp-Q-1; Mon, 29 Jan 2024 07:47:58 -0500 X-MC-Unique: -x5t9X58Pm--BZadaKlp-Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C50FF85A58A; Mon, 29 Jan 2024 12:47:56 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9ED7E8B; Mon, 29 Jan 2024 12:47:51 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 10/15] powerpc/mm: use pte_next_pfn() in set_ptes() Date: Mon, 29 Jan 2024 13:46:44 +0100 Message-ID: <20240129124649.189745-11-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" Let's use our handy new helper. Note that the implementation is slightly different, but shouldn't really make a difference in practice. Reviewed-by: Christophe Leroy Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- arch/powerpc/mm/pgtable.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c index a04ae4449a02..549a440ed7f6 100644 --- a/arch/powerpc/mm/pgtable.c +++ b/arch/powerpc/mm/pgtable.c @@ -220,10 +220,7 @@ void set_ptes(struct mm_struct *mm, unsigned long addr= , pte_t *ptep, break; ptep++; addr +=3D PAGE_SIZE; - /* - * increment the pfn. - */ - pte =3D pfn_pte(pte_pfn(pte) + 1, pte_pgprot((pte))); + pte =3D pte_next_pfn(pte); } } =20 --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 405A864CF2 for ; Mon, 29 Jan 2024 12:48:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532493; cv=none; b=DtBwQ5UwxW5NOPlAeb9FN9cQKBVYTqFoowsj5p9hhDbiavvDbi47y+Tx9meH3Ib106KZVxKT/Vzw5ZHNTJTc8D8ezAhwfSM5Tpwrxi5h5mT8OvrLeDQy7Li8ije/mXkifJgmZUXOoWsuTG07Rk1vPdyYB25m5stRDrObv8orJ/Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532493; c=relaxed/simple; bh=ByrEYHigCz/kRN5i/X/hP7nAwoCtSTm/annNcKKZ7M4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eyPcwsLpmcLC3BmVdLvPO6piS8XKopDAgmhSiBTGiMKeR2XoQbvRYUdBBzcuuKM0ztQB6bNe4mAOIdwRL1dLvFxvUioFaZfPAI5GZLJUMkGx3Jy7lH5IjtvBj/E+z1EZbj2IFdK4KKcDV3w+4zSXBaT7mmSY+muKC1QBpwcq/is= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BtC/Tpcn; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BtC/Tpcn" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532491; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d7wzDgEVUWhg5ZER45HLd710ycKB3fz8RPuXE32SMZA=; b=BtC/Tpcnz2LceVjkGy+IW7FvTPkE+B32bEYTnjYcJnanoRKCr1ky49rTmnyfUO8I2o/OW0 NbpYONU76lmBdpM3v77JF3zCi9gVMPPlhxGjGROfiApHcUUIPrM3/UMN9sl1XvdsMVnCll O348lKK9ichfw3xOyX9AkOfaoMnAZSM= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-140-9ly1pTIPMpeLAWW3OrQO1g-1; Mon, 29 Jan 2024 07:48:04 -0500 X-MC-Unique: 9ly1pTIPMpeLAWW3OrQO1g-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3AD4A3CBD50E; Mon, 29 Jan 2024 12:48:03 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2C0D28B; Mon, 29 Jan 2024 12:47:56 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 11/15] mm/memory: factor out copying the actual PTE in copy_present_pte() Date: Mon, 29 Jan 2024 13:46:45 +0100 Message-ID: <20240129124649.189745-12-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" Let's prepare for further changes. Reviewed-by: Ryan Roberts Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- mm/memory.c | 63 ++++++++++++++++++++++++++++------------------------- 1 file changed, 33 insertions(+), 30 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 8d14ba440929..a3bdb25f4c8d 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -930,6 +930,29 @@ copy_present_page(struct vm_area_struct *dst_vma, stru= ct vm_area_struct *src_vma return 0; } =20 +static inline void __copy_present_pte(struct vm_area_struct *dst_vma, + struct vm_area_struct *src_vma, pte_t *dst_pte, pte_t *src_pte, + pte_t pte, unsigned long addr) +{ + struct mm_struct *src_mm =3D src_vma->vm_mm; + + /* If it's a COW mapping, write protect it both processes. */ + if (is_cow_mapping(src_vma->vm_flags) && pte_write(pte)) { + ptep_set_wrprotect(src_mm, addr, src_pte); + pte =3D pte_wrprotect(pte); + } + + /* If it's a shared mapping, mark it clean in the child. */ + if (src_vma->vm_flags & VM_SHARED) + pte =3D pte_mkclean(pte); + pte =3D pte_mkold(pte); + + if (!userfaultfd_wp(dst_vma)) + pte =3D pte_clear_uffd_wp(pte); + + set_pte_at(dst_vma->vm_mm, addr, dst_pte, pte); +} + /* * Copy one pte. Returns 0 if succeeded, or -EAGAIN if one preallocated p= age * is required to copy this pte. @@ -939,23 +962,23 @@ copy_present_pte(struct vm_area_struct *dst_vma, stru= ct vm_area_struct *src_vma, pte_t *dst_pte, pte_t *src_pte, unsigned long addr, int *rss, struct folio **prealloc) { - struct mm_struct *src_mm =3D src_vma->vm_mm; - unsigned long vm_flags =3D src_vma->vm_flags; pte_t pte =3D ptep_get(src_pte); struct page *page; struct folio *folio; =20 page =3D vm_normal_page(src_vma, addr, pte); - if (page) - folio =3D page_folio(page); - if (page && folio_test_anon(folio)) { + if (unlikely(!page)) + goto copy_pte; + + folio =3D page_folio(page); + folio_get(folio); + if (folio_test_anon(folio)) { /* * If this page may have been pinned by the parent process, * copy the page immediately for the child so that we'll always * guarantee the pinned page won't be randomly replaced in the * future. */ - folio_get(folio); if (unlikely(folio_try_dup_anon_rmap_pte(folio, page, src_vma))) { /* Page may be pinned, we have to copy. */ folio_put(folio); @@ -963,34 +986,14 @@ copy_present_pte(struct vm_area_struct *dst_vma, stru= ct vm_area_struct *src_vma, addr, rss, prealloc, page); } rss[MM_ANONPAGES]++; - } else if (page) { - folio_get(folio); + VM_WARN_ON_FOLIO(PageAnonExclusive(page), folio); + } else { folio_dup_file_rmap_pte(folio, page); rss[mm_counter_file(folio)]++; } =20 - /* - * If it's a COW mapping, write protect it both - * in the parent and the child - */ - if (is_cow_mapping(vm_flags) && pte_write(pte)) { - ptep_set_wrprotect(src_mm, addr, src_pte); - pte =3D pte_wrprotect(pte); - } - VM_BUG_ON(page && folio_test_anon(folio) && PageAnonExclusive(page)); - - /* - * If it's a shared mapping, mark it clean in - * the child - */ - if (vm_flags & VM_SHARED) - pte =3D pte_mkclean(pte); - pte =3D pte_mkold(pte); - - if (!userfaultfd_wp(dst_vma)) - pte =3D pte_clear_uffd_wp(pte); - - set_pte_at(dst_vma->vm_mm, addr, dst_pte, pte); +copy_pte: + __copy_present_pte(dst_vma, src_vma, dst_pte, src_pte, pte, addr); return 0; } =20 --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97B6267E61 for ; Mon, 29 Jan 2024 12:48:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532497; cv=none; b=lTi92et+A/CIOc44grrUlRdl0x5HjFWEC1UwE3vGEuVnPDL39vSNSYDhX7MqvEJOW0EXRO8HVO64r77SneGnpyPb98Gls+jRgaOVzyf05jc6gx/T6bhdoTMwpRgwjjK9LGZ9IqFWog+VFPfUUaTNNl6szfJwzHyxP1p4I3Ivte0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532497; c=relaxed/simple; bh=WCHlJn12la69YWdL2YUOIZTol8Gam56seomR3cgdlf8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nDG7goiikzHUYHV5agBHLP3F9pI28q7kUCV8eFPzJCDakTMO9EQgFxd2UQ/3uAe5+WkEf64LmRMfBK9z+65pmb0AZS5Q4u7fcPlDActcxK41SJ0JkNjyTndJ72MQN2YgDY8eIYn6nNs72Wa+d2wok9Um3RxIJwGCWXEDOrr9DiY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=apaxodvu; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="apaxodvu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532494; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cS+fj/Ib0GI5otmzNrt4qLpnZeCkiOZS5Y+GTIa0gvY=; b=apaxodvu5Ho0lNvtdBYOSlY+///nImabbZg/KUjDpZzYNNy6HQZmIXeyUgTBFbKddQy3Er FV5Bsfw92j0Brddd0eo9WZf6QF8wwHk9PKIya6+GiZR5y6TdPWIcxXtdicOfSPAydzGtBI 0FvUvmVHlfbJSnu0pB/Iz2ItPEEP+l8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-639-uopDDWlrNyiX6spNRSMRFQ-1; Mon, 29 Jan 2024 07:48:10 -0500 X-MC-Unique: uopDDWlrNyiX6spNRSMRFQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D8CA6867943; Mon, 29 Jan 2024 12:48:08 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 99F798B; Mon, 29 Jan 2024 12:48:03 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 12/15] mm/memory: pass PTE to copy_present_pte() Date: Mon, 29 Jan 2024 13:46:46 +0100 Message-ID: <20240129124649.189745-13-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" We already read it, let's just forward it. This patch is based on work by Ryan Roberts. Reviewed-by: Ryan Roberts Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- mm/memory.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index a3bdb25f4c8d..41b24da5be38 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -959,10 +959,9 @@ static inline void __copy_present_pte(struct vm_area_s= truct *dst_vma, */ static inline int copy_present_pte(struct vm_area_struct *dst_vma, struct vm_area_struct *sr= c_vma, - pte_t *dst_pte, pte_t *src_pte, unsigned long addr, int *rss, - struct folio **prealloc) + pte_t *dst_pte, pte_t *src_pte, pte_t pte, unsigned long addr, + int *rss, struct folio **prealloc) { - pte_t pte =3D ptep_get(src_pte); struct page *page; struct folio *folio; =20 @@ -1103,7 +1102,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, } /* copy_present_pte() will clear `*prealloc' if consumed */ ret =3D copy_present_pte(dst_vma, src_vma, dst_pte, src_pte, - addr, rss, &prealloc); + ptent, addr, rss, &prealloc); /* * If we need a pre-allocated page for this pte, drop the * locks, allocate, and try again. --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84EDC651B5 for ; Mon, 29 Jan 2024 12:48:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532502; cv=none; b=i4Dw8Z3vYH6P06xo5Wyka7MEpXdHlvoTljFNKxGaeDgNfpJwpgxkn3moyQw3wv9FXWO9+pv+M/I7OEgdA7OVCWsUdarV/f1gHyE8wy9wpPvyLlplBos0AxP9Uby+I10DjQU+DCJYKt16XAPVJj5SL+dTbpxdIa2RDCLL0CAFCfA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532502; c=relaxed/simple; bh=BW1US5cMUO+3az2nkBk/Q2XXA2/iI656D4HsRTKejWY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=i90CLF45/cT7uJfKrjjqsicCy5fHsOLXPX/ARfhnkGjR17ksQCWQ8sPR6/N45+KJqUr5Rn/fGWD5B1ygLpYZgvvUODip9hUk9BHK+uYWG1lWp6aqneoXihBOOb9Yuv1LMR/NWosIQ+quu7+CTNlJR13mjOvnZt4K0fOjsX/uGmQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=SmzgwHJ1; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SmzgwHJ1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532499; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GLdn+T1R5PiYDW19VyAaeLNlt8B1zt8mn8YsDhIt5B8=; b=SmzgwHJ1XT60HCh60HP2Q68+1dzGgv20haSbIjHNuN0GgCTH1FXhgvEBH0d8f7IQ+B2Tkl SyZHwh5TculXYfZt3cRnrGEoCixz3Y7eE1ltQ6OAK20FtuLgd4xSDHIgOA5+frLc49A3MO tbkgcJXkjk9rX6orJo8W/He+hI3n/18= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-602-1IZZ7hGdOwG8M8ciVMufmA-1; Mon, 29 Jan 2024 07:48:16 -0500 X-MC-Unique: 1IZZ7hGdOwG8M8ciVMufmA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9B83A29AA3AC; Mon, 29 Jan 2024 12:48:14 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 277D3111E8; Mon, 29 Jan 2024 12:48:09 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 13/15] mm/memory: optimize fork() with PTE-mapped THP Date: Mon, 29 Jan 2024 13:46:47 +0100 Message-ID: <20240129124649.189745-14-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" Let's implement PTE batching when consecutive (present) PTEs map consecutive pages of the same large folio, and all other PTE bits besides the PFNs are equal. We will optimize folio_pte_batch() separately, to ignore selected PTE bits. This patch is based on work by Ryan Roberts. Use __always_inline for __copy_present_ptes() and keep the handling for single PTEs completely separate from the multi-PTE case: we really want the compiler to optimize for the single-PTE case with small folios, to not degrade performance. Note that PTE batching will never exceed a single page table and will always stay within VMA boundaries. Further, processing PTE-mapped THP that maybe pinned and have PageAnonExclusive set on at least one subpage should work as expected, but there is room for improvement: We will repeatedly (1) detect a PTE batch (2) detect that we have to copy a page (3) fall back and allocate a single page to copy a single page. For now we won't care as pinned pages are a corner case, and we should rather look into maintaining only a single PageAnonExclusive bit for large folios. Reviewed-by: Ryan Roberts Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) Tested-by: Ryan Roberts --- include/linux/pgtable.h | 31 +++++++++++ mm/memory.c | 112 +++++++++++++++++++++++++++++++++------- 2 files changed, 124 insertions(+), 19 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 351cd9dc7194..aab227e12493 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -650,6 +650,37 @@ static inline void ptep_set_wrprotect(struct mm_struct= *mm, unsigned long addres } #endif =20 +#ifndef wrprotect_ptes +/** + * wrprotect_ptes - Write-protect PTEs that map consecutive pages of the s= ame + * folio. + * @mm: Address space the pages are mapped into. + * @addr: Address the first page is mapped at. + * @ptep: Page table pointer for the first entry. + * @nr: Number of entries to write-protect. + * + * May be overridden by the architecture; otherwise, implemented as a simp= le + * loop over ptep_set_wrprotect(). + * + * Note that PTE bits in the PTE range besides the PFN can differ. For exa= mple, + * some PTEs might be write-protected. + * + * Context: The caller holds the page table lock. The PTEs map consecutive + * pages that belong to the same folio. The PTEs are all in the same PMD. + */ +static inline void wrprotect_ptes(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, unsigned int nr) +{ + for (;;) { + ptep_set_wrprotect(mm, addr, ptep); + if (--nr =3D=3D 0) + break; + ptep++; + addr +=3D PAGE_SIZE; + } +} +#endif + /* * On some architectures hardware does not set page access bit when access= ing * memory page, it is responsibility of software setting this bit. It brin= gs diff --git a/mm/memory.c b/mm/memory.c index 41b24da5be38..86f8a0021c8e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -930,15 +930,15 @@ copy_present_page(struct vm_area_struct *dst_vma, str= uct vm_area_struct *src_vma return 0; } =20 -static inline void __copy_present_pte(struct vm_area_struct *dst_vma, +static __always_inline void __copy_present_ptes(struct vm_area_struct *dst= _vma, struct vm_area_struct *src_vma, pte_t *dst_pte, pte_t *src_pte, - pte_t pte, unsigned long addr) + pte_t pte, unsigned long addr, int nr) { struct mm_struct *src_mm =3D src_vma->vm_mm; =20 /* If it's a COW mapping, write protect it both processes. */ if (is_cow_mapping(src_vma->vm_flags) && pte_write(pte)) { - ptep_set_wrprotect(src_mm, addr, src_pte); + wrprotect_ptes(src_mm, addr, src_pte, nr); pte =3D pte_wrprotect(pte); } =20 @@ -950,26 +950,93 @@ static inline void __copy_present_pte(struct vm_area_= struct *dst_vma, if (!userfaultfd_wp(dst_vma)) pte =3D pte_clear_uffd_wp(pte); =20 - set_pte_at(dst_vma->vm_mm, addr, dst_pte, pte); + set_ptes(dst_vma->vm_mm, addr, dst_pte, pte, nr); +} + +/* + * Detect a PTE batch: consecutive (present) PTEs that map consecutive + * pages of the same folio. + * + * All PTEs inside a PTE batch have the same PTE bits set, excluding the P= FN. + */ +static inline int folio_pte_batch(struct folio *folio, unsigned long addr, + pte_t *start_ptep, pte_t pte, int max_nr) +{ + unsigned long folio_end_pfn =3D folio_pfn(folio) + folio_nr_pages(folio); + const pte_t *end_ptep =3D start_ptep + max_nr; + pte_t expected_pte =3D pte_next_pfn(pte); + pte_t *ptep =3D start_ptep + 1; + + VM_WARN_ON_FOLIO(!pte_present(pte), folio); + + while (ptep !=3D end_ptep) { + pte =3D ptep_get(ptep); + + if (!pte_same(pte, expected_pte)) + break; + + /* + * Stop immediately once we reached the end of the folio. In + * corner cases the next PFN might fall into a different + * folio. + */ + if (pte_pfn(pte) =3D=3D folio_end_pfn) + break; + + expected_pte =3D pte_next_pfn(expected_pte); + ptep++; + } + + return ptep - start_ptep; } =20 /* - * Copy one pte. Returns 0 if succeeded, or -EAGAIN if one preallocated p= age - * is required to copy this pte. + * Copy one present PTE, trying to batch-process subsequent PTEs that map + * consecutive pages of the same folio by copying them as well. + * + * Returns -EAGAIN if one preallocated page is required to copy the next P= TE. + * Otherwise, returns the number of copied PTEs (at least 1). */ static inline int -copy_present_pte(struct vm_area_struct *dst_vma, struct vm_area_struct *sr= c_vma, +copy_present_ptes(struct vm_area_struct *dst_vma, struct vm_area_struct *s= rc_vma, pte_t *dst_pte, pte_t *src_pte, pte_t pte, unsigned long addr, - int *rss, struct folio **prealloc) + int max_nr, int *rss, struct folio **prealloc) { struct page *page; struct folio *folio; + int err, nr; =20 page =3D vm_normal_page(src_vma, addr, pte); if (unlikely(!page)) goto copy_pte; =20 folio =3D page_folio(page); + + /* + * If we likely have to copy, just don't bother with batching. Make + * sure that the common "small folio" case is as fast as possible + * by keeping the batching logic separate. + */ + if (unlikely(!*prealloc && folio_test_large(folio) && max_nr !=3D 1)) { + nr =3D folio_pte_batch(folio, addr, src_pte, pte, max_nr); + folio_ref_add(folio, nr); + if (folio_test_anon(folio)) { + if (unlikely(folio_try_dup_anon_rmap_ptes(folio, page, + nr, src_vma))) { + folio_ref_sub(folio, nr); + return -EAGAIN; + } + rss[MM_ANONPAGES] +=3D nr; + VM_WARN_ON_FOLIO(PageAnonExclusive(page), folio); + } else { + folio_dup_file_rmap_ptes(folio, page, nr); + rss[mm_counter_file(folio)] +=3D nr; + } + __copy_present_ptes(dst_vma, src_vma, dst_pte, src_pte, pte, + addr, nr); + return nr; + } + folio_get(folio); if (folio_test_anon(folio)) { /* @@ -981,8 +1048,9 @@ copy_present_pte(struct vm_area_struct *dst_vma, struc= t vm_area_struct *src_vma, if (unlikely(folio_try_dup_anon_rmap_pte(folio, page, src_vma))) { /* Page may be pinned, we have to copy. */ folio_put(folio); - return copy_present_page(dst_vma, src_vma, dst_pte, src_pte, - addr, rss, prealloc, page); + err =3D copy_present_page(dst_vma, src_vma, dst_pte, src_pte, + addr, rss, prealloc, page); + return err ? err : 1; } rss[MM_ANONPAGES]++; VM_WARN_ON_FOLIO(PageAnonExclusive(page), folio); @@ -992,8 +1060,8 @@ copy_present_pte(struct vm_area_struct *dst_vma, struc= t vm_area_struct *src_vma, } =20 copy_pte: - __copy_present_pte(dst_vma, src_vma, dst_pte, src_pte, pte, addr); - return 0; + __copy_present_ptes(dst_vma, src_vma, dst_pte, src_pte, pte, addr, 1); + return 1; } =20 static inline struct folio *folio_prealloc(struct mm_struct *src_mm, @@ -1030,10 +1098,11 @@ copy_pte_range(struct vm_area_struct *dst_vma, stru= ct vm_area_struct *src_vma, pte_t *src_pte, *dst_pte; pte_t ptent; spinlock_t *src_ptl, *dst_ptl; - int progress, ret =3D 0; + int progress, max_nr, ret =3D 0; int rss[NR_MM_COUNTERS]; swp_entry_t entry =3D (swp_entry_t){0}; struct folio *prealloc =3D NULL; + int nr; =20 again: progress =3D 0; @@ -1064,6 +1133,8 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, arch_enter_lazy_mmu_mode(); =20 do { + nr =3D 1; + /* * We are holding two locks at this point - either of them * could generate latencies in another task on another CPU. @@ -1100,9 +1171,10 @@ copy_pte_range(struct vm_area_struct *dst_vma, struc= t vm_area_struct *src_vma, */ WARN_ON_ONCE(ret !=3D -ENOENT); } - /* copy_present_pte() will clear `*prealloc' if consumed */ - ret =3D copy_present_pte(dst_vma, src_vma, dst_pte, src_pte, - ptent, addr, rss, &prealloc); + /* copy_present_ptes() will clear `*prealloc' if consumed */ + max_nr =3D (end - addr) / PAGE_SIZE; + ret =3D copy_present_ptes(dst_vma, src_vma, dst_pte, src_pte, + ptent, addr, max_nr, rss, &prealloc); /* * If we need a pre-allocated page for this pte, drop the * locks, allocate, and try again. @@ -1119,8 +1191,10 @@ copy_pte_range(struct vm_area_struct *dst_vma, struc= t vm_area_struct *src_vma, folio_put(prealloc); prealloc =3D NULL; } - progress +=3D 8; - } while (dst_pte++, src_pte++, addr +=3D PAGE_SIZE, addr !=3D end); + nr =3D ret; + progress +=3D 8 * nr; + } while (dst_pte +=3D nr, src_pte +=3D nr, addr +=3D PAGE_SIZE * nr, + addr !=3D end); =20 arch_leave_lazy_mmu_mode(); pte_unmap_unlock(orig_src_pte, src_ptl); @@ -1141,7 +1215,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, prealloc =3D folio_prealloc(src_mm, src_vma, addr, false); if (!prealloc) return -ENOMEM; - } else if (ret) { + } else if (ret < 0) { VM_WARN_ON_ONCE(1); } =20 --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99CFE67E9A for ; Mon, 29 Jan 2024 12:48:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532511; cv=none; b=nndM2iEJvS6107jty7m9G0ydacSgin1pI6uPLrMiWly8s+/71PhnXCIiOt8qNyTEzGWt19l3NMb0KDuvhJNO0vj0TnQ7QmOWFWwg1ci7eSnBDBxPNZYJN+de5lQc71bPhx8fDSp5S5pXvOWbwow4XXfDKG7d2uVZE//5Unqh8VA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532511; c=relaxed/simple; bh=HqyS3uvSq9oyOXHSShmhRtcbYKCwhrCOyj2V83p6M5c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pPsU/nTRHxrcr3BFtG68jSkLLyoIUcAdPXOerg6dBeJxj6bhUCQtVb8R76wVydE2Iba3O82HBtpIVwdLC7xIStgM6KJydgFejmLJWZ/cYKzpmYKz7EY5a/NR38KnKRdFZBmg9+1f2EOpmn8ZR0GJ8eqUUTkWp4FlvIgiQGkphP4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cJzO0cqx; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cJzO0cqx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=exEdDlpMqX/TLP6WakzBspRTLcII6GNZP9MIcL4XseU=; b=cJzO0cqxhK5avAha6o6hk2niwd7zGRBHlpgAav7pgjKN6U3VFdLi+pYkOOesmxKONz2sJV 5CHNKxcgayYTvQaJzAMWFyGdnTrh7VpXDjn7UOdNZ+w/TqEKTdCUfvsRJn0vG2p3eppg/1 OIjuEQ6Asj78aNJzApIRDXXbPqX0PTk= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-455-q1tX4gb6M5-rogvUSokYZw-1; Mon, 29 Jan 2024 07:48:21 -0500 X-MC-Unique: q1tX4gb6M5-rogvUSokYZw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0A9C628B72E2; Mon, 29 Jan 2024 12:48:20 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id D9D738B; Mon, 29 Jan 2024 12:48:14 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 14/15] mm/memory: ignore dirty/accessed/soft-dirty bits in folio_pte_batch() Date: Mon, 29 Jan 2024 13:46:48 +0100 Message-ID: <20240129124649.189745-15-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" Let's always ignore the accessed/young bit: we'll always mark the PTE as old in our child process during fork, and upcoming users will similarly not care. Ignore the dirty bit only if we don't want to duplicate the dirty bit into the child process during fork. Maybe, we could just set all PTEs in the child dirty if any PTE is dirty. For now, let's keep the behavior unchanged, this can be optimized later if required. Ignore the soft-dirty bit only if the bit doesn't have any meaning in the src vma, and similarly won't have any in the copied dst vma. For now, we won't bother with the uffd-wp bit. Reviewed-by: Ryan Roberts Signed-off-by: David Hildenbrand Tested-by: Ryan Roberts --- mm/memory.c | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 86f8a0021c8e..b2ec2b6b54c7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -953,24 +953,44 @@ static __always_inline void __copy_present_ptes(struc= t vm_area_struct *dst_vma, set_ptes(dst_vma->vm_mm, addr, dst_pte, pte, nr); } =20 +/* Flags for folio_pte_batch(). */ +typedef int __bitwise fpb_t; + +/* Compare PTEs after pte_mkclean(), ignoring the dirty bit. */ +#define FPB_IGNORE_DIRTY ((__force fpb_t)BIT(0)) + +/* Compare PTEs after pte_clear_soft_dirty(), ignoring the soft-dirty bit.= */ +#define FPB_IGNORE_SOFT_DIRTY ((__force fpb_t)BIT(1)) + +static inline pte_t __pte_batch_clear_ignored(pte_t pte, fpb_t flags) +{ + if (flags & FPB_IGNORE_DIRTY) + pte =3D pte_mkclean(pte); + if (likely(flags & FPB_IGNORE_SOFT_DIRTY)) + pte =3D pte_clear_soft_dirty(pte); + return pte_mkold(pte); +} + /* * Detect a PTE batch: consecutive (present) PTEs that map consecutive * pages of the same folio. * - * All PTEs inside a PTE batch have the same PTE bits set, excluding the P= FN. + * All PTEs inside a PTE batch have the same PTE bits set, excluding the P= FN, + * the accessed bit, dirty bit (with FPB_IGNORE_DIRTY) and soft-dirty bit + * (with FPB_IGNORE_SOFT_DIRTY). */ static inline int folio_pte_batch(struct folio *folio, unsigned long addr, - pte_t *start_ptep, pte_t pte, int max_nr) + pte_t *start_ptep, pte_t pte, int max_nr, fpb_t flags) { unsigned long folio_end_pfn =3D folio_pfn(folio) + folio_nr_pages(folio); const pte_t *end_ptep =3D start_ptep + max_nr; - pte_t expected_pte =3D pte_next_pfn(pte); + pte_t expected_pte =3D __pte_batch_clear_ignored(pte_next_pfn(pte), flags= ); pte_t *ptep =3D start_ptep + 1; =20 VM_WARN_ON_FOLIO(!pte_present(pte), folio); =20 while (ptep !=3D end_ptep) { - pte =3D ptep_get(ptep); + pte =3D __pte_batch_clear_ignored(ptep_get(ptep), flags); =20 if (!pte_same(pte, expected_pte)) break; @@ -1004,6 +1024,7 @@ copy_present_ptes(struct vm_area_struct *dst_vma, str= uct vm_area_struct *src_vma { struct page *page; struct folio *folio; + fpb_t flags =3D 0; int err, nr; =20 page =3D vm_normal_page(src_vma, addr, pte); @@ -1018,7 +1039,12 @@ copy_present_ptes(struct vm_area_struct *dst_vma, st= ruct vm_area_struct *src_vma * by keeping the batching logic separate. */ if (unlikely(!*prealloc && folio_test_large(folio) && max_nr !=3D 1)) { - nr =3D folio_pte_batch(folio, addr, src_pte, pte, max_nr); + if (src_vma->vm_flags & VM_SHARED) + flags |=3D FPB_IGNORE_DIRTY; + if (!vma_soft_dirty_enabled(src_vma)) + flags |=3D FPB_IGNORE_SOFT_DIRTY; + + nr =3D folio_pte_batch(folio, addr, src_pte, pte, max_nr, flags); folio_ref_add(folio, nr); if (folio_test_anon(folio)) { if (unlikely(folio_try_dup_anon_rmap_ptes(folio, page, --=20 2.43.0 From nobody Wed Dec 24 01:33:17 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FE6661691 for ; Mon, 29 Jan 2024 12:48:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532512; cv=none; b=RAGdcn3IHTh+mZz2MD4l8ke5X55tvM9VA4yUrRejWLyor2Nz9fW6SOHX+im4DS3yyuJ/r6Y2HYPxGy0AiyvkiMhn0s55psYVgW947z0YWFsW+hwdtg1lgiqn7ObENAVSMOdAWNQ6durKvxHuB5bYe9RJYzUaNPBVXp4YP7uJMrE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706532512; c=relaxed/simple; bh=WijXHNN6cEbo4+hvn2lkZ0k2/r3mW8vSMOQ+qEBDy2U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hOWCFrFUr4r8bnh52Ih+oyE0tJTiJ7szV2UV3brm5EuQz8CYdbl7OLumxYaf4L29SIoi0r2T0D4zZaBEXcYioeO+Fp4GtSJGTxiTRt8pfXzlf3qWxbDDiba4G+pAB71aqEGM90uo9GJlD6vLPqGUvszjMfMh6MWWoFQRwgeojak= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=KjeRfVvG; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KjeRfVvG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1706532510; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kPofd6LlL0FntZsVxpMLUzci6vAQKc1aG/cPn5S11O0=; b=KjeRfVvG+5QpX2CYiPT2SE6634umI5mDUdNheSMhDv0jw9AOx1JJXHZTWJwuSTQXpFzU+L 8BUmSoVQ8cOop6J49klZIrAlGwqjDZlLKVZbiyoXzrlWP9B7XCUA0JOxmuZ9WWyC2q/Wmf 8Ec0Eb/dlzQUNIwNa/Lic4sRwMRwQk0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-413-PehoyRVrMfeAXOFdLIlxVA-1; Mon, 29 Jan 2024 07:48:26 -0500 X-MC-Unique: PehoyRVrMfeAXOFdLIlxVA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 898A186F129; Mon, 29 Jan 2024 12:48:25 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.194.46]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4CC6D8B; Mon, 29 Jan 2024 12:48:20 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v3 15/15] mm/memory: ignore writable bit in folio_pte_batch() Date: Mon, 29 Jan 2024 13:46:49 +0100 Message-ID: <20240129124649.189745-16-david@redhat.com> In-Reply-To: <20240129124649.189745-1-david@redhat.com> References: <20240129124649.189745-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.5 Content-Type: text/plain; charset="utf-8" ... and conditionally return to the caller if any PTE except the first one is writable. fork() has to make sure to properly write-protect in case any PTE is writable. Other users (e.g., page unmaping) are expected to not care. Reviewed-by: Ryan Roberts Signed-off-by: David Hildenbrand Tested-by: Ryan Roberts --- mm/memory.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index b2ec2b6b54c7..b05fd28dbce1 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -968,7 +968,7 @@ static inline pte_t __pte_batch_clear_ignored(pte_t pte= , fpb_t flags) pte =3D pte_mkclean(pte); if (likely(flags & FPB_IGNORE_SOFT_DIRTY)) pte =3D pte_clear_soft_dirty(pte); - return pte_mkold(pte); + return pte_wrprotect(pte_mkold(pte)); } =20 /* @@ -976,21 +976,32 @@ static inline pte_t __pte_batch_clear_ignored(pte_t p= te, fpb_t flags) * pages of the same folio. * * All PTEs inside a PTE batch have the same PTE bits set, excluding the P= FN, - * the accessed bit, dirty bit (with FPB_IGNORE_DIRTY) and soft-dirty bit - * (with FPB_IGNORE_SOFT_DIRTY). + * the accessed bit, writable bit, dirty bit (with FPB_IGNORE_DIRTY) and + * soft-dirty bit (with FPB_IGNORE_SOFT_DIRTY). + * + * If "any_writable" is set, it will indicate if any other PTE besides the + * first (given) PTE is writable. */ static inline int folio_pte_batch(struct folio *folio, unsigned long addr, - pte_t *start_ptep, pte_t pte, int max_nr, fpb_t flags) + pte_t *start_ptep, pte_t pte, int max_nr, fpb_t flags, + bool *any_writable) { unsigned long folio_end_pfn =3D folio_pfn(folio) + folio_nr_pages(folio); const pte_t *end_ptep =3D start_ptep + max_nr; pte_t expected_pte =3D __pte_batch_clear_ignored(pte_next_pfn(pte), flags= ); pte_t *ptep =3D start_ptep + 1; + bool writable; + + if (any_writable) + *any_writable =3D false; =20 VM_WARN_ON_FOLIO(!pte_present(pte), folio); =20 while (ptep !=3D end_ptep) { - pte =3D __pte_batch_clear_ignored(ptep_get(ptep), flags); + pte =3D ptep_get(ptep); + if (any_writable) + writable =3D !!pte_write(pte); + pte =3D __pte_batch_clear_ignored(pte, flags); =20 if (!pte_same(pte, expected_pte)) break; @@ -1003,6 +1014,9 @@ static inline int folio_pte_batch(struct folio *folio= , unsigned long addr, if (pte_pfn(pte) =3D=3D folio_end_pfn) break; =20 + if (any_writable) + *any_writable |=3D writable; + expected_pte =3D pte_next_pfn(expected_pte); ptep++; } @@ -1024,6 +1038,7 @@ copy_present_ptes(struct vm_area_struct *dst_vma, str= uct vm_area_struct *src_vma { struct page *page; struct folio *folio; + bool any_writable; fpb_t flags =3D 0; int err, nr; =20 @@ -1044,7 +1059,8 @@ copy_present_ptes(struct vm_area_struct *dst_vma, str= uct vm_area_struct *src_vma if (!vma_soft_dirty_enabled(src_vma)) flags |=3D FPB_IGNORE_SOFT_DIRTY; =20 - nr =3D folio_pte_batch(folio, addr, src_pte, pte, max_nr, flags); + nr =3D folio_pte_batch(folio, addr, src_pte, pte, max_nr, flags, + &any_writable); folio_ref_add(folio, nr); if (folio_test_anon(folio)) { if (unlikely(folio_try_dup_anon_rmap_ptes(folio, page, @@ -1058,6 +1074,8 @@ copy_present_ptes(struct vm_area_struct *dst_vma, str= uct vm_area_struct *src_vma folio_dup_file_rmap_ptes(folio, page, nr); rss[mm_counter_file(folio)] +=3D nr; } + if (any_writable) + pte =3D pte_mkwrite(pte, src_vma); __copy_present_ptes(dst_vma, src_vma, dst_pte, src_pte, pte, addr, nr); return nr; --=20 2.43.0