From nobody Wed Jul 1 00:47:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 881D1C433F5 for ; Wed, 5 Jan 2022 21:48:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235115AbiAEVs3 (ORCPT ); Wed, 5 Jan 2022 16:48:29 -0500 Received: from new4-smtp.messagingengine.com ([66.111.4.230]:54785 "EHLO new4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234117AbiAEVsY (ORCPT ); Wed, 5 Jan 2022 16:48:24 -0500 Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailnew.nyi.internal (Postfix) with ESMTP id 0152D580510; Wed, 5 Jan 2022 16:48:24 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Wed, 05 Jan 2022 16:48:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=from :to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; s=fm2; bh=ssIavQd0t027n j+SJdZ+R89EjFohei8nsJIZDjq+gAc=; b=S0iFzugiHfWc3pkd+CZdBT6nLqkU1 O293WHqzAfo4z2NG7KeyjrOwbHMOOslgSqp/nzgDYhgtHbQy1TP5B107IQwfKrV8 sEDHbFP1mzRmf/bsMkZZ7+2pcaq0OjVwXSzmeIevlmdj0dGB74IDIhFKz9IeN+9N iEOV2a9f3zb+Q4MA+mVXYAU6GfHusjfsdpI8K78PATIFGaq7Y/d6Mq7pnJoAcUYA 9t/UBNDiCl8irKYbP4TG3sHTQN1b3IZRkuFRYdHXRZtYVMgM7MLwU70CNVDKvBw8 beX2PYbUiLK11zp70mQu+GnKdoHILPBnmmksVpxTSZHVqyVMG6Jdp07+g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:reply-to:subject :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=ssIavQd0t027nj+SJdZ+R89EjFohei8nsJIZDjq+gAc=; b=LJN2Ozob Zz3Qf1m9F8UnBOu76cytEY9asp+rvN52Rrd3ey/iblpRlyPSEJP8LwIE09PVGXL1 HkPLjcnDoTYkde2g6ZdmiTjinrkRQchiWLtI0g86iG7g0bP3QS7t+ridmDGtdgrE C7k2cvCS03Lqd4KYR9k5z9ysNwlR1BFEwFeJoN4uFz3/XQ4C+/mXMtj2J/4nnRf3 CGhK7Gqw5TiVw9sJW5LYNs2M+hd4v/dIjr9UmUd8Ofc85jh8ZMgWmMcJkeUwA361 Mn8Zye4hve17YR89Pg0yBCRGikTbApdB58nmr7svo5XlODiH2tKtbvrz4pIM/xsp IfqqtAyzNHJAVA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddrudefjedgieekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfrhgggfestdhqredtredttdenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepkeeiue elvddtteeujeehtdegheejledvtdetgfeileejfeeghfeftdeuffefleevnecuffhomhgr ihhnpehkvghrnhgvlhdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpeiiihdrhigrnhesshgvnhhtrdgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Jan 2022 16:48:23 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Marek Szyprowski , Robin Murphy , linuxppc-dev@lists.ozlabs.org, virtualization@lists.linux-foundation.org, iommu@lists.linux-foundation.org, Vlastimil Babka , Mel Gorman , Eric Ren , Zi Yan Subject: [RFC PATCH v3 1/8] mm: page_alloc: avoid merging non-fallbackable pageblocks with others. Date: Wed, 5 Jan 2022 16:47:49 -0500 Message-Id: <20220105214756.91065-2-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220105214756.91065-1-zi.yan@sent.com> References: <20220105214756.91065-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Zi Yan This is done in addition to MIGRATE_ISOLATE pageblock merge avoidance. It prepares for the upcoming removal of the MAX_ORDER-1 alignment requirement for CMA and alloc_contig_range(). MIGRARTE_HIGHATOMIC should not merge with other migratetypes like MIGRATE_ISOLATE and MIGRARTE_CMA[1], so this commit prevents that too. Also add MIGRARTE_HIGHATOMIC to fallbacks array for completeness. [1] https://lore.kernel.org/linux-mm/20211130100853.GP3366@techsingularity.= net/ Signed-off-by: Zi Yan --- include/linux/mmzone.h | 6 ++++++ mm/page_alloc.c | 28 ++++++++++++++++++---------- 2 files changed, 24 insertions(+), 10 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index aed44e9b5d89..0aa549653e4e 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -83,6 +83,12 @@ static inline bool is_migrate_movable(int mt) return is_migrate_cma(mt) || mt =3D=3D MIGRATE_MOVABLE; } =20 +/* See fallbacks[MIGRATE_TYPES][3] in page_alloc.c */ +static inline bool migratetype_has_fallback(int mt) +{ + return mt < MIGRATE_PCPTYPES; +} + #define for_each_migratetype_order(order, type) \ for (order =3D 0; order < MAX_ORDER; order++) \ for (type =3D 0; type < MIGRATE_TYPES; type++) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 8dd6399bafb5..5193c953dbf8 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1042,6 +1042,12 @@ buddy_merge_likely(unsigned long pfn, unsigned long = buddy_pfn, return page_is_buddy(higher_page, higher_buddy, order + 1); } =20 +static inline bool has_non_fallback_pageblock(struct zone *zone) +{ + return has_isolate_pageblock(zone) || zone_cma_pages(zone) !=3D 0 || + zone->nr_reserved_highatomic !=3D 0; +} + /* * Freeing function for a buddy system allocator. * @@ -1117,14 +1123,15 @@ static inline void __free_one_page(struct page *pag= e, } if (order < MAX_ORDER - 1) { /* If we are here, it means order is >=3D pageblock_order. - * We want to prevent merge between freepages on isolate - * pageblock and normal pageblock. Without this, pageblock - * isolation could cause incorrect freepage or CMA accounting. + * We want to prevent merge between freepages on pageblock + * without fallbacks and normal pageblock. Without this, + * pageblock isolation could cause incorrect freepage or CMA + * accounting or HIGHATOMIC accounting. * * We don't want to hit this code for the more frequent * low-order merging. */ - if (unlikely(has_isolate_pageblock(zone))) { + if (unlikely(has_non_fallback_pageblock(zone))) { int buddy_mt; =20 buddy_pfn =3D __find_buddy_pfn(pfn, order); @@ -1132,8 +1139,8 @@ static inline void __free_one_page(struct page *page, buddy_mt =3D get_pageblock_migratetype(buddy); =20 if (migratetype !=3D buddy_mt - && (is_migrate_isolate(migratetype) || - is_migrate_isolate(buddy_mt))) + && (!migratetype_has_fallback(migratetype) || + !migratetype_has_fallback(buddy_mt))) goto done_merging; } max_order =3D order + 1; @@ -2484,6 +2491,7 @@ static int fallbacks[MIGRATE_TYPES][3] =3D { [MIGRATE_UNMOVABLE] =3D { MIGRATE_RECLAIMABLE, MIGRATE_MOVABLE, MIGRA= TE_TYPES }, [MIGRATE_MOVABLE] =3D { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE, MIGRA= TE_TYPES }, [MIGRATE_RECLAIMABLE] =3D { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE, MIGRA= TE_TYPES }, + [MIGRATE_HIGHATOMIC] =3D { MIGRATE_TYPES }, /* Never used */ #ifdef CONFIG_CMA [MIGRATE_CMA] =3D { MIGRATE_TYPES }, /* Never used */ #endif @@ -2795,8 +2803,8 @@ static void reserve_highatomic_pageblock(struct page = *page, struct zone *zone, =20 /* Yoink! */ mt =3D get_pageblock_migratetype(page); - if (!is_migrate_highatomic(mt) && !is_migrate_isolate(mt) - && !is_migrate_cma(mt)) { + /* Only reserve normal pageblock */ + if (migratetype_has_fallback(mt)) { zone->nr_reserved_highatomic +=3D pageblock_nr_pages; set_pageblock_migratetype(page, MIGRATE_HIGHATOMIC); move_freepages_block(zone, page, MIGRATE_HIGHATOMIC, NULL); @@ -3545,8 +3553,8 @@ int __isolate_free_page(struct page *page, unsigned i= nt order) struct page *endpage =3D page + (1 << order) - 1; for (; page < endpage; page +=3D pageblock_nr_pages) { int mt =3D get_pageblock_migratetype(page); - if (!is_migrate_isolate(mt) && !is_migrate_cma(mt) - && !is_migrate_highatomic(mt)) + /* Only change normal pageblock */ + if (migratetype_has_fallback(mt)) set_pageblock_migratetype(page, MIGRATE_MOVABLE); } --=20 2.34.1 From nobody Wed Jul 1 00:47:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61F09C4332F for ; Wed, 5 Jan 2022 21:48:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235735AbiAEVse (ORCPT ); Wed, 5 Jan 2022 16:48:34 -0500 Received: from new4-smtp.messagingengine.com ([66.111.4.230]:35175 "EHLO new4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234125AbiAEVsZ (ORCPT ); Wed, 5 Jan 2022 16:48:25 -0500 Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailnew.nyi.internal (Postfix) with ESMTP id B286B580514; Wed, 5 Jan 2022 16:48:24 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Wed, 05 Jan 2022 16:48:24 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=from :to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; s=fm2; bh=F+6ytW8tADp2L NXytpAkKv+4a//pcBSmjTaldQvhu2o=; b=zJFe7Ao9MCOK+TZjq/K/TbeuXuhub uEMymMtNy7GDdtEO7HZkK2k7I0QJefpyNTc6QIDRv3VbTOQEz0mNoaKnX14u+Qdb o2OErLMPr8nbMPNViHntkADNEWvCU58ZNRNcUUUmv7lhKZFOrqn1ybsMWLAGNaxm lUcKl2Ja/YD7ICRFcBTTQK5B8G882NbVn06HGziGFLC39Gegp3cWIj7DDkwJ7odw C+Q7WHZw32/+23uQlB375cehRMhlM7Ev0KcGrO73+dc0fxhwnorPVP9V7xPUULlD Bm5XJPhUX0VfbeXD3jZhrIUp323BLLWatMr3g4u8XfgsAcn1XBlhLyzSw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:reply-to:subject :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=F+6ytW8tADp2LNXytpAkKv+4a//pcBSmjTaldQvhu2o=; b=I9TcvzqS Rw1NjBo+afbuM8HtqRLcbw5Y9/BUmNxengQ94iNfi4ETD2oDiTzizNxDWj0LPQ2B OfoQMTZ8GfnHobghTSW7URMh5JN5eZgGZ1w1YXl8AjktYUMIuDvEgCbRBWcFvfmk 0IByqySd67Ky97vYXjeCnQL5mgBwYo6aKzFN7QmZ5aHMGZ6AdspxurgVt+kzf4fz 5lvpapJNbk7lbPeWTK66P7UyBJJmRF+mSGy4zhtshTg3cFHYh2Mf0BZtzjnCEePw eIGV2ZIxnul9Y/lJzUQVQC08PkN8ojSSITPC+8d7mVxj8WOETGsGkuW690Y2Ehkt n6DTyTq83VG7/w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddrudefjedgieekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfrhgggfestdhqredtredttdenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepieejue dvueduuefhgefhheeiuedvtedvuefgieegveetueeiueehtdegudehfeelnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepiihirdihrghnsehsvg hnthdrtghomh X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Jan 2022 16:48:24 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Marek Szyprowski , Robin Murphy , linuxppc-dev@lists.ozlabs.org, virtualization@lists.linux-foundation.org, iommu@lists.linux-foundation.org, Vlastimil Babka , Mel Gorman , Eric Ren , Zi Yan Subject: [RFC PATCH v3 2/8] mm: compaction: handle non-lru compound pages properly in isolate_migratepages_block(). Date: Wed, 5 Jan 2022 16:47:50 -0500 Message-Id: <20220105214756.91065-3-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220105214756.91065-1-zi.yan@sent.com> References: <20220105214756.91065-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Zi Yan In isolate_migratepages_block(), a !PageLRU tail page can be encountered when the page is larger than a pageblock. Use compound head page for the checks inside and skip the entire compound page when isolation succeeds. Signed-off-by: Zi Yan --- mm/compaction.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index b4e94cda3019..ad9053fbbe06 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -979,19 +979,23 @@ isolate_migratepages_block(struct compact_control *cc= , unsigned long low_pfn, * Skip any other type of page */ if (!PageLRU(page)) { + struct page *head =3D compound_head(page); /* * __PageMovable can return false positive so we need * to verify it under page_lock. */ - if (unlikely(__PageMovable(page)) && - !PageIsolated(page)) { + if (unlikely(__PageMovable(head)) && + !PageIsolated(head)) { if (locked) { unlock_page_lruvec_irqrestore(locked, flags); locked =3D NULL; } =20 - if (!isolate_movable_page(page, isolate_mode)) + if (!isolate_movable_page(head, isolate_mode)) { + low_pfn +=3D (1 << compound_order(head)) - 1 - (page - head); + page =3D head; goto isolate_success; + } } =20 goto isolate_fail; --=20 2.34.1 From nobody Wed Jul 1 00:47:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A23D3C433EF for ; Wed, 5 Jan 2022 21:48:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244602AbiAEVsq (ORCPT ); Wed, 5 Jan 2022 16:48:46 -0500 Received: from new4-smtp.messagingengine.com ([66.111.4.230]:48101 "EHLO new4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234142AbiAEVs0 (ORCPT ); Wed, 5 Jan 2022 16:48:26 -0500 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id 82E62580516; Wed, 5 Jan 2022 16:48:25 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Wed, 05 Jan 2022 16:48:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=from :to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; s=fm2; bh=pLhfLGipzI8++ qIYMIVnkD5rDV8QoVpF5W3LR+3/IwA=; b=Gnw5kDEvRJbOmlvoEvQ4vpKG7S8SK j1uu9JA2H2vtlHmY099jZo9uhUqNplL221UZxxWi+kHEMyPceoZkdK2p7+Tx3hze GMvmLX15SjHzUzKPoJC+pamMGxn73Tui3GMN+JAO5QTamS5hCjQ8dBhpcXXHjQG9 Myz+ZVi8CB6lYahF5KghMzqj1aDHAj05t5Ub8VuHPGI+Hm2n6zveQGMe7MG/sHOJ vj9NG7hfG8ue3GU6uoDuNLBvY5MQXtPIOUjzLxW8yYfZ3DCBhZnMMntbBcuer2Ko BkaWE4NqlvBkEKtNV5dymaUNWqZBZSy1mb2x0cZyBeSYF2+iy7PjoxtKw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:reply-to:subject :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=pLhfLGipzI8++qIYMIVnkD5rDV8QoVpF5W3LR+3/IwA=; b=AEIybQwU Bc1S4r9mo8e/ZMtEZcXVTF6muGnmR9mczNKkfx5ZlTQPpJs4piw9GhQkuFHUFjGM yNQ1nU/dsySLhoLyiFzdNy8bQaqc2HT0vSOQNqYd4+N0o5UWBInRVpGOw/oiNKS0 lmFkv5kU3BZTCm/osmv0mkgFBzh0m3pmkbMLMUcHuOh7dxWwoNKMazib8+2Mq1K2 UM+mpLiLDibOWX5C7oMRIZSbTojBAtkWPJ/at9ESR8huXzTBFzHMWcH/98q5WvAP X6Iahi7RbyxXhocqm5DWjf04XK0Xr/kX2mzTsbKmbd41fBtSDUgwzaeQGxUAhTsA NJ0myAio5iyJ+A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddrudefjedgieekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfrhgggfestdhqredtredttdenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepieejue dvueduuefhgefhheeiuedvtedvuefgieegveetueeiueehtdegudehfeelnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepiihirdihrghnsehsvg hnthdrtghomh X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Jan 2022 16:48:24 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Marek Szyprowski , Robin Murphy , linuxppc-dev@lists.ozlabs.org, virtualization@lists.linux-foundation.org, iommu@lists.linux-foundation.org, Vlastimil Babka , Mel Gorman , Eric Ren , Zi Yan Subject: [RFC PATCH v3 3/8] mm: migrate: allocate the right size of non hugetlb or THP compound pages. Date: Wed, 5 Jan 2022 16:47:51 -0500 Message-Id: <20220105214756.91065-4-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220105214756.91065-1-zi.yan@sent.com> References: <20220105214756.91065-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Zi Yan alloc_migration_target() is used by alloc_contig_range() and non-LRU movable compound pages can be migrated. Current code does not allocate the right page size for such pages. Check THP precisely using is_transparent_huge() and add allocation support for non-LRU compound pages. Signed-off-by: Zi Yan --- mm/migrate.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index c7da064b4781..b1851ffb8576 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1546,9 +1546,7 @@ struct page *alloc_migration_target(struct page *page= , unsigned long private) =20 gfp_mask =3D htlb_modify_alloc_mask(h, gfp_mask); return alloc_huge_page_nodemask(h, nid, mtc->nmask, gfp_mask); - } - - if (PageTransHuge(page)) { + } else if (is_transparent_hugepage(page)) { /* * clear __GFP_RECLAIM to make the migration callback * consistent with regular THP allocations. @@ -1556,14 +1554,19 @@ struct page *alloc_migration_target(struct page *pa= ge, unsigned long private) gfp_mask &=3D ~__GFP_RECLAIM; gfp_mask |=3D GFP_TRANSHUGE; order =3D HPAGE_PMD_ORDER; + } else if (PageCompound(page)) { + /* for non-LRU movable compound pages */ + gfp_mask |=3D __GFP_COMP; + order =3D compound_order(page); } + zidx =3D zone_idx(page_zone(page)); if (is_highmem_idx(zidx) || zidx =3D=3D ZONE_MOVABLE) gfp_mask |=3D __GFP_HIGHMEM; =20 new_page =3D __alloc_pages(gfp_mask, order, nid, mtc->nmask); =20 - if (new_page && PageTransHuge(new_page)) + if (new_page && is_transparent_hugepage(page)) prep_transhuge_page(new_page); =20 return new_page; --=20 2.34.1 From nobody Wed Jul 1 00:47:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B21B9C433EF for ; Wed, 5 Jan 2022 21:48:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244623AbiAEVsx (ORCPT ); Wed, 5 Jan 2022 16:48:53 -0500 Received: from new4-smtp.messagingengine.com ([66.111.4.230]:39141 "EHLO new4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234261AbiAEVs0 (ORCPT ); Wed, 5 Jan 2022 16:48:26 -0500 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailnew.nyi.internal (Postfix) with ESMTP id 45284580547; Wed, 5 Jan 2022 16:48:26 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Wed, 05 Jan 2022 16:48:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=from :to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; s=fm2; bh=kmVQ9SzMySc6E PFCmvu67pL9VTKp1qAMRW9N2THgimw=; b=uGku6gIPQ976fWYy0UtYKjRbH41+H 0L+xBjqmOvSSsDwqjpqbHm4wqA5t+434o953ZVROcfxTrqJ+BVsZaaEk4AfvaueA b6zDUM97GTtVl+ASBvlckfeUGKOQsQ396od7R+1jLZfuWlNNs5CEGj2cTaVfYxez dTaFfU/eWpO8IVroIVCpbsXl5jaMsO/x6DC03qX10eVNfpu3abYOR82Ys+69DOJ2 RNdPOsLRLGiaVQ0saKCcPePPFaoWzUJnjZbrP9EfSWe3CiZHBxE6pNzltRs+sMp3 fcXcnZH/uE3mW0efpROuG/hw/dnBn6CNKZdQSAb0E1cRT2FT3zgc2zShA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:reply-to:subject :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=kmVQ9SzMySc6EPFCmvu67pL9VTKp1qAMRW9N2THgimw=; b=GmjX6tbX x4XxUA5uUpw9kU9UY7Z5FqgNWpAIXI2Pz52G1hVKmlDQKJpwTEL1tih1fnaWoDPx dYBdNdFy/kmQPlf90kmYhvXKAqRTPcyQEbPAzRl/GTJ3jKZc0HE2gVec8Qh7gCAH J12e8d7iHK7JsiaDywFNb40fug0PCNq3qqSmI25g4zREk4tFU7CUNBz0x07XQepk 3oeDeSe7V5O7B3v1LUkoiD5yPkTHoVHYc6oapKIvtcxtQ/YWiTtZtHP4MQzY8Tnm q+spLxLpKs4ollVYxR4oafUeKTVEK0b6R3Iv+Rt5PB3L+zsRuYZ0kezCyhR+Fx2+ uPuYVPLN8id7OQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddrudefjedgieekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfrhgggfestdhqredtredttdenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepieejue dvueduuefhgefhheeiuedvtedvuefgieegveetueeiueehtdegudehfeelnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepiihirdihrghnsehsvg hnthdrtghomh X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Jan 2022 16:48:25 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Marek Szyprowski , Robin Murphy , linuxppc-dev@lists.ozlabs.org, virtualization@lists.linux-foundation.org, iommu@lists.linux-foundation.org, Vlastimil Babka , Mel Gorman , Eric Ren , Zi Yan Subject: [RFC PATCH v3 4/8] mm: make alloc_contig_range work at pageblock granularity Date: Wed, 5 Jan 2022 16:47:52 -0500 Message-Id: <20220105214756.91065-5-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220105214756.91065-1-zi.yan@sent.com> References: <20220105214756.91065-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Zi Yan alloc_contig_range() worked at MAX_ORDER-1 granularity to avoid merging pageblocks with different migratetypes. It might unnecessarily convert extra pageblocks at the beginning and at the end of the range. Change alloc_contig_range() to work at pageblock granularity. It is done by restoring pageblock types and split >pageblock_order free pages after isolating at MAX_ORDER-1 granularity and migrating pages away at pageblock granularity. The reason for this process is that during isolation, some pages, either free or in-use, might have >pageblock sizes and isolating part of them can cause free accounting issues. Restoring the migratetypes of the pageblocks not in the interesting range later is much easier. Signed-off-by: Zi Yan --- mm/page_alloc.c | 174 ++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 154 insertions(+), 20 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 5193c953dbf8..e1c09ae54e31 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -8986,8 +8986,8 @@ struct page *has_unmovable_pages(struct zone *zone, s= truct page *page, #ifdef CONFIG_CONTIG_ALLOC static unsigned long pfn_max_align_down(unsigned long pfn) { - return pfn & ~(max_t(unsigned long, MAX_ORDER_NR_PAGES, - pageblock_nr_pages) - 1); + return ALIGN_DOWN(pfn, max_t(unsigned long, MAX_ORDER_NR_PAGES, + pageblock_nr_pages)); } =20 static unsigned long pfn_max_align_up(unsigned long pfn) @@ -9076,6 +9076,52 @@ static int __alloc_contig_migrate_range(struct compa= ct_control *cc, return 0; } =20 +static inline int save_migratetypes(unsigned char *migratetypes, + unsigned long start_pfn, unsigned long end_pfn) +{ + unsigned long pfn =3D start_pfn; + int num =3D 0; + + while (pfn < end_pfn) { + migratetypes[num] =3D get_pageblock_migratetype(pfn_to_page(pfn)); + num++; + pfn +=3D pageblock_nr_pages; + } + return num; +} + +static inline int restore_migratetypes(unsigned char *migratetypes, + unsigned long start_pfn, unsigned long end_pfn) +{ + unsigned long pfn =3D start_pfn; + int num =3D 0; + + while (pfn < end_pfn) { + set_pageblock_migratetype(pfn_to_page(pfn), migratetypes[num]); + num++; + pfn +=3D pageblock_nr_pages; + } + return num; +} + +static inline void split_free_page_into_pageblocks(struct page *free_page, + int order, struct zone *zone) +{ + unsigned long pfn; + + spin_lock(&zone->lock); + del_page_from_free_list(free_page, zone, order); + for (pfn =3D page_to_pfn(free_page); + pfn < page_to_pfn(free_page) + (1UL << order); + pfn +=3D pageblock_nr_pages) { + int mt =3D get_pfnblock_migratetype(pfn_to_page(pfn), pfn); + + __free_one_page(pfn_to_page(pfn), pfn, zone, pageblock_order, + mt, FPI_NONE); + } + spin_unlock(&zone->lock); +} + /** * alloc_contig_range() -- tries to allocate given range of pages * @start: start PFN to allocate @@ -9101,8 +9147,15 @@ int alloc_contig_range(unsigned long start, unsigned= long end, unsigned migratetype, gfp_t gfp_mask) { unsigned long outer_start, outer_end; + unsigned long isolate_start =3D pfn_max_align_down(start); + unsigned long isolate_end =3D pfn_max_align_up(end); + unsigned long alloc_start =3D ALIGN_DOWN(start, pageblock_nr_pages); + unsigned long alloc_end =3D ALIGN(end, pageblock_nr_pages); + unsigned long num_pageblock_to_save; unsigned int order; int ret =3D 0; + unsigned char *saved_mt; + int num; =20 struct compact_control cc =3D { .nr_migratepages =3D 0, @@ -9116,11 +9169,30 @@ int alloc_contig_range(unsigned long start, unsigne= d long end, }; INIT_LIST_HEAD(&cc.migratepages); =20 + /* + * TODO: make MIGRATE_ISOLATE a standalone bit to avoid overwriting + * the exiting migratetype. Then, we will not need the save and restore + * process here. + */ + + /* Save the migratepages of the pageblocks before start and after end */ + num_pageblock_to_save =3D (alloc_start - isolate_start) / pageblock_nr_pa= ges + + (isolate_end - alloc_end) / pageblock_nr_pages; + saved_mt =3D + kmalloc_array(num_pageblock_to_save, + sizeof(unsigned char), GFP_KERNEL); + if (!saved_mt) + return -ENOMEM; + + num =3D save_migratetypes(saved_mt, isolate_start, alloc_start); + + num =3D save_migratetypes(&saved_mt[num], alloc_end, isolate_end); + /* * What we do here is we mark all pageblocks in range as * MIGRATE_ISOLATE. Because pageblock and max order pages may * have different sizes, and due to the way page allocator - * work, we align the range to biggest of the two pages so + * work, we align the isolation range to biggest of the two so * that page allocator won't try to merge buddies from * different pageblocks and change MIGRATE_ISOLATE to some * other migration type. @@ -9130,6 +9202,20 @@ int alloc_contig_range(unsigned long start, unsigned= long end, * we are interested in). This will put all the pages in * range back to page allocator as MIGRATE_ISOLATE. * + * Afterwards, we restore the migratetypes of the pageblocks not + * in range, split free pages spanning outside the range, + * and put split free pages (at pageblock_order) to the right + * migratetype list. + * + * NOTE: the above approach is used because it can cause free + * page accounting issues during isolation, if a page, either + * free or in-use, contains multiple pageblocks and we only + * isolate a subset of them. For example, if only the second + * pageblock is isolated from a page with 2 pageblocks, after + * the page is free, it will be put in the first pageblock + * migratetype list instead of having 2 pageblocks in two + * separate migratetype lists. + * * When this is done, we take the pages in range from page * allocator removing them from the buddy system. This way * page allocator will never consider using them. @@ -9140,10 +9226,9 @@ int alloc_contig_range(unsigned long start, unsigned= long end, * put back to page allocator so that buddy can use them. */ =20 - ret =3D start_isolate_page_range(pfn_max_align_down(start), - pfn_max_align_up(end), migratetype, 0); + ret =3D start_isolate_page_range(isolate_start, isolate_end, migratetype,= 0); if (ret) - return ret; + goto done; =20 drain_all_pages(cc.zone); =20 @@ -9179,6 +9264,19 @@ int alloc_contig_range(unsigned long start, unsigned= long end, * isolated thus they won't get removed from buddy. */ =20 + /* + * Restore migratetypes of pageblocks outside [start, end) + * TODO: remove it when MIGRATE_ISOLATE becomes a standalone bit + */ + + num =3D restore_migratetypes(saved_mt, isolate_start, alloc_start); + + num =3D restore_migratetypes(&saved_mt[num], alloc_end, isolate_end); + + /* + * Split free page spanning [isolate_start, alloc_start) and put the + * pageblocks in the right migratetype lists. + */ order =3D 0; outer_start =3D start; while (!PageBuddy(pfn_to_page(outer_start))) { @@ -9193,37 +9291,73 @@ int alloc_contig_range(unsigned long start, unsigne= d long end, order =3D buddy_order(pfn_to_page(outer_start)); =20 /* - * outer_start page could be small order buddy page and - * it doesn't include start page. Adjust outer_start - * in this case to report failed page properly - * on tracepoint in test_pages_isolated() + * split the free page has start page and put the pageblocks + * in the right migratetype list */ - if (outer_start + (1UL << order) <=3D start) - outer_start =3D start; + if (outer_start + (1UL << order) > start) { + struct page *free_page =3D pfn_to_page(outer_start); + + split_free_page_into_pageblocks(free_page, order, cc.zone); + } + } + + /* + * Split free page spanning [alloc_end, isolate_end) and put the + * pageblocks in the right migratetype list + */ + for (outer_end =3D alloc_end; outer_end < isolate_end;) { + unsigned long begin_pfn =3D outer_end; + + order =3D 0; + while (!PageBuddy(pfn_to_page(outer_end))) { + if (++order >=3D MAX_ORDER) { + outer_end =3D begin_pfn; + break; + } + outer_end &=3D ~0UL << order; + } + + if (outer_end !=3D begin_pfn) { + order =3D buddy_order(pfn_to_page(outer_end)); + + /* + * split the free page has start page and put the pageblocks + * in the right migratetype list + */ + VM_BUG_ON(outer_end + (1UL << order) <=3D begin_pfn); + { + struct page *free_page =3D pfn_to_page(outer_end); + + split_free_page_into_pageblocks(free_page, order, cc.zone); + } + outer_end +=3D 1UL << order; + } else + outer_end =3D begin_pfn + 1; } =20 /* Make sure the range is really isolated. */ - if (test_pages_isolated(outer_start, end, 0)) { + if (test_pages_isolated(alloc_start, alloc_end, 0)) { ret =3D -EBUSY; goto done; } =20 /* Grab isolated pages from freelists. */ - outer_end =3D isolate_freepages_range(&cc, outer_start, end); + outer_end =3D isolate_freepages_range(&cc, alloc_start, alloc_end); if (!outer_end) { ret =3D -EBUSY; goto done; } =20 /* Free head and tail (if any) */ - if (start !=3D outer_start) - free_contig_range(outer_start, start - outer_start); - if (end !=3D outer_end) - free_contig_range(end, outer_end - end); + if (start !=3D alloc_start) + free_contig_range(alloc_start, start - alloc_start); + if (end !=3D alloc_end) + free_contig_range(end, alloc_end - end); =20 done: - undo_isolate_page_range(pfn_max_align_down(start), - pfn_max_align_up(end), migratetype); + kfree(saved_mt); + undo_isolate_page_range(alloc_start, + alloc_end, migratetype); return ret; } EXPORT_SYMBOL(alloc_contig_range); --=20 2.34.1 From nobody Wed Jul 1 00:47:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D17AC433EF for ; Wed, 5 Jan 2022 21:48:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235306AbiAEVs4 (ORCPT ); Wed, 5 Jan 2022 16:48:56 -0500 Received: from new4-smtp.messagingengine.com ([66.111.4.230]:58991 "EHLO new4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234612AbiAEVs1 (ORCPT ); Wed, 5 Jan 2022 16:48:27 -0500 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailnew.nyi.internal (Postfix) with ESMTP id ECF0E580561; Wed, 5 Jan 2022 16:48:26 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Wed, 05 Jan 2022 16:48:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=from :to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; s=fm2; bh=i5PR4FWm5EakP MkM2WD3s3D7Gp7BwGYZgyto5AB1H+A=; b=jGom0jRZSFHXrg1CS6RUip8HsmYj0 3PheJTpvIcw5TEN3ohqR/Ma28+eS3iu6zlhcTP7cpjlNbSRIE7YjpML56aHPrvIA +4GCENqlIs1mrOA8cTTe6ODjaj/IIYn3v8Qtnh4HttRhILcr/pd8ruEjGU4hNGLV 1/U3mzoPK1PlgfFENaYnLxCYwzg0lcObzxYKbQIClfbgKOIq4/g7ESkYp4G16GaS SkUw4/Cd2XElHlNC+nSdsqJ6mfysMNjUhgm69pVRho2izlvpuUSfRQA2u3WNXQdi /OMBHAnq7TMG7/Anti0xBbKSe8Qv5J0Q0JemZrb5BY048UH5QAAWRht0g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:reply-to:subject :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=i5PR4FWm5EakPMkM2WD3s3D7Gp7BwGYZgyto5AB1H+A=; b=SO02xALW XLfRTAl0GBWZ21+X0VJGIktyQxLwkvOh1ddauSqlMPL0vAt1wDSerfK+IGkyIKzq quhPthzVNMPD+LyBukxDsTP89RoxSq3jwYOKrLqBdShkg5IdkzsDrloNQLCcPSHA 35JMuHFblanwxMyV6jrcK1PHNQI5NU9br75DPp971SoUx/EyUuh14KUtFmg0PYj6 szv/SLRDYtYTeDwj7DdjY5ErT9qlO8lrryqEQCOQYj8oMiUBsKPEqpAOahGpIoHm zSDFrgc/nfTo5K45SbfshUEKCsL4hLo/lSqJ3n8MsUibeourM4SwPREC65DN/96f 2WqsXmvVvF9HUg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddrudefjedgieekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfrhgggfestdhqredtredttdenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepieejue dvueduuefhgefhheeiuedvtedvuefgieegveetueeiueehtdegudehfeelnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepiihirdihrghnsehsvg hnthdrtghomh X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Jan 2022 16:48:26 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Marek Szyprowski , Robin Murphy , linuxppc-dev@lists.ozlabs.org, virtualization@lists.linux-foundation.org, iommu@lists.linux-foundation.org, Vlastimil Babka , Mel Gorman , Eric Ren , Zi Yan Subject: [RFC PATCH v3 5/8] mm: page_isolation: check specified range for unmovable pages during isolation. Date: Wed, 5 Jan 2022 16:47:53 -0500 Message-Id: <20220105214756.91065-6-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220105214756.91065-1-zi.yan@sent.com> References: <20220105214756.91065-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Zi Yan Enable set_migratetype_isolate() to check specified sub-range for unmovable pages during isolation. Page isolation is done at max(MAX_ORDER_NR_PAEGS, pageblock_nr_pages) granularity, but not all pages within that granularity are intended to be isolated. For example, alloc_contig_range(), which uses page isolation, allows ranges without alignment. This commit makes unmovable page check only look for interesting pages, so that page isolation can succeed for any non-overlapping ranges. has_unmovable_pages() is moved to mm/page_isolation.c since it is only used by page isolation. Signed-off-by: Zi Yan --- include/linux/page-isolation.h | 3 +- mm/memory_hotplug.c | 12 ++- mm/page_alloc.c | 122 +-------------------------- mm/page_isolation.c | 148 +++++++++++++++++++++++++++++++-- 4 files changed, 153 insertions(+), 132 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 572458016331..a4d2687ed4e6 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -33,8 +33,6 @@ static inline bool is_migrate_isolate(int migratetype) #define MEMORY_OFFLINE 0x1 #define REPORT_FAILURE 0x2 =20 -struct page *has_unmovable_pages(struct zone *zone, struct page *page, - int migratetype, int flags); void set_pageblock_migratetype(struct page *page, int migratetype); int move_freepages_block(struct zone *zone, struct page *page, int migratetype, int *num_movable); @@ -44,6 +42,7 @@ int move_freepages_block(struct zone *zone, struct page *= page, */ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pfn, + unsigned long isolate_start, unsigned long isolate_end, unsigned migratetype, int flags); =20 /* diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 0139b77c51d5..5db84c3fa882 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1901,8 +1901,18 @@ int __ref offline_pages(unsigned long start_pfn, uns= igned long nr_pages, zone_pcp_disable(zone); lru_cache_disable(); =20 - /* set above range as isolated */ + /* + * set above range as isolated + * + * start_pfn and end_pfn are the same as isolate_start and isolate_end, + * because start_pfn and end_pfn are already PAGES_PER_SECTION + * (>=3D MAX_ORDER_NR_PAGES) aligned; if start_pfn is + * pageblock_nr_pages aligned in memmap_on_memory case, there is no + * need to isolate pages before start_pfn, since they are used by + * memmap thus not user visible. + */ ret =3D start_isolate_page_range(start_pfn, end_pfn, + start_pfn, end_pfn, MIGRATE_MOVABLE, MEMORY_OFFLINE | REPORT_FAILURE); if (ret) { diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e1c09ae54e31..faee7637740a 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -8864,125 +8864,6 @@ void *__init alloc_large_system_hash(const char *ta= blename, return table; } =20 -/* - * This function checks whether pageblock includes unmovable pages or not. - * - * PageLRU check without isolation or lru_lock could race so that - * MIGRATE_MOVABLE block might include unmovable pages. And __PageMovable - * check without lock_page also may miss some movable non-lru pages at - * race condition. So you can't expect this function should be exact. - * - * Returns a page without holding a reference. If the caller wants to - * dereference that page (e.g., dumping), it has to make sure that it - * cannot get removed (e.g., via memory unplug) concurrently. - * - */ -struct page *has_unmovable_pages(struct zone *zone, struct page *page, - int migratetype, int flags) -{ - unsigned long iter =3D 0; - unsigned long pfn =3D page_to_pfn(page); - unsigned long offset =3D pfn % pageblock_nr_pages; - - if (is_migrate_cma_page(page)) { - /* - * CMA allocations (alloc_contig_range) really need to mark - * isolate CMA pageblocks even when they are not movable in fact - * so consider them movable here. - */ - if (is_migrate_cma(migratetype)) - return NULL; - - return page; - } - - for (; iter < pageblock_nr_pages - offset; iter++) { - page =3D pfn_to_page(pfn + iter); - - /* - * Both, bootmem allocations and memory holes are marked - * PG_reserved and are unmovable. We can even have unmovable - * allocations inside ZONE_MOVABLE, for example when - * specifying "movablecore". - */ - if (PageReserved(page)) - return page; - - /* - * If the zone is movable and we have ruled out all reserved - * pages then it should be reasonably safe to assume the rest - * is movable. - */ - if (zone_idx(zone) =3D=3D ZONE_MOVABLE) - continue; - - /* - * Hugepages are not in LRU lists, but they're movable. - * THPs are on the LRU, but need to be counted as #small pages. - * We need not scan over tail pages because we don't - * handle each tail page individually in migration. - */ - if (PageHuge(page) || PageTransCompound(page)) { - struct page *head =3D compound_head(page); - unsigned int skip_pages; - - if (PageHuge(page)) { - if (!hugepage_migration_supported(page_hstate(head))) - return page; - } else if (!PageLRU(head) && !__PageMovable(head)) { - return page; - } - - skip_pages =3D compound_nr(head) - (page - head); - iter +=3D skip_pages - 1; - continue; - } - - /* - * We can't use page_count without pin a page - * because another CPU can free compound page. - * This check already skips compound tails of THP - * because their page->_refcount is zero at all time. - */ - if (!page_ref_count(page)) { - if (PageBuddy(page)) - iter +=3D (1 << buddy_order(page)) - 1; - continue; - } - - /* - * The HWPoisoned page may be not in buddy system, and - * page_count() is not 0. - */ - if ((flags & MEMORY_OFFLINE) && PageHWPoison(page)) - continue; - - /* - * We treat all PageOffline() pages as movable when offlining - * to give drivers a chance to decrement their reference count - * in MEM_GOING_OFFLINE in order to indicate that these pages - * can be offlined as there are no direct references anymore. - * For actually unmovable PageOffline() where the driver does - * not support this, we will fail later when trying to actually - * move these pages that still have a reference count > 0. - * (false negatives in this function only) - */ - if ((flags & MEMORY_OFFLINE) && PageOffline(page)) - continue; - - if (__PageMovable(page) || PageLRU(page)) - continue; - - /* - * If there are RECLAIMABLE pages, we need to check - * it. But now, memory offline itself doesn't call - * shrink_node_slabs() and it still to be fixed. - */ - return page; - } - return NULL; -} - #ifdef CONFIG_CONTIG_ALLOC static unsigned long pfn_max_align_down(unsigned long pfn) { @@ -9226,7 +9107,8 @@ int alloc_contig_range(unsigned long start, unsigned = long end, * put back to page allocator so that buddy can use them. */ =20 - ret =3D start_isolate_page_range(isolate_start, isolate_end, migratetype,= 0); + ret =3D start_isolate_page_range(start, end, isolate_start, isolate_end, + migratetype, 0); if (ret) goto done; =20 diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 6a0ddda6b3c5..7a7991460eb9 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -15,12 +15,143 @@ #define CREATE_TRACE_POINTS #include =20 -static int set_migratetype_isolate(struct page *page, int migratetype, int= isol_flags) +/* + * This function checks whether pageblock within [start_pfn, end_pfn) incl= udes + * unmovable pages or not. + * + * PageLRU check without isolation or lru_lock could race so that + * MIGRATE_MOVABLE block might include unmovable pages. And __PageMovable + * check without lock_page also may miss some movable non-lru pages at + * race condition. So you can't expect this function should be exact. + * + * Returns a page without holding a reference. If the caller wants to + * dereference that page (e.g., dumping), it has to make sure that it + * cannot get removed (e.g., via memory unplug) concurrently. + * + */ +static struct page *has_unmovable_pages(struct zone *zone, struct page *pa= ge, + int migratetype, int flags, + unsigned long start_pfn, unsigned long end_pfn) +{ + unsigned long first_pfn =3D max(page_to_pfn(page), start_pfn); + unsigned long pfn =3D first_pfn; + unsigned long last_pfn =3D min(ALIGN(pfn + 1, pageblock_nr_pages), end_pf= n); + + page =3D pfn_to_page(pfn); + + if (is_migrate_cma_page(page)) { + /* + * CMA allocations (alloc_contig_range) really need to mark + * isolate CMA pageblocks even when they are not movable in fact + * so consider them movable here. + */ + if (is_migrate_cma(migratetype)) + return NULL; + + return page; + } + + for (pfn =3D first_pfn; pfn < last_pfn; pfn++) { + page =3D pfn_to_page(pfn); + + /* + * Both, bootmem allocations and memory holes are marked + * PG_reserved and are unmovable. We can even have unmovable + * allocations inside ZONE_MOVABLE, for example when + * specifying "movablecore". + */ + if (PageReserved(page)) + return page; + + /* + * If the zone is movable and we have ruled out all reserved + * pages then it should be reasonably safe to assume the rest + * is movable. + */ + if (zone_idx(zone) =3D=3D ZONE_MOVABLE) + continue; + + /* + * Hugepages are not in LRU lists, but they're movable. + * THPs are on the LRU, but need to be counted as #small pages. + * We need not scan over tail pages because we don't + * handle each tail page individually in migration. + */ + if (PageHuge(page) || PageTransCompound(page)) { + struct page *head =3D compound_head(page); + unsigned int skip_pages; + + if (PageHuge(page)) { + if (!hugepage_migration_supported(page_hstate(head))) + return page; + } else if (!PageLRU(head) && !__PageMovable(head)) { + return page; + } + + skip_pages =3D compound_nr(head) - (page - head); + pfn +=3D skip_pages - 1; + continue; + } + + /* + * We can't use page_count without pin a page + * because another CPU can free compound page. + * This check already skips compound tails of THP + * because their page->_refcount is zero at all time. + */ + if (!page_ref_count(page)) { + if (PageBuddy(page)) + pfn +=3D (1 << buddy_order(page)) - 1; + continue; + } + + /* + * The HWPoisoned page may be not in buddy system, and + * page_count() is not 0. + */ + if ((flags & MEMORY_OFFLINE) && PageHWPoison(page)) + continue; + + /* + * We treat all PageOffline() pages as movable when offlining + * to give drivers a chance to decrement their reference count + * in MEM_GOING_OFFLINE in order to indicate that these pages + * can be offlined as there are no direct references anymore. + * For actually unmovable PageOffline() where the driver does + * not support this, we will fail later when trying to actually + * move these pages that still have a reference count > 0. + * (false negatives in this function only) + */ + if ((flags & MEMORY_OFFLINE) && PageOffline(page)) + continue; + + if (__PageMovable(page) || PageLRU(page)) + continue; + + /* + * If there are RECLAIMABLE pages, we need to check + * it. But now, memory offline itself doesn't call + * shrink_node_slabs() and it still to be fixed. + */ + return page; + } + return NULL; +} + +/* + * This function set pageblock migratetype to isolate if no unmovable page= is + * present in [start_pfn, end_pfn). The pageblock must be within + * [start_pfn, end_pfn). + */ +static int set_migratetype_isolate(struct page *page, int migratetype, int= isol_flags, + unsigned long start_pfn, unsigned long end_pfn) { struct zone *zone =3D page_zone(page); struct page *unmovable; unsigned long flags; =20 + VM_BUG_ON(page_to_pfn(page) < start_pfn || page_to_pfn(page) >=3D end_pfn= ); + spin_lock_irqsave(&zone->lock, flags); =20 /* @@ -37,7 +168,7 @@ static int set_migratetype_isolate(struct page *page, in= t migratetype, int isol_ * FIXME: Now, memory hotplug doesn't call shrink_slab() by itself. * We just check MOVABLE pages. */ - unmovable =3D has_unmovable_pages(zone, page, migratetype, isol_flags); + unmovable =3D has_unmovable_pages(zone, page, migratetype, isol_flags, st= art_pfn, end_pfn); if (!unmovable) { unsigned long nr_pages; int mt =3D get_pageblock_migratetype(page); @@ -185,20 +316,19 @@ __first_valid_page(unsigned long pfn, unsigned long n= r_pages) * Return: 0 on success and -EBUSY if any part of range cannot be isolated. */ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, + unsigned long isolate_start, unsigned long isolate_end, unsigned migratetype, int flags) { unsigned long pfn; struct page *page; =20 - BUG_ON(!IS_ALIGNED(start_pfn, pageblock_nr_pages)); - BUG_ON(!IS_ALIGNED(end_pfn, pageblock_nr_pages)); - - for (pfn =3D start_pfn; - pfn < end_pfn; + for (pfn =3D isolate_start; + pfn < isolate_end; pfn +=3D pageblock_nr_pages) { page =3D __first_valid_page(pfn, pageblock_nr_pages); - if (page && set_migratetype_isolate(page, migratetype, flags)) { - undo_isolate_page_range(start_pfn, pfn, migratetype); + if (page && set_migratetype_isolate(page, migratetype, flags, + start_pfn, end_pfn)) { + undo_isolate_page_range(isolate_start, pfn, migratetype); return -EBUSY; } } --=20 2.34.1 From nobody Wed Jul 1 00:47:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC811C433EF for ; Wed, 5 Jan 2022 21:49:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244637AbiAEVtC (ORCPT ); Wed, 5 Jan 2022 16:49:02 -0500 Received: from new4-smtp.messagingengine.com ([66.111.4.230]:60953 "EHLO new4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234070AbiAEVs2 (ORCPT ); Wed, 5 Jan 2022 16:48:28 -0500 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailnew.nyi.internal (Postfix) with ESMTP id B93E1580565; Wed, 5 Jan 2022 16:48:27 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Wed, 05 Jan 2022 16:48:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=from :to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; s=fm2; bh=cdqW9WkR9DiZi 1wR67hfIXu60iQ9l4qjc+vr1lXC0xM=; b=YzFUupbOB0mt3rO5G0b92O16rBLLK hRO1PcYzYgQFEzEHWizWsOe1vicVABwqhoHyAFiPb2rBhKP9ENunSSLLxXR2LOqL w4G/dvr0VvF9Wi8ocsZ3WCwk1hWCjoRALIrsXYOT2NcdAyDPXaaPaL2Il0R7CYMT iAUgh9u6HRpE7fX2sShq3A81ezaOag/v8XsqGvaQqNx8WqPjgOXabPv5vHAtmbLc ZByizcysMtdU3kPJY5iTYZOLgWRB7E1jaRcM2txtfyKO+bK2SMYzfm4gNwFj7nmB KJ5bZ1GGgH0RuE5fRBa5K0YVVk1Rt/NA4m50Ur6E40t2BqJ0/PmHI5iyQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:reply-to:subject :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=cdqW9WkR9DiZi1wR67hfIXu60iQ9l4qjc+vr1lXC0xM=; b=GdqAoGtO xU/Ega15jpW53pP7JWCIiLIS+MSEkLp1y/6DIHhkoX6f5mdHYjhpIY6/B7kFJAYU MJjbSQ6qd6BTwCSgzj68oBNGCMsXnQIemUzlghy28uS0PpaeDwrkLO0xQ7M02ePq s26YNSmGBdNYE79yhlXZh5K8Az9DedUHScDL3d30e9hKGw9MEEGYR7W9xXOtwwQY 0sqA4PJVWq4O5+CaBlP4PYKFbzYfkWd56S1Yh+hKTCTUGxK9YK0L9V65KWZloKPd 2lFAkT+4M7HOS1BRvJ6AZdAnuUnm9snqSPFIU1xAvLjpCQei2ncVNWqvIzUYYvhO bXTjpr0jzSFAaw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddrudefjedgieekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfrhgggfestdhqredtredttdenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepieejue dvueduuefhgefhheeiuedvtedvuefgieegveetueeiueehtdegudehfeelnecuvehluhhs thgvrhfuihiivgepvdenucfrrghrrghmpehmrghilhhfrhhomhepiihirdihrghnsehsvg hnthdrtghomh X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Jan 2022 16:48:26 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Marek Szyprowski , Robin Murphy , linuxppc-dev@lists.ozlabs.org, virtualization@lists.linux-foundation.org, iommu@lists.linux-foundation.org, Vlastimil Babka , Mel Gorman , Eric Ren , Zi Yan Subject: [RFC PATCH v3 6/8] mm: cma: use pageblock_order as the single alignment Date: Wed, 5 Jan 2022 16:47:54 -0500 Message-Id: <20220105214756.91065-7-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220105214756.91065-1-zi.yan@sent.com> References: <20220105214756.91065-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Zi Yan Now alloc_contig_range() works at pageblock granularity. Change CMA allocation, which uses alloc_contig_range(), to use pageblock_order alignment. Signed-off-by: Zi Yan --- include/linux/mmzone.h | 5 +---- kernel/dma/contiguous.c | 2 +- mm/cma.c | 6 ++---- mm/page_alloc.c | 6 +++--- 4 files changed, 7 insertions(+), 12 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 0aa549653e4e..d28a02a893d6 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -54,10 +54,7 @@ enum migratetype { * * The way to use it is to change migratetype of a range of * pageblocks to MIGRATE_CMA which can be done by - * __free_pageblock_cma() function. What is important though - * is that a range of pageblocks must be aligned to - * MAX_ORDER_NR_PAGES should biggest page be bigger than - * a single pageblock. + * __free_pageblock_cma() function. */ MIGRATE_CMA, #endif diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c index 3d63d91cba5c..ac35b14b0786 100644 --- a/kernel/dma/contiguous.c +++ b/kernel/dma/contiguous.c @@ -399,7 +399,7 @@ static const struct reserved_mem_ops rmem_cma_ops =3D { =20 static int __init rmem_cma_setup(struct reserved_mem *rmem) { - phys_addr_t align =3D PAGE_SIZE << max(MAX_ORDER - 1, pageblock_order); + phys_addr_t align =3D PAGE_SIZE << pageblock_order; phys_addr_t mask =3D align - 1; unsigned long node =3D rmem->fdt_node; bool default_cma =3D of_get_flat_dt_prop(node, "linux,cma-default", NULL); diff --git a/mm/cma.c b/mm/cma.c index bc9ca8f3c487..d171158bd418 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -180,8 +180,7 @@ int __init cma_init_reserved_mem(phys_addr_t base, phys= _addr_t size, return -EINVAL; =20 /* ensure minimal alignment required by mm core */ - alignment =3D PAGE_SIZE << - max_t(unsigned long, MAX_ORDER - 1, pageblock_order); + alignment =3D PAGE_SIZE << pageblock_order; =20 /* alignment should be aligned with order_per_bit */ if (!IS_ALIGNED(alignment >> PAGE_SHIFT, 1 << order_per_bit)) @@ -268,8 +267,7 @@ int __init cma_declare_contiguous_nid(phys_addr_t base, * migratetype page by page allocator's buddy algorithm. In the case, * you couldn't get a contiguous memory, which is not what we want. */ - alignment =3D max(alignment, (phys_addr_t)PAGE_SIZE << - max_t(unsigned long, MAX_ORDER - 1, pageblock_order)); + alignment =3D max(alignment, (phys_addr_t)PAGE_SIZE << pageblock_order); if (fixed && base & (alignment - 1)) { ret =3D -EINVAL; pr_err("Region at %pa must be aligned to %pa bytes\n", diff --git a/mm/page_alloc.c b/mm/page_alloc.c index faee7637740a..63d76f436ed1 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -9013,8 +9013,8 @@ static inline void split_free_page_into_pageblocks(st= ruct page *free_page, * be either of the two. * @gfp_mask: GFP mask to use during compaction * - * The PFN range does not have to be pageblock or MAX_ORDER_NR_PAGES - * aligned. The PFN range must belong to a single zone. + * The PFN range does not have to be pageblock aligned. The PFN range must + * belong to a single zone. * * The first thing this routine does is attempt to MIGRATE_ISOLATE all * pageblocks in the range. Once isolated, the pageblocks should not @@ -9130,7 +9130,7 @@ int alloc_contig_range(unsigned long start, unsigned = long end, ret =3D 0; =20 /* - * Pages from [start, end) are within a MAX_ORDER_NR_PAGES + * Pages from [start, end) are within a pageblock_nr_pages * aligned blocks that are marked as MIGRATE_ISOLATE. What's * more, all pages in [start, end) are free in page allocator. * What we are going to do is to allocate all pages from --=20 2.34.1 From nobody Wed Jul 1 00:47:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4BBFC433F5 for ; Wed, 5 Jan 2022 21:49:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244611AbiAEVtb (ORCPT ); Wed, 5 Jan 2022 16:49:31 -0500 Received: from new4-smtp.messagingengine.com ([66.111.4.230]:60899 "EHLO new4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235063AbiAEVs3 (ORCPT ); Wed, 5 Jan 2022 16:48:29 -0500 Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailnew.nyi.internal (Postfix) with ESMTP id 73DD3580569; Wed, 5 Jan 2022 16:48:28 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Wed, 05 Jan 2022 16:48:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=from :to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; s=fm2; bh=kJxpjApBzoVvC kd49IN11sRCxgdbdXjwUm6fGe2ajOg=; b=W9rMnsZRyKPY8hxaUtB1n45xh/WyS fg4Hd6Wv7LLs3JFu45mfiZqlQkvgxYzRQXfumen4MEH/Jwzlli73/5/pvMeraycO 9eKEoyFnpmYyLHznxRj6xK++R6NCUXFAVU8oXlPGXgxRR2Wgg3+oF6b6ZDvMahUV GpxYVYWNEkOoKLXIkG1rE+Bw44KmtUlQZzw9Ofns2eNpZrNWcpNgy6pLxxJrTvgU TVw/gddpMjYKFZxlwg7hzcuzu3rmYBYAnkK+TzJM6TLXsTeaFrdv+ytH17+QQyu5 EiFrxm4fmZSmrLIj9T2ZcfRdSl4RWpD9X5ZT7XnA6zxCyKnNedvOIXE4Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:reply-to:subject :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=kJxpjApBzoVvCkd49IN11sRCxgdbdXjwUm6fGe2ajOg=; b=kO2uSSXr joqeXbL1xK0ixZIGTvPiqETF+2nWYNLoYKVIwY6GFJd5q7eyqoaLTSADlAqszsgE 4xC/fNTrHhSs9Nwjdv6/bJBZ/OOKplWUVcto1VtAd7FNtVL1wsjtsyWNEW6PzXDr lq7NT6H1BtpUP0ZnYTMnoJrAdSVOE/AdkxGW6HtbYC8YoY759LdF8HcZNChLieK9 3eu4QU+b1qTGTIXi7R5S+K8CnkmU5UWP4pugbG6M3uA0XOPIPbcYtzJ19FwsUBsv 5QqK9zyUkYcjCvYWg+6Pz8yZ2OWlB7yPjORFhAaPt5LxBT7V2xhsma2gSya5PG3R xRuXzouDZOqy8g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddrudefjedgieekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfrhgggfestdhqredtredttdenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepieejue dvueduuefhgefhheeiuedvtedvuefgieegveetueeiueehtdegudehfeelnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepiihirdihrghnsehsvg hnthdrtghomh X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Jan 2022 16:48:27 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Marek Szyprowski , Robin Murphy , linuxppc-dev@lists.ozlabs.org, virtualization@lists.linux-foundation.org, iommu@lists.linux-foundation.org, Vlastimil Babka , Mel Gorman , Eric Ren , Zi Yan Subject: [RFC PATCH v3 7/8] drivers: virtio_mem: use pageblock size as the minimum virtio_mem size. Date: Wed, 5 Jan 2022 16:47:55 -0500 Message-Id: <20220105214756.91065-8-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220105214756.91065-1-zi.yan@sent.com> References: <20220105214756.91065-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Zi Yan alloc_contig_range() now only needs to be aligned to pageblock_order, drop virtio_mem size requirement that it needs to be the max of pageblock_order and MAX_ORDER. Signed-off-by: Zi Yan --- drivers/virtio/virtio_mem.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index a6a78685cfbe..2664dc16d0f9 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -2481,8 +2481,7 @@ static int virtio_mem_init_hotplug(struct virtio_mem = *vm) * - Is required for now for alloc_contig_range() to work reliably - * it doesn't properly handle smaller granularity on ZONE_NORMAL. */ - sb_size =3D max_t(uint64_t, MAX_ORDER_NR_PAGES, - pageblock_nr_pages) * PAGE_SIZE; + sb_size =3D pageblock_nr_pages * PAGE_SIZE; sb_size =3D max_t(uint64_t, vm->device_block_size, sb_size); =20 if (sb_size < memory_block_size_bytes() && !force_bbm) { --=20 2.34.1 From nobody Wed Jul 1 00:47:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27B73C433F5 for ; Wed, 5 Jan 2022 21:49:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244671AbiAEVtQ (ORCPT ); Wed, 5 Jan 2022 16:49:16 -0500 Received: from new4-smtp.messagingengine.com ([66.111.4.230]:34605 "EHLO new4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234117AbiAEVs3 (ORCPT ); Wed, 5 Jan 2022 16:48:29 -0500 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.nyi.internal (Postfix) with ESMTP id 4AB4D580571; Wed, 5 Jan 2022 16:48:29 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Wed, 05 Jan 2022 16:48:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=from :to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; s=fm2; bh=8yuexR1+nrtHG SO8wGNcV0MvB+tGU4p0S4nQ2u6/h9Y=; b=kNcgqXMU4/n+0YyMRK0TyVMysDqOh yGnxu3188w1RZak/r6C5GGCMGkYAvYZrGUXVsCm5C/Jb9fk2x8P+Zhkbv2juEiHF Xx9yXwsMcFZDgw14URi8KOX4Gq3BjiqSdCn8GKzv3NQXWRkliOB0JVHVM8uS81kz FXWoyZvcC8pdi2IY2HIDM9KJjv4NSShQ3V/ljJO+UE1AAlyWOJHG3t1nZ3upZM6c K6hHMhVO16bR2AATD8DVfvQ/aYyij27BJTE7NIUx0TvMRg4mNtuYCz5coZkOImzw Rby2nrBx/WVcBJxX1xpgf8EFekNemKmfyEiUBjB3mQtzUXARZkQ3mfEPA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:reply-to:subject :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; bh=8yuexR1+nrtHGSO8wGNcV0MvB+tGU4p0S4nQ2u6/h9Y=; b=PhZUATg9 FmZKLL8OPTeqjkwnPLyfUAOaHv5EhgMImn/cYWCeGVksMiYYzIciYyMSBtwa1hDH zCMqdscamUNJ9VbKXUa+IXBcuZ4nNEh60LtYJw1Gyy7ZrarTM7wCpYBEirVm+okP YES7YEJcbaR5e1IzVbtjM+zqax4CfbcEuaxE6NvYyGcm+/aJJ5AVvkeBkZgiJuFq 6idpjFtqRH2qzA1FEI/Uln2s5XO78fxP3GhFfLNoCsG+v6WvjpSukS8udRI4XDeu yJj7Gmw41fOmt1fCiz7dqFk+yuPnwHiyM29QUQlFw2sp8YKAXCccMuYCYuzG/BA2 DQReGxxAcIFXcg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddrudefjedgieekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfrhgggfestdhqredtredttdenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepieejue dvueduuefhgefhheeiuedvtedvuefgieegveetueeiueehtdegudehfeelnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepiihirdihrghnsehsvg hnthdrtghomh X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 5 Jan 2022 16:48:28 -0500 (EST) From: Zi Yan To: David Hildenbrand , linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Marek Szyprowski , Robin Murphy , linuxppc-dev@lists.ozlabs.org, virtualization@lists.linux-foundation.org, iommu@lists.linux-foundation.org, Vlastimil Babka , Mel Gorman , Eric Ren , Zi Yan Subject: [RFC PATCH v3 8/8] arch: powerpc: adjust fadump alignment to be pageblock aligned. Date: Wed, 5 Jan 2022 16:47:56 -0500 Message-Id: <20220105214756.91065-9-zi.yan@sent.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220105214756.91065-1-zi.yan@sent.com> References: <20220105214756.91065-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: Zi Yan CMA only requires pageblock alignment now. Change CMA alignment in fadump too. Signed-off-by: Zi Yan --- arch/powerpc/include/asm/fadump-internal.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/powerpc/include/asm/fadump-internal.h b/arch/powerpc/incl= ude/asm/fadump-internal.h index 52189928ec08..fbfca85b4200 100644 --- a/arch/powerpc/include/asm/fadump-internal.h +++ b/arch/powerpc/include/asm/fadump-internal.h @@ -20,9 +20,7 @@ #define memblock_num_regions(memblock_type) (memblock.memblock_type.cnt) =20 /* Alignment per CMA requirement. */ -#define FADUMP_CMA_ALIGNMENT (PAGE_SIZE << \ - max_t(unsigned long, MAX_ORDER - 1, \ - pageblock_order)) +#define FADUMP_CMA_ALIGNMENT (PAGE_SIZE << pageblock_order) =20 /* FAD commands */ #define FADUMP_REGISTER 1 --=20 2.34.1