From nobody Sun Jun 28 07:39:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E8BDC433EF for ; Thu, 10 Feb 2022 12:37:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241604AbiBJMh0 (ORCPT ); Thu, 10 Feb 2022 07:37:26 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:54898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241599AbiBJMhY (ORCPT ); Thu, 10 Feb 2022 07:37:24 -0500 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73CEF260D for ; Thu, 10 Feb 2022 04:37:25 -0800 (PST) Received: by mail-pj1-x1031.google.com with SMTP id r64-20020a17090a43c600b001b8854e682eso5460819pjg.0 for ; Thu, 10 Feb 2022 04:37:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gKFpQhrIc0qg1OGn2vjlbFux2cfH4pwiDVgfg/+s2Gk=; b=JjFFmiBFFspTkoWluz4Y/2OhOC+2cDtbHwUi6Z8G3s5O6jLaI4K+hCvdUcs90RTnTb VZzbBD0KpisBRj8k0XTR/DImFMEN6lbYnM9WDekRGuOQXnKm5v/KFFCA1wP7yp5PeI0v C+DYAamNxNXL8z4lSF6NDBsFbDFTsDn5cWDibf8rPJgyTMdpR/aegPTr3LScOIw320ZH sqJSZSjApl195zpCodCKCZUZ3KK2Il7lDKn9IX13DkmKxsDDQPAnQ+a9yOjE2qFn5wCo RCS7uBuIFUpHMKaPGF2+nbn7Z+voZtzaqSo80Xtu69Fr9csc3qXRTPKMyfc2amzuhODn LyZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gKFpQhrIc0qg1OGn2vjlbFux2cfH4pwiDVgfg/+s2Gk=; b=VgbKy9RE6zJRDYTJjchwnV4jGV1Uu2W6/VHQY4+KZH4bbX60qYl0IhdycO18fgu7Rl tbWNDi4Og/vpkPbv9w/2da4lHB3kSdqnacW/I1RjSctisq0ByFvQ3syR2D9RDToBdJRd /2XgxVG6SSggZ19NKpI47LRiROxI/lMp8A+M6vU2k+L0aXEpa8NYi+FI6dDBcGIW6Wtd gXcevSa/RRb1bxQKtPaHHFZxDJRelqX2piRoKQeXz5RM698uqTAQ9mUB9XIpgWTeAovq oyCeGLUI1qzoxb/X8SfO313+wqBSPZI5C0p9WaQYJzelz83vgHbwafScyPSDVvpnrYWt M80A== X-Gm-Message-State: AOAM533vdMEO1OSAA76D3gKHSgkBlOyF8rmrQGAltg5V+Wobj9OOEuo2 SMngJao4vTwirNLo8zWL5infYw== X-Google-Smtp-Source: ABdhPJzv8qdSlKpXUXZEnltPAAV/8T3YIxP2tHLDjv5DmIpbC3EC8oCGfwcWkp6oBYuakLpHJH3riQ== X-Received: by 2002:a17:902:cf0c:: with SMTP id i12mr7387028plg.64.1644496645014; Thu, 10 Feb 2022 04:37:25 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.237]) by smtp.gmail.com with ESMTPSA id i8sm11767812pgf.94.2022.02.10.04.37.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 04:37:24 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH v5 1/7] mm: thp: fix wrong cache flush in remove_migration_pmd() Date: Thu, 10 Feb 2022 20:30:52 +0800 Message-Id: <20220210123058.79206-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220210123058.79206-1-songmuchun@bytedance.com> References: <20220210123058.79206-1-songmuchun@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The flush_cache_range() is supposed to be justified only if the page is already placed in process page table, and that is done right after flush_cache_range(). So using this interface is wrong. And there is no need to invalite cache since it was non-present before in remove_migration_pmd(). So just to remove it. Signed-off-by: Muchun Song Reviewed-by: Zi Yan --- mm/huge_memory.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f58524394dc1..45ede45b11f5 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3207,7 +3207,6 @@ void remove_migration_pmd(struct page_vma_mapped_walk= *pvmw, struct page *new) if (pmd_swp_uffd_wp(*pvmw->pmd)) pmde =3D pmd_wrprotect(pmd_mkuffd_wp(pmde)); =20 - flush_cache_range(vma, mmun_start, mmun_start + HPAGE_PMD_SIZE); if (PageAnon(new)) page_add_anon_rmap(new, vma, mmun_start, true); else @@ -3215,6 +3214,8 @@ void remove_migration_pmd(struct page_vma_mapped_walk= *pvmw, struct page *new) set_pmd_at(mm, mmun_start, pvmw->pmd, pmde); if ((vma->vm_flags & VM_LOCKED) && !PageDoubleMap(new)) mlock_vma_page(new); + + /* No need to invalidate - it was non-present before */ update_mmu_cache_pmd(vma, address, pvmw->pmd); } #endif --=20 2.11.0 From nobody Sun Jun 28 07:39:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9EAAC433F5 for ; Thu, 10 Feb 2022 12:37:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241613AbiBJMhc (ORCPT ); Thu, 10 Feb 2022 07:37:32 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:54940 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241599AbiBJMh2 (ORCPT ); Thu, 10 Feb 2022 07:37:28 -0500 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3D5A725F9 for ; Thu, 10 Feb 2022 04:37:30 -0800 (PST) Received: by mail-pf1-x431.google.com with SMTP id a39so9130073pfx.7 for ; Thu, 10 Feb 2022 04:37:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=I/ARb1Au1FcGhHT5OSe5P8uQ767qWIf49sbqK0ZF8no=; b=vH5ZsS3PT2z/Kj1kRUC91t3vQ/3r4wzvj9kOFOgiv2XuirXjyqCONSJ/51PQ+9kRrK me7yzHej8KCOSngIj9caZ8TYmD6yjv+OipS0Zh++ztGJnNTQv16Q7RRYXhTYPGAmroqD 8m5j3krUJ5T7ngC61kzmUoO7LQUaIz2LEx+1pTMuFaVjauZ+az7YgVOMXYNt/EgUzglJ CeNzs5L0I0SGoqAREdYhRiJAtewRFVJhmU7qPU751vHJEyUnYbv9HmWNYc2VVD/9t+9P RXlLPEDx6qiWtpgfwcD3oePkZ1C9Ng9PvZ1jiTdIVXdZgfWlIljgvFq369BTQ+OK00/n CUjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I/ARb1Au1FcGhHT5OSe5P8uQ767qWIf49sbqK0ZF8no=; b=nyXqtTbk8CIqfjpIi5y4HFHjnzd9S2ZqMDcSBLa5zWOx4nsHuJe2D26oUyF/TYYUzC PRaqs5WuDzN6smLcRwbpaWbXVE3KfPq/aEp0J9/bgT8VdqdjydorrFBCifJFIPv3rIF5 X35599DAWCU7nWf0Nto7bmL4qZQm8urCau5seeR2DzWPRzXViH9TOYhrg8NILJkdbzpT IKqFq1FGcuOJqDdlzX6TivOsS+J/QKVrksLN+vhJdpJ7jUAAcOjJgM75dZVxbnEBhtmg +rLH+jv6scpx70fsvk8IbKdRTc2kj8Or/x8TOqtnnUxov4fuTPNamluGh3ERBocEJN0d lSzQ== X-Gm-Message-State: AOAM532FrBfpgr3tr/joy7Rt9CeAq5Uc7iIClEX/h1JDs+fVaHHI3nLk Gsyl9RkCGxj+YZAC5k/0rOVzsA== X-Google-Smtp-Source: ABdhPJwl1X1ncYwB3dcpyVNk3GZyIIA7yD1CzabbhXJpnxAOqkc97+qhRQDMOf2nz7oIH9laIqf+NA== X-Received: by 2002:a65:681a:: with SMTP id l26mr5933154pgt.365.1644496649737; Thu, 10 Feb 2022 04:37:29 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.237]) by smtp.gmail.com with ESMTPSA id i8sm11767812pgf.94.2022.02.10.04.37.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 04:37:29 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH v5 2/7] mm: fix missing cache flush for all tail pages of compound page Date: Thu, 10 Feb 2022 20:30:53 +0800 Message-Id: <20220210123058.79206-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220210123058.79206-1-songmuchun@bytedance.com> References: <20220210123058.79206-1-songmuchun@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The D-cache maintenance inside move_to_new_page() only consider one page, there is still D-cache maintenance issue for tail pages of compound page (e.g. THP or HugeTLB). THP migration is only enabled on x86_64, ARM64 and powerpc, while powerpc and arm64 need to maintain the consistency between I-Cache and D-Cache, which depends on flush_dcache_page() to maintain the consistency between I-Cache and D-Cache. But there is no issues on arm64 and powerpc since they already considers the compound page cache flushing in their icache flush function. HugeTLB migration is enabled on arm, arm64, mips, parisc, powerpc, riscv, s390 and sh, while arm has handled the compound page cache flush in flush_dcache_page(), but most others do not. In theory, the issue exists on many architectures. Fix this by not using flush_dcache_folio() since it is not backportable. Fixes: 290408d4a250 ("hugetlb: hugepage migration core") Signed-off-by: Muchun Song Reviewed-by: Zi Yan --- mm/migrate.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index c9296d63878d..c418e8d92b9c 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -933,9 +933,12 @@ static int move_to_new_page(struct page *newpage, stru= ct page *page, if (!PageMappingFlags(page)) page->mapping =3D NULL; =20 - if (likely(!is_zone_device_page(newpage))) - flush_dcache_page(newpage); + if (likely(!is_zone_device_page(newpage))) { + int i, nr =3D compound_nr(newpage); =20 + for (i =3D 0; i < nr; i++) + flush_dcache_page(newpage + i); + } } out: return rc; --=20 2.11.0 From nobody Sun Jun 28 07:39:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18FFCC433EF for ; Thu, 10 Feb 2022 12:37:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241621AbiBJMhh (ORCPT ); Thu, 10 Feb 2022 07:37:37 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:54972 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241614AbiBJMhd (ORCPT ); Thu, 10 Feb 2022 07:37:33 -0500 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2CB825FB for ; Thu, 10 Feb 2022 04:37:34 -0800 (PST) Received: by mail-pj1-x1030.google.com with SMTP id h14-20020a17090a130e00b001b88991a305so8352458pja.3 for ; Thu, 10 Feb 2022 04:37:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kCvGgcRI4BCb4bo+PLBmo+rPOq2UtF9zlalPG6igjOk=; b=nk3w1/O8ruce/KxIcFhyUO1pVaNj82bb2o0RtC+Eqn+gFpTDRTDEEO+SUjkKii+bMA 6/XUIHV6zMhbdvk0KD8AaMfYxOjdrDE/7BPf2Z1eS3SWvotZrlr+0FHt9//kPbH5xExr QBMsk7iVC+lbtURShOMph5ee0edzKKy8tr0ltF1A0FHWFQrbjSd6iU3q6WGP9e8BVY17 9SsD6HPxj3Z6fmn3OcjrV8uLz3kbddDZFxPxIcLS9FKpC8iZfZiXhVY5IdjLNbA2qjSP kyJe8qybWut+rnxiXLz/UUmADHvH367cikWiEbu+7LMtZO3nb/X5hv0TeKzr6J8tdMIc UEdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kCvGgcRI4BCb4bo+PLBmo+rPOq2UtF9zlalPG6igjOk=; b=hSgO4ZRHOdT+o4wIDSZD1PBFnb6iGSytkz0+khDFZ/piI3lekmkj8aoLtkyZ9LteeG QN8NBgqSZnOQiurdirscRqm2K9Y3HmiUUo0gGR3X8Mlo+iX70J3aTiT3h5aDVrsJ+gQZ 9ERyx7RthboDHzhsuLrMgc+qpYcfLup09MmTicA25EjMX0vczLx5Ids4Om0x7EVP00SK yM4ikJY/h9vgwHlRP4qA9jwUM1ZzXggAU5csolFwzoz51YKOgNtsUNSDUM3rKe1jdH1t db4cLM0uJ+A11Qlft/Sx1L5n/zNAUsHwHqkkBKVSSm9ZkK871LSLY4LxqwoEOGCAtal1 DQIw== X-Gm-Message-State: AOAM533+dKXSV04ehEKaLQRwUQHqN/4VNo02u6fTdeme6hV/d4Hgb/ha 8RcSSvcAiWd3f9WWSlW/jV/bgg== X-Google-Smtp-Source: ABdhPJydgYDL68pV4tg6C9n6s9P9xqUtx6BfUfSjlx18xcMF5d9QufoEkJRRr1Q5wwnwW+Wqm5H7Zw== X-Received: by 2002:a17:902:e743:: with SMTP id p3mr1888043plf.152.1644496654350; Thu, 10 Feb 2022 04:37:34 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.237]) by smtp.gmail.com with ESMTPSA id i8sm11767812pgf.94.2022.02.10.04.37.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 04:37:34 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH v5 3/7] mm: hugetlb: fix missing cache flush in copy_huge_page_from_user() Date: Thu, 10 Feb 2022 20:30:54 +0800 Message-Id: <20220210123058.79206-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220210123058.79206-1-songmuchun@bytedance.com> References: <20220210123058.79206-1-songmuchun@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The userfaultfd calls copy_huge_page_from_user() which does not do any cache flushing for the target page. Then the target page will be mapped to the user space with a different address (user address), which might have an alias issue with the kernel address used to copy the data from the user to. Fix this issue by flushing dcache in copy_huge_page_from_user(). Fixes: fa4d75c1de13 ("userfaultfd: hugetlbfs: add copy_huge_page_from_user = for hugetlb userfaultfd support") Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz --- mm/memory.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/memory.c b/mm/memory.c index e8ce066be5f2..eb027da68aa7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5405,6 +5405,8 @@ long copy_huge_page_from_user(struct page *dst_page, if (rc) break; =20 + flush_dcache_page(subpage); + cond_resched(); } return ret_val; --=20 2.11.0 From nobody Sun Jun 28 07:39:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 516CBC433F5 for ; Thu, 10 Feb 2022 12:37:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241628AbiBJMhl (ORCPT ); Thu, 10 Feb 2022 07:37:41 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:55074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241614AbiBJMhh (ORCPT ); Thu, 10 Feb 2022 07:37:37 -0500 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6515AB3 for ; Thu, 10 Feb 2022 04:37:39 -0800 (PST) Received: by mail-pl1-x633.google.com with SMTP id w1so1700268plb.6 for ; Thu, 10 Feb 2022 04:37:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kILCWGYQN4aCmOVzGkGgpbDqf3vflThkp2WsD9ah7ls=; b=fG95xjXcp5pb/4AZm3qvm7w7s5VMHbtcG0Lx0tYYXqTPxFq3Uc9uBYT7LAv2r+KKbS O4ERCsf/+TVyMN0TI/YJ5AQEXti7JI7zzwU0mLvt4C3cBqleBqOzGfuRZcC1OF+y8qK2 vSoUE7Hf9EA/yx8S3k+43lmfcwEiQw/pT+Qqytcy93VzMeV9T5V1d8vGtrZWnnkTZ83m gsLyBS9KN5r5acorSo3QXiEsgVjacHClEVUDMD4sG3kLdlBslDcPD75Vy3XQBiXZPJMa t7zXdHWqMkokITwK0wwNIOyWHuSRyTfwh6IqmbAlxzQ9Gwc1zYBjlhUYgd302H5unjo9 UwFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kILCWGYQN4aCmOVzGkGgpbDqf3vflThkp2WsD9ah7ls=; b=MC//iyPS0roU4ey1pYO5Kx16K/VhM8qesCrTtuy+4Nq8faQ27LBtOgFuUPAYzvoqLI hbA1XLFFo66i1jatXnV+jU6BC6BN73+ITOMn7MeU7CfaIVLrYLnQ1BQcweRg8Zs6PzLI WlYHFN44M0FPYTqsSa1VOOzthwak0b3uQQA1dHIZvChf1kuw3rx4ici6Etf3gwDIEvG6 FARajU9RPGDmmJgv7kisPhusRoJ7ZYYxFw+6vL+Zzo8pG4DBBldSC6MVFPgAYVWDgXpM TfRMVJ+etTRHLVXaruLe9jsApRbI1VpJgjxkZUxEYQjz9vkSJxGEImdkLLrMRKpkEafm JCsQ== X-Gm-Message-State: AOAM5337Do5IJc5Xx6KnjqyJH9H/IJ+f7QRzb9G8ciyPhcyC3SBToh7j DdOsqXt0A5Fn9DoNqCVxu8KEA7MPztMoJWQP X-Google-Smtp-Source: ABdhPJxXO3PdREGl7iYGtPoZwu9ZnEuV9DWQ2+VdIf1bDfCvRiQrzRC1gMtC9rR0cVrUjm5h5t4w8g== X-Received: by 2002:a17:903:1211:: with SMTP id l17mr7635562plh.11.1644496658991; Thu, 10 Feb 2022 04:37:38 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.237]) by smtp.gmail.com with ESMTPSA id i8sm11767812pgf.94.2022.02.10.04.37.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 04:37:38 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH v5 4/7] mm: hugetlb: fix missing cache flush in hugetlb_mcopy_atomic_pte() Date: Thu, 10 Feb 2022 20:30:55 +0800 Message-Id: <20220210123058.79206-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220210123058.79206-1-songmuchun@bytedance.com> References: <20220210123058.79206-1-songmuchun@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" folio_copy() will copy the data from one page to the target page, then the target page will be mapped to the user space address, which might have an alias issue with the kernel address used to copy the data from the page to. There are 2 ways to fix this issue. 1) insert flush_dcache_page() after folio_copy(). 2) replace folio_copy() with copy_user_huge_page() which already considers the cache maintenance. We chose 2) way to fix the issue since architectures can optimize this situation. It is also make backports easier. Fixes: 8cc5fcbb5be8 ("mm, hugetlb: fix racy resv_huge_pages underflow on UF= FDIO_COPY") Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz --- mm/hugetlb.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a1baa198519a..eba7681d15d0 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5818,7 +5818,8 @@ int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, *pagep =3D NULL; goto out; } - folio_copy(page_folio(page), page_folio(*pagep)); + copy_user_huge_page(page, *pagep, dst_addr, dst_vma, + pages_per_huge_page(h)); put_page(*pagep); *pagep =3D NULL; } --=20 2.11.0 From nobody Sun Jun 28 07:39:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D10CC433FE for ; Thu, 10 Feb 2022 12:37:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241640AbiBJMhs (ORCPT ); Thu, 10 Feb 2022 07:37:48 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:55174 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241614AbiBJMhm (ORCPT ); Thu, 10 Feb 2022 07:37:42 -0500 Received: from mail-pl1-x634.google.com (mail-pl1-x634.google.com [IPv6:2607:f8b0:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D991E10A for ; Thu, 10 Feb 2022 04:37:43 -0800 (PST) Received: by mail-pl1-x634.google.com with SMTP id z17so1694124plb.9 for ; Thu, 10 Feb 2022 04:37:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tKpVSiHHtaj87FMK/TeB9fab8FVNXtwv8Xc4GuIj/Kg=; b=fQy8vfqpU9cS9VF0UkKIV2NNCg0tQEdPXZgAQyveKt+IqZMhPhSLL2L91pt18Zg06N vNvRvBopSQe6m1BOl262fpfSHiT8PTxbkyLfR2s8UUCSsGqv6jhJr5tF046QxWGKPvQ4 j1luq5AgDAl2iOiKPsSWugj44vi8gpSPdoJjaafW25tBKv8lpwq+uDQF3SBoV0mX0uWh oeE64v4w2/BhLSM2qEBvn2JtkZhtGWhP1r03BRWqGEilJv2WHtsOlbBI0BeMeSkYlft6 BtL8Es/U26EonkHfhoxdHnW89rwm0do83Rj77+xgClJ16smYtj669p2PtIeIaa0K/Otd MINg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tKpVSiHHtaj87FMK/TeB9fab8FVNXtwv8Xc4GuIj/Kg=; b=yoVEf88DcTJ74cz4vt9x/l0d1MkyWPRwFuAnMKI8qb59kYUSSeMtP+fk/P32/or5TG eeYu7wMWDX8KXSUBg+2p66VWgcZ5XUKJzPuIysaqxwNpPCpqTs8QeEEb/hvV9xA7gQVL fqHSFe27jbB/dTNpiHlBWOLxcegKhQku/+aCu6xLL4LhwtEV9DrEMeMMrCvIaRKyC7ID Sq34f8vVywI1BV7IzYx6WfkSwqchPkW2/2H0c2lXve9jrrKcWSHsjM8MN5g3t/nNjTJp pryMSaR9fwYj07LxEvbU3fwp/yF9BO/iqD7nYOmzgQ3YpI8oNqr44NzvsRze3YPRUHBL MSWQ== X-Gm-Message-State: AOAM533BI3X79YgxEATOWj2I7Wum7v749AFzWOTDZdT5ZsYy7e9mJcF4 vaW9lxqfuOciOFxXBK1IOCgFFQ== X-Google-Smtp-Source: ABdhPJyGp2GsZvSDMqpR5iLkDxZjE53dEk3dm5rB0VkaJA74+KantoyMJ9LJ6pDdNdp3d3NZ3dXk6A== X-Received: by 2002:a17:90b:1c0e:: with SMTP id oc14mr2638965pjb.2.1644496663463; Thu, 10 Feb 2022 04:37:43 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.237]) by smtp.gmail.com with ESMTPSA id i8sm11767812pgf.94.2022.02.10.04.37.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 04:37:43 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH v5 5/7] mm: shmem: fix missing cache flush in shmem_mfill_atomic_pte() Date: Thu, 10 Feb 2022 20:30:56 +0800 Message-Id: <20220210123058.79206-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220210123058.79206-1-songmuchun@bytedance.com> References: <20220210123058.79206-1-songmuchun@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The userfaultfd calls shmem_mfill_atomic_pte() which does not do any cache flushing for the target page. Then the target page will be mapped to the user space with a different address (user address), which might have an alias issue with the kernel address used to copy the data from the user to. Insert flush_dcache_page() in non-zero-page case. And replace clear_highpage() with clear_user_highpage() which already considers the cache maintenance. Fixes: 8d1039634206 ("userfaultfd: shmem: add shmem_mfill_zeropage_pte for = userfaultfd support") Fixes: 4c27fe4c4c84 ("userfaultfd: shmem: add shmem_mcopy_atomic_pte for us= erfaultfd support") Signed-off-by: Muchun Song --- mm/shmem.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index eb0fd9001130..2e17ec9231a2 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2371,8 +2371,10 @@ int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, /* don't free the page */ goto out_unacct_blocks; } + + flush_dcache_page(page); } else { /* ZEROPAGE */ - clear_highpage(page); + clear_user_highpage(page, dst_addr); } } else { page =3D *pagep; --=20 2.11.0 From nobody Sun Jun 28 07:39:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51447C433EF for ; Thu, 10 Feb 2022 12:37:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241647AbiBJMhv (ORCPT ); Thu, 10 Feb 2022 07:37:51 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:55346 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240214AbiBJMhr (ORCPT ); Thu, 10 Feb 2022 07:37:47 -0500 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CAAB5DF5 for ; Thu, 10 Feb 2022 04:37:48 -0800 (PST) Received: by mail-pf1-x433.google.com with SMTP id x15so7547782pfr.5 for ; Thu, 10 Feb 2022 04:37:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kGvpK6UTEMeGdDCg9a68QTimGRB3Wg5V4hLO/H8ovOk=; b=8R2KQc0VHwfvgfFLuPp0xcs/i0bFi3ZpoOvzA4bg66nMtTPqQzqxTX+Hj1tWbZt+HU 01u5lq16jDHas70gMpnZO5BBiMuTF+W5pva0r6kivMj+a2RM3QWGQAUXznhUqBxpKPkf Vv/Tu1P5u0Zi/cQioKebx7XyAGlUXUXMibP1SqIBSj3UGsVQaJUelPy5p4+rwTHdnNhy imoRrsXIOlM97bvi9gNvPrSM9JSeP4ZkPvsAYpxNIA9hXjYHh50pMLoZIGcA0ttGpuMA Y/Xsnao5EZKEuqby/5+bnXTVOdcLa1PFjq8mBuhVnryqOZu7iQRstCzev9b3DNb+X1dy uUqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kGvpK6UTEMeGdDCg9a68QTimGRB3Wg5V4hLO/H8ovOk=; b=I8gk0jyGWPahOJo5fSOK72IWtfoKYVLUeDjoaT5p6u5E/Y3ZAl0V+0N891Ukjkmudd mRByGzFkLm+m1oCyc4CGBZ97TTPD35a1pSdoEoHOl5PUKIhUX3ha35d8DEfONrtV4QSg 4xqCAG/2af44cOrwo6WxhmoXWMEfG2jTA+NI4bYlQmYmwqeh+OxhIwiJNxFwPCt9DezB gtXMsYuR9QdmONLaiya4MEF6l8j3mFhanVvrs70NXR4dtObOAQmQ74+Eau50b+gf427h fH19UEkboW+mdINrCu1rXO6VyeBuXSgVX+C+nODwVQsJo7B86AuXMlo0wxv+oCg2Gcq6 GNMA== X-Gm-Message-State: AOAM530NmLw1DhSP/KRcTdw9EkWu+Y5RiKXtja00VMn+LdyFQDyJPyCG jy8OLnlPYP5pBPNWTkYropToJA== X-Google-Smtp-Source: ABdhPJxuA+mcAk8N3GxFOFt4YcmS2sQMINZgJ8h6aEo3yg4idUSXIkOyJ07Dtpti42YAT/Z5DU8wrQ== X-Received: by 2002:a63:4e0e:: with SMTP id c14mr6120150pgb.490.1644496668324; Thu, 10 Feb 2022 04:37:48 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.237]) by smtp.gmail.com with ESMTPSA id i8sm11767812pgf.94.2022.02.10.04.37.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 04:37:48 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH v5 6/7] mm: userfaultfd: fix missing cache flush in mcopy_atomic_pte() and __mcopy_atomic() Date: Thu, 10 Feb 2022 20:30:57 +0800 Message-Id: <20220210123058.79206-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220210123058.79206-1-songmuchun@bytedance.com> References: <20220210123058.79206-1-songmuchun@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The userfaultfd calls mcopy_atomic_pte() and __mcopy_atomic() which do not do any cache flushing for the target page. Then the target page will be mapped to the user space with a different address (user address), which mig= ht have an alias issue with the kernel address used to copy the data from the user to. Fix this by insert flush_dcache_page() after copy_from_user() succeeds. Fixes: b6ebaedb4cb1 ("userfaultfd: avoid mmap_sem read recursion in mcopy_a= tomic") Fixes: c1a4de99fada ("userfaultfd: mcopy_atomic|mfill_zeropage: UFFDIO_COPY= |UFFDIO_ZEROPAGE preparation") Signed-off-by: Muchun Song --- mm/userfaultfd.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 0780c2a57ff1..6ccc534d1c1c 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -150,6 +150,8 @@ static int mcopy_atomic_pte(struct mm_struct *dst_mm, /* don't free the page */ goto out; } + + flush_dcache_page(page); } else { page =3D *pagep; *pagep =3D NULL; @@ -625,6 +627,7 @@ static __always_inline ssize_t __mcopy_atomic(struct mm= _struct *dst_mm, err =3D -EFAULT; goto out; } + flush_dcache_page(page); goto retry; } else BUG_ON(page); --=20 2.11.0 From nobody Sun Jun 28 07:39:30 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4378C433EF for ; Thu, 10 Feb 2022 12:37:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241661AbiBJMh4 (ORCPT ); Thu, 10 Feb 2022 07:37:56 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:55422 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240214AbiBJMhv (ORCPT ); Thu, 10 Feb 2022 07:37:51 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C255FDA for ; Thu, 10 Feb 2022 04:37:53 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id t14-20020a17090a3e4e00b001b8f6032d96so5417143pjm.2 for ; Thu, 10 Feb 2022 04:37:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oDahdnw06IsGwB/7M26luCqPUnRrkijqY+ZjOYkno4A=; b=tpGNdIY2hwVGwRhbVAEv6E5VyLyH6hl7LUw7SLZyLIw8PMoLNcP/DUGn0A+TA43MA3 DAp3ueB1y4QZ01hqgxcV1mb3FD1UWVfLo9aGY6MkmnYv1rOx2LYVOFu5ECBqxxAlZ6h7 +fYloX/Dkjb81uKdqULv2aHbXuLal3uHsATiys2WgJE7mplGe4YGnpxk3Z0z9gm5fdvJ Fvl59LuYztWU/omTw0zQNUWaMxPJB3cNt2Hzp19Bnjfm+NtW+p+D6/+bjn6Ikjry2ySE 7IJqRMvlWxRjZy4PkrIHOFY8qSvSNocEuSXW3CRF3ibl5YTyGYBL49XvaPC+KX6yN7Yk TAAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oDahdnw06IsGwB/7M26luCqPUnRrkijqY+ZjOYkno4A=; b=vm1Vw5M+DAhTLyr03lnimxr6L/0Kx/5rcYNCuRGPncDcDVLzu9/C8HsUa2Lot/vLTb TTFwVgOTrrklyorowQoFCkBU9dEwcFYRrigR2PZKeZqv3tjGZ9hcpXKuWmrCDdLWY/rA slx/oqDuQhazwLY4qBNMZpA66SUADBaw0DHimXo3WAPbQJMRvrI93c1z2yB5dhFQJdv1 UK9vsEFB5Tx+st2mTsGgDXYHqPQzaeEwI7HhydmR5wsO34amWY7gToPT1P0jvbEDywzR QDKKlQ7XIa5Rw5slB/WgQH7kQ5r+7DufMzlUWgRrHhweqbL7cDl9KkoyA0Ao+lmiHpgS 23WQ== X-Gm-Message-State: AOAM531sk1V/Sq3/5jxciL9eGmSlcZllujmZouu75SBnXGRd5EzmxIfO BrZrha0R0N6HSoUZqz1K9JYUZw== X-Google-Smtp-Source: ABdhPJzn0l92vEIhPNZDK6wnPCBAAD+wPaRAZsukzdXyxqRwg+i/PIhMukVTcN6WwYR8rEu1ZtzJfQ== X-Received: by 2002:a17:90a:fe15:: with SMTP id ck21mr2602609pjb.95.1644496673004; Thu, 10 Feb 2022 04:37:53 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.237]) by smtp.gmail.com with ESMTPSA id i8sm11767812pgf.94.2022.02.10.04.37.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Feb 2022 04:37:52 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH v5 7/7] mm: replace multiple dcache flush with flush_dcache_folio() Date: Thu, 10 Feb 2022 20:30:58 +0800 Message-Id: <20220210123058.79206-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220210123058.79206-1-songmuchun@bytedance.com> References: <20220210123058.79206-1-songmuchun@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Simplify the code by using flush_dcache_folio(). Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz --- mm/migrate.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index c418e8d92b9c..daf2b3508670 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -933,12 +933,8 @@ static int move_to_new_page(struct page *newpage, stru= ct page *page, if (!PageMappingFlags(page)) page->mapping =3D NULL; =20 - if (likely(!is_zone_device_page(newpage))) { - int i, nr =3D compound_nr(newpage); - - for (i =3D 0; i < nr; i++) - flush_dcache_page(newpage + i); - } + if (likely(!is_zone_device_page(newpage))) + flush_dcache_folio(page_folio(newpage)); } out: return rc; --=20 2.11.0