From nobody Mon Apr 6 13:29:10 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69387C54EE9 for ; Tue, 27 Sep 2022 16:28:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232798AbiI0Q24 (ORCPT ); Tue, 27 Sep 2022 12:28:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231126AbiI0Q20 (ORCPT ); Tue, 27 Sep 2022 12:28:26 -0400 Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD03212613 for ; Tue, 27 Sep 2022 09:28:09 -0700 (PDT) Received: by mail-pl1-x62a.google.com with SMTP id iw17so9597264plb.0 for ; Tue, 27 Sep 2022 09:28:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=4E5FbfG/IuR+E8W1emv8V3grpVK/EnZv9IRpD1bffsk=; b=R6qS6gl1BdIlAy4Pc+DF6Tnk7r09A1Hzh3RhF+jg8a3vWDHO/Z8Z0F/8KLFmNT4UZz p/zFN3RfTqWi2DRtOVtFciqpCdIdmtpT6DJVVtxBQq6UgU6IFoSyjC2EQZwD12QjI2rP N6sKt/2wIihFM0xukfEBIwASir6D+SsWWfgEpZfNYQlep+bU6+y1VL+nU1RUxcpcZbzj HDJDak4eIuiXg+OZKVbWp7xuXqSnIxgyQjH4X/XP702GEzT5h+djQNgxeLZCe8y8DeX2 1QEo27PMnsy6vuvoS48eRrJKETldKeSTA/zrFTpSxL5TIlzzEkP0pCRtT4W1+NWE4eY6 RLZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=4E5FbfG/IuR+E8W1emv8V3grpVK/EnZv9IRpD1bffsk=; b=MQdP8XC1GQoGKtqs2Bchn22pj3SuAWYohzZFX2VIghs6N7zU1I8K0hHeY3N5ydMNIr FoOLZAsOL+vJheLd2Xod24UsFjRU+Bw1lC8jt/viWE0iUcOZbnW3q8CknseO3pfHT9jV Q+LPhEYY1jok3/4sCfHkxXmdNer2cjuBokfMawzBCp1CoeB1K338GLTtAODEWWDhS7+T IQROBRgBhlkVKUdAglp5qqYV3OcBfGzVbVqu+E/zhtbF+tetL3tzPBGVAKU4BIu++B9E ItcApWO57hFNA6xWSfYfjoGysgTitOp4ErwlfAS64kJrznSGdJVqlhORiJnjG0+Gjq1n 8n+g== X-Gm-Message-State: ACrzQf3+2nDPeTFqsFG4XqlSsGqGiRjbrrO0vKirMGzHqYVWFVyJLB07 crv3eJmvi/EHovdYC621jUw= X-Google-Smtp-Source: AMsMyM4f2hvZ08Sspjy753JXqV501MbKNQc+IDnf+X9RFXObXCyHmtTBbnBpVaxK4l6Y+BwjCYmecg== X-Received: by 2002:a17:902:ce81:b0:179:f3fe:7fef with SMTP id f1-20020a170902ce8100b00179f3fe7fefmr2502835plg.119.1664296089240; Tue, 27 Sep 2022 09:28:09 -0700 (PDT) Received: from archlinux.localdomain ([140.121.198.213]) by smtp.googlemail.com with ESMTPSA id 9-20020a17090a0f0900b001f333fab3d6sm8602360pjy.18.2022.09.27.09.28.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Sep 2022 09:28:08 -0700 (PDT) From: Chih-En Lin To: Andrew Morton , Qi Zheng , David Hildenbrand , Matthew Wilcox , Christophe Leroy Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Luis Chamberlain , Kees Cook , Iurii Zaikin , Vlastimil Babka , William Kucharski , "Kirill A . Shutemov" , Peter Xu , Suren Baghdasaryan , Arnd Bergmann , Tong Tiangen , Pasha Tatashin , Li kunyu , Nadav Amit , Anshuman Khandual , Minchan Kim , Yang Shi , Song Liu , Miaohe Lin , Thomas Gleixner , Sebastian Andrzej Siewior , Andy Lutomirski , Fenghua Yu , Dinglan Peng , Pedro Fonseca , Jim Huang , Huichun Feng , Chih-En Lin Subject: [RFC PATCH v2 6/9] mm, pgtable: Add COW_PTE_OWNER_EXCLUSIVE flag Date: Wed, 28 Sep 2022 00:29:54 +0800 Message-Id: <20220927162957.270460-7-shiyn.lin@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20220927162957.270460-1-shiyn.lin@gmail.com> References: <20220927162957.270460-1-shiyn.lin@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" For present COW logic (physical page), in some situations (e.g., pinned page), we cannot share those pages. To make the COW PTE consistent with current logic, introduce the COW_PTE_OWNER_EXCLUSIVE flag to avoid doing COW to the PTE table during fork(). The following is a list of the exclusive flag used. - GUP pinnig with COW physical page will get in trouble. Currently, it will not do COW when GUP works. Follow the rule here. Signed-off-by: Chih-En Lin --- include/linux/pgtable.h | 18 ++++++++++++++++++ mm/gup.c | 13 +++++++++++-- 2 files changed, 29 insertions(+), 2 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 8b497d7d800ed..9b08a3361d490 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -656,6 +656,24 @@ static inline int cow_pte_count(pmd_t *pmd) return page_count(pmd_page(*pmd)); } =20 +/* Keep the first bit clear. See more detail in the comments of struct pag= e. */ +#define COW_PTE_OWNER_EXCLUSIVE ((pmd_t *) 0x02UL) + +static inline void pmd_cow_pte_mkexclusive(pmd_t *pmd) +{ + set_cow_pte_owner(pmd, COW_PTE_OWNER_EXCLUSIVE); +} + +static inline bool pmd_cow_pte_exclusive(pmd_t *pmd) +{ + return cow_pte_owner_is_same(pmd, COW_PTE_OWNER_EXCLUSIVE); +} + +static inline void pmd_cow_pte_clear_mkexclusive(pmd_t *pmd) +{ + set_cow_pte_owner(pmd, NULL); +} + #ifndef pte_access_permitted #define pte_access_permitted(pte, write) \ (pte_present(pte) && (!(write) || pte_write(pte))) diff --git a/mm/gup.c b/mm/gup.c index 5abdaf4874605..4949c8d42a400 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -634,6 +634,11 @@ static struct page *follow_page_pte(struct vm_area_str= uct *vma, mark_page_accessed(page); } out: + /* + * We don't share the PTE when any other pinned page exists. And + * let the exclusive flag stick around until the table is freed. + */ + pmd_cow_pte_mkexclusive(pmd); pte_unmap_unlock(ptep, ptl); return page; no_page: @@ -932,6 +937,7 @@ static int get_gate_page(struct mm_struct *mm, unsigned= long address, pte =3D pte_offset_map(pmd, address); if (pte_none(*pte)) goto unmap; + pmd_cow_pte_clear_mkexclusive(pmd); *vma =3D get_gate_vma(mm); if (!page) goto out; @@ -2764,8 +2770,11 @@ static int gup_pmd_range(pud_t *pudp, pud_t pud, uns= igned long addr, unsigned lo if (!gup_huge_pd(__hugepd(pmd_val(pmd)), addr, PMD_SHIFT, next, flags, pages, nr)) return 0; - } else if (!gup_pte_range(pmd, addr, next, flags, pages, nr)) - return 0; + } else { + if (!gup_pte_range(pmd, addr, next, flags, pages, nr)) + return 0; + pmd_cow_pte_mkexclusive(&pmd); + } } while (pmdp++, addr =3D next, addr !=3D end); =20 return 1; --=20 2.37.3