From nobody Sun Feb 8 21:46:32 2026 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 85EE4320A06 for ; Mon, 29 Dec 2025 14:38:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767019105; cv=none; b=BVGz4xDzWxwedAQRIvQ2gbV6kQLSLlMI12HUJUGMs2TjUlePtSjdfg4S5ruIqZxo+tGEH8etwz6lD/LYP4qF37NDMpI3UWbo0RFnOr5MkuGU4laXNlzqM0fQmDaBWaK+ZgSmCNuZT0Xp8LYq+SrN+dAYVPJHGLJg3nZvdb1xCPw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767019105; c=relaxed/simple; bh=QELGmcYo0okd7XeZ+8OIMQ2Ox7GoXMWa9Ttw3qK5W4U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rgnhRxbW5MVqHL172yhUFS7Ds6rFbyVJU4MxB0DXMVddC+dsQ+xb9KuldIIMVF41sRuyJIEikcoZWIHS0nGB7+cb47e0BajXX4kHq67/qQwdDWfEDoiYgHZ37aOUA1DWjRrjzXkzju5Efvtl8a0MdKyvaRIrrMuTtjKofAZDnXA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2a0834769f0so90062975ad.2 for ; Mon, 29 Dec 2025 06:38:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1767019098; x=1767623898; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=K2pBV4vyT2c4GBUcrUp/G3rP2JcNWZOiRH8LtGLdyis=; b=lheDW/dQpPOFeiussPFhtys0OBEZzZj98RbH+BApom4ikyawt1YaKDQzs+sGCuqcJz SQEGttT1Pdh6u68L+UsBvymsj7wZslbhuJX6mTwCfaKrOBWq84/JU1uFjTFHObxD/pr3 Xzj0bFORNEdoB9WZer6M71KNLU0NZTftNa0Zr9PzRcdyFkcJEvlk6c4dqKIxMZrmWyXE xxeEZYdLqlmepEht2TSNJVaXMCYX94/9HboN4/kq8LFMX1+doKgnrjN0My9bCenEdWf/ cAplYn8S9QNvg3w59IgTBlFcMqDsnWHWJEKuHjO6ZnXbJIuBXnhwdAiD6/3RPyvN1xne BlxA== X-Forwarded-Encrypted: i=1; AJvYcCXKG+cBJuQpORlJByMGqVLaaYoZOushLWJRxPH+2p8XbzCs3FYtO23CWtm9moxAc4awOAtF8axQvQORUGA=@vger.kernel.org X-Gm-Message-State: AOJu0Yz/19TIC7yS5eQk53lFbyqemfQ5E2eODIDVE/HtQV7mkr2xs82F ff+Sp9LIIG+dBg26DGHpoYrOF/Xgg611DXMqceZ48L9XE12Ex7c1HYfG X-Gm-Gg: AY/fxX6zcTHMgV+2Mkj0mReDtIOGXAuSVX6n1rhTEcKmdoeh6V09UJL3dMI9Hp9OOOd GTEHNXzeEHcKklPua0Jv/z4wvOdZBvQbJtypXav1VAkMgIDwf7U++Ef0Z3Sk4OQ/JXrGNnQ97+C LTkR643ERtqZg/FtfqFGL3NX+UtGkGZztsu98fjR0CYw6RF9HIO8pWW5VMWPtmyfFn7riaj+tW3 uoIckWGg3Lr5uPegmWBXGjp7JJB+pDT+XkdSk1pa5J+sTvsuzlsNNkkv5pI2QDJI3t6CQLhoYHl UxZA1av1QXnfQvnMoKyr9dmEsQrTwoGHfpfClDoBDhdlChQL2RgXGfOMZTU+URu3bPmcbOFAl/T 0iN4xWGxKis8N2Qe1lHfGcQC50kTmGGnPdV0PIQbI79B4V/C6VqVBcXtKUSYswQukyFqjqTsUbF sqVPC+yGKDiFO814TIZD9f X-Google-Smtp-Source: AGHT+IGdJt3lkU5juysv6cnzuaDtBicBqS5Uo+6zRrWiZ1emfZCDgGtYUFWFKyKinXKOayvqccQJSQ== X-Received: by 2002:a17:902:ea11:b0:290:91d2:9304 with SMTP id d9443c01a7336-2a2f22052f9mr332042505ad.4.1767019098192; Mon, 29 Dec 2025 06:38:18 -0800 (PST) Received: from EBJ9932692.tcent.cn ([45.8.220.167]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2a2f3d7736fsm279669625ad.92.2025.12.29.06.38.09 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 29 Dec 2025 06:38:17 -0800 (PST) From: Lance Yang To: akpm@linux-foundation.org Cc: will@kernel.org, aneesh.kumar@kernel.org, npiggin@gmail.com, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, arnd@arndb.de, david@kernel.org, lorenzo.stoakes@oracle.com, ziy@nvidia.com, baolin.wang@linux.alibaba.com, Liam.Howlett@oracle.com, npache@redhat.com, ryan.roberts@arm.com, dev.jain@arm.com, baohua@kernel.org, ioworker0@gmail.com, shy828301@gmail.com, riel@surriel.com, jannh@google.com, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Lance Yang Subject: [PATCH RESEND v1 3/3] mm: embed TLB flush IPI check in tlb_remove_table_sync_one() Date: Mon, 29 Dec 2025 22:36:57 +0800 Message-ID: <20251229143657.76968-4-lance.yang@linux.dev> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20251229143657.76968-1-lance.yang@linux.dev> References: <20251229143657.76968-1-lance.yang@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lance Yang Embed the tlb_table_flush_implies_ipi_broadcast() check directly inside tlb_remove_table_sync_one() instead of requiring every caller to check it explicitly. This relies on callers to do the right thing: flush with freed_tables=3Dtrue or unshared_tables=3Dtrue beforehand. All existing callers satisfy this requirement: 1. mm/khugepaged.c:1188 (collapse_huge_page): pmdp_collapse_flush(vma, address, pmd) -> flush_tlb_range(vma, address, address + HPAGE_PMD_SIZE) -> flush_tlb_mm_range(mm, ..., freed_tables =3D true) -> flush_tlb_multi(mm_cpumask(mm), info) So freed_tables=3Dtrue before calling tlb_remove_table_sync_one(). 2. include/asm-generic/tlb.h:861 (tlb_flush_unshared_tables): tlb_flush_mmu_tlbonly(tlb) -> tlb_flush(tlb) -> flush_tlb_mm_range(mm, ..., unshared_tables =3D true) -> flush_tlb_multi(mm_cpumask(mm), info) unshared_tables=3Dtrue (equivalent to freed_tables for sending IPIs). 3. mm/mmu_gather.c:341 (__tlb_remove_table_one): When we can't allocate a batch page in tlb_remove_table(), we do: tlb_table_invalidate(tlb) -> tlb_flush_mmu_tlbonly(tlb) -> flush_tlb_mm_range(mm, ..., freed_tables =3D true) -> flush_tlb_multi(mm_cpumask(mm), info) Then: tlb_remove_table_one(table) -> __tlb_remove_table_one(table) // if !CONFIG_PT_RECLAIM -> tlb_remove_table_sync_one() freed_tables=3Dtrue, and this should work too. Why is tlb->freed_tables guaranteed? Because callers like pte_free_tlb() (via free_pte_range) set freed_tables=3Dtrue before calling __pte_free_tlb(), which then calls tlb_remove_table(). We cannot free page tables without freed_tables=3Dtrue. Note that tlb_remove_table_sync_one() was a NOP on bare metal x86 (CONFIG_MMU_GATHER_RCU_TABLE_FREE=3Dn) before commit a37259732a7d ("x86/mm: Make MMU_GATHER_RCU_TABLE_FREE unconditional"). 4-5. mm/khugepaged.c:1683,1819 (pmdp_get_lockless_sync macro): Same as #1. These also use pmdp_collapse_flush() beforehand. Suggested-by: David Hildenbrand (Red Hat) Signed-off-by: Lance Yang --- mm/mmu_gather.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c index 7468ec388455..7b588643cbae 100644 --- a/mm/mmu_gather.c +++ b/mm/mmu_gather.c @@ -276,6 +276,10 @@ static void tlb_remove_table_smp_sync(void *arg) =20 void tlb_remove_table_sync_one(void) { + /* Skip the IPI if the TLB flush already synchronized with other CPUs. */ + if (tlb_table_flush_implies_ipi_broadcast()) + return; + /* * This isn't an RCU grace period and hence the page-tables cannot be * assumed to be actually RCU-freed. --=20 2.49.0