From nobody Fri Dec 19 14:22:43 2025 Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 776612D9481 for ; Wed, 5 Nov 2025 17:30:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762363822; cv=none; b=QU85WA83Vl1YlSMKLacnh7PzUB4RHirFE8drUQrxRUYQDc5cpdfFewYVwmgya/aXicEAHOVmiAaDxlTiMFjGiKqk85+9ABkfrruOCkygme3yLkgwSfUzEjJ2VSp19CkNb2TFxFbJLAkOktjVUivHptVEyMravRj14LEMx+UGc80= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762363822; c=relaxed/simple; bh=Vw3oDB87MEUxRUWFvQMjoC2affq/wtOrOfd1yATVqF4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kSJRRFueokUO76bW1B95JaSgGz5HzM4xni4T46/alnAP9eK26citGYE0jrEtInIUQMJyYldLE7VsUm/z91rslyuGNggs/46+OPTW3TZ42E4WLhK2WfAr2CI2mL+WmPZ6HPEqKZUTemtfmk6yKSehUhK8ndJo0w3dvsgDM+jVSME= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hA0RpS2s; arc=none smtp.client-ip=209.85.210.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hA0RpS2s" Received: by mail-ot1-f54.google.com with SMTP id 46e09a7af769-7c284d4867eso1945965a34.3 for ; Wed, 05 Nov 2025 09:30:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1762363819; x=1762968619; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G/i7MZkNHnwcGwYWKTXHoQSCF+TqXVB+QQ7fHjYAukA=; b=hA0RpS2s3Y6gRNhwvn2lmC+jUO4VqBA+x2gOtqe3r3rOMy2D6Q/ETxCkd+fn683lhI a/LpUwDSPBn2mNmHKwLaXT/XLCH4uYwGQO4Z4wz5rGCtm307Y0VStdSTfB+I48VFmCbI Zi4psZXVDY5d3C4Xz4d5CIjpcPEvOAO1J41oDU5mVu2rWLukzu87FRc0MOxCmcYnPqtC H/GGWx7X3mOhmPD2q0Tpy09E8oJCvOa7GurHBDG5ybafKGwW/flezz6Rq5WlRi7LHVJX N0g4yuL3kIyGmfC37LZVbeO9sQBaQwBPFrqHm2UJmB9AgUiyKjxFUt+MDn8hmAPCawSk YM9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762363819; x=1762968619; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G/i7MZkNHnwcGwYWKTXHoQSCF+TqXVB+QQ7fHjYAukA=; b=c2UUjiDFO8VngJ9smg0DVXCpnplh7JXxcNmT8BE1Mjn2uhGsAikIXGCzuetmA25wIx RcfrN4twRrENGiqLEMBpVBME38Ilo221TZ5iv73HC4Dor4IlXU3yF+hAOr30XhkKcWbH 0R/DARTkUygPOdLX/PDQt//kJJiuVf1r30X4maaUw+SW0Th4eUVY6kfu3VQzuLgaMvVN dpWCvsGBZp5WEJJ10tU5EEav8KgpnsnJfLBm/KaUQi/JOfWKh1dX/hI4l4VwmpDxChe9 9tK/48w5QqB5N/FyGQPyru15u0kfo/SZ1cEJzSAdEUqG5kbhJNpPyrGKSd7sRVUNEJrZ bIOA== X-Forwarded-Encrypted: i=1; AJvYcCXERdgKNXH1P4GAKHHIR9+XwXO82blNG9VgsI+ZX7PgY4QWGSAxO6Zbi3+1S4A9oqCPXcXKF95zsfjygto=@vger.kernel.org X-Gm-Message-State: AOJu0YwAGWqMQu7jjRjBVSDAOPsPZTrlGjkdPka2RaXn7jvfatLRzuM7 DCs3pZ6vlWfV6QHGOAeXCPG3bVfipyBycvh9Ej+h1AKHErhjW9YWyi6/HS8Vhf1tDyc= X-Gm-Gg: ASbGnctMlCjjawcwKd+lkGdWhl2IauaMBoLhgDDDnp8t/BOWitsSRcBEMbtptn2JMHn cIwWAZt8KpicMj6CG0TAoUOVuNWIFKenO42wbD7qcxdnNbmDSAyA6oUAvdfJaVsC/EWIAgfUbHL hQOJiYYpc9spEEyO+sF8vr4KnTXkKnMspJ9o/+FxxQn/IPv8bng78pSkcGUSoBvcGc//GGmep8J DaB3alPfJ1UlDCAVQ/kOMPuYcDrYHWZfwx4rrIrS69WkhpdJ5rlII3avVUpQT7W2GQdEgbyWZBa Cnny56WyhalOy9nk6XenE584l5oaqA5euZDo9oRxq6jeoset9jgZLIOsoWl309pFg3llznJkdq4 KrnKGwhfzmoi2Bdxo4po7pZexWXrL4jnSpOBG6SCWhXAoWcYfw2iFE1K7hKQ/V1ypO7uJYmlVV8 sIbfZbtMTM0gWd1Dl6jKPH8yQXmQPHcP1hyT62f36mZtyzn97NP+Ag0t+T X-Google-Smtp-Source: AGHT+IGbfNFoEK9x1jFcWstI6BiHgzpe4FTe/whRBvh9s/FjxTvOCYLIbaIAvuJbi6r52ipnPV3d7Q== X-Received: by 2002:a05:6830:620c:b0:7c6:8c48:924a with SMTP id 46e09a7af769-7c6d1498595mr2812045a34.25.1762363819369; Wed, 05 Nov 2025 09:30:19 -0800 (PST) Received: from weg-ThinkPad-P16v-Gen-2.. ([177.73.136.69]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7c6c25108b3sm2248313a34.30.2025.11.05.09.30.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Nov 2025 09:30:18 -0800 (PST) From: Pedro Demarchi Gomes To: David Hildenbrand , Andrew Morton Cc: Xu Xin , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Pedro Demarchi Gomes Subject: [PATCH v3 1/3] Revert "mm/ksm: convert break_ksm() from walk_page_range_vma() to folio_walk" Date: Wed, 5 Nov 2025 14:27:54 -0300 Message-ID: <20251105172756.167009-2-pedrodemargomes@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251105172756.167009-1-pedrodemargomes@gmail.com> References: <20251105172756.167009-1-pedrodemargomes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This reverts commit e317a8d8b4f600fc7ec9725e26417030ee594f52 and changes function break_ksm_pmd_entry() to use folios. This reverts break_ksm() to use walk_page_range_vma() instead of folio_walk_start(). Change break_ksm_pmd_entry() to call is_ksm_zero_pte() only if we know the folio is present, and also rename variable ret to found. This will make it easier to later modify break_ksm() to perform a proper range walk. Suggested-by: David Hildenbrand Signed-off-by: Pedro Demarchi Gomes Acked-by: David Hildenbrand (Red Hat) --- mm/ksm.c | 64 ++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 48 insertions(+), 16 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 4f672f4f2140..de10ebcf3509 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -607,6 +607,48 @@ static inline bool ksm_test_exit(struct mm_struct *mm) return atomic_read(&mm->mm_users) =3D=3D 0; } =20 +static int break_ksm_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned lo= ng next, + struct mm_walk *walk) +{ + struct folio *folio =3D NULL; + spinlock_t *ptl; + pte_t *pte; + pte_t ptent; + int found; + + pte =3D pte_offset_map_lock(walk->mm, pmd, addr, &ptl); + if (!pte) + return 0; + ptent =3D ptep_get(pte); + if (pte_present(ptent)) { + folio =3D vm_normal_folio(walk->vma, addr, ptent); + } else if (!pte_none(ptent)) { + swp_entry_t entry =3D pte_to_swp_entry(ptent); + + /* + * As KSM pages remain KSM pages until freed, no need to wait + * here for migration to end. + */ + if (is_migration_entry(entry)) + folio =3D pfn_swap_entry_folio(entry); + } + /* return 1 if the page is an normal ksm page or KSM-placed zero page */ + found =3D (folio && folio_test_ksm(folio)) || (pte_present(ptent) + && is_ksm_zero_pte(ptent)); + pte_unmap_unlock(pte, ptl); + return found; +} + +static const struct mm_walk_ops break_ksm_ops =3D { + .pmd_entry =3D break_ksm_pmd_entry, + .walk_lock =3D PGWALK_RDLOCK, +}; + +static const struct mm_walk_ops break_ksm_lock_vma_ops =3D { + .pmd_entry =3D break_ksm_pmd_entry, + .walk_lock =3D PGWALK_WRLOCK, +}; + /* * We use break_ksm to break COW on a ksm page by triggering unsharing, * such that the ksm page will get replaced by an exclusive anonymous page. @@ -623,26 +665,16 @@ static inline bool ksm_test_exit(struct mm_struct *mm) static int break_ksm(struct vm_area_struct *vma, unsigned long addr, bool = lock_vma) { vm_fault_t ret =3D 0; - - if (lock_vma) - vma_start_write(vma); + const struct mm_walk_ops *ops =3D lock_vma ? + &break_ksm_lock_vma_ops : &break_ksm_ops; =20 do { - bool ksm_page =3D false; - struct folio_walk fw; - struct folio *folio; + int ksm_page; =20 cond_resched(); - folio =3D folio_walk_start(&fw, vma, addr, - FW_MIGRATION | FW_ZEROPAGE); - if (folio) { - /* Small folio implies FW_LEVEL_PTE. */ - if (!folio_test_large(folio) && - (folio_test_ksm(folio) || is_ksm_zero_pte(fw.pte))) - ksm_page =3D true; - folio_walk_end(&fw, vma); - } - + ksm_page =3D walk_page_range_vma(vma, addr, addr + 1, ops, NULL); + if (WARN_ON_ONCE(ksm_page < 0)) + return ksm_page; if (!ksm_page) return 0; ret =3D handle_mm_fault(vma, addr, --=20 2.43.0 From nobody Fri Dec 19 14:22:43 2025 Received: from mail-ot1-f46.google.com (mail-ot1-f46.google.com [209.85.210.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFA6E31577E for ; Wed, 5 Nov 2025 17:30:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.46 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762363834; cv=none; b=WvSWSWCZ9cqXW2L4Ry+ZrBk8V7dp7feaKkzmDQ/f0pHTHOt2w1HKv9zHr5+sp+ThjawIs9MSo6sRnNUOoDW7x/QsOqbXSf8psq2vXAJEpU+KPdbh4dRm3OLYOiROavDdCb+EkYBAI89t2fp+kOHgHItMqTXy2UIxQ+iyVnGiiOA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762363834; c=relaxed/simple; bh=qO/NLuJPuUMS5aquCLYqrzoe93p2xYxMPVgKO2WNdgo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=G9hwnH/Nh5HKP1gn0HrMaSjC/dT8j2laMCzUz7iKo+jm7zJ5M0HHRJxmD3MzRqlaRCuZtGuUGhJhzcpqxINdLtDZui5U6wBv/Mnij+XE4VCWM0NQq6r+EPjPfDMSubez+eNGt2+h42eSK3ph2v9nOMcVrDVOHf51BRRY0lNcx4M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XfpoHwsq; arc=none smtp.client-ip=209.85.210.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XfpoHwsq" Received: by mail-ot1-f46.google.com with SMTP id 46e09a7af769-7c6d13986f8so87235a34.0 for ; Wed, 05 Nov 2025 09:30:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1762363832; x=1762968632; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9T7mC5XsBejwVN/MD7TteppoeD76dolBCKVexUhUEGA=; b=XfpoHwsqfXQTs12oUAE3ACqx8oHXSU7v4HeZsu2bDVP9JmzTG9z+vYYpapDYbgwBR3 fHYmaP3Hffvo2Lt/F+vdWH8PG3iWsCNUarNw/HgzifrcQvvfRxp8KwXpH9k5gsak1Mnk iyp9pdmkync6ystxbSX66hMVm1nnTAI35lgHPgc+J/iuDYCOjYAaN+Wddf1jtrZw4i6e OJIxVbeEMtKfBJ3ucqfMgIf91S9Ev9duLbHW9rcLWn0M3NBrZjq4c3eUVO67NKTvDBKv H5xeeV1hejS/ryg7omlb149m98eOiTVsTV88tfEj+VH4do426Wz9N0rJ/INpfHLQ4Uso lGaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762363832; x=1762968632; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=9T7mC5XsBejwVN/MD7TteppoeD76dolBCKVexUhUEGA=; b=T+4dYBWwMvtdPmM8Cj9nZnDn98BpuzkeSsggTLnrEqgSlezKiyfGKCGxHC7kTn6M/4 VsohhWV9ETdxVBbCgmdJAgmhk24p5oLepIZpDJmff8A8r4RIHUFqwP+Ez6tDJKBu1V6M amtt2cF28oAwKlK0RDyNvvmtl2I83GOjfEmWsIIJjWnF/AE9+yo+2QDiUZk9+d7pwpGD v4kfqj9eUzJUzO/EN/KF8js7fa0ktgDzAje5R7Qiu8dU1F5yDGt+j/ecq88I8MAQWwz/ O/tqewbqHcyKZ8kHX5QchDASQfyRwvUPyB9R4ej+wxYiYrIf6pNpEGvw8y2hNY+Z2Qgc kUcA== X-Forwarded-Encrypted: i=1; AJvYcCWAXtRFucL8Pf7fQx3Fw1/0cBRKeS3rP/wq96Sxy7viaI1C2B9Wd5ve28HQSHzgq9l6huLlVA8qOZsiXdw=@vger.kernel.org X-Gm-Message-State: AOJu0YzEO7u71BbcMpVcf0+fmB7cBApetrdqvz3qpl173oGzhKfn/l5X FHsO/7XklXTxHuYkjkyaMWo4rlX3GxaAU3k+qOklNYB4MtgfTpc2kEdR X-Gm-Gg: ASbGncuuCXpR07Fqq6p4UcFfo3WclYSD0RQ7OfVulHxY7w6NLvuN51vI5rlUVasfJWL zjdaOInEyQXJx6QYApLudZImxx1qRWfi4/rieBqF39rqxAxVLx1QyTIVXlTFZQV+xcnJTFND52u FbZ+g7V6fj8vj47OW0H+fR4bCsTRSFQyZr1PlCxE9z2dB67LQRS6Qx93TWCr2nzzrYOvjnWoix/ PlrvBLWh8W/+IbgCgLOr/d6c12eJIizp+UTv4aOGLI1LAoUBXraqHHt9zLfJbyA9HwBHdpGxe4a W8hWU3KmtZjmKCxDaPz2jlyeF303hUr6eRXIL1nPl0l2aryCZyyAi8HT+fBxLtzy3hM39pMF4f8 SmzGWpaFINhLbihZhX5TC6P05yoCXq8VQVvZx4del0WP6mR2xRDdP1RradFKp4tevPIfD8rUN0h QPWd5Tvdem0iPOsyFNku9SLMDiv5uNuSSs2B/tYx16xZpMIGFpPMaXtBqB X-Google-Smtp-Source: AGHT+IEBiljDpXhOdYTX7ZvPWxKHLbWSS4eTdnsshXb5F5CF5p8DGEANRiLmMLmNxqqbwLiDoOJOfw== X-Received: by 2002:a05:6830:2a86:b0:7c5:3b4e:3020 with SMTP id 46e09a7af769-7c6e1404c5emr168244a34.12.1762363831770; Wed, 05 Nov 2025 09:30:31 -0800 (PST) Received: from weg-ThinkPad-P16v-Gen-2.. ([177.73.136.69]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7c6c25108b3sm2248313a34.30.2025.11.05.09.30.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Nov 2025 09:30:31 -0800 (PST) From: Pedro Demarchi Gomes To: David Hildenbrand , Andrew Morton Cc: Xu Xin , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Pedro Demarchi Gomes Subject: [PATCH v3 2/3] ksm: perform a range-walk in break_ksm Date: Wed, 5 Nov 2025 14:27:55 -0300 Message-ID: <20251105172756.167009-3-pedrodemargomes@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251105172756.167009-1-pedrodemargomes@gmail.com> References: <20251105172756.167009-1-pedrodemargomes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make break_ksm() receive an address range and change break_ksm_pmd_entry() to perform a range-walk and return the address of the first ksm page found. This change allows break_ksm() to skip unmapped regions instead of iterating every page address. When unmerging large sparse VMAs, this significantly reduces runtime. In a benchmark unmerging a 32 TiB sparse virtual address space where only one page was populated, the runtime dropped from 9 minutes to less then 5 seconds. Suggested-by: David Hildenbrand Signed-off-by: Pedro Demarchi Gomes Acked-by: David Hildenbrand (Red Hat) --- mm/ksm.c | 83 +++++++++++++++++++++++++++++--------------------------- 1 file changed, 43 insertions(+), 40 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index de10ebcf3509..5fe3d9c2cd20 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -607,35 +607,50 @@ static inline bool ksm_test_exit(struct mm_struct *mm) return atomic_read(&mm->mm_users) =3D=3D 0; } =20 -static int break_ksm_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned lo= ng next, +static int break_ksm_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned l= ong end, struct mm_walk *walk) { - struct folio *folio =3D NULL; + unsigned long *found_addr =3D (unsigned long *) walk->private; + struct mm_struct *mm =3D walk->mm; + pte_t *start_ptep, *ptep; spinlock_t *ptl; - pte_t *pte; - pte_t ptent; - int found; + int found =3D 0; =20 - pte =3D pte_offset_map_lock(walk->mm, pmd, addr, &ptl); - if (!pte) + if (ksm_test_exit(walk->mm)) return 0; - ptent =3D ptep_get(pte); - if (pte_present(ptent)) { - folio =3D vm_normal_folio(walk->vma, addr, ptent); - } else if (!pte_none(ptent)) { - swp_entry_t entry =3D pte_to_swp_entry(ptent); + if (signal_pending(current)) + return -ERESTARTSYS; =20 - /* - * As KSM pages remain KSM pages until freed, no need to wait - * here for migration to end. - */ - if (is_migration_entry(entry)) - folio =3D pfn_swap_entry_folio(entry); + start_ptep =3D pte_offset_map_lock(mm, pmdp, addr, &ptl); + if (!start_ptep) + return 0; + + for (ptep =3D start_ptep; addr < end; ptep++, addr +=3D PAGE_SIZE) { + pte_t pte =3D ptep_get(ptep); + struct folio *folio =3D NULL; + + if (pte_present(pte)) { + folio =3D vm_normal_folio(walk->vma, addr, pte); + } else if (!pte_none(pte)) { + swp_entry_t entry =3D pte_to_swp_entry(pte); + + /* + * As KSM pages remain KSM pages until freed, no need to wait + * here for migration to end. + */ + if (is_migration_entry(entry)) + folio =3D pfn_swap_entry_folio(entry); + } + /* return 1 if the page is an normal ksm page or KSM-placed zero page */ + found =3D (folio && folio_test_ksm(folio)) || (pte_present(pte) + && is_ksm_zero_pte(pte)); + if (found) { + *found_addr =3D addr; + goto out_unlock; + } } - /* return 1 if the page is an normal ksm page or KSM-placed zero page */ - found =3D (folio && folio_test_ksm(folio)) || (pte_present(ptent) - && is_ksm_zero_pte(ptent)); - pte_unmap_unlock(pte, ptl); +out_unlock: + pte_unmap_unlock(ptep, ptl); return found; } =20 @@ -662,7 +677,8 @@ static const struct mm_walk_ops break_ksm_lock_vma_ops = =3D { * of the process that owns 'vma'. We also do not want to enforce * protection keys here anyway. */ -static int break_ksm(struct vm_area_struct *vma, unsigned long addr, bool = lock_vma) +static int break_ksm(struct vm_area_struct *vma, unsigned long addr, + unsigned long end, bool lock_vma) { vm_fault_t ret =3D 0; const struct mm_walk_ops *ops =3D lock_vma ? @@ -672,11 +688,9 @@ static int break_ksm(struct vm_area_struct *vma, unsig= ned long addr, bool lock_v int ksm_page; =20 cond_resched(); - ksm_page =3D walk_page_range_vma(vma, addr, addr + 1, ops, NULL); - if (WARN_ON_ONCE(ksm_page < 0)) + ksm_page =3D walk_page_range_vma(vma, addr, end, ops, &addr); + if (ksm_page <=3D 0) return ksm_page; - if (!ksm_page) - return 0; ret =3D handle_mm_fault(vma, addr, FAULT_FLAG_UNSHARE | FAULT_FLAG_REMOTE, NULL); @@ -762,7 +776,7 @@ static void break_cow(struct ksm_rmap_item *rmap_item) mmap_read_lock(mm); vma =3D find_mergeable_vma(mm, addr); if (vma) - break_ksm(vma, addr, false); + break_ksm(vma, addr, addr + PAGE_SIZE, false); mmap_read_unlock(mm); } =20 @@ -1073,18 +1087,7 @@ static void remove_trailing_rmap_items(struct ksm_rm= ap_item **rmap_list) static int unmerge_ksm_pages(struct vm_area_struct *vma, unsigned long start, unsigned long end, bool lock_vma) { - unsigned long addr; - int err =3D 0; - - for (addr =3D start; addr < end && !err; addr +=3D PAGE_SIZE) { - if (ksm_test_exit(vma->vm_mm)) - break; - if (signal_pending(current)) - err =3D -ERESTARTSYS; - else - err =3D break_ksm(vma, addr, lock_vma); - } - return err; + return break_ksm(vma, start, end, lock_vma); } =20 static inline --=20 2.43.0 From nobody Fri Dec 19 14:22:43 2025 Received: from mail-ot1-f42.google.com (mail-ot1-f42.google.com [209.85.210.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E990533B97B for ; Wed, 5 Nov 2025 17:30:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762363841; cv=none; b=J1qkyK554TwZ/OnUgVEeAwtqCrce8H+88dnr+7lOE9XWWehYektWWZG6DZS1aFO9i277gpEnMZ2tpOT3sdEqDC5J9s99fu4csjuRt6zVJLNQABPhd4dkjlSxWEL4ZoX59APmnt/PhVDvDhOHx4ZdwoIaKJ+dzsymsd+SBYAZYkc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762363841; c=relaxed/simple; bh=quGAGyZ78djsPo2+I+REOlFLU8wZEyYPOcHKcfLEtww=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iaNBQd/g29ppTy4UBGcF+kNyevcJo+i243VxKa+k0sQ8CKVPNxNv28ySE7FSj9MXrmDexVMhppcUy5FO5IOvjtL62NmsEGLWJgxsY7+L5666fseZiqhs/VtknrfunlRXmkL9vlbG21/OBmIEp8y4fOgodmGGhoXK4kOeK9oMC7E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Qyit8yh9; arc=none smtp.client-ip=209.85.210.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Qyit8yh9" Received: by mail-ot1-f42.google.com with SMTP id 46e09a7af769-7c5333e7099so4150122a34.3 for ; Wed, 05 Nov 2025 09:30:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1762363839; x=1762968639; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HtG8m3lKC5YzUVxybdhftXK279x1q4y+tfqJb0ojn9g=; b=Qyit8yh9n8KtGvhEdAqlGxkeDSESaeurDnbd0ZPpSnhPQjZLbDVcRi5st0ajQS5hr+ a0AjtPcWh7DzAe8T4OH1/5fYq3PiiQWWzPyUbUzHkIiPB7kmXdLlqNMezh/kJXUHLUWa LXGbapvUtkz9y42GG2dR/AvRX8cryeeI4EhI6wzylAoBKV88zyq45zxL54zBYseTiDT+ SojiEOnBxWxQ+qT4YoXz+E3FFPu44Yc7PFSmKxRPNPoHoCZTQ6otmOngE2+4jj62Anaz gPTCEIIJeuVlle/Ck554dO9prjIVP6Mplspj6Bw/XcjclF+aHqUvljVrP47cbWTq00Re iTCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762363839; x=1762968639; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HtG8m3lKC5YzUVxybdhftXK279x1q4y+tfqJb0ojn9g=; b=a53EeqBtc63AFOBg62y5bbLe2SxRTNycL2dFjGFxN9740Fp3lfFX1Z25ZWJ1utVy5B kh1otlP6Yuw68NR4whZmRnRh7PcBAQfcIUqg/zQ5lF9knewI6xdMZ8OZguSt+QfUgX+J w42uxynaVbkGU0riIFuqQieSgtKXBlhpDzoTonMXRYSVeKxMQ2g/uxewrak6pCoAoJeu 4T/JgDd2lvejYwJ4NKCKN0rI7Nn152i0NqPkUE4dQibCCgI7myovyvVPmJWML0V+I3Ex FvI3UDXbMdRx0ayUHpezeTu5d4y3bv++QX+L4GqaIZ2OCTKgIn5P58VlVT2kz9Lqio4R M8dg== X-Forwarded-Encrypted: i=1; AJvYcCUsS5ZXfETfkd/ubLtC8bH+1R9M30ntiCleImjtQHnvZov7cOOENQJDua2axUYpk5QY5GJZhbIr0rGTeHM=@vger.kernel.org X-Gm-Message-State: AOJu0YzQBjB8fcnK8FVunccMkeNW1mWUNbf4+xUcGKRHWG5u1yGBznDJ 1Q38+unyPnvvzqV8LmwArw6Um64y4r1xY1LafinjQO+45wjH/zcyRQJz X-Gm-Gg: ASbGncsVkFqpy1n/KjWIjSKfwWI8oO6xuJCwHaXeGU/hflAU7HPwv+DbYU1fM4uDyfE ooIEI99ttePldBreFP+pTorYxoIOIb1DoP2NGpCIenWWoGPhDE2O8D4jyW/phexq1OwkNGZt2lh zP8OGf4oaw9lHIX6x044FwLSOJrCQ8WihDzsqagWv2l3eJIx9qHxocGe8yAyuBWThsEHSwPJ2rJ 0EwXg4BONSoTzvX0yENtR/gFDSpY4M4rsmC0mv2fURwJ1iAqHiplxa8hJH5OGyD4pMAsS3IDQq+ sgncsQHb7IjW2X8gctyEJ/+7A5X3wAZgXUc5DR7HtosfrKs5h9fey5th1WNx20L4LLjYxGNhi5i iHXsX1krPzR/zNq4goyqep+YTzkelkyJuoNpb+hFj9P+8Hxbl7LelZwtzlOPqnlw/AL7p4PNSrS 2LJ6cmK0ytUKNoV67VuPgoEt3nwaM1D9AjkD3Bu4IflTt+0X1zU1Ru9HiI X-Google-Smtp-Source: AGHT+IFsqa3f21t/TPiouRwC1NNyll3RzLZxNHIyUXMLTJDkC0oghDxR3kOoLF+y4ihKIeUnycY3mA== X-Received: by 2002:a05:6830:3155:b0:7c5:411a:9204 with SMTP id 46e09a7af769-7c6d1106516mr2563202a34.0.1762363838805; Wed, 05 Nov 2025 09:30:38 -0800 (PST) Received: from weg-ThinkPad-P16v-Gen-2.. ([177.73.136.69]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7c6c25108b3sm2248313a34.30.2025.11.05.09.30.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Nov 2025 09:30:38 -0800 (PST) From: Pedro Demarchi Gomes To: David Hildenbrand , Andrew Morton Cc: Xu Xin , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Pedro Demarchi Gomes Subject: [PATCH v3 3/3] ksm: replace function unmerge_ksm_pages with break_ksm Date: Wed, 5 Nov 2025 14:27:56 -0300 Message-ID: <20251105172756.167009-4-pedrodemargomes@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251105172756.167009-1-pedrodemargomes@gmail.com> References: <20251105172756.167009-1-pedrodemargomes@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Function unmerge_ksm_pages() is unnecessary since now break_ksm() walks an address range. So replace it with break_ksm(). Suggested-by: David Hildenbrand Signed-off-by: Pedro Demarchi Gomes Acked-by: David Hildenbrand (Red Hat) --- mm/ksm.c | 40 ++++++++++++++++------------------------ 1 file changed, 16 insertions(+), 24 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 5fe3d9c2cd20..a78346bfeb91 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -665,6 +665,18 @@ static const struct mm_walk_ops break_ksm_lock_vma_ops= =3D { }; =20 /* + * Though it's very tempting to unmerge rmap_items from stable tree rather + * than check every pte of a given vma, the locking doesn't quite work for + * that - an rmap_item is assigned to the stable tree after inserting ksm + * page and upping mmap_lock. Nor does it fit with the way we skip dup'ing + * rmap_items from parent to child at fork time (so as not to waste time + * if exit comes before the next scan reaches it). + * + * Similarly, although we'd like to remove rmap_items (so updating counts + * and freeing memory) when unmerging an area, it's easier to leave that + * to the next pass of ksmd - consider, for example, how ksmd might be + * in cmp_and_merge_page on one of the rmap_items we would be removing. + * * We use break_ksm to break COW on a ksm page by triggering unsharing, * such that the ksm page will get replaced by an exclusive anonymous page. * @@ -1071,25 +1083,6 @@ static void remove_trailing_rmap_items(struct ksm_rm= ap_item **rmap_list) } } =20 -/* - * Though it's very tempting to unmerge rmap_items from stable tree rather - * than check every pte of a given vma, the locking doesn't quite work for - * that - an rmap_item is assigned to the stable tree after inserting ksm - * page and upping mmap_lock. Nor does it fit with the way we skip dup'ing - * rmap_items from parent to child at fork time (so as not to waste time - * if exit comes before the next scan reaches it). - * - * Similarly, although we'd like to remove rmap_items (so updating counts - * and freeing memory) when unmerging an area, it's easier to leave that - * to the next pass of ksmd - consider, for example, how ksmd might be - * in cmp_and_merge_page on one of the rmap_items we would be removing. - */ -static int unmerge_ksm_pages(struct vm_area_struct *vma, - unsigned long start, unsigned long end, bool lock_vma) -{ - return break_ksm(vma, start, end, lock_vma); -} - static inline struct ksm_stable_node *folio_stable_node(const struct folio *folio) { @@ -1227,8 +1220,7 @@ static int unmerge_and_remove_all_rmap_items(void) for_each_vma(vmi, vma) { if (!(vma->vm_flags & VM_MERGEABLE) || !vma->anon_vma) continue; - err =3D unmerge_ksm_pages(vma, - vma->vm_start, vma->vm_end, false); + err =3D break_ksm(vma, vma->vm_start, vma->vm_end, false); if (err) goto error; } @@ -2855,7 +2847,7 @@ static int __ksm_del_vma(struct vm_area_struct *vma) return 0; =20 if (vma->anon_vma) { - err =3D unmerge_ksm_pages(vma, vma->vm_start, vma->vm_end, true); + err =3D break_ksm(vma, vma->vm_start, vma->vm_end, true); if (err) return err; } @@ -3007,7 +2999,7 @@ int ksm_madvise(struct vm_area_struct *vma, unsigned = long start, return 0; /* just ignore the advice */ =20 if (vma->anon_vma) { - err =3D unmerge_ksm_pages(vma, start, end, true); + err =3D break_ksm(vma, start, end, true); if (err) return err; } @@ -3389,7 +3381,7 @@ static int ksm_memory_callback(struct notifier_block = *self, * Prevent ksm_do_scan(), unmerge_and_remove_all_rmap_items() * and remove_all_stable_nodes() while memory is going offline: * it is unsafe for them to touch the stable tree at this time. - * But unmerge_ksm_pages(), rmap lookups and other entry points + * But break_ksm(), rmap lookups and other entry points * which do not need the ksm_thread_mutex are all safe. */ mutex_lock(&ksm_thread_mutex); --=20 2.43.0