From nobody Thu Sep 24 12:53:51 2026 Received: from m16.mail.163.com (m16.mail.163.com [220.197.31.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B852372EF0 for ; Thu, 24 Sep 2026 04:13:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=220.197.31.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790223207; cv=none; b=TI/eFiBJBawXdabNC8i0UeZQ1DVrkyGJ+1iMQs/OyDfw4OSa9nHtizkqzt+7UXHPHrBZYV6Tl6sD5YFX8DGSoDx3ALuw3nMN1Ax0YKvbGBO33k/N2vFwiSRjcKBCiYQgAfp3veElrg/f7t//IjK7LhyQ3AQz/F2hntMQEcoSm0o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790223207; c=relaxed/simple; bh=8+RdLEb/EU5kXwYDXztHKTzJVyeDHgxCzgwtiszcjes=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type; b=L1BEKbvkqjcGMz6VMzmEsqC/GR0YTpixrx4sJ+z1PfGA/GIaNvjmtnJwJ6MlIcLx1PhVrLT75/LvVekc53QeYMoGS4CzQw7kj16xZ8gF/4bXioljIINu2SP5D6iZk8XhahKUEEjz0M3X2e/vnXHxwxND93gr4f90aHzxXUc/JuE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com; spf=pass smtp.mailfrom=163.com; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b=ZjqTNrHw; arc=none smtp.client-ip=220.197.31.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=163.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=163.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=163.com header.i=@163.com header.b="ZjqTNrHw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:To:Subject:Date:Message-ID:MIME-Version: Content-Type; bh=iv81RNA485adQFvrL4+HZjzDjbdgSHwKqirIAdW4HY4=; b=ZjqTNrHwAlJ2M/cArk9hKPvbBufLBaJtptxYdQb7413DMPTmJwjJeoRfyEviU0 kuF3Wh4Sw1iOEGSb/5uIz6/8tK+l+S0QaDGZkwe9a8AsQovXL95HSAl3Oiif0x8N r9T7e/mOzeJIIH7r3m4cch0AkYv/DJzJOUyYtiwsyQ3B8= Received: from localhost (unknown []) by gzga-smtp-mtada-g1-4 (Coremail) with SMTP id _____wD3X9Y0o7RqWqn_AQ--.41231S2; Thu, 24 Sep 2026 12:12:37 +0800 (CST) From: Hui Su To: Andrew Morton , David Hildenbrand Cc: Hui Su , Balbir Singh , Matthew Brost , Zi Yan , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v4] mm/migrate_device: consolidate compound folio handling Date: Thu, 24 Sep 2026 13:12:10 +0900 Message-ID: <20260924041210.13d04b8be24495fc593665283d2e4453d449502f@163.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: _____wD3X9Y0o7RqWqn_AQ--.41231S2 X-Coremail-Antispam: 1Uf129KBjvJXoW3GFW3WFyDCF47Ww48uFWktFb_yoWxKFy3pF 40gasrJrZrWryjkr13Aa18Ar13urZ3Xa13GFZxGrn7uFZ8JFyfuw1Iq3Z5XFs8CrZ7XFyx Zay7tFy7Z3WDXaUanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x0piWlk3UUUUU= X-CM-SenderInfo: xvkbvvri6rljoofrz/xtbC6ha9HWq0ozZl7AAA3e Content-Type: text/plain; charset="utf-8" Commit dc41e961a269 ("mm/migrate_device: avoid out-of-bounds writes for compound folios") added handling for compound folios that do not fit in the remaining PFN array. migrate_device_range() and migrate_device_pfns() duplicate the logic for locking device PFNs, encoding compound folios, and handling this boundary condition. A compound folio cannot be represented partially for migration. Use WARN_ON_ONCE() when one does not fit in the remaining PFN array, while retaining the defensive handling: release the current folio's lock and reference, clear the remaining entries, and stop collecting. Move the shared collection and encoding logic into a helper so both interfaces handle compound folios consistently. Keep the reference, lock, size lookup, and PFN encoding together so cleanup acts on the folio whose reference and lock were acquired. Also use memset() for compound folio tail entries instead of open-coding the clearing loop. If a PFN cannot be referenced or its folio cannot be locked, leave that source entry clear and continue at page granularity. Compound tail pages have a zero page->_refcount, so folio_get_nontail_page() cannot acquire them. They are therefore skipped one slot at a time. This preserves collection of later allocated pages in a device chunk containing free PFNs. Document that an encountered compound folio must fit entirely in the remaining range or PFN array. Link: https://lore.kernel.org/r/c99ca53a-73ef-4a0c-8738-eba1cc89bea2@kernel= .org Link: https://lore.kernel.org/r/60634ab5-3fcc-42fb-8722-4ba6771acfda@kernel= .org Suggested-by: David Hildenbrand Signed-off-by: Hui Su --- Tested on x86_64 with KASAN: - 7/7 private-device HMM THP cases passed on the exact v4 tree. - Local KUnit validation covered success, get/trylock failure, and truncated compound-folio paths. Changes in v4: - Fold migrate_device_pfn_lock() into migrate_device_collect_folio() so the folio whose reference and lock are acquired is also used for size and cleanup decisions. - When a PFN cannot be referenced or its folio cannot be locked, leave only the current entry clear and continue at page granularity. Compound tails have a zero page->_refcount, so folio_get_nontail_page() cannot acquire t= hem. - Use WARN_ON_ONCE() with defensive cleanup when a compound folio does not fit in the remaining array, as suggested by David Hildenbrand. - Document that a compound folio occupies consecutive page-granular entries and must fit entirely in the remaining PFN array. Changes in v3: - Keep the source entry zero when migrate_device_pfn_lock() fails instead of setting MIGRATE_PFN_COMPOUND, while still consuming and clearing all slots belonging to the compound folio. - Use VM_WARN_ON_ONCE() for the truncated compound-folio invariant while keeping defensive cleanup independent of CONFIG_DEBUG_VM, as suggested by Balbir Singh. Changes in v2: - Fix the helper parameter indentation and keep the declaration to two lines, as suggested by David Hildenbrand. mm/migrate_device.c | 92 +++++++++++++++++++++++---------------------- 1 file changed, 48 insertions(+), 44 deletions(-) diff --git a/mm/migrate_device.c b/mm/migrate_device.c index 009bfa8b212d..34d687882d85 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -1376,20 +1376,47 @@ void migrate_vma_finalize(struct migrate_vma *migra= te) } EXPORT_SYMBOL(migrate_vma_finalize); =20 -static unsigned long migrate_device_pfn_lock(unsigned long pfn) +/* + * Collect a device folio into the page-granular PFN array. + * + * Return the number of entries consumed. Return 1 with a clear source ent= ry + * if the current PFN cannot be referenced or locked. Return 0 if a compou= nd + * folio does not fit in the remaining array and collection should stop. + */ +static unsigned int migrate_device_collect_folio(unsigned long *src_pfn, + unsigned long pfn, unsigned long remaining) { struct folio *folio; + unsigned int nr; + + *src_pfn =3D 0; =20 folio =3D folio_get_nontail_page(pfn_to_page(pfn)); if (!folio) - return 0; + return 1; =20 if (!folio_trylock(folio)) { folio_put(folio); + return 1; + } + + nr =3D folio_nr_pages(folio); + + if (WARN_ON_ONCE(nr > remaining)) { + folio_unlock(folio); + folio_put(folio); + memset(src_pfn, 0, remaining * sizeof(*src_pfn)); return 0; } =20 - return migrate_pfn(pfn) | MIGRATE_PFN_MIGRATE; + *src_pfn =3D migrate_pfn(pfn) | MIGRATE_PFN_MIGRATE; + + if (nr > 1) { + *src_pfn |=3D MIGRATE_PFN_COMPOUND; + memset(src_pfn + 1, 0, (nr - 1) * sizeof(*src_pfn)); + } + + return nr; } =20 /** @@ -1410,35 +1437,22 @@ static unsigned long migrate_device_pfn_lock(unsign= ed long pfn) * migrating pages that aren't free before unmapping them. Drivers may then * allocate destination pages and start copying data from the device to CPU * memory before calling migrate_device_pages(). + * + * A compound folio must fit entirely in the remaining range. */ int migrate_device_range(unsigned long *src_pfns, unsigned long start, unsigned long npages) { - unsigned long i, j, pfn; + unsigned long i, pfn; =20 for (pfn =3D start, i =3D 0; i < npages; pfn++, i++) { - struct page *page =3D pfn_to_page(pfn); - struct folio *folio =3D page_folio(page); - unsigned int nr =3D 1; + unsigned int nr; =20 - src_pfns[i] =3D migrate_device_pfn_lock(pfn); - nr =3D folio_nr_pages(folio); - if (nr > npages - i) { - if (src_pfns[i] & MIGRATE_PFN_MIGRATE) { - folio_unlock(folio); - folio_put(folio); - } - memset(&src_pfns[i], 0, - (npages - i) * sizeof(*src_pfns)); + nr =3D migrate_device_collect_folio(&src_pfns[i], pfn, npages - i); + if (!nr) break; - } - if (nr > 1) { - src_pfns[i] |=3D MIGRATE_PFN_COMPOUND; - for (j =3D 1; j < nr; j++) - src_pfns[i+j] =3D 0; - i +=3D j - 1; - pfn +=3D j - 1; - } + i +=3D nr - 1; + pfn +=3D nr - 1; } =20 migrate_device_unmap(src_pfns, npages, NULL); @@ -1454,33 +1468,23 @@ EXPORT_SYMBOL(migrate_device_range); * * Similar to migrate_device_range() but supports non-contiguous pre-popul= ated * array of device pages to migrate. + * + * Entries for different folios may be non-contiguous, but a compound folio + * must occupy consecutive page-granular entries and fit entirely in the + * remaining PFN array. */ int migrate_device_pfns(unsigned long *src_pfns, unsigned long npages) { - unsigned long i, j; + unsigned long i; =20 for (i =3D 0; i < npages; i++) { - struct page *page =3D pfn_to_page(src_pfns[i]); - struct folio *folio =3D page_folio(page); - unsigned int nr =3D 1; + unsigned long pfn =3D src_pfns[i]; + unsigned int nr; =20 - src_pfns[i] =3D migrate_device_pfn_lock(src_pfns[i]); - nr =3D folio_nr_pages(folio); - if (nr > npages - i) { - if (src_pfns[i] & MIGRATE_PFN_MIGRATE) { - folio_unlock(folio); - folio_put(folio); - } - memset(&src_pfns[i], 0, - (npages - i) * sizeof(*src_pfns)); + nr =3D migrate_device_collect_folio(&src_pfns[i], pfn, npages - i); + if (!nr) break; - } - if (nr > 1) { - src_pfns[i] |=3D MIGRATE_PFN_COMPOUND; - for (j =3D 1; j < nr; j++) - src_pfns[i+j] =3D 0; - i +=3D j - 1; - } + i +=3D nr - 1; } =20 migrate_device_unmap(src_pfns, npages, NULL); base-commit: fe2ec83746e501645709761605c2464a44fd2929 --=20 2.55.0