From nobody Tue Feb 10 15:43:36 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9C482505DC for ; Mon, 10 Feb 2025 19:39:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739216347; cv=none; b=U5XUfyJI2DiCM2qiCpl4kDy8mLV+ymmtVrxKMBxy22JH5pRTgGgRVeVvNyq2mqZHd0QIFHNVK7dKbtqf9R9WVKePCtwPk8FT5AUoZ8wwZVK3qON+rvk4z7F4l9F7gbO40mMINPNfXZKqfB6Kf4OegB+GJDxTc/4HFWGN04bP2Zw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739216347; c=relaxed/simple; bh=eeTQic3U3Pz3ShuuvH4rmh3qdHr2tcndU+mn29E8LNI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BiJnbeRQZzA+uPVoJ4eAJvvBq6VF4qlR4fHdjYzUC1r5ZTmcwEVZizTJHwJD/K0jOsQVLkPGF92nFwJHg3C30r4/eoi/hb/tJBwq1Q33qaNM888IxAVE7O0c3JWjtgTDPt/0qdi1WBH/t+zs6mQlsX8u9ERB0xts+93PztfwOEU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=GIf9EIBn; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="GIf9EIBn" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1739216344; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oNpxFEEEyxoH6lwF3K+DQE9+aA/s7jzY02gjXCRiBPg=; b=GIf9EIBn800qN8VHy/2XREBmN6R5F0adch97q/LfMYsUscssVKg/l3JjSI6U+i/GbpwRES rIhLWnGtc310tcF1GdqaNnsWkeGLJ/zonzpPLva9jrv5Ewy2ckvxNAXf2wH430W5PNaHvS 7ag4kxrkQJt81nwFebuCflIHl4volCc= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-324-pcfXIJzPMnaBcKq21PO25Q-1; Mon, 10 Feb 2025 14:39:03 -0500 X-MC-Unique: pcfXIJzPMnaBcKq21PO25Q-1 X-Mimecast-MFC-AGG-ID: pcfXIJzPMnaBcKq21PO25Q Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-38ddee8329eso708600f8f.1 for ; Mon, 10 Feb 2025 11:39:03 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739216342; x=1739821142; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oNpxFEEEyxoH6lwF3K+DQE9+aA/s7jzY02gjXCRiBPg=; b=PivDaOEY9Y/uaiwsbex23V0awlRJDhIMIHgLLUFDiyOkev71BpRNH/YLDqcnDdCJ3Q VIkIGmqYy8VDM3Kii8xeE5El78KNKybi0SQD04B0rrNfhuVzQqmfTx8sbF3hri7n7TeQ gLOQ/Sph83szSldHVcSX6l3zrXbv/fu3aN2pQtGLEQRbyC+pgB5VQULVcL8C84/4Sq8t N/bXJCpcu+haYe8/A5s06bR4MIkUh6nE6aao4i8k6QhR6yfOdTp9scucMmtt6f59K6hc xd36A5hJNWeAKpoGL7VS8MpnhGsvU6T+z86bzWKZZN0hMUiWkc69v31dBQggOAG9fz++ 6MrQ== X-Gm-Message-State: AOJu0Ywle9SuviiTY29Jby/FL0bjXlr4qrjsK+zCdZp1paIUMW2Y3g82 PtN4JTFVvzRYk085BtcO6jUhrwSGBAur5y8TkW8ApFdj0xP8dvfCB7Qt6EoqHIXLRroNCa8GNhG i3dL2Y3RETUtApyE5GaWVPI5peYCI7LhoUMbjBj1deN8IU13k+J1AM3K636SbRb5m6LA7NG1uzt dnh8xIBM+tNU/r0fQatizDkClT1+lPkoA2BYxDOpkBiM/Z X-Gm-Gg: ASbGncsbwuw8P69zDR0HNRCsTSU3bpWT0yl6rr68/DoZhz99IkN9iNzKud4pKU7wxce xMvs3NFTa3lhYVnuc7UpxsTlzBEb+VH8Iv4J9FNV0+nLHZ1FRNCJCJG12WW0sfvW9QrITD6Y5Qi 2cQWmskV1C2lSGZ8AHxhzHzBIvc2gfr1omD6dHtsTiKTPhc+eb8cSgTZ6b4l+X5Kkn9U7AM5ehs nhtke9cmorQvWB65Ga1kfAusYNO7HNfO8c/8yHD0PR2HCu+rYQ6hkSKqFAtQrObnb4qlAa27kUh IbT0PJcw8tB4pJDd618ydQiKG1KPE82G66ZB9DhcIpryDrXjwsLx8cP2dfpLxBTayg== X-Received: by 2002:a05:6000:1813:b0:38a:418e:21c7 with SMTP id ffacd0b85a97d-38dc935246fmr8277038f8f.53.1739216342203; Mon, 10 Feb 2025 11:39:02 -0800 (PST) X-Google-Smtp-Source: AGHT+IHTiiTPkwAyfuLx0qL+LO6PapdaXuVNjUwBeGg/Z0ah/0RffyIckymQJ3LkKa1NUzxInLX6yQ== X-Received: by 2002:a05:6000:1813:b0:38a:418e:21c7 with SMTP id ffacd0b85a97d-38dc935246fmr8276996f8f.53.1739216341643; Mon, 10 Feb 2025 11:39:01 -0800 (PST) Received: from localhost (p200300cbc734b80012c465cd348aaee6.dip0.t-ipconnect.de. [2003:cb:c734:b800:12c4:65cd:348a:aee6]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-38dc4d00645sm11916376f8f.66.2025.02.10.11.38.58 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 10 Feb 2025 11:39:00 -0800 (PST) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-doc@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-mm@kvack.org, nouveau@lists.freedesktop.org, linux-trace-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, damon@lists.linux.dev, David Hildenbrand , Andrew Morton , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Jonathan Corbet , Alex Shi , Yanteng Si , Karol Herbst , Lyude Paul , Danilo Krummrich , David Airlie , Simona Vetter , Masami Hiramatsu , Oleg Nesterov , Peter Zijlstra , SeongJae Park , "Liam R. Howlett" , Lorenzo Stoakes , Vlastimil Babka , Jann Horn , Pasha Tatashin , Peter Xu , Alistair Popple , Jason Gunthorpe Subject: [PATCH v2 15/17] mm/damon: handle device-exclusive entries correctly in damon_folio_mkold_one() Date: Mon, 10 Feb 2025 20:37:57 +0100 Message-ID: <20250210193801.781278-16-david@redhat.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250210193801.781278-1-david@redhat.com> References: <20250210193801.781278-1-david@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Ever since commit b756a3b5e7ea ("mm: device exclusive memory access") we can return with a device-exclusive entry from page_vma_mapped_walk(). damon_folio_mkold_one() is not prepared for that and calls damon_ptep_mkold() with PFN swap PTEs. Teach damon_ptep_mkold() to deal with these PFN swap PTEs. Note that device-private entries are so far not applicable on that path, as damon_get_folio() filters out non-lru folios. Should we just skip PFN swap PTEs completely? Possible, but it seems straight forward to just handle it correctly. Note that we could currently only run into this case with device-exclusive entries on THPs. We still adjust the mapcount on conversion to device-exclusive; this makes the rmap walk abort early for small folios, because we'll always have !folio_mapped() with a single device-exclusive entry. We'll adjust the mapcount logic once all page_vma_mapped_walk() users can properly handle device-exclusive entries. Signed-off-by: David Hildenbrand Reviewed-by: SeongJae Park --- mm/damon/ops-common.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index d25d99cb5f2bb..86a50e8fbc806 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -9,6 +9,8 @@ #include #include #include +#include +#include =20 #include "ops-common.h" =20 @@ -39,12 +41,29 @@ struct folio *damon_get_folio(unsigned long pfn) =20 void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned lon= g addr) { - struct folio *folio =3D damon_get_folio(pte_pfn(ptep_get(pte))); + pte_t pteval =3D ptep_get(pte); + struct folio *folio; + bool young =3D false; + unsigned long pfn; + + if (likely(pte_present(pteval))) + pfn =3D pte_pfn(pteval); + else + pfn =3D swp_offset_pfn(pte_to_swp_entry(pteval)); =20 + folio =3D damon_get_folio(pfn); if (!folio) return; =20 - if (ptep_clear_young_notify(vma, addr, pte)) + /* + * PFN swap PTEs, such as device-exclusive ones, that actually map pages + * are "old" from a CPU perspective. The MMU notifier takes care of any + * device aspects. + */ + if (likely(pte_present(pteval))) + young |=3D ptep_test_and_clear_young(vma, addr, pte); + young |=3D mmu_notifier_clear_young(vma->vm_mm, addr, addr + PAGE_SIZE); + if (young) folio_set_young(folio); =20 folio_set_idle(folio); --=20 2.48.1