From nobody Sat Feb 7 11:42:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6693F4D9F2; Sun, 12 May 2024 17:54:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715536495; cv=none; b=ZY/XV/9k4/mittFtm6m//c65fm5a0ibL8Qr5TtOYk7Fmfcx6gwKfWQ+GQMELNhRIMRCz2RxPBPUTrFkTc1mgXAQk1IUmDzCVk7BbJaQ6GJEcfjky2xdNSUUs/Tz3S3UQt6q4tjrFXBVz2Krkao8e80YflqiBjcJWOSF6PuaQrfo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715536495; c=relaxed/simple; bh=dAqk9DDy2iHhz/atR+7qN/JQ2SHCLI1jeIZ9SIxQ32U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=K+FgJ61MNBHeBOP6CCBR1dcybslWAvjCOrYhF7Mjpq9PkZYYEgv+Aad2xr+rc9tsMwdimTsqkua9h3n6TgPqSTk4bhZvwsEWnXSkRBB33aeuch7YUMrxd92gPx3m63U36dcUaogqBS3XGA1+ML2yiVPER5z1dZl+SbUUGJtp+wE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YzfZLIO0; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YzfZLIO0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 83FBFC4AF07; Sun, 12 May 2024 17:54:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715536495; bh=dAqk9DDy2iHhz/atR+7qN/JQ2SHCLI1jeIZ9SIxQ32U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YzfZLIO0xU4xk2uGqYmJFr7f3seXBey3bXrqIjZdiw/SFtMH+fG1I9q4CFgFopxqd IVCnp0TttZkN0fqtW1I0HRDOz9zY7Hqm+1AvNaxbdN/NJXc7SOWQOtXo2Ro+ts6agQ t0HwAzSIReX9f0lWCnagZvNE5sfntr6LCRIzsAUroGijBR6Pxq2vnA9Y4IS9H2QvAG L1cQBBw0OLhgd7HoABJbPLpial7xLk3jDLhb3Z1BR9aYlaIGTYdAWCcoHKkSnrIGyA yJRHG4yG8JyLOix4tNXkKsQHuWXDuTQcrpMKOM/c6OV6IW9l/86wjhbPIkQu181ai+ HaBDUF3ADFFGw== From: SeongJae Park To: Cc: Honggyu Kim , Andrew Morton , linux-mm@kvack.org, damon@lists.linux.dev, linux-kernel@vger.kernel.org, SeongJae Park Subject: [RFC PATCH v4 1/5] mm: make alloc_demote_folio externally invokable for migration Date: Sun, 12 May 2024 10:54:43 -0700 Message-Id: <20240512175447.75943-2-sj@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240512175447.75943-1-sj@kernel.org> References: <20240512175447.75943-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Honggyu Kim The alloc_demote_folio can be used out of vmscan.c so it'd be better to remove static keyword from it. This function can also be used for both demotion and promotion so it'd be better to rename it from alloc_demote_folio to alloc_migrate_folio. Signed-off-by: Honggyu Kim Reviewed-by: SeongJae Park Signed-off-by: SeongJae Park --- mm/internal.h | 1 + mm/vmscan.c | 3 +-- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index b2c75b12014e..b3ca996a4efc 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1052,6 +1052,7 @@ extern unsigned long __must_check vm_mmap_pgoff(stru= ct file *, unsigned long, unsigned long, unsigned long); =20 extern void set_pageblock_order(void); +struct folio *alloc_demote_folio(struct folio *src, unsigned long private); unsigned long reclaim_pages(struct list_head *folio_list); unsigned int reclaim_clean_pages_from_list(struct zone *zone, struct list_head *folio_list); diff --git a/mm/vmscan.c b/mm/vmscan.c index 6981a71c8ef0..27269bc2bcc5 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -933,8 +933,7 @@ static void folio_check_dirty_writeback(struct folio *f= olio, mapping->a_ops->is_dirty_writeback(folio, dirty, writeback); } =20 -static struct folio *alloc_demote_folio(struct folio *src, - unsigned long private) +struct folio *alloc_demote_folio(struct folio *src, unsigned long private) { struct folio *dst; nodemask_t *allowed_mask; --=20 2.39.2 From nobody Sat Feb 7 11:42:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E6EC4F205; Sun, 12 May 2024 17:54:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715536496; cv=none; b=YIalF5Ef2j7+ajaVHSWpNXrwiGT/9USiYc/6PxtgqK/rleJpwAv+fY0snALAJ4jshBUGcrPQRpA6jgIyN/HlzaRs+vghpqmpPaayonjERKLUhscYxbk0hEQschw5Nntw29AuC1douXUOkpB8gum4NMam6hixI1ISv4lg0cUaYZY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715536496; c=relaxed/simple; bh=vBax8IQjCZahbUBgsA8N3gO+pf43TzfEVXtgesAQ6bE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XDuUHui7sxa+9NjiuxNPZ6NluyEejx2HEth7kUn8Fm3zNxfruKGeCW3Fs6xTBHWE2DQQ/SghvWw03aG9I/AH5HCf9+DPkN2EzVlJMUVLL4mWdd76lZ4be6E7EcXm1E6FWxJdZ907Q1tZZTUMJYpmikktqgxBPmsLWlr8h50y6Ao= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lyBAV9AV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lyBAV9AV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2FABCC116B1; Sun, 12 May 2024 17:54:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715536495; bh=vBax8IQjCZahbUBgsA8N3gO+pf43TzfEVXtgesAQ6bE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lyBAV9AVGCq6yXdy/aW3PmZ8JWsV8XngJMaFr7osxhdC0ekeLwPTvVw6Wiyg+PCPu eRdHFPEp6poLsUEDvio7nw1h6oX84fhBy8yyV3VTGioQRBvofhvpNZ/cr+QUwZt4Rm pUvI4R9YYYfVpasOVMitrr2pQ84mVKGNXWtcqGNsBlyTzKdaXWqP9V/AytmDCZhrSY XlD98FWNAgZwMEllxKOh3k50efUoMsnKpn/bHWSM/1hMMb78X7ec/04u6ZTypiJYrd J3r0o3mEJc4aPWXNhaE0GUdgdX6YF5L7DmnS1X8HpAe3KmPcqsp01gq1vNl4lYLwCA EOT9JC90/lCDw== From: SeongJae Park To: Cc: Hyeongtak Ji , SeongJae Park , Andrew Morton , damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Honggyu Kim Subject: [RFC PATCH v4 2/5] mm/damon/sysfs-schemes: add target_nid on sysfs-schemes Date: Sun, 12 May 2024 10:54:44 -0700 Message-Id: <20240512175447.75943-3-sj@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240512175447.75943-1-sj@kernel.org> References: <20240512175447.75943-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeongtak Ji This patch adds target_nid under /sys/kernel/mm/damon/admin/kdamonds//contexts//schemes// The 'target_nid' can be used as the destination node for DAMOS actions such as DAMOS_MIGRATE_{HOT,COLD} in the follow up patches. Signed-off-by: Hyeongtak Ji Signed-off-by: Honggyu Kim Signed-off-by: SeongJae Park --- include/linux/damon.h | 11 ++++++++++- mm/damon/core.c | 5 ++++- mm/damon/dbgfs.c | 2 +- mm/damon/lru_sort.c | 3 ++- mm/damon/reclaim.c | 3 ++- mm/damon/sysfs-schemes.c | 33 ++++++++++++++++++++++++++++++++- 6 files changed, 51 insertions(+), 6 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index f7da65e1ac04..21d6b69a015c 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -374,6 +374,7 @@ struct damos_access_pattern { * @apply_interval_us: The time between applying the @action. * @quota: Control the aggressiveness of this scheme. * @wmarks: Watermarks for automated (in)activation of this scheme. + * @target_nid: Destination node if @action is "migrate_{hot,cold}". * @filters: Additional set of &struct damos_filter for &action. * @stat: Statistics of this scheme. * @list: List head for siblings. @@ -389,6 +390,10 @@ struct damos_access_pattern { * monitoring context are inactive, DAMON stops monitoring either, and just * repeatedly checks the watermarks. * + * @target_nid is used to set the migration target node for migrate_hot or + * migrate_cold actions, which means it's only meaningful when @action is = either + * "migrate_hot" or "migrate_cold". + * * Before applying the &action to a memory region, &struct damon_operations * implementation could check pages of the region and skip &action to resp= ect * &filters @@ -410,6 +415,9 @@ struct damos { /* public: */ struct damos_quota quota; struct damos_watermarks wmarks; + union { + int target_nid; + }; struct list_head filters; struct damos_stat stat; struct list_head list; @@ -726,7 +734,8 @@ struct damos *damon_new_scheme(struct damos_access_patt= ern *pattern, enum damos_action action, unsigned long apply_interval_us, struct damos_quota *quota, - struct damos_watermarks *wmarks); + struct damos_watermarks *wmarks, + int target_nid); void damon_add_scheme(struct damon_ctx *ctx, struct damos *s); void damon_destroy_scheme(struct damos *s); =20 diff --git a/mm/damon/core.c b/mm/damon/core.c index e1a0c2a11007..32c261aeff88 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -364,7 +364,8 @@ struct damos *damon_new_scheme(struct damos_access_patt= ern *pattern, enum damos_action action, unsigned long apply_interval_us, struct damos_quota *quota, - struct damos_watermarks *wmarks) + struct damos_watermarks *wmarks, + int target_nid) { struct damos *scheme; =20 @@ -391,6 +392,8 @@ struct damos *damon_new_scheme(struct damos_access_patt= ern *pattern, scheme->wmarks =3D *wmarks; scheme->wmarks.activated =3D true; =20 + scheme->target_nid =3D target_nid; + return scheme; } =20 diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index 2461cfe2e968..51a6f1cac385 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -281,7 +281,7 @@ static struct damos **str_to_schemes(const char *str, s= size_t len, =20 pos +=3D parsed; scheme =3D damon_new_scheme(&pattern, action, 0, "a, - &wmarks); + &wmarks, NUMA_NO_NODE); if (!scheme) goto fail; =20 diff --git a/mm/damon/lru_sort.c b/mm/damon/lru_sort.c index 3de2916a65c3..3775f0f2743d 100644 --- a/mm/damon/lru_sort.c +++ b/mm/damon/lru_sort.c @@ -163,7 +163,8 @@ static struct damos *damon_lru_sort_new_scheme( /* under the quota. */ "a, /* (De)activate this according to the watermarks. */ - &damon_lru_sort_wmarks); + &damon_lru_sort_wmarks, + NUMA_NO_NODE); } =20 /* Create a DAMON-based operation scheme for hot memory regions */ diff --git a/mm/damon/reclaim.c b/mm/damon/reclaim.c index 9bd341d62b4c..a05ccb41749b 100644 --- a/mm/damon/reclaim.c +++ b/mm/damon/reclaim.c @@ -177,7 +177,8 @@ static struct damos *damon_reclaim_new_scheme(void) /* under the quota. */ &damon_reclaim_quota, /* (De)activate this according to the watermarks. */ - &damon_reclaim_wmarks); + &damon_reclaim_wmarks, + NUMA_NO_NODE); } =20 static void damon_reclaim_copy_quota_status(struct damos_quota *dst, diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index bea5bc52846a..0632d28b67f8 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -6,6 +6,7 @@ */ =20 #include +#include =20 #include "sysfs-common.h" =20 @@ -1445,6 +1446,7 @@ struct damon_sysfs_scheme { struct damon_sysfs_scheme_filters *filters; struct damon_sysfs_stats *stats; struct damon_sysfs_scheme_regions *tried_regions; + int target_nid; }; =20 /* This should match with enum damos_action */ @@ -1470,6 +1472,7 @@ static struct damon_sysfs_scheme *damon_sysfs_scheme_= alloc( scheme->kobj =3D (struct kobject){}; scheme->action =3D action; scheme->apply_interval_us =3D apply_interval_us; + scheme->target_nid =3D NUMA_NO_NODE; return scheme; } =20 @@ -1692,6 +1695,28 @@ static ssize_t apply_interval_us_store(struct kobjec= t *kobj, return err ? err : count; } =20 +static ssize_t target_nid_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + struct damon_sysfs_scheme *scheme =3D container_of(kobj, + struct damon_sysfs_scheme, kobj); + + return sysfs_emit(buf, "%d\n", scheme->target_nid); +} + +static ssize_t target_nid_store(struct kobject *kobj, + struct kobj_attribute *attr, const char *buf, size_t count) +{ + struct damon_sysfs_scheme *scheme =3D container_of(kobj, + struct damon_sysfs_scheme, kobj); + int err =3D 0; + + /* TODO: error handling for target_nid range. */ + err =3D kstrtoint(buf, 0, &scheme->target_nid); + + return err ? err : count; +} + static void damon_sysfs_scheme_release(struct kobject *kobj) { kfree(container_of(kobj, struct damon_sysfs_scheme, kobj)); @@ -1703,9 +1728,13 @@ static struct kobj_attribute damon_sysfs_scheme_acti= on_attr =3D static struct kobj_attribute damon_sysfs_scheme_apply_interval_us_attr =3D __ATTR_RW_MODE(apply_interval_us, 0600); =20 +static struct kobj_attribute damon_sysfs_scheme_target_nid_attr =3D + __ATTR_RW_MODE(target_nid, 0600); + static struct attribute *damon_sysfs_scheme_attrs[] =3D { &damon_sysfs_scheme_action_attr.attr, &damon_sysfs_scheme_apply_interval_us_attr.attr, + &damon_sysfs_scheme_target_nid_attr.attr, NULL, }; ATTRIBUTE_GROUPS(damon_sysfs_scheme); @@ -2031,7 +2060,8 @@ static struct damos *damon_sysfs_mk_scheme( }; =20 scheme =3D damon_new_scheme(&pattern, sysfs_scheme->action, - sysfs_scheme->apply_interval_us, "a, &wmarks); + sysfs_scheme->apply_interval_us, "a, &wmarks, + sysfs_scheme->target_nid); if (!scheme) return NULL; =20 @@ -2068,6 +2098,7 @@ static void damon_sysfs_update_scheme(struct damos *s= cheme, =20 scheme->action =3D sysfs_scheme->action; scheme->apply_interval_us =3D sysfs_scheme->apply_interval_us; + scheme->target_nid =3D sysfs_scheme->target_nid; =20 scheme->quota.ms =3D sysfs_quotas->ms; scheme->quota.sz =3D sysfs_quotas->sz; --=20 2.39.2 From nobody Sat Feb 7 11:42:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28DD25028C; Sun, 12 May 2024 17:54:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715536497; cv=none; b=B+/vGS4oU5ZFjkCICzkJWoHTWm0JYA8ax97CHY8U4oZD9/LWx2IZZMDSlgh1u7GMrOTnIHZ73NrZQPGceTbHhwCAHpC7qu+G7Y/59toMQI5a82k1oCkIMEAirRmYoLmwLzqM39pDXH5xQm0DDrJrwuT/SUtHJz+YSh+QhEC0+cU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715536497; c=relaxed/simple; bh=0nYpdJNOcVsAetSwCRBcYWpfg7K4GaKZMal/LiW2aDo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Xop1mDoxZy+RAxFYZgzr1B/9/kQk0lRwvjf8bsCB6i8n/xf9X4H8yTArNUWncOCntj7jHXW7gqzCjr8f24RT0lakobIiqp+eyu9F050ypB7WHA9VnBxD/38w5cb1uuWLeGMNqczkhWkVc6ev6vjJ6K6aAA2yfQtlB1op04RE6yw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZiCeiPqc; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZiCeiPqc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EFC4CC4AF08; Sun, 12 May 2024 17:54:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715536496; bh=0nYpdJNOcVsAetSwCRBcYWpfg7K4GaKZMal/LiW2aDo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZiCeiPqcqV3HR6k9NGEGmv4MbUCDzmuxGtuizXoVn0cPcZ2oH7D0dAD2qaL9JevbA 2/PofgQ5iNSARyYqrbxl48ZFFfT0CGlMaaT0tgpiFcmpGWnUYnlZsmMxAnBupiQA5B 8ZUMqtWE2Gpr66DPiyuAbwLu///LkQbyiD29BfRtrL40kVnknmVPavydG9qYWh6U+E AINgYWGQP/LzlQjCbobJc7kgKHZeh8a4B1Y2QTxPaqPnmTu4n4hmdswrnyM1A3JsK9 SWEZD9x+rYdlDmywxX7FvHgs7UlmCh8pgn5y2ZzCwA02OkVzGfOjj22voQEi+7xy7x ZuQzZOOofKOIg== From: SeongJae Park To: Cc: Honggyu Kim , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , linux-kernel@vger.kernel.org, damon@lists.linux.dev, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, SeongJae Park Subject: [RFC PATCH v4 3/5] mm/migrate: add MR_DAMON to migrate_reason Date: Sun, 12 May 2024 10:54:45 -0700 Message-Id: <20240512175447.75943-4-sj@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240512175447.75943-1-sj@kernel.org> References: <20240512175447.75943-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Honggyu Kim The current patch series introduces DAMON based migration across NUMA nodes so it'd be better to have a new migrate_reason in trace events. Signed-off-by: Honggyu Kim Reviewed-by: SeongJae Park Signed-off-by: SeongJae Park --- include/linux/migrate_mode.h | 1 + include/trace/events/migrate.h | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/include/linux/migrate_mode.h b/include/linux/migrate_mode.h index f37cc03f9369..cec36b7e7ced 100644 --- a/include/linux/migrate_mode.h +++ b/include/linux/migrate_mode.h @@ -29,6 +29,7 @@ enum migrate_reason { MR_CONTIG_RANGE, MR_LONGTERM_PIN, MR_DEMOTION, + MR_DAMON, MR_TYPES }; =20 diff --git a/include/trace/events/migrate.h b/include/trace/events/migrate.h index 0190ef725b43..cd01dd7b3640 100644 --- a/include/trace/events/migrate.h +++ b/include/trace/events/migrate.h @@ -22,7 +22,8 @@ EM( MR_NUMA_MISPLACED, "numa_misplaced") \ EM( MR_CONTIG_RANGE, "contig_range") \ EM( MR_LONGTERM_PIN, "longterm_pin") \ - EMe(MR_DEMOTION, "demotion") + EM( MR_DEMOTION, "demotion") \ + EMe(MR_DAMON, "damon") =20 /* * First define the enums in the above macros to be exported to userspace --=20 2.39.2 From nobody Sat Feb 7 11:42:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D0393524A6; Sun, 12 May 2024 17:54:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715536497; cv=none; b=XrSNfi7UozdHOzX3MR4ffNa3t6WkOKXTlXF7jadv5jI27vSTJUzjYEjQ0vUVuOvSZCdhVuWnEA1HsDoUtWifMFeZKqC++JUZ3rsyA6cQiasIC4BR9LvY8hJT9OHrStsJzR7iwzSk7l7jMY4BQdoH2NlNytKspaQ+1X/8lqu9LDA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715536497; c=relaxed/simple; bh=pJ6vzvCtZUPWSFq3LZUFfl+zSubCIvDIafn053zECpU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=csRQUqaP515vWa2mgUoB5mxmc0oOoHy64v9jVOlFQRvbOlZCzVuNqNQa6XBfcQ4WXMFXbFqqzu+TIhRTqCVzTuFvbW30kr7T21ybFg3A07tCU9HyVvmoXa/162XVD2idi+nwmIEUkOx/TLJCSJhhRzvnzYAFjf2fwPXcCEROPuo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Cc70FMlK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Cc70FMlK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CC9FFC4AF07; Sun, 12 May 2024 17:54:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715536497; bh=pJ6vzvCtZUPWSFq3LZUFfl+zSubCIvDIafn053zECpU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Cc70FMlKxLi1JuD51HmpqEsx8453y79uRyuCpKWF1nFmbmpT3wquNTxEgmigdAGeP vOBagAGtYXvLeuZDb7OAbeB7Bg/rsRmffuka3NZMTyvrPgBuKUCuH6zS64yc8uIegg eo9hlLQMmqvopMR8rbRj2YnhWwtjtZotQ5AgJi26Y5BHdKGHt7b7iqvcnrIV+0XH5h 5Ay0c7jBCrdo0JwwpVwLuqqcqJQx6Ur+54tZHTranalXvTOFgw1NHZQYObYyribLXj HySwsRHduSQvTrYwk/d08i69oLkf8HWJkllCj9u+M2yPaM6ABrwmmwrU9Wo0GCzIZN 3zOv8UL9vgZ0w== From: SeongJae Park To: Cc: Honggyu Kim , SeongJae Park , Andrew Morton , damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Hyeongtak Ji Subject: [RFC PATCH v4 4/5] mm/damon/paddr: introduce DAMOS_MIGRATE_COLD action for demotion Date: Sun, 12 May 2024 10:54:46 -0700 Message-Id: <20240512175447.75943-5-sj@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240512175447.75943-1-sj@kernel.org> References: <20240512175447.75943-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Honggyu Kim This patch introduces DAMOS_MIGRATE_COLD action, which is similar to DAMOS_PAGEOUT, but migrate folios to the given 'target_nid' in the sysfs instead of swapping them out. The 'target_nid' sysfs knob informs the migration target node ID. Here is one of the example usage of this 'migrate_cold' action. $ cd /sys/kernel/mm/damon/admin/kdamonds/ $ cat contexts//schemes//action migrate_cold $ echo 2 > contexts//schemes//target_nid $ echo commit > state $ numactl -p 0 ./hot_cold 500M 600M & $ numastat -c -p hot_cold Per-node process memory usage (in MBs) PID Node 0 Node 1 Node 2 Total -------------- ------ ------ ------ ----- 701 (hot_cold) 501 0 601 1101 Since there are some common routines with pageout, many functions have similar logics between pageout and migrate cold. damon_pa_migrate_folio_list() is a minimized version of shrink_folio_list(). Signed-off-by: Honggyu Kim Signed-off-by: Hyeongtak Ji Signed-off-by: SeongJae Park --- include/linux/damon.h | 2 + mm/damon/paddr.c | 154 +++++++++++++++++++++++++++++++++++++++ mm/damon/sysfs-schemes.c | 1 + 3 files changed, 157 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index 21d6b69a015c..56714b6eb0d7 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -105,6 +105,7 @@ struct damon_target { * @DAMOS_NOHUGEPAGE: Call ``madvise()`` for the region with MADV_NOHUGEPA= GE. * @DAMOS_LRU_PRIO: Prioritize the region on its LRU lists. * @DAMOS_LRU_DEPRIO: Deprioritize the region on its LRU lists. + * @DAMOS_MIGRATE_COLD: Migrate the regions prioritizing colder regions. * @DAMOS_STAT: Do nothing but count the stat. * @NR_DAMOS_ACTIONS: Total number of DAMOS actions * @@ -122,6 +123,7 @@ enum damos_action { DAMOS_NOHUGEPAGE, DAMOS_LRU_PRIO, DAMOS_LRU_DEPRIO, + DAMOS_MIGRATE_COLD, DAMOS_STAT, /* Do nothing but only record the stat */ NR_DAMOS_ACTIONS, }; diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 18797c1b419b..a468792d9162 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -12,6 +12,9 @@ #include #include #include +#include +#include +#include =20 #include "../internal.h" #include "ops-common.h" @@ -325,6 +328,153 @@ static unsigned long damon_pa_deactivate_pages(struct= damon_region *r, return damon_pa_mark_accessed_or_deactivate(r, s, false); } =20 +static unsigned int __damon_pa_migrate_folio_list( + struct list_head *migrate_folios, struct pglist_data *pgdat, + int target_nid) +{ + unsigned int nr_succeeded; + nodemask_t allowed_mask =3D NODE_MASK_NONE; + struct migration_target_control mtc =3D { + /* + * Allocate from 'node', or fail quickly and quietly. + * When this happens, 'page' will likely just be discarded + * instead of migrated. + */ + .gfp_mask =3D (GFP_HIGHUSER_MOVABLE & ~__GFP_RECLAIM) | + __GFP_NOWARN | __GFP_NOMEMALLOC | GFP_NOWAIT, + .nid =3D target_nid, + .nmask =3D &allowed_mask + }; + + if (pgdat->node_id =3D=3D target_nid || target_nid =3D=3D NUMA_NO_NODE) + return 0; + + if (list_empty(migrate_folios)) + return 0; + + /* Migration ignores all cpuset and mempolicy settings */ + migrate_pages(migrate_folios, alloc_demote_folio, NULL, + (unsigned long)&mtc, MIGRATE_ASYNC, MR_DAMON, + &nr_succeeded); + + return nr_succeeded; +} + +static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_li= st, + struct pglist_data *pgdat, + int target_nid) +{ + unsigned int nr_migrated =3D 0; + struct folio *folio; + LIST_HEAD(ret_folios); + LIST_HEAD(migrate_folios); + + while (!list_empty(folio_list)) { + struct folio *folio; + + cond_resched(); + + folio =3D lru_to_folio(folio_list); + list_del(&folio->lru); + + if (!folio_trylock(folio)) + goto keep; + + /* Relocate its contents to another node. */ + list_add(&folio->lru, &migrate_folios); + folio_unlock(folio); + continue; +keep: + list_add(&folio->lru, &ret_folios); + } + /* 'folio_list' is always empty here */ + + /* Migrate folios selected for migration */ + nr_migrated +=3D __damon_pa_migrate_folio_list( + &migrate_folios, pgdat, target_nid); + /* + * Folios that could not be migrated are still in @migrate_folios. Add + * those back on @folio_list + */ + if (!list_empty(&migrate_folios)) + list_splice_init(&migrate_folios, folio_list); + + try_to_unmap_flush(); + + list_splice(&ret_folios, folio_list); + + while (!list_empty(folio_list)) { + folio =3D lru_to_folio(folio_list); + list_del(&folio->lru); + folio_putback_lru(folio); + } + + return nr_migrated; +} + +static unsigned long damon_pa_migrate_pages(struct list_head *folio_list, + int target_nid) +{ + int nid; + unsigned long nr_migrated =3D 0; + LIST_HEAD(node_folio_list); + unsigned int noreclaim_flag; + + if (list_empty(folio_list)) + return nr_migrated; + + noreclaim_flag =3D memalloc_noreclaim_save(); + + nid =3D folio_nid(lru_to_folio(folio_list)); + do { + struct folio *folio =3D lru_to_folio(folio_list); + + if (nid =3D=3D folio_nid(folio)) { + list_move(&folio->lru, &node_folio_list); + continue; + } + + nr_migrated +=3D damon_pa_migrate_folio_list(&node_folio_list, + NODE_DATA(nid), + target_nid); + nid =3D folio_nid(lru_to_folio(folio_list)); + } while (!list_empty(folio_list)); + + nr_migrated +=3D damon_pa_migrate_folio_list(&node_folio_list, + NODE_DATA(nid), + target_nid); + + memalloc_noreclaim_restore(noreclaim_flag); + + return nr_migrated; +} + +static unsigned long damon_pa_migrate(struct damon_region *r, struct damos= *s) +{ + unsigned long addr, applied; + LIST_HEAD(folio_list); + + for (addr =3D r->ar.start; addr < r->ar.end; addr +=3D PAGE_SIZE) { + struct folio *folio =3D damon_get_folio(PHYS_PFN(addr)); + + if (!folio) + continue; + + if (damos_pa_filter_out(s, folio)) + goto put_folio; + + if (!folio_isolate_lru(folio)) + goto put_folio; + list_add(&folio->lru, &folio_list); +put_folio: + folio_put(folio); + } + applied =3D damon_pa_migrate_pages(&folio_list, s->target_nid); + cond_resched(); + return applied * PAGE_SIZE; +} + + static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, struct damon_target *t, struct damon_region *r, struct damos *scheme) @@ -336,6 +486,8 @@ static unsigned long damon_pa_apply_scheme(struct damon= _ctx *ctx, return damon_pa_mark_accessed(r, scheme); case DAMOS_LRU_DEPRIO: return damon_pa_deactivate_pages(r, scheme); + case DAMOS_MIGRATE_COLD: + return damon_pa_migrate(r, scheme); case DAMOS_STAT: break; default: @@ -356,6 +508,8 @@ static int damon_pa_scheme_score(struct damon_ctx *cont= ext, return damon_hot_score(context, r, scheme); case DAMOS_LRU_DEPRIO: return damon_cold_score(context, r, scheme); + case DAMOS_MIGRATE_COLD: + return damon_cold_score(context, r, scheme); default: break; } diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index 0632d28b67f8..880015d5b5ea 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -1458,6 +1458,7 @@ static const char * const damon_sysfs_damos_action_st= rs[] =3D { "nohugepage", "lru_prio", "lru_deprio", + "migrate_cold", "stat", }; =20 --=20 2.39.2 From nobody Sat Feb 7 11:42:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7FD0953804; Sun, 12 May 2024 17:54:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715536498; cv=none; b=ngL2mn4KPxXJmvFnwiBa53Jip2bfRI5Tv40vrZXZt4TaeCJT7W6oGGPFmfouWqGniZ60tGwuql6zeOXJ0y+suvGmkOFe+l5YlyZZ9XXxOLkYClekl/fPYM99YU/aHp0aXQtABjQ7gJfxJWxVpLuCNI4BY73UlQyZDWFOTU6S20Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715536498; c=relaxed/simple; bh=HjyWpv2l8S24lCKx7/me+IFm1iDwuNsUj8wbeOXFYY0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bSRsFcKnnn6pv6w5sCjAUkfc0F8t/nIbTxgebOeeDyvJ7Ep7TmqhrNxg45XrCHW8ucbwc0IJ3LnZsquvVgeEILYJeezJTUG8gPLqnqvplb5dn/zynNQECtktVXdQwLAAkhnzVA8vNE3y9CHqkHuHzeZb6y+8zrcKGRNH6aKslXc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fOCDVk1p; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fOCDVk1p" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9859EC32781; Sun, 12 May 2024 17:54:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1715536498; bh=HjyWpv2l8S24lCKx7/me+IFm1iDwuNsUj8wbeOXFYY0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fOCDVk1p+QvsfPxFZGFuR7GWACBBP/DSQaaw966CnXovAR+eXd/oKQbhPIvBEGNJW WTQHUMZ9Tir6K0G1me1gLjTnFp8gxKD5G54IzI7k95vWlEOnVljY1wuxU7/QDBBCl8 TDHLXrizha9Gkc88qZSd6RHLzJj7UZOJJ94y5SfZp0Bmlp1AxP2d6Ny1kiFKaOUi65 ZwJmbuljvTEJPIEv2qWLpvi27KZPbt50udjXkXo31GtgNtfkvlqK9vP6z8JLVmTpW/ +PKLUiobEZ0l3l+PoNCQZwFIZOS6ibjGA+OecvuI6B5vyICO5mrBbMg2/WaES2jZx9 gnXanyNFEMxhw== From: SeongJae Park To: Cc: Hyeongtak Ji , SeongJae Park , Andrew Morton , damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Honggyu Kim Subject: [RFC PATCH v4 5/5] mm/damon/paddr: introduce DAMOS_MIGRATE_HOT action for promotion Date: Sun, 12 May 2024 10:54:47 -0700 Message-Id: <20240512175447.75943-6-sj@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240512175447.75943-1-sj@kernel.org> References: <20240512175447.75943-1-sj@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Hyeongtak Ji This patch introduces DAMOS_MIGRATE_HOT action, which is similar to DAMOS_MIGRATE_COLD, but proritizes hot pages. It migrates pages inside the given region to the 'target_nid' NUMA node in the sysfs. Here is one of the example usage of this 'migrate_hot' action. $ cd /sys/kernel/mm/damon/admin/kdamonds/ $ cat contexts//schemes//action migrate_hot $ echo 0 > contexts//schemes//target_nid $ echo commit > state $ numactl -p 2 ./hot_cold 500M 600M & $ numastat -c -p hot_cold Per-node process memory usage (in MBs) PID Node 0 Node 1 Node 2 Total -------------- ------ ------ ------ ----- 701 (hot_cold) 501 0 601 1101 Signed-off-by: Hyeongtak Ji Signed-off-by: Honggyu Kim Signed-off-by: SeongJae Park --- include/linux/damon.h | 2 ++ mm/damon/paddr.c | 3 +++ mm/damon/sysfs-schemes.c | 1 + 3 files changed, 6 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index 56714b6eb0d7..3d62d98d6359 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -105,6 +105,7 @@ struct damon_target { * @DAMOS_NOHUGEPAGE: Call ``madvise()`` for the region with MADV_NOHUGEPA= GE. * @DAMOS_LRU_PRIO: Prioritize the region on its LRU lists. * @DAMOS_LRU_DEPRIO: Deprioritize the region on its LRU lists. + * @DAMOS_MIGRATE_HOT: Migrate the regions prioritizing warmer regions. * @DAMOS_MIGRATE_COLD: Migrate the regions prioritizing colder regions. * @DAMOS_STAT: Do nothing but count the stat. * @NR_DAMOS_ACTIONS: Total number of DAMOS actions @@ -123,6 +124,7 @@ enum damos_action { DAMOS_NOHUGEPAGE, DAMOS_LRU_PRIO, DAMOS_LRU_DEPRIO, + DAMOS_MIGRATE_HOT, DAMOS_MIGRATE_COLD, DAMOS_STAT, /* Do nothing but only record the stat */ NR_DAMOS_ACTIONS, diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index a468792d9162..81163206e70c 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -486,6 +486,7 @@ static unsigned long damon_pa_apply_scheme(struct damon= _ctx *ctx, return damon_pa_mark_accessed(r, scheme); case DAMOS_LRU_DEPRIO: return damon_pa_deactivate_pages(r, scheme); + case DAMOS_MIGRATE_HOT: case DAMOS_MIGRATE_COLD: return damon_pa_migrate(r, scheme); case DAMOS_STAT: @@ -508,6 +509,8 @@ static int damon_pa_scheme_score(struct damon_ctx *cont= ext, return damon_hot_score(context, r, scheme); case DAMOS_LRU_DEPRIO: return damon_cold_score(context, r, scheme); + case DAMOS_MIGRATE_HOT: + return damon_hot_score(context, r, scheme); case DAMOS_MIGRATE_COLD: return damon_cold_score(context, r, scheme); default: diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index 880015d5b5ea..66fccfa776d7 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -1458,6 +1458,7 @@ static const char * const damon_sysfs_damos_action_st= rs[] =3D { "nohugepage", "lru_prio", "lru_deprio", + "migrate_hot", "migrate_cold", "stat", }; --=20 2.39.2