From nobody Mon Apr 27 12:15:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1813CCA47B for ; Mon, 13 Jun 2022 20:33:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235083AbiFMUda (ORCPT ); Mon, 13 Jun 2022 16:33:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35202 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229796AbiFMUc7 (ORCPT ); Mon, 13 Jun 2022 16:32:59 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F259C4E3B7 for ; Mon, 13 Jun 2022 12:23:10 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0D3AA614E3 for ; Mon, 13 Jun 2022 19:23:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EBC34C341C5; Mon, 13 Jun 2022 19:23:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655148189; bh=kMFgX2OiG0wP+m/airbACgnFus0Mqohfs1JXeb3bcyc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cEHxWCWcDQnzLU9nrgqKVHPe8OuFolxLzHQbWGm0FDF8s2xyR+UTbdrrPSpXxDWb+ 2gja07x61ylmEov3ngjkYe9zoxwsLiRsSLnjjcuXyTF1V+f2m0lVxOgZUrUBy+RKLl HoyjSta5L7tdYRjPc5ynMnYKljYgVJoGjtPLFVephMY8pxOffmSghJkInlXkTdzq7j EQ0n+oOBMJcI9B9+LUGI/G25iNl58r5iAnolPTravhz/k2gCL6FnuyQlG3VTunei/h CU1/VdnwPkiKUvgic43cy3ptsNJfigiY8JonZzcVNLWAvemNXegxPvstsoUcuQP2t0 o3V9PurM1s9Nw== From: SeongJae Park To: SeongJae Park , Andrew Morton Cc: damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/8] mm/damon/dbgfs: add mappings between 'schemes' file's action inputs and 'damos_action' values Date: Mon, 13 Jun 2022 19:22:54 +0000 Message-Id: <20220613192301.8817-3-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220613192301.8817-1-sj@kernel.org> References: <20220613192301.8817-1-sj@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" DAMON debugfs interface assumes users will write 'damos_action' value directly to the 'schemes' file. This makes adding new 'damos_action' in the middle of its definition breaks the backward compatibility of DAMON debugfs interface, as values of some 'damos_action' could be changed. To support backward compatibility in the cases, this commit adds mappings between the user inputs and 'damos_action' value and makes DAMON debugfs code uses those. Signed-off-by: SeongJae Park --- mm/damon/dbgfs.c | 64 +++++++++++++++++++++++++++++++++++++----------- 1 file changed, 50 insertions(+), 14 deletions(-) diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index 5ae810927309..cb8a7e9926a4 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -97,6 +97,31 @@ static ssize_t dbgfs_attrs_write(struct file *file, return ret; } =20 +/* + * Return corresponding dbgfs' scheme action value (int) for the given + * damos_action if the given damos_action value is valid and supported by + * dbgfs, negative error code otherwise. + */ +static int damos_action_to_dbgfs_scheme_action(enum damos_action action) +{ + switch (action) { + case DAMOS_WILLNEED: + return 0; + case DAMOS_COLD: + return 1; + case DAMOS_PAGEOUT: + return 2; + case DAMOS_HUGEPAGE: + return 3; + case DAMOS_NOHUGEPAGE: + return 4; + case DAMOS_STAT: + return 5; + default: + return -EINVAL; + } +} + static ssize_t sprint_schemes(struct damon_ctx *c, char *buf, ssize_t len) { struct damos *s; @@ -109,7 +134,7 @@ static ssize_t sprint_schemes(struct damon_ctx *c, char= *buf, ssize_t len) s->min_sz_region, s->max_sz_region, s->min_nr_accesses, s->max_nr_accesses, s->min_age_region, s->max_age_region, - s->action, + damos_action_to_dbgfs_scheme_action(s->action), s->quota.ms, s->quota.sz, s->quota.reset_interval, s->quota.weight_sz, @@ -160,18 +185,27 @@ static void free_schemes_arr(struct damos **schemes, = ssize_t nr_schemes) kfree(schemes); } =20 -static bool damos_action_valid(int action) +/* + * Return corresponding damos_action for the given dbgfs input for a scheme + * action if the input is valid, negative error code otherwise. + */ +static enum damos_action dbgfs_scheme_action_to_damos_action(int dbgfs_act= ion) { - switch (action) { - case DAMOS_WILLNEED: - case DAMOS_COLD: - case DAMOS_PAGEOUT: - case DAMOS_HUGEPAGE: - case DAMOS_NOHUGEPAGE: - case DAMOS_STAT: - return true; + switch (dbgfs_action) { + case 0: + return DAMOS_WILLNEED; + case 1: + return DAMOS_COLD; + case 2: + return DAMOS_PAGEOUT; + case 3: + return DAMOS_HUGEPAGE; + case 4: + return DAMOS_NOHUGEPAGE; + case 5: + return DAMOS_STAT; default: - return false; + return -EINVAL; } } =20 @@ -189,7 +223,8 @@ static struct damos **str_to_schemes(const char *str, s= size_t len, int pos =3D 0, parsed, ret; unsigned long min_sz, max_sz; unsigned int min_nr_a, max_nr_a, min_age, max_age; - unsigned int action; + unsigned int action_input; + enum damos_action action; =20 schemes =3D kmalloc_array(max_nr_schemes, sizeof(scheme), GFP_KERNEL); @@ -204,7 +239,7 @@ static struct damos **str_to_schemes(const char *str, s= size_t len, ret =3D sscanf(&str[pos], "%lu %lu %u %u %u %u %u %lu %lu %lu %u %u %u %u %lu %lu %lu %lu%n", &min_sz, &max_sz, &min_nr_a, &max_nr_a, - &min_age, &max_age, &action, "a.ms, + &min_age, &max_age, &action_input, "a.ms, "a.sz, "a.reset_interval, "a.weight_sz, "a.weight_nr_accesses, "a.weight_age, &wmarks.metric, @@ -212,7 +247,8 @@ static struct damos **str_to_schemes(const char *str, s= size_t len, &wmarks.low, &parsed); if (ret !=3D 18) break; - if (!damos_action_valid(action)) + action =3D dbgfs_scheme_action_to_damos_action(action_input); + if ((int)action < 0) goto fail; =20 if (min_sz > max_sz || min_nr_a > max_nr_a || min_age > max_age) --=20 2.25.1 From nobody Mon Apr 27 12:15:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2978FC43334 for ; Mon, 13 Jun 2022 20:35:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239541AbiFMUdf (ORCPT ); Mon, 13 Jun 2022 16:33:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35214 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232570AbiFMUdA (ORCPT ); Mon, 13 Jun 2022 16:33:00 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E1B528726 for ; Mon, 13 Jun 2022 12:23:12 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id E2B67B8128D for ; Mon, 13 Jun 2022 19:23:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 454CBC3411F; Mon, 13 Jun 2022 19:23:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655148188; bh=XOWacsSFy4qQVUe4yC1Lr3xCRJ6lmmqOdxcVjKLkxx8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YCizNenf2u5d4ieuePcVEjzZ2J39Sj48RNI7hGwMQwP/Abq9boO2PmAU5PmHVHtos K5zCk0uhTbHp67T5LrOHdPQirivVTyL+mIvseF7cg/vJOrabOuc9GBmWbd90836pWq 80AdC/J8slOkYQUhjnYmijIIcNH46See1cJo/ApvzWcC+dlTcwICTFJjEgsTLgpumq zhRoiHq0nEHIAs+a8Tg5cXkc04tSfs/P9Czxgry3qfzQnD0ELzWBhGFQl6u5FvIPxB /L61PzRRY849tV0+hguoIHWX/5IzklIs3N5PskB/9YNMI13TArwYX9VDHYDvdNiENf OjDhaCgR8CI+w== From: SeongJae Park To: SeongJae Park , Andrew Morton Cc: damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/8] mm/damon/dbgfs: add and use mappings between 'schemes' action inputs and 'damos_action' values Date: Mon, 13 Jun 2022 19:22:53 +0000 Message-Id: <20220613192301.8817-2-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220613192301.8817-1-sj@kernel.org> References: <20220613192301.8817-1-sj@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" DAMON debugfs interface assumes users will write 'damos_action' value directly to the 'schemes' file. This makes adding new 'damos_action' in the middle of its definition breaks the backward compatibility of DAMON debugfs interface, as values of some 'damos_action' could be changed. To mitigate the situation, this commit adds mappings between the user inputs and 'damos_action' value and makes DAMON debugfs code uses those. Signed-off-by: SeongJae Park --- mm/damon/dbgfs.c | 64 +++++++++++++++++++++++++++++++++++++----------- 1 file changed, 50 insertions(+), 14 deletions(-) diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index 5ae810927309..cb8a7e9926a4 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -97,6 +97,31 @@ static ssize_t dbgfs_attrs_write(struct file *file, return ret; } =20 +/* + * Return corresponding dbgfs' scheme action value (int) for the given + * damos_action if the given damos_action value is valid and supported by + * dbgfs, negative error code otherwise. + */ +static int damos_action_to_dbgfs_scheme_action(enum damos_action action) +{ + switch (action) { + case DAMOS_WILLNEED: + return 0; + case DAMOS_COLD: + return 1; + case DAMOS_PAGEOUT: + return 2; + case DAMOS_HUGEPAGE: + return 3; + case DAMOS_NOHUGEPAGE: + return 4; + case DAMOS_STAT: + return 5; + default: + return -EINVAL; + } +} + static ssize_t sprint_schemes(struct damon_ctx *c, char *buf, ssize_t len) { struct damos *s; @@ -109,7 +134,7 @@ static ssize_t sprint_schemes(struct damon_ctx *c, char= *buf, ssize_t len) s->min_sz_region, s->max_sz_region, s->min_nr_accesses, s->max_nr_accesses, s->min_age_region, s->max_age_region, - s->action, + damos_action_to_dbgfs_scheme_action(s->action), s->quota.ms, s->quota.sz, s->quota.reset_interval, s->quota.weight_sz, @@ -160,18 +185,27 @@ static void free_schemes_arr(struct damos **schemes, = ssize_t nr_schemes) kfree(schemes); } =20 -static bool damos_action_valid(int action) +/* + * Return corresponding damos_action for the given dbgfs input for a scheme + * action if the input is valid, negative error code otherwise. + */ +static enum damos_action dbgfs_scheme_action_to_damos_action(int dbgfs_act= ion) { - switch (action) { - case DAMOS_WILLNEED: - case DAMOS_COLD: - case DAMOS_PAGEOUT: - case DAMOS_HUGEPAGE: - case DAMOS_NOHUGEPAGE: - case DAMOS_STAT: - return true; + switch (dbgfs_action) { + case 0: + return DAMOS_WILLNEED; + case 1: + return DAMOS_COLD; + case 2: + return DAMOS_PAGEOUT; + case 3: + return DAMOS_HUGEPAGE; + case 4: + return DAMOS_NOHUGEPAGE; + case 5: + return DAMOS_STAT; default: - return false; + return -EINVAL; } } =20 @@ -189,7 +223,8 @@ static struct damos **str_to_schemes(const char *str, s= size_t len, int pos =3D 0, parsed, ret; unsigned long min_sz, max_sz; unsigned int min_nr_a, max_nr_a, min_age, max_age; - unsigned int action; + unsigned int action_input; + enum damos_action action; =20 schemes =3D kmalloc_array(max_nr_schemes, sizeof(scheme), GFP_KERNEL); @@ -204,7 +239,7 @@ static struct damos **str_to_schemes(const char *str, s= size_t len, ret =3D sscanf(&str[pos], "%lu %lu %u %u %u %u %u %lu %lu %lu %u %u %u %u %lu %lu %lu %lu%n", &min_sz, &max_sz, &min_nr_a, &max_nr_a, - &min_age, &max_age, &action, "a.ms, + &min_age, &max_age, &action_input, "a.ms, "a.sz, "a.reset_interval, "a.weight_sz, "a.weight_nr_accesses, "a.weight_age, &wmarks.metric, @@ -212,7 +247,8 @@ static struct damos **str_to_schemes(const char *str, s= size_t len, &wmarks.low, &parsed); if (ret !=3D 18) break; - if (!damos_action_valid(action)) + action =3D dbgfs_scheme_action_to_damos_action(action_input); + if ((int)action < 0) goto fail; =20 if (min_sz > max_sz || min_nr_a > max_nr_a || min_age > max_age) --=20 2.25.1 From nobody Mon Apr 27 12:15:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20288C43334 for ; Mon, 13 Jun 2022 20:33:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232632AbiFMUdR (ORCPT ); Mon, 13 Jun 2022 16:33:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234129AbiFMUc6 (ORCPT ); Mon, 13 Jun 2022 16:32:58 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7263F6D394 for ; Mon, 13 Jun 2022 12:23:11 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A149A614ED for ; Mon, 13 Jun 2022 19:23:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E22AC341C4; Mon, 13 Jun 2022 19:23:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655148190; bh=n6pIJardcDOsM0Vty4Jd2+LmdyUc2TSqs3BgnjMqBM8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZU4tSULFHm6FsvW+iSnmeOF25EuoA37Y1Q7CK6fSJNEpgPnkR2oqvoVXTbM1DXOKk 0309qTloTvFcZz49r6f/2AVtvrXlKCH005FkekFR746N66hY3QujklydIZpzauQ8sI /fxtWRoCjt56xVP3Tq1VpSE7IWzJrMgWKvmxsrFplcztBs5DG73e0+qszzEia4COJq mk/79vYNlULn+hT+qgm0Dz+eBjcLp15rzxCsD8XZhMkDWP+5zwPfRKikazB5zqhKqk oQWNCCqpwDJ31kF9vSfBuQeBkzZIQ2WcKvJHUbrWWggRgb9Xyn4RPqsuGnfBoqJNjM QlrpeStxf/IhA== From: SeongJae Park To: SeongJae Park , Andrew Morton Cc: damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/8] mm/damon/paddr: use a separate function for 'DAMOS_PAGEOUT' handling Date: Mon, 13 Jun 2022 19:22:55 +0000 Message-Id: <20220613192301.8817-4-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220613192301.8817-1-sj@kernel.org> References: <20220613192301.8817-1-sj@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit moves code for 'DAMOS_PAGEOUT' handling of the physical address space monitoring operations set to a separate function so that its caller, 'damon_pa_apply_scheme()', can be more easily extended for additional DAMOS actions later. Signed-off-by: SeongJae Park --- mm/damon/paddr.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index b40ff5811bb2..7bcd48066b43 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -204,16 +204,11 @@ static unsigned int damon_pa_check_accesses(struct da= mon_ctx *ctx) return max_nr_accesses; } =20 -static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, - struct damon_target *t, struct damon_region *r, - struct damos *scheme) +static unsigned long damon_pa_pageout(struct damon_region *r) { unsigned long addr, applied; LIST_HEAD(page_list); =20 - if (scheme->action !=3D DAMOS_PAGEOUT) - return 0; - for (addr =3D r->ar.start; addr < r->ar.end; addr +=3D PAGE_SIZE) { struct page *page =3D damon_get_page(PHYS_PFN(addr)); =20 @@ -238,6 +233,19 @@ static unsigned long damon_pa_apply_scheme(struct damo= n_ctx *ctx, return applied * PAGE_SIZE; } =20 +static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, + struct damon_target *t, struct damon_region *r, + struct damos *scheme) +{ + switch (scheme->action) { + case DAMOS_PAGEOUT: + return damon_pa_pageout(r); + default: + break; + } + return 0; +} + static int damon_pa_scheme_score(struct damon_ctx *context, struct damon_target *t, struct damon_region *r, struct damos *scheme) --=20 2.25.1 From nobody Mon Apr 27 12:15:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F30B8CCA47D for ; Mon, 13 Jun 2022 20:34:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231229AbiFMUeG (ORCPT ); Mon, 13 Jun 2022 16:34:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33622 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232406AbiFMUdG (ORCPT ); Mon, 13 Jun 2022 16:33:06 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5550B183A9 for ; Mon, 13 Jun 2022 12:23:15 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id EA47DB812A2 for ; Mon, 13 Jun 2022 19:23:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4293BC341C0; Mon, 13 Jun 2022 19:23:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655148190; bh=uKUGIZGPQZchKZAH92MEAi0WmMFjIc8qS5qYmL1UAXQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dK6TFvLEVwmXnr86/42/FgXjvzbKYeaAFVTHq+OA3hBpVVNQDCmG6yj4xyYosJlpO 3HbCMV2snUEAbqP+mr2h7GsZuOMYWVrchP3JJW3gl7ZOR+isfJCza0/wEphct07O36 m+Uz/LwRBP2cPaMZXDSi6CoZZf4BC7IH8kdLulq8mZUJt1/o1nCRBug8S8WRxnKSJ4 shA6ZH2CwhBt13c8rn42G/ppl9ZdMS8ZiXsiew65Y4tagAYXCjOK86SI9DfSHCI0RC LQQe48kkhO6Nw+higB9rZQCVP9QdVti3gPJaQoK92LKhDSrzcZ+CSDj4ThpN1dWh+w kW6kPFM+piwBg== From: SeongJae Park To: SeongJae Park , Andrew Morton Cc: damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/8] mm/damon/schemes: add 'LRU_PRIO' DAMOS action Date: Mon, 13 Jun 2022 19:22:56 +0000 Message-Id: <20220613192301.8817-5-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220613192301.8817-1-sj@kernel.org> References: <20220613192301.8817-1-sj@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit adds a new DAMOS action called 'LRU_PRIO' for the physical address space. The action prioritizes pages in the memory regions of the user-specified target access pattern on their LRU lists. This is hence supposed to be used for frequently accessed (hot) memory regions so that hot pages could be more likely protected under memory pressure. Internally, it simply calls 'mark_page_accessed()'. Signed-off-by: SeongJae Park --- include/linux/damon.h | 2 ++ mm/damon/ops-common.c | 42 ++++++++++++++++++++++++++++++++++++++++++ mm/damon/ops-common.h | 2 ++ mm/damon/paddr.c | 20 ++++++++++++++++++++ mm/damon/sysfs.c | 1 + 5 files changed, 67 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index b9aae19fab3e..4c64e03e94d8 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -86,6 +86,7 @@ struct damon_target { * @DAMOS_PAGEOUT: Call ``madvise()`` for the region with MADV_PAGEOUT. * @DAMOS_HUGEPAGE: Call ``madvise()`` for the region with MADV_HUGEPAGE. * @DAMOS_NOHUGEPAGE: Call ``madvise()`` for the region with MADV_NOHUGEPA= GE. + * @DAMOS_LRU_PRIO: Prioritize the region on its LRU lists. * @DAMOS_STAT: Do nothing but count the stat. * @NR_DAMOS_ACTIONS: Total number of DAMOS actions */ @@ -95,6 +96,7 @@ enum damos_action { DAMOS_PAGEOUT, DAMOS_HUGEPAGE, DAMOS_NOHUGEPAGE, + DAMOS_LRU_PRIO, DAMOS_STAT, /* Do nothing but only record the stat */ NR_DAMOS_ACTIONS, }; diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index 10ef20b2003f..b1335de200e7 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -130,3 +130,45 @@ int damon_pageout_score(struct damon_ctx *c, struct da= mon_region *r, /* Return coldness of the region */ return DAMOS_MAX_SCORE - hotness; } + +int damon_hot_score(struct damon_ctx *c, struct damon_region *r, + struct damos *s) +{ + unsigned int max_nr_accesses; + int freq_subscore; + unsigned int age_in_sec; + int age_in_log, age_subscore; + unsigned int freq_weight =3D s->quota.weight_nr_accesses; + unsigned int age_weight =3D s->quota.weight_age; + int hotness; + + max_nr_accesses =3D c->aggr_interval / c->sample_interval; + freq_subscore =3D r->nr_accesses * DAMON_MAX_SUBSCORE / max_nr_accesses; + + age_in_sec =3D (unsigned long)r->age * c->aggr_interval / 1000000; + for (age_in_log =3D 0; age_in_log < DAMON_MAX_AGE_IN_LOG && age_in_sec; + age_in_log++, age_in_sec >>=3D 1) + ; + + /* If frequency is 0, higher age means it's colder */ + if (freq_subscore =3D=3D 0) + age_in_log *=3D -1; + + /* + * Now age_in_log is in [-DAMON_MAX_AGE_IN_LOG, DAMON_MAX_AGE_IN_LOG]. + * Scale it to be in [0, 100] and set it as age subscore. + */ + age_in_log +=3D DAMON_MAX_AGE_IN_LOG; + age_subscore =3D age_in_log * DAMON_MAX_SUBSCORE / + DAMON_MAX_AGE_IN_LOG / 2; + + hotness =3D (freq_weight * freq_subscore + age_weight * age_subscore); + if (freq_weight + age_weight) + hotness /=3D freq_weight + age_weight; + /* + * Transform it to fit in [0, DAMOS_MAX_SCORE] + */ + hotness =3D hotness * DAMOS_MAX_SCORE / DAMON_MAX_SUBSCORE; + + return hotness; +} diff --git a/mm/damon/ops-common.h b/mm/damon/ops-common.h index e790cb5f8fe0..52329ff361cd 100644 --- a/mm/damon/ops-common.h +++ b/mm/damon/ops-common.h @@ -14,3 +14,5 @@ void damon_pmdp_mkold(pmd_t *pmd, struct mm_struct *mm, u= nsigned long addr); =20 int damon_pageout_score(struct damon_ctx *c, struct damon_region *r, struct damos *s); +int damon_hot_score(struct damon_ctx *c, struct damon_region *r, + struct damos *s); diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 7bcd48066b43..f145b1d51e13 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -233,6 +233,22 @@ static unsigned long damon_pa_pageout(struct damon_reg= ion *r) return applied * PAGE_SIZE; } =20 +static unsigned long damon_pa_mark_accessed(struct damon_region *r) +{ + unsigned long addr, applied =3D 0; + + for (addr =3D r->ar.start; addr < r->ar.end; addr +=3D PAGE_SIZE) { + struct page *page =3D damon_get_page(PHYS_PFN(addr)); + + if (!page) + continue; + mark_page_accessed(page); + put_page(page); + applied++; + } + return applied * PAGE_SIZE; +} + static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, struct damon_target *t, struct damon_region *r, struct damos *scheme) @@ -240,6 +256,8 @@ static unsigned long damon_pa_apply_scheme(struct damon= _ctx *ctx, switch (scheme->action) { case DAMOS_PAGEOUT: return damon_pa_pageout(r); + case DAMOS_LRU_PRIO: + return damon_pa_mark_accessed(r); default: break; } @@ -253,6 +271,8 @@ static int damon_pa_scheme_score(struct damon_ctx *cont= ext, switch (scheme->action) { case DAMOS_PAGEOUT: return damon_pageout_score(context, r, scheme); + case DAMOS_LRU_PRIO: + return damon_hot_score(context, r, scheme); default: break; } diff --git a/mm/damon/sysfs.c b/mm/damon/sysfs.c index c35809c6087c..86c69f980927 100644 --- a/mm/damon/sysfs.c +++ b/mm/damon/sysfs.c @@ -762,6 +762,7 @@ static const char * const damon_sysfs_damos_action_strs= [] =3D { "pageout", "hugepage", "nohugepage", + "lru_prio", "stat", }; =20 --=20 2.25.1 From nobody Mon Apr 27 12:15:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2BDCCCA47B for ; Mon, 13 Jun 2022 20:33:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233520AbiFMUdv (ORCPT ); Mon, 13 Jun 2022 16:33:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233219AbiFMUdI (ORCPT ); Mon, 13 Jun 2022 16:33:08 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 98C4B25E8D; Mon, 13 Jun 2022 12:23:15 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id C8F18B81260; Mon, 13 Jun 2022 19:23:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E8645C341C6; Mon, 13 Jun 2022 19:23:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655148191; bh=VBsgQVJb305tu+x5Ezk9xdnNcUiCg8aWk5S50W2VGCw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nwkY5iB/FBsFoMBVuUTRoDxNVkViDcsprfVa78XaY+e7pn0E6ofcJAjKfa3FiTuYV ZAvtKRakoy2Pw1Ka4WbCimxUtoupKTxE20bGydUF+qaBNqxK/oP1M6tGBim0b2riWz cGhqciMQKJiHntWylNAwhcH/oNXNkvpcJEaUyQdEvtpD+RWTJqwYRFItCqj5M5Tkvp UWOf5ZuGcGnRJtioModUaO7SA5r/B+//cRZaUe9JKD4P2fu9GTWgjb33Em7uXhzCx5 PkFPWPiOXYDWYA9uKF8XmQkIwVCXFXHp9VlLNw5VW9Szz+j2t5rCRuHGQuxZ3vIWGo 32ujOHqyEAA1A== From: SeongJae Park To: SeongJae Park , Jonathan Corbet Cc: damon@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/8] Docs/admin-guide/damon/sysfs: document 'LRU_PRIO' scheme action Date: Mon, 13 Jun 2022 19:22:57 +0000 Message-Id: <20220613192301.8817-6-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220613192301.8817-1-sj@kernel.org> References: <20220613192301.8817-1-sj@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit documents the 'lru_prio' scheme action for DAMON sysfs interface. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/usage.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/a= dmin-guide/mm/damon/usage.rst index 1bb7b72414b2..af4e15ee81cd 100644 --- a/Documentation/admin-guide/mm/damon/usage.rst +++ b/Documentation/admin-guide/mm/damon/usage.rst @@ -264,6 +264,7 @@ that can be written to and read from the file and their= meaning are as below. - ``pageout``: Call ``madvise()`` for the region with ``MADV_PAGEOUT`` - ``hugepage``: Call ``madvise()`` for the region with ``MADV_HUGEPAGE`` - ``nohugepage``: Call ``madvise()`` for the region with ``MADV_NOHUGEPAG= E`` + - ``lru_prio``: Prioritize the region on its LRU lists. - ``stat``: Do nothing but count the statistics =20 schemes//access_pattern/ --=20 2.25.1 From nobody Mon Apr 27 12:15:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 074CFCCA47B for ; Mon, 13 Jun 2022 20:34:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234345AbiFMUd7 (ORCPT ); Mon, 13 Jun 2022 16:33:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34734 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232903AbiFMUdH (ORCPT ); Mon, 13 Jun 2022 16:33:07 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 98D063A5EF for ; Mon, 13 Jun 2022 12:23:15 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 3D7C4B81289 for ; Mon, 13 Jun 2022 19:23:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9B180C341C5; Mon, 13 Jun 2022 19:23:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655148192; bh=sOSbjIcAeZVbU2/qKMJLnCAeX6/kYK4mBHuJd9HgKYw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RFg5Rpysf9zbj5CtehWxzO10QOsIcqZFMAVehvaCj3ken4joxESy5KiDIEht0Ewdm zM2aLSGCzJ9xYnTsvdov8f30Ez/QfGpjDuaXq9Q5EXlwxWuiRsOp4EGJGYK9Sm1MYn NLr2uMG31AI4D7Wi2+I2cEK3TVzrifqxZ7FfbmCasCdj0PxBhCHRly5PmHnDrxI+nf rz75VQTOAywnAB6tvsLpIVCV3iCRPw1YYPkoesHUHMHLW0228C1/BiFdAAKnPSZLqp QqFWbG/Ltkm/xR6bAEBKwdAlDpNdyY3RasJcTuV9NxwNKLRwwYfoOLMWKNyNLrG0E0 MoVQ7WfKoaC9g== From: SeongJae Park To: SeongJae Park , Andrew Morton Cc: damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/8] mm/damon/schemes: add 'LRU_DEPRIO' action Date: Mon, 13 Jun 2022 19:22:58 +0000 Message-Id: <20220613192301.8817-7-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220613192301.8817-1-sj@kernel.org> References: <20220613192301.8817-1-sj@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit adds a new DAMON-based operation scheme action called 'LRU_DEPRIO' for physical address space. The action deprioritizes pages in the memory area of the target access pattern on their LRU lists. This is hence supposed to be used for rarely accessed (cold) memory regions so that cold pages could be more likely reclaimed first under memory pressure. Internally, it simply calls 'lru_deactivate()'. Using this with 'LRU_PRIO' action for hot pages, users can proactively sort LRU lists based on the access pattern. That is, it can make the LRU lists somewhat more trustworthy source of access temperature. As a result, efficiency of LRU-lists based mechanisms including the reclamation target selection could be improved. Signed-off-by: SeongJae Park --- include/linux/damon.h | 2 ++ mm/damon/paddr.c | 20 ++++++++++++++++++++ mm/damon/sysfs.c | 1 + 3 files changed, 23 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index 4c64e03e94d8..7b1f4a488230 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -87,6 +87,7 @@ struct damon_target { * @DAMOS_HUGEPAGE: Call ``madvise()`` for the region with MADV_HUGEPAGE. * @DAMOS_NOHUGEPAGE: Call ``madvise()`` for the region with MADV_NOHUGEPA= GE. * @DAMOS_LRU_PRIO: Prioritize the region on its LRU lists. + * @DAMOS_LRU_DEPRIO: Deprioritize the region on its LRU lists. * @DAMOS_STAT: Do nothing but count the stat. * @NR_DAMOS_ACTIONS: Total number of DAMOS actions */ @@ -97,6 +98,7 @@ enum damos_action { DAMOS_HUGEPAGE, DAMOS_NOHUGEPAGE, DAMOS_LRU_PRIO, + DAMOS_LRU_DEPRIO, DAMOS_STAT, /* Do nothing but only record the stat */ NR_DAMOS_ACTIONS, }; diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index f145b1d51e13..dc131c6a5403 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -249,6 +249,22 @@ static unsigned long damon_pa_mark_accessed(struct dam= on_region *r) return applied * PAGE_SIZE; } =20 +static unsigned long damon_pa_deactivate_pages(struct damon_region *r) +{ + unsigned long addr, applied =3D 0; + + for (addr =3D r->ar.start; addr < r->ar.end; addr +=3D PAGE_SIZE) { + struct page *page =3D damon_get_page(PHYS_PFN(addr)); + + if (!page) + continue; + deactivate_page(page); + put_page(page); + applied++; + } + return applied * PAGE_SIZE; +} + static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, struct damon_target *t, struct damon_region *r, struct damos *scheme) @@ -258,6 +274,8 @@ static unsigned long damon_pa_apply_scheme(struct damon= _ctx *ctx, return damon_pa_pageout(r); case DAMOS_LRU_PRIO: return damon_pa_mark_accessed(r); + case DAMOS_LRU_DEPRIO: + return damon_pa_deactivate_pages(r); default: break; } @@ -273,6 +291,8 @@ static int damon_pa_scheme_score(struct damon_ctx *cont= ext, return damon_pageout_score(context, r, scheme); case DAMOS_LRU_PRIO: return damon_hot_score(context, r, scheme); + case DAMOS_LRU_DEPRIO: + return damon_pageout_score(context, r, scheme); default: break; } diff --git a/mm/damon/sysfs.c b/mm/damon/sysfs.c index 86c69f980927..7488e27c87c3 100644 --- a/mm/damon/sysfs.c +++ b/mm/damon/sysfs.c @@ -763,6 +763,7 @@ static const char * const damon_sysfs_damos_action_strs= [] =3D { "hugepage", "nohugepage", "lru_prio", + "lru_deprio", "stat", }; =20 --=20 2.25.1 From nobody Mon Apr 27 12:15:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31205C433EF for ; Mon, 13 Jun 2022 20:34:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236684AbiFMUeO (ORCPT ); Mon, 13 Jun 2022 16:34:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232288AbiFMUdG (ORCPT ); Mon, 13 Jun 2022 16:33:06 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90CF93631E; Mon, 13 Jun 2022 12:23:13 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4EBC2614EF; Mon, 13 Jun 2022 19:23:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 43251C341C8; Mon, 13 Jun 2022 19:23:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655148192; bh=/ZIOfIXjUyDL/E317WytkrmVVIeFLDByyTBTkw3FhHs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TQbrkfuor7ndKRkLJR3NUU84zVZPei00GRr2otDJiT+VZlwEryFoOHCX1Syn6t9Xe 8oJZnO+PJBBUrPCdzUv8PWssdHiL3vSPi+rOYH1y/sHRyySHTWhGs4oZjVgKlELfip ziKC4IXJ9o+lMKCzodXTrfhm4w38F9ZlbpWSs4rJ32h8Uc5vbUaYWRSQ17NcxjPaQ9 ARph6Dsherb3IPwUe7Nh+g/K1qPB0pVoMg5BfE7hiuar8OmAy0tFaQx5YNm8UMDP5Q j8G/4Md/ii7Ni/TvrE3v22zCWouPhR0BzoRkAjiC+m+1jcjbKWHoGwElUnqYzizREd Xd/6ZOkRAY6Vg== From: SeongJae Park To: SeongJae Park , Jonathan Corbet Cc: damon@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/8] Docs/admin-guide/damon/sysfs: document 'LRU_DEPRIO' scheme action Date: Mon, 13 Jun 2022 19:22:59 +0000 Message-Id: <20220613192301.8817-8-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220613192301.8817-1-sj@kernel.org> References: <20220613192301.8817-1-sj@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit documents the 'LRU_DEPRIO' scheme action for DAMON sysfs interface.` Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/usage.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/a= dmin-guide/mm/damon/usage.rst index af4e15ee81cd..d822bf6355ce 100644 --- a/Documentation/admin-guide/mm/damon/usage.rst +++ b/Documentation/admin-guide/mm/damon/usage.rst @@ -265,6 +265,7 @@ that can be written to and read from the file and their= meaning are as below. - ``hugepage``: Call ``madvise()`` for the region with ``MADV_HUGEPAGE`` - ``nohugepage``: Call ``madvise()`` for the region with ``MADV_NOHUGEPAG= E`` - ``lru_prio``: Prioritize the region on its LRU lists. + - ``lru_deprio``: Deprioritize the region on its LRU lists. - ``stat``: Do nothing but count the statistics =20 schemes//access_pattern/ --=20 2.25.1 From nobody Mon Apr 27 12:15:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32020C433EF for ; Mon, 13 Jun 2022 20:34:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233924AbiFMUeL (ORCPT ); Mon, 13 Jun 2022 16:34:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232328AbiFMUdG (ORCPT ); Mon, 13 Jun 2022 16:33:06 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87EC662C2 for ; Mon, 13 Jun 2022 12:23:14 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0BA9A614F1 for ; Mon, 13 Jun 2022 19:23:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E97FFC341CA; Mon, 13 Jun 2022 19:23:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655148193; bh=1yVVksVGWWxWxGAbOlcBQDM1gy3N2GV1Y1IErEq9oDc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MnKDhubN2lSVAB568H3rk5QrD+L3+41JfvM9VOzd9Y6jglvH5Omqj4KZJoct5Q3C8 2u12xwRdaAu+QSk1cvwfZtrd6wjYKioIJEQVpfIRU4dHZzwx1Nc4pXa/wXrcznJPu9 /OTj7vEZx+4xrK1KqPXLhLwh/FtV0lm+cdNVz1Dav0fjS69g1vYW+Vqm3fCgO8eChY 9ykvjmdbPTquq8fnEFtTm0N5UWxR4gMGu1xo1MTSxHcRfM0NP5Gn3a/jaGW5PG8Jje ufPaCj1Smv7dtkh3XbSgpzNAD6mJwlRkShvrpVZkFpooV+o2YeX0qhWysonYJpJ96p /EO88DGGxdjYQ== From: SeongJae Park To: SeongJae Park , Andrew Morton Cc: linux-kernel@vger.kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Subject: [PATCH 7/8] mm/damon: introduce DAMON-based LRU-lists Sorting Date: Mon, 13 Jun 2022 19:23:00 +0000 Message-Id: <20220613192301.8817-9-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220613192301.8817-1-sj@kernel.org> References: <20220613192301.8817-1-sj@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Users can do data access-aware LRU-lists sorting using 'LRU_PRIO' and 'LRU_DEPRIO' DAMOS actions. However, finding best parameters including the hotness/coldness thresholds, CPU quota, and watermarks could be challenging for some users. To make the scheme easy to be used without complex tuning for common situations, this commit implements a static kernel module called 'DAMON_LRU_SORT' using the 'LRU_PRIO' and 'LRU_DEPRIO' DAMOS actions. It proactively sorts LRU-lists using DAMON with conservatively chosen default values of the parameters. That is, the module under its default parameters will make no harm for common situations but provide some level of efficiency improvements for systems having clear hot/cold access pattern under a level of memory pressure while consuming only a limited small portion of CPU time. Signed-off-by: SeongJae Park --- mm/damon/Kconfig | 8 + mm/damon/Makefile | 1 + mm/damon/lru_sort.c | 546 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 555 insertions(+) create mode 100644 mm/damon/lru_sort.c diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig index 9b559c76d6dd..66265e3a9c65 100644 --- a/mm/damon/Kconfig +++ b/mm/damon/Kconfig @@ -92,4 +92,12 @@ config DAMON_RECLAIM reclamation under light memory pressure, while the traditional page scanning-based reclamation is used for heavy pressure. =20 +config DAMON_LRU_SORT + bool "Build DAMON-based LRU-lists sorting (DAMON_LRU_SORT)" + depends on DAMON_PADDR + help + This builds the DAMON-based LRU-lists sorting subsystem. It tries to + protect frequently accessed (hot) pages while rarely accessed (cold) + pages reclaimed first under memory pressure. + endmenu diff --git a/mm/damon/Makefile b/mm/damon/Makefile index dbf7190b4144..3e6b8ad73858 100644 --- a/mm/damon/Makefile +++ b/mm/damon/Makefile @@ -6,3 +6,4 @@ obj-$(CONFIG_DAMON_PADDR) +=3D ops-common.o paddr.o obj-$(CONFIG_DAMON_SYSFS) +=3D sysfs.o obj-$(CONFIG_DAMON_DBGFS) +=3D dbgfs.o obj-$(CONFIG_DAMON_RECLAIM) +=3D reclaim.o +obj-$(CONFIG_DAMON_LRU_SORT) +=3D lru_sort.o diff --git a/mm/damon/lru_sort.c b/mm/damon/lru_sort.c new file mode 100644 index 000000000000..c276736a071c --- /dev/null +++ b/mm/damon/lru_sort.c @@ -0,0 +1,546 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * DAMON-based LRU-lists Sorting + * + * Author: SeongJae Park + */ + +#define pr_fmt(fmt) "damon-lru-sort: " fmt + +#include +#include +#include +#include +#include + +#ifdef MODULE_PARAM_PREFIX +#undef MODULE_PARAM_PREFIX +#endif +#define MODULE_PARAM_PREFIX "damon_lru_sort." + +/* + * Enable or disable DAMON_LRU_SORT. + * + * You can enable DAMON_LRU_SORT by setting the value of this parameter as + * ``Y``. Setting it as ``N`` disables DAMON_LRU_SORT. Note that + * DAMON_LRU_SORT could do no real monitoring and LRU-lists sorting due to= the + * watermarks-based activation condition. Refer to below descriptions for= the + * watermarks parameter for this. + */ +static bool enabled __read_mostly; + +/* + * Make DAMON_LRU_SORT reads the input parameters again, except ``enabled`= `. + * + * Input parameters that updated while DAMON_LRU_SORT is running are not + * applied by default. Once this parameter is set as ``Y``, DAMON_LRU_SORT + * reads values of parametrs except ``enabled`` again. Once the re-readin= g is + * done, this parameter is set as ``N``. If invalid parameters are found = while + * the re-reading, DAMON_LRU_SORT will be disabled. + */ +static bool commit_inputs __read_mostly; +module_param(commit_inputs, bool, 0600); + +/* + * Access frequency threshold for hot memory regions identification in per= mil. + * + * If a memory region is accessed in frequency of this or higher, + * DAMON_LRU_SORT identifies the region as hot, and mark it as accessed on= the + * LRU list, so that it could not be reclaimed under memory pressure. 50%= by + * default. + */ +static unsigned long hot_thres_access_freq =3D 500; +module_param(hot_thres_access_freq, ulong, 0600); + +/* + * Time threshold for cold memory regions identification in microseconds. + * + * If a memory region is not accessed for this or longer time, DAMON_LRU_S= ORT + * identifies the region as cold, and mark it as unaccessed on the LRU lis= t, so + * that it could be reclaimed first under memory pressure. 120 seconds by + * default. + */ +static unsigned long cold_min_age __read_mostly =3D 120000000; +module_param(cold_min_age, ulong, 0600); + +/* + * Limit of time for trying the LRU lists sorting in milliseconds. + * + * DAMON_LRU_SORT tries to use only up to this time within a time window + * (quota_reset_interval_ms) for trying LRU lists sorting. This can be us= ed + * for limiting CPU consumption of DAMON_LRU_SORT. If the value is zero, = the + * limit is disabled. + * + * 10 ms by default. + */ +static unsigned long quota_ms __read_mostly =3D 10; +module_param(quota_ms, ulong, 0600); + +/* + * The time quota charge reset interval in milliseconds. + * + * The charge reset interval for the quota of time (quota_ms). That is, + * DAMON_LRU_SORT does not try LRU-lists sorting for more than quota_ms + * milliseconds or quota_sz bytes within quota_reset_interval_ms milliseco= nds. + * + * 1 second by default. + */ +static unsigned long quota_reset_interval_ms __read_mostly =3D 1000; +module_param(quota_reset_interval_ms, ulong, 0600); + +/* + * The watermarks check time interval in microseconds. + * + * Minimal time to wait before checking the watermarks, when DAMON_LRU_SOR= T is + * enabled but inactive due to its watermarks rule. 5 seconds by default. + */ +static unsigned long wmarks_interval __read_mostly =3D 5000000; +module_param(wmarks_interval, ulong, 0600); + +/* + * Free memory rate (per thousand) for the high watermark. + * + * If free memory of the system in bytes per thousand bytes is higher than + * this, DAMON_LRU_SORT becomes inactive, so it does nothing but periodica= lly + * checks the watermarks. 200 (20%) by default. + */ +static unsigned long wmarks_high __read_mostly =3D 200; +module_param(wmarks_high, ulong, 0600); + +/* + * Free memory rate (per thousand) for the middle watermark. + * + * If free memory of the system in bytes per thousand bytes is between thi= s and + * the low watermark, DAMON_LRU_SORT becomes active, so starts the monitor= ing + * and the LRU-lists sorting. 150 (15%) by default. + */ +static unsigned long wmarks_mid __read_mostly =3D 150; +module_param(wmarks_mid, ulong, 0600); + +/* + * Free memory rate (per thousand) for the low watermark. + * + * If free memory of the system in bytes per thousand bytes is lower than = this, + * DAMON_LRU_SORT becomes inactive, so it does nothing but periodically ch= ecks + * the watermarks. 50 (5%) by default. + */ +static unsigned long wmarks_low __read_mostly =3D 50; +module_param(wmarks_low, ulong, 0600); + +/* + * Sampling interval for the monitoring in microseconds. + * + * The sampling interval of DAMON for the hot/cold memory monitoring. Ple= ase + * refer to the DAMON documentation for more detail. 5 ms by default. + */ +static unsigned long sample_interval __read_mostly =3D 5000; +module_param(sample_interval, ulong, 0600); + +/* + * Aggregation interval for the monitoring in microseconds. + * + * The aggregation interval of DAMON for the hot/cold memory monitoring. + * Please refer to the DAMON documentation for more detail. 100 ms by def= ault. + */ +static unsigned long aggr_interval __read_mostly =3D 100000; +module_param(aggr_interval, ulong, 0600); + +/* + * Minimum number of monitoring regions. + * + * The minimal number of monitoring regions of DAMON for the hot/cold memo= ry + * monitoring. This can be used to set lower-bound of the monitoring qual= ity. + * But, setting this too high could result in increased monitoring overhea= d. + * Please refer to the DAMON documentation for more detail. 10 by default. + */ +static unsigned long min_nr_regions __read_mostly =3D 10; +module_param(min_nr_regions, ulong, 0600); + +/* + * Maximum number of monitoring regions. + * + * The maximum number of monitoring regions of DAMON for the hot/cold memo= ry + * monitoring. This can be used to set upper-bound of the monitoring over= head. + * However, setting this too low could result in bad monitoring quality. + * Please refer to the DAMON documentation for more detail. 1000 by defau= lt. + */ +static unsigned long max_nr_regions __read_mostly =3D 1000; +module_param(max_nr_regions, ulong, 0600); + +/* + * Start of the target memory region in physical address. + * + * The start physical address of memory region that DAMON_LRU_SORT will do= work + * against. By default, biggest System RAM is used as the region. + */ +static unsigned long monitor_region_start __read_mostly; +module_param(monitor_region_start, ulong, 0600); + +/* + * End of the target memory region in physical address. + * + * The end physical address of memory region that DAMON_LRU_SORT will do w= ork + * against. By default, biggest System RAM is used as the region. + */ +static unsigned long monitor_region_end __read_mostly; +module_param(monitor_region_end, ulong, 0600); + +/* + * PID of the DAMON thread + * + * If DAMON_LRU_SORT is enabled, this becomes the PID of the worker thread. + * Else, -1. + */ +static int kdamond_pid __read_mostly =3D -1; +module_param(kdamond_pid, int, 0400); + +/* + * Number of hot memory regions that tried to be LRU-sorted. + */ +static unsigned long nr_lru_sort_tried_hot_regions __read_mostly; +module_param(nr_lru_sort_tried_hot_regions, ulong, 0400); + +/* + * Total bytes of hot memory regions that tried to be LRU-sorted. + */ +static unsigned long bytes_lru_sort_tried_hot_regions __read_mostly; +module_param(bytes_lru_sort_tried_hot_regions, ulong, 0400); + +/* + * Number of hot memory regions that successfully be LRU-sorted. + */ +static unsigned long nr_lru_sorted_hot_regions __read_mostly; +module_param(nr_lru_sorted_hot_regions, ulong, 0400); + +/* + * Total bytes of hot memory regions that successfully be LRU-sorted. + */ +static unsigned long bytes_lru_sorted_hot_regions __read_mostly; +module_param(bytes_lru_sorted_hot_regions, ulong, 0400); + +/* + * Number of times that the time quota limit for hot regions have exceeded + */ +static unsigned long nr_hot_quota_exceeds __read_mostly; +module_param(nr_hot_quota_exceeds, ulong, 0400); + +/* + * Number of cold memory regions that tried to be LRU-sorted. + */ +static unsigned long nr_lru_sort_tried_cold_regions __read_mostly; +module_param(nr_lru_sort_tried_cold_regions, ulong, 0400); + +/* + * Total bytes of cold memory regions that tried to be LRU-sorted. + */ +static unsigned long bytes_lru_sort_tried_cold_regions __read_mostly; +module_param(bytes_lru_sort_tried_cold_regions, ulong, 0400); + +/* + * Number of cold memory regions that successfully be LRU-sorted. + */ +static unsigned long nr_lru_sorted_cold_regions __read_mostly; +module_param(nr_lru_sorted_cold_regions, ulong, 0400); + +/* + * Total bytes of cold memory regions that successfully be LRU-sorted. + */ +static unsigned long bytes_lru_sorted_cold_regions __read_mostly; +module_param(bytes_lru_sorted_cold_regions, ulong, 0400); + +/* + * Number of times that the time quota limit for cold regions have exceeded + */ +static unsigned long nr_cold_quota_exceeds __read_mostly; +module_param(nr_cold_quota_exceeds, ulong, 0400); + +static struct damon_ctx *ctx; +static struct damon_target *target; + +struct damon_lru_sort_ram_walk_arg { + unsigned long start; + unsigned long end; +}; + +static int walk_system_ram(struct resource *res, void *arg) +{ + struct damon_lru_sort_ram_walk_arg *a =3D arg; + + if (a->end - a->start < resource_size(res)) { + a->start =3D res->start; + a->end =3D res->end; + } + return 0; +} + +/* + * Find biggest 'System RAM' resource and store its start and end address = in + * @start and @end, respectively. If no System RAM is found, returns fals= e. + */ +static bool get_monitoring_region(unsigned long *start, unsigned long *end) +{ + struct damon_lru_sort_ram_walk_arg arg =3D {}; + + walk_system_ram_res(0, ULONG_MAX, &arg, walk_system_ram); + if (arg.end <=3D arg.start) + return false; + + *start =3D arg.start; + *end =3D arg.end; + return true; +} + +/* Create a DAMON-based operation scheme for hot memory regions */ +static struct damos *damon_lru_sort_new_hot_scheme(unsigned int hot_thres) +{ + struct damos_watermarks wmarks =3D { + .metric =3D DAMOS_WMARK_FREE_MEM_RATE, + .interval =3D wmarks_interval, + .high =3D wmarks_high, + .mid =3D wmarks_mid, + .low =3D wmarks_low, + }; + struct damos_quota quota =3D { + /* + * Do not try LRU-lists sorting of hot pages for more than half + * of quota_ms milliseconds within quota_reset_interval_ms. + */ + .ms =3D quota_ms / 2, + .sz =3D 0, + .reset_interval =3D quota_reset_interval_ms, + /* Within the quota, mark hotter regions accessed first. */ + .weight_sz =3D 0, + .weight_nr_accesses =3D 1, + .weight_age =3D 0, + }; + struct damos *scheme =3D damon_new_scheme( + /* Find regions having PAGE_SIZE or larger size */ + PAGE_SIZE, ULONG_MAX, + /* and accessed for more than the threshold */ + hot_thres, UINT_MAX, + /* no matter its age */ + 0, UINT_MAX, + /* prioritize those on LRU lists, as soon as found */ + DAMOS_LRU_PRIO, + /* under the quota. */ + "a, + /* (De)activate this according to the watermarks. */ + &wmarks); + + return scheme; +} + +/* Create a DAMON-based operation scheme for cold memory regions */ +static struct damos *damon_lru_sort_new_cold_scheme(unsigned int cold_thre= s) +{ + struct damos_watermarks wmarks =3D { + .metric =3D DAMOS_WMARK_FREE_MEM_RATE, + .interval =3D wmarks_interval, + .high =3D wmarks_high, + .mid =3D wmarks_mid, + .low =3D wmarks_low, + }; + struct damos_quota quota =3D { + /* + * Do not try LRU-lists sorting of cold pages for more than + * half of quota_ms milliseconds within + * quota_reset_interval_ms. + */ + .ms =3D quota_ms / 2, + .sz =3D 0, + .reset_interval =3D quota_reset_interval_ms, + /* Within the quota, mark colder regions not accessed first. */ + .weight_sz =3D 0, + .weight_nr_accesses =3D 0, + .weight_age =3D 1, + }; + struct damos *scheme =3D damon_new_scheme( + /* Find regions having PAGE_SIZE or larger size */ + PAGE_SIZE, ULONG_MAX, + /* and not accessed at all */ + 0, 0, + /* for cold_thres or more micro-seconds, and */ + cold_thres, UINT_MAX, + /* mark those as not accessed, as soon as found */ + DAMOS_LRU_DEPRIO, + /* under the quota. */ + "a, + /* (De)activate this according to the watermarks. */ + &wmarks); + + return scheme; +} + +static int damon_lru_sort_apply_parameters(void) +{ + struct damos *scheme, *next_scheme; + struct damon_addr_range addr_range; + unsigned int hot_thres, cold_thres; + int err =3D 0; + + err =3D damon_set_attrs(ctx, sample_interval, aggr_interval, 0, + min_nr_regions, max_nr_regions); + if (err) + return err; + + /* free previously set schemes */ + damon_for_each_scheme_safe(scheme, next_scheme, ctx) + damon_destroy_scheme(scheme); + + /* aggr_interval / sample_interval is the maximum nr_accesses */ + hot_thres =3D aggr_interval / sample_interval * hot_thres_access_freq / + 1000; + scheme =3D damon_lru_sort_new_hot_scheme(hot_thres); + if (!scheme) + return -ENOMEM; + damon_add_scheme(ctx, scheme); + + cold_thres =3D cold_min_age / aggr_interval; + scheme =3D damon_lru_sort_new_cold_scheme(cold_thres); + if (!scheme) + return -ENOMEM; + damon_add_scheme(ctx, scheme); + + if (monitor_region_start > monitor_region_end) + return -EINVAL; + if (!monitor_region_start && !monitor_region_end && + !get_monitoring_region(&monitor_region_start, + &monitor_region_end)) + return -EINVAL; + addr_range.start =3D monitor_region_start; + addr_range.end =3D monitor_region_end; + return damon_set_regions(target, &addr_range, 1); +} + +static int damon_lru_sort_turn(bool on) +{ + int err; + + if (!on) { + err =3D damon_stop(&ctx, 1); + if (!err) + kdamond_pid =3D -1; + return err; + } + + err =3D damon_lru_sort_apply_parameters(); + if (err) + return err; + + err =3D damon_start(&ctx, 1, true); + if (err) + return err; + kdamond_pid =3D ctx->kdamond->pid; + return 0; +} + +static struct delayed_work damon_lru_sort_timer; +static void damon_lru_sort_timer_fn(struct work_struct *work) +{ + static bool last_enabled; + bool now_enabled; + + now_enabled =3D enabled; + if (last_enabled !=3D now_enabled) { + if (!damon_lru_sort_turn(now_enabled)) + last_enabled =3D now_enabled; + else + enabled =3D last_enabled; + } +} +static DECLARE_DELAYED_WORK(damon_lru_sort_timer, damon_lru_sort_timer_fn); + +static bool damon_lru_sort_initialized; + +static int damon_lru_sort_enabled_store(const char *val, + const struct kernel_param *kp) +{ + int rc =3D param_set_bool(val, kp); + + if (rc < 0) + return rc; + + if (!damon_lru_sort_initialized) + return rc; + + schedule_delayed_work(&damon_lru_sort_timer, 0); + + return 0; +} + +static const struct kernel_param_ops enabled_param_ops =3D { + .set =3D damon_lru_sort_enabled_store, + .get =3D param_get_bool, +}; + +module_param_cb(enabled, &enabled_param_ops, &enabled, 0600); +MODULE_PARM_DESC(enabled, + "Enable or disable DAMON_LRU_SORT (default: disabled)"); + +static int damon_lru_sort_handle_commit_inputs(void) +{ + int err; + + if (!commit_inputs) + return 0; + + err =3D damon_lru_sort_apply_parameters(); + commit_inputs =3D false; + return err; +} + +static int damon_lru_sort_after_aggregation(struct damon_ctx *c) +{ + struct damos *s; + + /* update the stats parameter */ + damon_for_each_scheme(s, c) { + if (s->action =3D=3D DAMOS_LRU_PRIO) { + nr_lru_sort_tried_hot_regions =3D s->stat.nr_tried; + bytes_lru_sort_tried_hot_regions =3D s->stat.sz_tried; + nr_lru_sorted_hot_regions =3D s->stat.nr_applied; + bytes_lru_sorted_hot_regions =3D s->stat.sz_applied; + nr_hot_quota_exceeds =3D s->stat.qt_exceeds; + } else if (s->action =3D=3D DAMOS_LRU_DEPRIO) { + nr_lru_sort_tried_cold_regions =3D s->stat.nr_tried; + bytes_lru_sort_tried_cold_regions =3D s->stat.sz_tried; + nr_lru_sorted_cold_regions =3D s->stat.nr_applied; + bytes_lru_sorted_cold_regions =3D s->stat.sz_applied; + nr_cold_quota_exceeds =3D s->stat.qt_exceeds; + } + } + + return damon_lru_sort_handle_commit_inputs(); +} + +static int damon_lru_sort_after_wmarks_check(struct damon_ctx *c) +{ + return damon_lru_sort_handle_commit_inputs(); +} + +static int __init damon_lru_sort_init(void) +{ + ctx =3D damon_new_ctx(); + if (!ctx) + return -ENOMEM; + + if (damon_select_ops(ctx, DAMON_OPS_PADDR)) + return -EINVAL; + + ctx->callback.after_wmarks_check =3D damon_lru_sort_after_wmarks_check; + ctx->callback.after_aggregation =3D damon_lru_sort_after_aggregation; + + target =3D damon_new_target(); + if (!target) { + damon_destroy_ctx(ctx); + return -ENOMEM; + } + damon_add_target(ctx, target); + + schedule_delayed_work(&damon_lru_sort_timer, 0); + + damon_lru_sort_initialized =3D true; + return 0; +} + +module_init(damon_lru_sort_init); --=20 2.25.1 From nobody Mon Apr 27 12:15:09 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D10E1CCA482 for ; Mon, 13 Jun 2022 20:33:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231759AbiFMUdl (ORCPT ); Mon, 13 Jun 2022 16:33:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33504 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232866AbiFMUdG (ORCPT ); Mon, 13 Jun 2022 16:33:06 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2998B843; Mon, 13 Jun 2022 12:23:15 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 65CC8614F6; Mon, 13 Jun 2022 19:23:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A9936C341CB; Mon, 13 Jun 2022 19:23:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655148194; bh=OcttZG3BikFqfAlshS+woLV21AAsp+0FLs1PXY7sox4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jcz+cyns6QrAxysGrweD0IGhn6x+BczwAs1ebqS79sLKHItDmnvq2dUmj8Km4eIM1 GtVBP+V2I8bPXllQunRvC8g6AVlesylG96c6zF1kFke7Q2GT3MFoxARZZMM11A4bmM hYaELrTYCa2IwGbIuYjvAxOkhCALGobRNUKUEgxAE0rBqMJiR1w5uKgtUQ7l+Z6i/X sn5oF4IHBOFatpRKo4ogQ4ASvVV9zd1VVavqtgdMRCcO7+KiFCY1Lr58kroRV1dPVa HmXSW6PNOqPWVNVXUmMk+/KphhcXcdenNvmEeoiEVS3n0IaOAsMaxmwe+pUGa9QMBl csCP/f9DegfgQ== From: SeongJae Park To: SeongJae Park , Jonathan Corbet Cc: damon@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 8/8] Docs/admin-guide/damon: add a document for DAMON_LRU_SORT Date: Mon, 13 Jun 2022 19:23:01 +0000 Message-Id: <20220613192301.8817-10-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220613192301.8817-1-sj@kernel.org> References: <20220613192301.8817-1-sj@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit documents the usage of DAMON_LRU_SORT for admins. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/index.rst | 1 + .../admin-guide/mm/damon/lru_sort.rst | 294 ++++++++++++++++++ 2 files changed, 295 insertions(+) create mode 100644 Documentation/admin-guide/mm/damon/lru_sort.rst diff --git a/Documentation/admin-guide/mm/damon/index.rst b/Documentation/a= dmin-guide/mm/damon/index.rst index 61aff88347f3..53762770e0e4 100644 --- a/Documentation/admin-guide/mm/damon/index.rst +++ b/Documentation/admin-guide/mm/damon/index.rst @@ -14,3 +14,4 @@ optimize those. start usage reclaim + lru_sort diff --git a/Documentation/admin-guide/mm/damon/lru_sort.rst b/Documentatio= n/admin-guide/mm/damon/lru_sort.rst new file mode 100644 index 000000000000..c09cace80651 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/lru_sort.rst @@ -0,0 +1,294 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D +DAMON-based LRU-lists Sorting +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D + +DAMON-based LRU-lists Sorting (DAMON_LRU_SORT) is a static kernel module t= hat +aimed to be used for proactive and lightweight data access pattern based +(de)prioritization of pages on their LRU-lists for making LRU-lists a more +trusworthy data access pattern source. + +Where Proactive LRU-lists Sorting is Required? +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +As page-granularity access checking overhead could be significant on huge +systems, LRU lists are normally not proactively sorted but partially and +reactively sorted for special events including specific user requests, sys= tem +calls and memory pressure. As a result, LRU lists are sometimes not so +perfectly prepared to be used as a trustworthy access pattern source for s= ome +situations including reclamation target pages selection under sudden memory +pressure. + +Because DAMON can identify access patterns of best-effort accuracy while +inducing only user-specified range of overhead, proactively running +DAMON_LRU_SORT could be helpful for making LRU lists more trustworthy acce= ss +pattern source with low and controlled overhead. + +How It Works? +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +DAMON_LRU_SORT finds hot pages (pages of memory regions that showing access +rates that higher than a user-specified threshold) and cold pages (pages of +memory regions that showing no access for a time that longer than a +user-specified threshold) using DAMON, and prioritizes hot pages while +deprioritizing cold pages on their LRU-lists. To avoid it consuming too m= uch +CPU for the prioritizations, a CPU time usage limit can be configured. Un= der +the limit, it prioritizes and deprioritizes more hot and cold pages first, +respectively. System administrators can also configure under what situati= on +this scheme should automatically activated and deactivated with three memo= ry +pressure watermarks. + +Its default parameters for hotness/coldness thresholds and CPU quota limit= are +conservatively chosen. That is, the module under its default parameters c= ould +be widely used without harm for common situations while providing a level = of +benefits for systems having clear hot/cold access patterns under memory +pressure while consuming only a limited small portion of CPU time. + +Interface: Module Parameters +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D + +To use this feature, you should first ensure your system is running on a k= ernel +that is built with ``CONFIG_DAMON_LRU_SORT=3Dy``. + +To let sysadmins enable or disable it and tune for the given system, +DAMON_LRU_SORT utilizes module parameters. That is, you can put +``damon_lru_sort.=3D`` on the kernel boot command line o= r write +proper values to ``/sys/modules/damon_lru_sort/parameters/`` fi= les. + +Below are the description of each parameter. + +enabled +------- + +Enable or disable DAMON_LRU_SORT. + +You can enable DAMON_LRU_SORT by setting the value of this parameter as ``= Y``. +Setting it as ``N`` disables DAMON_LRU_SORT. Note that DAMON_LRU_SORT cou= ld do +no real monitoring and LRU-lists sorting due to the watermarks-based activ= ation +condition. Refer to below descriptions for the watermarks parameter for t= his. + +commit_inputs +------------- + +Make DAMON_LRU_SORT reads the input parameters again, except ``enabled``. + +Input parameters that updated while DAMON_LRU_SORT is running are not appl= ied +by default. Once this parameter is set as ``Y``, DAMON_LRU_SORT reads val= ues +of parametrs except ``enabled`` again. Once the re-reading is done, this +parameter is set as ``N``. If invalid parameters are found while the +re-reading, DAMON_LRU_SORT will be disabled. + +hot_thres_access_freq +--------------------- + +Access frequency threshold for hot memory regions identification in permil. + +If a memory region is accessed in frequency of this or higher, DAMON_LRU_S= ORT +identifies the region as hot, and mark it as accessed on the LRU list, so = that +it could not be reclaimed under memory pressure. 50% by default. + +cold_min_age +------------ + +Time threshold for cold memory regions identification in microseconds. + +If a memory region is not accessed for this or longer time, DAMON_LRU_SORT +identifies the region as cold, and mark it as unaccessed on the LRU list, = so +that it could be reclaimed first under memory pressure. 120 seconds by +default. + +quota_ms +-------- + +Limit of time for trying the LRU lists sorting in milliseconds. + +DAMON_LRU_SORT tries to use only up to this time within a time window +(quota_reset_interval_ms) for trying LRU lists sorting. This can be used +for limiting CPU consumption of DAMON_LRU_SORT. If the value is zero, the +limit is disabled. + +10 ms by default. + +quota_reset_interval_ms +----------------------- + +The time quota charge reset interval in milliseconds. + +The charge reset interval for the quota of time (quota_ms). That is, +DAMON_LRU_SORT does not try LRU-lists sorting for more than quota_ms +milliseconds or quota_sz bytes within quota_reset_interval_ms milliseconds. + +1 second by default. + +wmarks_interval +--------------- + +The watermarks check time interval in microseconds. + +Minimal time to wait before checking the watermarks, when DAMON_LRU_SORT is +enabled but inactive due to its watermarks rule. 5 seconds by default. + +wmarks_high +----------- + +Free memory rate (per thousand) for the high watermark. + +If free memory of the system in bytes per thousand bytes is higher than th= is, +DAMON_LRU_SORT becomes inactive, so it does nothing but periodically check= s the +watermarks. 200 (20%) by default. + +wmarks_mid +---------- + +Free memory rate (per thousand) for the middle watermark. + +If free memory of the system in bytes per thousand bytes is between this a= nd +the low watermark, DAMON_LRU_SORT becomes active, so starts the monitoring= and +the LRU-lists sorting. 150 (15%) by default. + +wmarks_low +---------- + +Free memory rate (per thousand) for the low watermark. + +If free memory of the system in bytes per thousand bytes is lower than thi= s, +DAMON_LRU_SORT becomes inactive, so it does nothing but periodically check= s the +watermarks. 50 (5%) by default. + +sample_interval +--------------- + +Sampling interval for the monitoring in microseconds. + +The sampling interval of DAMON for the cold memory monitoring. Please ref= er to +the DAMON documentation (:doc:`usage`) for more detail. 5ms by default. + +aggr_interval +------------- + +Aggregation interval for the monitoring in microseconds. + +The aggregation interval of DAMON for the cold memory monitoring. Please +refer to the DAMON documentation (:doc:`usage`) for more detail. 100ms by +default. + +min_nr_regions +-------------- + +Minimum number of monitoring regions. + +The minimal number of monitoring regions of DAMON for the cold memory +monitoring. This can be used to set lower-bound of the monitoring quality. +But, setting this too high could result in increased monitoring overhead. +Please refer to the DAMON documentation (:doc:`usage`) for more detail. 1= 0 by +default. + +max_nr_regions +-------------- + +Maximum number of monitoring regions. + +The maximum number of monitoring regions of DAMON for the cold memory +monitoring. This can be used to set upper-bound of the monitoring overhea= d. +However, setting this too low could result in bad monitoring quality. Ple= ase +refer to the DAMON documentation (:doc:`usage`) for more detail. 1000 by +defaults. + +monitor_region_start +-------------------- + +Start of target memory region in physical address. + +The start physical address of memory region that DAMON_LRU_SORT will do wo= rk +against. By default, biggest System RAM is used as the region. + +monitor_region_end +------------------ + +End of target memory region in physical address. + +The end physical address of memory region that DAMON_LRU_SORT will do work +against. By default, biggest System RAM is used as the region. + +kdamond_pid +----------- + +PID of the DAMON thread. + +If DAMON_LRU_SORT is enabled, this becomes the PID of the worker thread. = Else, +-1. + +nr_lru_sort_tried_hot_regions +----------------------------- + +Number of hot memory regions that tried to be LRU-sorted. + +bytes_lru_sort_tried_hot_regions +-------------------------------- + +Total bytes of hot memory regions that tried to be LRU-sorted. + +nr_lru_sorted_hot_regions +------------------------- + +Number of hot memory regions that successfully be LRU-sorted. + +bytes_lru_sorted_hot_regions +---------------------------- + +Total bytes of hot memory regions that successfully be LRU-sorted. + +nr_hot_quota_exceeds +-------------------- + +Number of times that the time quota limit for hot regions have exceeded. + +nr_lru_sort_tried_cold_regions +------------------------------ + +Number of cold memory regions that tried to be LRU-sorted. + +bytes_lru_sort_tried_cold_regions +--------------------------------- + +Total bytes of cold memory regions that tried to be LRU-sorted. + +nr_lru_sorted_cold_regions +-------------------------- + +Number of cold memory regions that successfully be LRU-sorted. + +bytes_lru_sorted_cold_regions +----------------------------- + +Total bytes of cold memory regions that successfully be LRU-sorted. + +nr_cold_quota_exceeds +--------------------- + +Number of times that the time quota limit for cold regions have exceeded. + +Example +=3D=3D=3D=3D=3D=3D=3D + +Below runtime example commands make DAMON_LRU_SORT to find memory regions +having >=3D50% access frequency and LRU-prioritize while LRU-deprioritizing +memory regions that not accessed for 120 seconds. The prioritization and +deprioritization is limited to be done using only up to 1% CPU time to avo= id +DAMON_LRU_SORT consuming too much CPU time for the (de)prioritization. It= also +asks DAMON_LRU_SORT to do nothing if the system's free memory rate is more= than +50%, but start the real works if it becomes lower than 40%. If DAMON_RECL= AIM +doesn't make progress and therefore the free memory rate becomes lower than +20%, it asks DAMON_LRU_SORT to do nothing again, so that we can fall back = to +the LRU-list based page granularity reclamation. :: + + # cd /sys/modules/damon_lru_sort/parameters + # echo 500 > hot_thres_access_freq + # echo 120000000 > cold_min_age + # echo 10 > quota_ms + # echo 1000 > quota_reset_interval_ms + # echo 500 > wmarks_high + # echo 400 > wmarks_mid + # echo 200 > wmarks_low + # echo Y > enabled --=20 2.25.1