From nobody Wed Apr 8 10:13:38 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EDF7C4332F for ; Thu, 13 Oct 2022 22:45:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229852AbiJMWpS (ORCPT ); Thu, 13 Oct 2022 18:45:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33302 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229776AbiJMWpE (ORCPT ); Thu, 13 Oct 2022 18:45:04 -0400 Received: from resqmta-c1p-024063.sys.comcast.net (resqmta-c1p-024063.sys.comcast.net [IPv6:2001:558:fd00:56::8]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B45F65C96E for ; Thu, 13 Oct 2022 15:44:56 -0700 (PDT) Received: from resomta-c1p-023267.sys.comcast.net ([96.102.18.232]) by resqmta-c1p-024063.sys.comcast.net with ESMTP id j5u4o8WmbopRHj6uGozIg7; Thu, 13 Oct 2022 22:42:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcastmailservice.net; s=20211018a; t=1665700944; bh=RmsPdp6XB2/XHXf0fJ7VeuVLgpyyn/dl4lgDaA8Nfi4=; h=Received:Received:From:To:Subject:Date:Message-Id:MIME-Version; b=o+aFDZmt1HIJr8SdJmyUSe1F6OhddikpPnTNE/7TWKuY8ZeOlKKMXW8mb6qExDBmg Ttlz1va1XslPHu/tqqsY2297fd1RWO0KlPDdjY8ql8JSq4zd9Vo8mmoBaDRHA2gfEF QyUIWDPw6ru4Pb81MLiDsC5QHYxnPXpg+p5Mvl10BT90kN5pL244H06/i0DjOO5lDa BSfSPGFGgKEFr3SQc73vO9wlddOHLttZ3jDC2szN8r1N6f3C31ZXIJqA6CZ1RxmkrI vF2aWwE0qOB8BR1TSu3x0O+QL8kvymFjzy2GJVcqWQD2ZL9EfJGL+kW8NLGHqDAwoF ct0H9LC9SYUxw== Received: from jderrick-mobl4.amr.corp.intel.com ([71.205.181.50]) by resomta-c1p-023267.sys.comcast.net with ESMTPA id j6toofOVmA6uYj6tuozg6q; Thu, 13 Oct 2022 22:42:02 +0000 X-Xfinity-VAAS: gggruggvucftvghtrhhoucdtuddrgedvfedrfeekuddgudefucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuvehomhgtrghsthdqtfgvshhipdfqfgfvpdfpqffurfetoffkrfenuceurghilhhouhhtmecufedtudenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomheplfhonhgrthhhrghnucffvghrrhhitghkuceojhhonhgrthhhrghnrdguvghrrhhitghksehlihhnuhigrdguvghvqeenucggtffrrghtthgvrhhnpedtteeljeffgfffveehhfetveefuedvheevffffhedtjeeuvdevgfeftddtheeftdenucfkphepjedurddvtdehrddukedurdehtdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhephhgvlhhopehjuggvrhhrihgtkhdqmhhosghlgedrrghmrhdrtghorhhprdhinhhtvghlrdgtohhmpdhinhgvthepjedurddvtdehrddukedurdehtddpmhgrihhlfhhrohhmpehjohhnrghthhgrnhdruggvrhhrihgtkheslhhinhhugidruggvvhdpnhgspghrtghpthhtohepjedprhgtphhtthhopehsohhngheskhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqrhgrihgusehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqkhgvrhhnvghlsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepjhhonhgrthhhrghnrdguvghrrhhitghkse hsohhlihguihhgmhdrtghomhdprhgtphhtthhopehjohhnrghthhgrnhigrdhskhdruggvrhhrihgtkhesihhnthgvlhdrtghomhdprhgtphhtthhopehmrghrihhushiirdhtkhgrtgiihihksehlihhnuhigrdhinhhtvghlrdgtohhmpdhrtghpthhtohepjhhonhgrthhhrghnrdguvghrrhhitghksehlihhnuhigrdguvghv X-Xfinity-VMeta: sc=-100.00;st=legit From: Jonathan Derrick To: Song Liu Cc: , , jonathan.derrick@solidigm.com, jonathanx.sk.derrick@intel.com, Mariusz Tkaczyk , Jonathan Derrick Subject: [PATCH v2 1/3] md/bitmap: Add chunk-threshold unplugging Date: Thu, 13 Oct 2022 16:41:49 -0600 Message-Id: <20221013224151.300-2-jonathan.derrick@linux.dev> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221013224151.300-1-jonathan.derrick@linux.dev> References: <20221013224151.300-1-jonathan.derrick@linux.dev> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a mechanism to allow bitmap unplugging and flushing to wait until it has surpassed a defined threshold of dirty chunks. This allows certain high I/O write workloads to make good forward progress between bitmap updates or provide reliable bitmap consistency. The default behavior is previous behavior of always unplugging when called. Signed-off-by: Jonathan Derrick --- drivers/md/md-bitmap.c | 35 +++++++++++++++++++++++++++++++---- drivers/md/md-bitmap.h | 1 + drivers/md/md.h | 1 + 3 files changed, 33 insertions(+), 4 deletions(-) diff --git a/drivers/md/md-bitmap.c b/drivers/md/md-bitmap.c index bf6dffadbe6f..c5c77f8371a8 100644 --- a/drivers/md/md-bitmap.c +++ b/drivers/md/md-bitmap.c @@ -1004,7 +1004,7 @@ static int md_bitmap_file_test_bit(struct bitmap *bit= map, sector_t block) /* this gets called when the md device is ready to unplug its underlying * (slave) device queues -- before we let any writes go down, we need to * sync the dirty pages of the bitmap file to disk */ -void md_bitmap_unplug(struct bitmap *bitmap) +static void __md_bitmap_unplug(struct bitmap *bitmap) { unsigned long i; int dirty, need_write; @@ -1038,6 +1038,33 @@ void md_bitmap_unplug(struct bitmap *bitmap) if (test_bit(BITMAP_WRITE_ERROR, &bitmap->flags)) md_bitmap_file_kick(bitmap); } + +/* + * Conditional unplug based on user-defined parameter + * Defaults to unconditional behavior + */ +void md_bitmap_unplug(struct bitmap *bitmap) +{ + unsigned int flush_threshold =3D bitmap->mddev->bitmap_info.flush_thresho= ld; + + if (!flush_threshold) { + __md_bitmap_unplug(bitmap); + } else { + struct bitmap_page *bp =3D bitmap->counts.bp; + unsigned long pages =3D bitmap->counts.pages; + unsigned long k, count =3D 0; + + for (k =3D 0; k < pages; k++) + if (bp[k].map && !bp[k].hijacked) + count +=3D bp[k].count; + + if (count - bitmap->unplugged_count > flush_threshold) { + bitmap->unplugged_count =3D count; + md_bitmap_daemon_work(&bitmap->mddev->daemon_timer); + __md_bitmap_unplug(bitmap); + } + } +} EXPORT_SYMBOL(md_bitmap_unplug); =20 static void md_bitmap_set_memory_bits(struct bitmap *bitmap, sector_t offs= et, int needed); @@ -2012,9 +2039,9 @@ int md_bitmap_copy_from_slot(struct mddev *mddev, int= slot, for (i =3D 0; i < bitmap->storage.file_pages; i++) if (test_page_attr(bitmap, i, BITMAP_PAGE_PENDING)) set_page_attr(bitmap, i, BITMAP_PAGE_NEEDWRITE); - md_bitmap_unplug(bitmap); + __md_bitmap_unplug(bitmap); } - md_bitmap_unplug(mddev->bitmap); + __md_bitmap_unplug(mddev->bitmap); *low =3D lo; *high =3D hi; md_bitmap_free(bitmap); @@ -2246,7 +2273,7 @@ int md_bitmap_resize(struct bitmap *bitmap, sector_t = blocks, spin_unlock_irq(&bitmap->counts.lock); =20 if (!init) { - md_bitmap_unplug(bitmap); + __md_bitmap_unplug(bitmap); bitmap->mddev->pers->quiesce(bitmap->mddev, 0); } ret =3D 0; diff --git a/drivers/md/md-bitmap.h b/drivers/md/md-bitmap.h index cfd7395de8fd..49a93d8ff307 100644 --- a/drivers/md/md-bitmap.h +++ b/drivers/md/md-bitmap.h @@ -223,6 +223,7 @@ struct bitmap { unsigned long daemon_lastrun; /* jiffies of last run */ unsigned long last_end_sync; /* when we lasted called end_sync to * update bitmap with resync progress */ + unsigned long unplugged_count; /* last dirty count from md_bitmap_unplug = */ =20 atomic_t pending_writes; /* pending writes to the bitmap file */ wait_queue_head_t write_wait; diff --git a/drivers/md/md.h b/drivers/md/md.h index b4e2d8b87b61..1a558cb18bd4 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -501,6 +501,7 @@ struct mddev { int external; int nodes; /* Maximum number of nodes in the cluster */ char cluster_name[64]; /* Name of the cluster */ + unsigned int flush_threshold; /* how many dirty chunks between updates = */ } bitmap_info; =20 atomic_t max_corr_read_errors; /* max read retries */ --=20 2.31.1 From nobody Wed Apr 8 10:13:38 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF068C4332F for ; Thu, 13 Oct 2022 22:45:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229747AbiJMWpD (ORCPT ); Thu, 13 Oct 2022 18:45:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229460AbiJMWpB (ORCPT ); Thu, 13 Oct 2022 18:45:01 -0400 X-Greylist: delayed 150 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Thu, 13 Oct 2022 15:44:56 PDT Received: from resqmta-c1p-023463.sys.comcast.net (resqmta-c1p-023463.sys.comcast.net [IPv6:2001:558:fd00:56::4]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5683F45F44 for ; Thu, 13 Oct 2022 15:44:55 -0700 (PDT) Received: from resomta-c1p-023267.sys.comcast.net ([96.102.18.232]) by resqmta-c1p-023463.sys.comcast.net with ESMTP id j6RpojqbljS9Rj6uGop7ul; Thu, 13 Oct 2022 22:42:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcastmailservice.net; s=20211018a; t=1665700944; bh=LV8/JcMHb+MmlhgjlVRQXW/dcI3Ijaatxb0kxbiP4wE=; h=Received:Received:From:To:Subject:Date:Message-Id:MIME-Version; b=Bf/IvOWLEP/fzD0l0knDUzF8vrErgOOJD7mWZY2FHQhI6LBMR/fPjf+eFdEltgIfL Vyg3e0db4ZRkJnbukq6EGFMsWcl32u2BPQxWFHjJkkDU43jKOYZuWSYV4mkF820JI0 xSbH5xIyOmXyN+ZIB86lmccpn6HbMMBy9XultRXTUIDTJpDng6yWI2lmlP5oCo97ZQ Pd843NeFvHzxM6Kr0LoDNxemzP4+nMd7LM1T4FnSahmm2pT+7TqSVkNq9c+ltu1EBl 9YbtibG3+8zXI96C9t/e2hBJ77q1y/UgxK21ogksiTsX0G3gXwfB3/slJxswyil1N3 F92l4sdjXwMQw== Received: from jderrick-mobl4.amr.corp.intel.com ([71.205.181.50]) by resomta-c1p-023267.sys.comcast.net with ESMTPA id j6toofOVmA6uYj6tuozg6t; Thu, 13 Oct 2022 22:42:03 +0000 X-Xfinity-VAAS: gggruggvucftvghtrhhoucdtuddrgedvfedrfeekuddgudefucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuvehomhgtrghsthdqtfgvshhipdfqfgfvpdfpqffurfetoffkrfenuceurghilhhouhhtmecufedtudenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomheplfhonhgrthhhrghnucffvghrrhhitghkuceojhhonhgrthhhrghnrdguvghrrhhitghksehlihhnuhigrdguvghvqeenucggtffrrghtthgvrhhnpedtteeljeffgfffveehhfetveefuedvheevffffhedtjeeuvdevgfeftddtheeftdenucfkphepjedurddvtdehrddukedurdehtdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhephhgvlhhopehjuggvrhhrihgtkhdqmhhosghlgedrrghmrhdrtghorhhprdhinhhtvghlrdgtohhmpdhinhgvthepjedurddvtdehrddukedurdehtddpmhgrihhlfhhrohhmpehjohhnrghthhgrnhdruggvrhhrihgtkheslhhinhhugidruggvvhdpnhgspghrtghpthhtohepjedprhgtphhtthhopehsohhngheskhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqrhgrihgusehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqkhgvrhhnvghlsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepjhhonhgrthhhrghnrdguvghrrhhitghkse hsohhlihguihhgmhdrtghomhdprhgtphhtthhopehjohhnrghthhgrnhigrdhskhdruggvrhhrihgtkhesihhnthgvlhdrtghomhdprhgtphhtthhopehmrghrihhushiirdhtkhgrtgiihihksehlihhnuhigrdhinhhtvghlrdgtohhmpdhrtghpthhtohepjhhonhgrthhhrghnrdguvghrrhhitghksehlihhnuhigrdguvghv X-Xfinity-VMeta: sc=-100.00;st=legit From: Jonathan Derrick To: Song Liu Cc: , , jonathan.derrick@solidigm.com, jonathanx.sk.derrick@intel.com, Mariusz Tkaczyk , Jonathan Derrick Subject: [PATCH v2 2/3] md/bitmap: Add sysfs interface for flush threshold Date: Thu, 13 Oct 2022 16:41:50 -0600 Message-Id: <20221013224151.300-3-jonathan.derrick@linux.dev> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221013224151.300-1-jonathan.derrick@linux.dev> References: <20221013224151.300-1-jonathan.derrick@linux.dev> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Adds a sysfs interface in the bitmap device for setting the chunk flush threshold. This is an unsigned integer value which defines the amount of dirty chunks allowed to be pending between bitmap flushes. Signed-off-by: Jonathan Derrick --- Documentation/admin-guide/md.rst | 5 +++++ drivers/md/md-bitmap.c | 33 ++++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/Documentation/admin-guide/md.rst b/Documentation/admin-guide/m= d.rst index d8fc9a59c086..d688ae4065cf 100644 --- a/Documentation/admin-guide/md.rst +++ b/Documentation/admin-guide/md.rst @@ -401,6 +401,11 @@ All md devices contain: once the array becomes non-degraded, and this fact has been recorded in the metadata. =20 + bitmap/flush_threshold + The number of outstanding dirty chunks that are allowed to be pending + before unplugging the bitmap queue. The default behavior is to always + unplugging the queue when requested. + consistency_policy This indicates how the array maintains consistency in case of unexpec= ted shutdown. It can be: diff --git a/drivers/md/md-bitmap.c b/drivers/md/md-bitmap.c index c5c77f8371a8..cd8250368860 100644 --- a/drivers/md/md-bitmap.c +++ b/drivers/md/md-bitmap.c @@ -2652,6 +2652,38 @@ static struct md_sysfs_entry max_backlog_used =3D __ATTR(max_backlog_used, S_IRUGO | S_IWUSR, behind_writes_used_show, behind_writes_used_reset); =20 +static ssize_t +bitmap_flush_threshold_show(struct mddev *mddev, char *page) +{ + ssize_t ret; + spin_lock(&mddev->lock); + if (mddev->bitmap =3D=3D NULL) + ret =3D sprintf(page, "0\n"); + else + ret =3D sprintf(page, "%u\n", + mddev->bitmap_info.flush_threshold); + spin_unlock(&mddev->lock); + return ret; +} + +static ssize_t +bitmap_flush_threshold_store(struct mddev *mddev, const char *buf, size_t = len) +{ + unsigned int thresh; + int ret; + if (!mddev->bitmap) + return -ENOENT; + ret =3D kstrtouint(buf, 10, &thresh); + if (ret) + return ret; + mddev->bitmap_info.flush_threshold =3D thresh; + return len; +} + +static struct md_sysfs_entry bitmap_flush_threshold =3D +__ATTR(flush_threshold, S_IRUGO | S_IWUSR, + bitmap_flush_threshold_show, bitmap_flush_threshold_store); + static struct attribute *md_bitmap_attrs[] =3D { &bitmap_location.attr, &bitmap_space.attr, @@ -2661,6 +2693,7 @@ static struct attribute *md_bitmap_attrs[] =3D { &bitmap_metadata.attr, &bitmap_can_clear.attr, &max_backlog_used.attr, + &bitmap_flush_threshold.attr, NULL }; const struct attribute_group md_bitmap_group =3D { --=20 2.31.1 From nobody Wed Apr 8 10:13:38 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFA0EC4332F for ; Thu, 13 Oct 2022 22:45:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229836AbiJMWpN (ORCPT ); Thu, 13 Oct 2022 18:45:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33300 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229773AbiJMWpE (ORCPT ); Thu, 13 Oct 2022 18:45:04 -0400 Received: from resqmta-c1p-023462.sys.comcast.net (resqmta-c1p-023462.sys.comcast.net [IPv6:2001:558:fd00:56::2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B454A5B05E for ; Thu, 13 Oct 2022 15:44:56 -0700 (PDT) Received: from resomta-c1p-023267.sys.comcast.net ([96.102.18.232]) by resqmta-c1p-023462.sys.comcast.net with ESMTP id j6RpomdrT03bPj6uGoD3ZD; Thu, 13 Oct 2022 22:42:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcastmailservice.net; s=20211018a; t=1665700944; bh=Vod9bfBwBD3lRXXsfZrFXG7EXQYLJ2cHVyFKENUAAzU=; h=Received:Received:From:To:Subject:Date:Message-Id:MIME-Version; b=J690zIjBgP0kPbDXXI+Neca+cVOheOOsmD6EbSP2jjoIDXf24bi9A40PBQMNAfYAH f3ELOCc4QbG3JzZTRCawGr1f4VqxcbPv8bWvI7LdOQ7NA4pW4kr+cYBR1qRd7e8N7h SWxTvgvbnZGxVGIIY+xzhNAgIglF793NBeWjsZ18UNXvMfes1vR6aBOYJNKA3Iaujc PpS/wBZTN3TJYdw6xQCk4McWlLRyqQRnvLh05sSWN9eWE7D7SDFhx4OtFX8QfAJxEf +jjzrTZfjKbqKUe2JuW3OwHxsMZSxj6KNnSpKdqkVLQTuDCtotW3xdvj9qzwVL8NQF Kg7TmB2D1jFqA== Received: from jderrick-mobl4.amr.corp.intel.com ([71.205.181.50]) by resomta-c1p-023267.sys.comcast.net with ESMTPA id j6toofOVmA6uYj6tvozg6y; Thu, 13 Oct 2022 22:42:04 +0000 X-Xfinity-VAAS: gggruggvucftvghtrhhoucdtuddrgedvfedrfeekuddgudefucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuvehomhgtrghsthdqtfgvshhipdfqfgfvpdfpqffurfetoffkrfenuceurghilhhouhhtmecufedtudenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurhephffvvefufffkofgjfhgggfestdekredtredttdenucfhrhhomheplfhonhgrthhhrghnucffvghrrhhitghkuceojhhonhgrthhhrghnrdguvghrrhhitghksehlihhnuhigrdguvghvqeenucggtffrrghtthgvrhhnpedtteeljeffgfffveehhfetveefuedvheevffffhedtjeeuvdevgfeftddtheeftdenucfkphepjedurddvtdehrddukedurdehtdenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmhephhgvlhhopehjuggvrhhrihgtkhdqmhhosghlgedrrghmrhdrtghorhhprdhinhhtvghlrdgtohhmpdhinhgvthepjedurddvtdehrddukedurdehtddpmhgrihhlfhhrohhmpehjohhnrghthhgrnhdruggvrhhrihgtkheslhhinhhugidruggvvhdpnhgspghrtghpthhtohepjedprhgtphhtthhopehsohhngheskhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqrhgrihgusehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtoheplhhinhhugidqkhgvrhhnvghlsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepjhhonhgrthhhrghnrdguvghrrhhitghkse hsohhlihguihhgmhdrtghomhdprhgtphhtthhopehjohhnrghthhgrnhigrdhskhdruggvrhhrihgtkhesihhnthgvlhdrtghomhdprhgtphhtthhopehmrghrihhushiirdhtkhgrtgiihihksehlihhnuhigrdhinhhtvghlrdgtohhmpdhrtghpthhtohepjhhonhgrthhhrghnrdguvghrrhhitghksehlihhnuhigrdguvghv X-Xfinity-VMeta: sc=-100.00;st=legit From: Jonathan Derrick To: Song Liu Cc: , , jonathan.derrick@solidigm.com, jonathanx.sk.derrick@intel.com, Mariusz Tkaczyk , Jonathan Derrick Subject: [PATCH v2 3/3] md/bitmap: Convert daemon_work to proper timer Date: Thu, 13 Oct 2022 16:41:51 -0600 Message-Id: <20221013224151.300-4-jonathan.derrick@linux.dev> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221013224151.300-1-jonathan.derrick@linux.dev> References: <20221013224151.300-1-jonathan.derrick@linux.dev> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" It was observed that with certain high I/O workloads that the daemon work may never run, preventing the bitmap from fully flushing. Ensure the bitmap fully flushes by converting the daemon worker to a proper timer. Signed-off-by: Jonathan Derrick --- drivers/md/md-bitmap.c | 30 +++++++++++++----------------- drivers/md/md-bitmap.h | 3 +-- drivers/md/md.c | 9 +++++++-- drivers/md/md.h | 1 + 4 files changed, 22 insertions(+), 21 deletions(-) diff --git a/drivers/md/md-bitmap.c b/drivers/md/md-bitmap.c index cd8250368860..34feb906243f 100644 --- a/drivers/md/md-bitmap.c +++ b/drivers/md/md-bitmap.c @@ -1250,8 +1250,9 @@ static bitmap_counter_t *md_bitmap_get_counter(struct= bitmap_counts *bitmap, * out to disk */ =20 -void md_bitmap_daemon_work(struct mddev *mddev) +void md_bitmap_daemon_work(struct timer_list *t) { + struct mddev *mddev =3D from_timer(mddev, t, daemon_timer); struct bitmap *bitmap; unsigned long j; unsigned long nextpage; @@ -1267,11 +1268,7 @@ void md_bitmap_daemon_work(struct mddev *mddev) mutex_unlock(&mddev->bitmap_info.mutex); return; } - if (time_before(jiffies, bitmap->daemon_lastrun - + mddev->bitmap_info.daemon_sleep)) - goto done; =20 - bitmap->daemon_lastrun =3D jiffies; if (bitmap->allclean) { mddev->thread->timeout =3D MAX_SCHEDULE_TIMEOUT; goto done; @@ -1372,6 +1369,7 @@ void md_bitmap_daemon_work(struct mddev *mddev) if (bitmap->allclean =3D=3D 0) mddev->thread->timeout =3D mddev->bitmap_info.daemon_sleep; + mod_timer(&mddev->daemon_timer, jiffies + mddev->bitmap_info.daemon_sleep= ); mutex_unlock(&mddev->bitmap_info.mutex); } =20 @@ -1735,21 +1733,16 @@ void md_bitmap_dirty_bits(struct bitmap *bitmap, un= signed long s, unsigned long void md_bitmap_flush(struct mddev *mddev) { struct bitmap *bitmap =3D mddev->bitmap; - long sleep; =20 if (!bitmap) /* there was no bitmap */ return; =20 /* run the daemon_work three time to ensure everything is flushed - * that can be - */ - sleep =3D mddev->bitmap_info.daemon_sleep * 2; - bitmap->daemon_lastrun -=3D sleep; - md_bitmap_daemon_work(mddev); - bitmap->daemon_lastrun -=3D sleep; - md_bitmap_daemon_work(mddev); - bitmap->daemon_lastrun -=3D sleep; - md_bitmap_daemon_work(mddev); + * that can be + */ + md_bitmap_daemon_work(&mddev->daemon_timer); + md_bitmap_daemon_work(&mddev->daemon_timer); + md_bitmap_daemon_work(&mddev->daemon_timer); if (mddev->bitmap_info.external) md_super_wait(mddev); md_bitmap_update_sb(bitmap); @@ -1826,7 +1819,7 @@ void md_bitmap_destroy(struct mddev *mddev) mutex_unlock(&mddev->bitmap_info.mutex); if (mddev->thread) mddev->thread->timeout =3D MAX_SCHEDULE_TIMEOUT; - + del_timer_sync(&mddev->daemon_timer); md_bitmap_free(bitmap); } =20 @@ -1904,7 +1897,10 @@ struct bitmap *md_bitmap_create(struct mddev *mddev,= int slot) if (err) goto error; =20 - bitmap->daemon_lastrun =3D jiffies; + timer_setup(&mddev->daemon_timer, md_bitmap_daemon_work, 0); + mddev->daemon_timer.expires =3D jiffies + mddev->bitmap_info.daemon_sleep; + add_timer(&mddev->daemon_timer); + err =3D md_bitmap_resize(bitmap, blocks, mddev->bitmap_info.chunksize, 1); if (err) goto error; diff --git a/drivers/md/md-bitmap.h b/drivers/md/md-bitmap.h index 49a93d8ff307..b7e8f2266bf2 100644 --- a/drivers/md/md-bitmap.h +++ b/drivers/md/md-bitmap.h @@ -220,7 +220,6 @@ struct bitmap { * the bitmap daemon - periodically wakes up and sweeps the bitmap * file, cleaning up bits and flushing out pages to disk as necessary */ - unsigned long daemon_lastrun; /* jiffies of last run */ unsigned long last_end_sync; /* when we lasted called end_sync to * update bitmap with resync progress */ unsigned long unplugged_count; /* last dirty count from md_bitmap_unplug = */ @@ -265,7 +264,7 @@ void md_bitmap_sync_with_cluster(struct mddev *mddev, sector_t new_lo, sector_t new_hi); =20 void md_bitmap_unplug(struct bitmap *bitmap); -void md_bitmap_daemon_work(struct mddev *mddev); +void md_bitmap_daemon_work(struct timer_list *t); =20 int md_bitmap_resize(struct bitmap *bitmap, sector_t blocks, int chunksize, int init); diff --git a/drivers/md/md.c b/drivers/md/md.c index afaf36b2f6ab..9f8a9f62b3db 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9265,8 +9265,13 @@ void md_check_recovery(struct mddev *mddev) if (mddev->suspended) return; =20 - if (mddev->bitmap) - md_bitmap_daemon_work(mddev); + if (mddev->bitmap) { + spin_lock(&pers_lock); + if (mddev->bitmap->allclean =3D=3D 0) + mddev->thread->timeout =3D + mddev->bitmap_info.daemon_sleep; + spin_unlock(&pers_lock); + } =20 if (signal_pending(current)) { if (mddev->pers->sync_request && !mddev->external) { diff --git a/drivers/md/md.h b/drivers/md/md.h index 1a558cb18bd4..578cc496c325 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -503,6 +503,7 @@ struct mddev { char cluster_name[64]; /* Name of the cluster */ unsigned int flush_threshold; /* how many dirty chunks between updates = */ } bitmap_info; + struct timer_list daemon_timer; =20 atomic_t max_corr_read_errors; /* max read retries */ struct list_head all_mddevs; --=20 2.31.1