From nobody Fri Sep 25 16:51:18 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A0F63B774B for ; Thu, 10 Sep 2026 08:11:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027882; cv=none; b=bYGTaxGBH8b4dwBYYAU6dTXyq/B5XONqi29hli1A4+z8pmLocWJ9WcNBdj54wtA5No3St044oBjFtm6vY2M9vXzmUTytgp47Yl2K80zk+Li6HpvtBU/2LOmSWl+yvvCit7MAwRFcvXC+aTc8V2fN+TiFp/4/yg5sZd11/VlCCpA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027882; c=relaxed/simple; bh=vvSIYJQObR1nDYB8lJ4RbBMp3hmCIa0DaCci/koCzZQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qdHTZtqpoDo5heASWGo8l1Z9Qoms1rDa5udeJcQUFegeuWR3G3a8H8xixzgOGkuaRsLq3eZgRGK6Jm4cYPaZahcR9F2JgLyL12payeHCLOKTp3zbMTChohCbCCGLwcy/yqXSijm75L2tDxnamDH1NGJqdXJrPQQKL9guuzz4Uf0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=ORqbxmC/; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="ORqbxmC/" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-49d0726cdbcso3153165e9.0 for ; Thu, 10 Sep 2026 01:11:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1789027878; x=1789632678; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=hZ0/qzbC4prOZzDrke+CQgOkg3IeDQ8d9wlQhN5ktvw=; b=ORqbxmC/PAWJtsxG3SHI5Ahe2sAXXPQWfkOeRFVR+3jRCpZ3/D8Q44r/dEu9xTgNXz OZAgxUk5WPft5onHT8qLs8GGoLLTAU8hW+wECZgkp2EQK0ozg5q8EEjNeb0dBR6j9tDs U5YJjFD4GMNrBcD9vpEwrP9CK3lpqR6Qud6Nem+QMH5WI9zkqshg/yCfyxawjgUNQ4J1 0JUU16TwJNEFhsoALr4X2LzAjjYBmM1hJJgzOD2euzuIOG2UdUlc9J4VDSVl+HHtrl/Q miqi9kysbexRPmOAQUs/ABpVzkJcjVwj0pjIEnJJmIkmaOLtgCnDYvz22jjXXeAmwXuY XdOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789027878; x=1789632678; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=hZ0/qzbC4prOZzDrke+CQgOkg3IeDQ8d9wlQhN5ktvw=; b=aQHT0Oco3xP3R0B90AL4x37XGE0+weyWve1aGLanzOrRdhYtUZZGZ8PeQCQSU7Xqiz nxwHqvxlWMnF8krF81VvdH7DlAyKnWsKfdbXtcI+bCH8ns2YSI1ivjFPopT4rA4XSYvN Hdulj3ChuXofhBw3Cn61F8DD2x8Ff1fy1Zlrj44iZPYGpHHcBtxjfU0AmdUMWBetJb9f m5agNkx9vmEpwe8cYgFRip5+gZtGNe3yAEDHpMukNXgFz5g6iKsIXj/DEB2SY8CBJLq6 5pl4zWHAvWz0a9Nvu0VbDf4oyMeybwIWxZ34EtmQPchIgabYi1QxZR/8A8UUfx1xePTW Yc3w== X-Forwarded-Encrypted: i=1; AKwUvBz6XQTV//59RDxZ5rWCiFnSIK3AWMcEckfJGk6+CIisliyRlcZ3AKmkzv/0YNH+wHI6LytPJuHZSiD7dD8=@vger.kernel.org X-Gm-Message-State: AFuF++lsj6Yj7fHIULTB5AlphYvr0He09pqaLiUo5yhLawM+sVHMD9MX jpPcHgldb1LMQiffxWaIl1nKcuNvmC1BtYoLYJCA/z6S58Hyq/MtIq9vcjkx1UepHxw= X-Gm-Gg: AYBFou0Ys1GBA/UuDEWLj0ZEyoNz5UMrEkNMuplx2fFkbJElOf/PUPXMSNM8G1JcGt/ Nrvca25KXFj4BwGfRWmPSfxUe4tG6HbObyWyLjjDUoEwNYEJMsSPjAzaoGMMVqvF/kUcw4BJpt3 kv4AFSSqnrY865lSjL7z0aOLwYq2aVvrXTtrLC0aIxGzWC87QtCLHnyPs6WRsVvZi+nXGMh/Bj5 1CmEVnncpa3Uphp2I0QIxkewac/RXXKmBUGB2R5qAGQZiM6k9IGpRLueqLsMOVKVVl8wwT0+h/P 5d2kN/tuqctrlHFBpN4xI2hBgi19uunuLN3449z+FMWpNEA9lVxY1oENXthR5mgSUfgRQcU/MLB lQFf4tFVnWEweOM8N10ZI5b/a/PF6Gr0CIq2uaUrEC3mHZDpdlr5Y+D+ByeToH6MO5++MsxmhU2 rLeWXO8IaLfasIPTjQN9RxdnqnmbIjsj6+vpGLkXVy/y7Cvo//OIY3ky79Egv67kXtNJRSw8sc7 otYm9Nz4sLGLFR9I6hwHYAUNr0XUhptF0xAQ0dDgTMA X-Received: by 2002:a05:600c:c48f:b0:49b:9241:7ff0 with SMTP id 5b1f17b1804b1-49d010c4f64mr288748745e9.0.1789027877469; Thu, 10 Sep 2026 01:11:17 -0700 (PDT) Received: from jwang-ThinkPad-T14-Gen-6.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d20fc1be3sm55261135e9.4.2026.09.10.01.11.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2026 01:11:16 -0700 (PDT) From: Jack Wang To: Song Liu , Yu Kuai , linux-raid@vger.kernel.org, Nilay Shroff , abd.masalkhi@gmail.com Cc: linux-block@vger.kernel.org, Jens Axboe , Christoph Hellwig , Damien Le Moal , Ming Lei , Xiao Ni , Li Nan , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, linux-kernel@vger.kernel.org, Jack Wang Subject: [PATCH v2 1/8] md: pass a queue_limits down to ->hot_add_disk() Date: Thu, 10 Sep 2026 10:11:06 +0200 Message-ID: <20260910081114.1605746-2-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260910081114.1605746-1-jinpu.wang@ionos.com> References: <20260910081114.1605746-1-jinpu.wang@ionos.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jack Wang Adding a leg stacks its queue limits, which mddev_stack_new_rdev() does by taking q->limits_lock itself. Callers holding reconfig_mutex or a suspended array cannot allow that, and must own the update instead. Give ->hot_add_disk(), remove_and_add_spares() and md_choose_sync_action() a struct queue_limits argument with three states: an update to stack into, NULL to let the personality take the lock as before, or MDDEV_STACK_SKIP to add the leg without touching the limits, for callers that can do neither. mddev_stack_rdev_into() stacks into a caller-owned update without the lock. Every caller still passes NULL and nothing passes the sentinel yet, so there is no functional change; the users follow. Assisted-by: LLM Signed-off-by: Jack Wang --- drivers/md/dm-raid.c | 2 +- drivers/md/md-linear.c | 28 ++++++++++++++---- drivers/md/md.c | 66 ++++++++++++++++++++++++++++++++---------- drivers/md/md.h | 11 ++++++- drivers/md/raid1.c | 10 +++++-- drivers/md/raid10.c | 19 +++++++++--- drivers/md/raid5.c | 5 ++-- 7 files changed, 110 insertions(+), 31 deletions(-) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index 8f5a5e1342a9..21a1922bee4f 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -3923,7 +3923,7 @@ static void attempt_restore_of_faulty_devices(struct = raid_set *rs) clear_bit(Faulty, &r->flags); clear_bit(WriteErrorSeen, &r->flags); =20 - if (mddev->pers->hot_add_disk(mddev, r)) { + if (mddev->pers->hot_add_disk(mddev, r, NULL)) { /* Failed to revive this device, try next */ r->raid_disk =3D r->saved_raid_disk =3D -1; r->flags =3D flags; diff --git a/drivers/md/md-linear.c b/drivers/md/md-linear.c index 73b367b61b87..da82c313d459 100644 --- a/drivers/md/md-linear.c +++ b/drivers/md/md-linear.c @@ -65,11 +65,16 @@ static sector_t linear_size(struct mddev *mddev, sector= _t sectors, int raid_disk return array_sectors; } =20 -static int linear_set_limits(struct mddev *mddev) +static int linear_set_limits(struct mddev *mddev, + struct queue_limits *caller_lim) { struct queue_limits lim; int err; =20 + /* the caller can neither stack nor take q->limits_lock */ + if (caller_lim =3D=3D MDDEV_STACK_SKIP) + return 0; + md_init_stacking_limits(&lim); lim.features |=3D BLK_FEAT_NOWAIT; lim.max_hw_sectors =3D mddev->chunk_sectors; @@ -82,10 +87,20 @@ static int linear_set_limits(struct mddev *mddev) if (err) return err; =20 + /* + * The caller owns an update and commits it itself; taking + * q->limits_lock here would take it a second time. + */ + if (caller_lim) { + *caller_lim =3D lim; + return 0; + } + return queue_limits_set(mddev->gendisk->queue, &lim); } =20 -static struct linear_conf *linear_conf(struct mddev *mddev, int raid_disks) +static struct linear_conf *linear_conf(struct mddev *mddev, int raid_disks, + struct queue_limits *lim) { struct linear_conf *conf; struct md_rdev *rdev; @@ -151,7 +166,7 @@ static struct linear_conf *linear_conf(struct mddev *md= dev, int raid_disks) conf->disks[i].rdev->sectors; =20 if (!mddev_is_dm(mddev)) { - ret =3D linear_set_limits(mddev); + ret =3D linear_set_limits(mddev, lim); if (ret) goto out; } @@ -171,7 +186,7 @@ static int linear_run(struct mddev *mddev) if (md_check_no_bitmap(mddev)) return -EINVAL; =20 - conf =3D linear_conf(mddev, mddev->raid_disks); + conf =3D linear_conf(mddev, mddev->raid_disks, NULL); if (IS_ERR(conf)) return PTR_ERR(conf); =20 @@ -186,7 +201,8 @@ static int linear_run(struct mddev *mddev) return ret; } =20 -static int linear_add(struct mddev *mddev, struct md_rdev *rdev) +static int linear_add(struct mddev *mddev, struct md_rdev *rdev, + struct queue_limits *lim) { /* Adding a drive to a linear array allows the array to grow. * It is permitted if the new drive has a matching superblock @@ -204,7 +220,7 @@ static int linear_add(struct mddev *mddev, struct md_rd= ev *rdev) rdev->raid_disk =3D rdev->saved_raid_disk; rdev->saved_raid_disk =3D -1; =20 - newconf =3D linear_conf(mddev, mddev->raid_disks + 1); + newconf =3D linear_conf(mddev, mddev->raid_disks + 1, lim); if (IS_ERR(newconf)) return PTR_ERR(newconf); =20 diff --git a/drivers/md/md.c b/drivers/md/md.c index 680b34a63cb3..28fc903ffeea 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -94,8 +94,8 @@ static DECLARE_WAIT_QUEUE_HEAD(resync_wait); */ static struct workqueue_struct *md_misc_wq; =20 -static int remove_and_add_spares(struct mddev *mddev, - struct md_rdev *this); +static int remove_and_add_spares(struct mddev *mddev, struct md_rdev *this, + struct queue_limits *lim); static void mddev_detach(struct mddev *mddev); static void export_rdev(struct md_rdev *rdev); static void md_wakeup_thread_directly(struct md_thread __rcu **thread); @@ -2994,7 +2994,7 @@ static int add_bound_rdev(struct md_rdev *rdev) */ super_types[mddev->major_version]. validate_super(mddev, NULL/*freshest*/, rdev); - err =3D mddev->pers->hot_add_disk(mddev, rdev); + err =3D mddev->pers->hot_add_disk(mddev, rdev, NULL); if (err) { md_kick_rdev_from_array(rdev); return err; @@ -3110,7 +3110,7 @@ state_store(struct md_rdev *rdev, const char *buf, si= ze_t len) } else if (cmd_match(buf, "remove")) { if (rdev->mddev->pers) { clear_bit(Blocked, &rdev->flags); - remove_and_add_spares(rdev->mddev, rdev); + remove_and_add_spares(rdev->mddev, rdev, NULL); } if (rdev->raid_disk >=3D 0) err =3D -EBUSY; @@ -3314,7 +3314,7 @@ slot_store(struct md_rdev *rdev, const char *buf, siz= e_t len) if (rdev->mddev->pers->hot_remove_disk =3D=3D NULL) return -EINVAL; clear_bit(Blocked, &rdev->flags); - remove_and_add_spares(rdev->mddev, rdev); + remove_and_add_spares(rdev->mddev, rdev, NULL); if (rdev->raid_disk >=3D 0) return -EBUSY; set_bit(MD_RECOVERY_NEEDED, &rdev->mddev->recovery); @@ -3344,7 +3344,8 @@ slot_store(struct md_rdev *rdev, const char *buf, siz= e_t len) rdev->saved_raid_disk =3D -1; clear_bit(In_sync, &rdev->flags); clear_bit(Bitmap_sync, &rdev->flags); - err =3D rdev->mddev->pers->hot_add_disk(rdev->mddev, rdev); + err =3D rdev->mddev->pers->hot_add_disk(rdev->mddev, rdev, + NULL); if (err) { rdev->raid_disk =3D -1; return err; @@ -6275,6 +6276,40 @@ int mddev_stack_new_rdev(struct mddev *mddev, struct= md_rdev *rdev) } EXPORT_SYMBOL_GPL(mddev_stack_new_rdev); =20 +/* + * Stack a new rdev into limits the caller already holds limits_lock for a= nd + * will commit itself. Used from paths that must take limits_lock before + * quiescing the array, see md_start_sync(). + */ +int mddev_stack_rdev_into(struct mddev *mddev, struct md_rdev *rdev, + struct queue_limits *lim) +{ + struct queue_limits tmp =3D *lim; + + if (mddev_is_dm(mddev)) + return 0; + + if (queue_logical_block_size(rdev->bdev->bd_disk->queue) > + queue_logical_block_size(mddev->gendisk->queue)) { + pr_err("%s: incompatible logical_block_size, can not add\n", + mdname(mddev)); + return -EINVAL; + } + + queue_limits_stack_bdev(&tmp, rdev->bdev, rdev->data_offset, + mddev->gendisk->disk_name); + + if (!queue_limits_stack_integrity_bdev(&tmp, rdev->bdev)) { + pr_err("%s: incompatible integrity profile for %pg\n", + mdname(mddev), rdev->bdev); + return -ENXIO; + } + + *lim =3D tmp; + return 0; +} +EXPORT_SYMBOL_GPL(mddev_stack_rdev_into); + /* update the optimal I/O size after a reshape */ void mddev_update_io_opt(struct mddev *mddev, unsigned int nr_stripes) { @@ -7706,7 +7741,7 @@ static int hot_remove_disk(struct mddev *mddev, dev_t= dev) goto kick_rdev; =20 clear_bit(Blocked, &rdev->flags); - remove_and_add_spares(mddev, rdev); + remove_and_add_spares(mddev, rdev, NULL); =20 if (rdev->raid_disk >=3D 0) goto busy; @@ -10167,8 +10202,8 @@ static int remove_spares(struct mddev *mddev, struc= t md_rdev *this) return removed; } =20 -static int remove_and_add_spares(struct mddev *mddev, - struct md_rdev *this) +static int remove_and_add_spares(struct mddev *mddev, struct md_rdev *this, + struct queue_limits *lim) { struct md_rdev *rdev; int spares =3D 0; @@ -10191,7 +10226,7 @@ static int remove_and_add_spares(struct mddev *mdde= v, continue; if (!test_bit(Journal, &rdev->flags)) rdev->recovery_offset =3D 0; - if (mddev->pers->hot_add_disk(mddev, rdev) =3D=3D 0) { + if (mddev->pers->hot_add_disk(mddev, rdev, lim) =3D=3D 0) { /* failure here is OK */ sysfs_link_rdev(mddev, rdev); if (!test_bit(Journal, &rdev->flags)) @@ -10206,7 +10241,8 @@ static int remove_and_add_spares(struct mddev *mdde= v, return spares; } =20 -static bool md_choose_sync_action(struct mddev *mddev, int *spares) +static bool md_choose_sync_action(struct mddev *mddev, int *spares, + struct queue_limits *lim) { /* Check if reshape is in progress first. */ if (mddev->reshape_position !=3D MaxSector) { @@ -10234,7 +10270,7 @@ static bool md_choose_sync_action(struct mddev *mdd= ev, int *spares) * also removed and re-added, to allow the personality to fail the * re-add. */ - *spares =3D remove_and_add_spares(mddev, NULL); + *spares =3D remove_and_add_spares(mddev, NULL, lim); if (*spares || test_bit(MD_RECOVERY_LAZY_RECOVER, &mddev->recovery)) { clear_bit(MD_RECOVERY_SYNC, &mddev->recovery); clear_bit(MD_RECOVERY_CHECK, &mddev->recovery); @@ -10294,11 +10330,11 @@ static void md_start_sync(struct work_struct *ws) * As we only add devices that are already in-sync, we can * activate the spares immediately. */ - remove_and_add_spares(mddev, NULL); + remove_and_add_spares(mddev, NULL, NULL); goto not_running; } =20 - if (!md_choose_sync_action(mddev, &spares)) + if (!md_choose_sync_action(mddev, &spares, NULL)) goto not_running; =20 if (!mddev->pers->sync_request) @@ -10849,7 +10885,7 @@ static void check_sb_changes(struct mddev *mddev, s= truct md_rdev *rdev) rdev2->saved_raid_disk =3D -1; else rdev2->saved_raid_disk =3D role; - ret =3D remove_and_add_spares(mddev, rdev2); + ret =3D remove_and_add_spares(mddev, rdev2, NULL); pr_info("Activated spare: %pg\n", rdev2->bdev); /* wakeup mddev->thread here, so array could diff --git a/drivers/md/md.h b/drivers/md/md.h index b6d2e8929a0f..ebdd57677062 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -765,7 +765,8 @@ struct md_personality * if appropriate, and should abort recovery if needed */ void (*error_handler)(struct mddev *mddev, struct md_rdev *rdev); - int (*hot_add_disk) (struct mddev *mddev, struct md_rdev *rdev); + int (*hot_add_disk)(struct mddev *mddev, struct md_rdev *rdev, + struct queue_limits *lim); int (*hot_remove_disk) (struct mddev *mddev, struct md_rdev *rdev); int (*spare_active) (struct mddev *mddev); sector_t (*sync_request)(struct mddev *mddev, sector_t sector_nr, @@ -1047,6 +1048,14 @@ int do_md_run(struct mddev *mddev); int mddev_stack_rdev_limits(struct mddev *mddev, struct queue_limits *lim, unsigned int flags); int mddev_stack_new_rdev(struct mddev *mddev, struct md_rdev *rdev); +int mddev_stack_rdev_into(struct mddev *mddev, struct md_rdev *rdev, + struct queue_limits *lim); +/* + * Sentinel for the queue_limits argument of ->hot_add_disk(). The caller= has + * no update to stack into and must not take q->limits_lock itself, so the= leg + * is added with the array's current limits. + */ +#define MDDEV_STACK_SKIP ((struct queue_limits *)ERR_PTR(-EAGAIN)) void mddev_update_io_opt(struct mddev *mddev, unsigned int nr_stripes); =20 extern const struct block_device_operations md_fops; diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index f0646fb24371..78effcac138d 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -1898,7 +1898,8 @@ static bool raid1_remove_conf(struct r1conf *conf, in= t disk) return true; } =20 -static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev) +static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev, + struct queue_limits *lim) { struct r1conf *conf =3D mddev->private; int err =3D -EEXIST; @@ -1923,7 +1924,12 @@ static int raid1_add_disk(struct mddev *mddev, struc= t md_rdev *rdev) for (mirror =3D first; mirror <=3D last; mirror++) { p =3D conf->mirrors + mirror; if (!p->rdev) { - err =3D mddev_stack_new_rdev(mddev, rdev); + if (lim =3D=3D MDDEV_STACK_SKIP) + err =3D 0; + else if (lim) + err =3D mddev_stack_rdev_into(mddev, rdev, lim); + else + err =3D mddev_stack_new_rdev(mddev, rdev); if (err) return err; =20 diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 1093c798d9dd..222bd7badcff 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -2095,7 +2095,8 @@ static int raid10_spare_active(struct mddev *mddev) return count; } =20 -static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev) +static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev, + struct queue_limits *lim) { struct r10conf *conf =3D mddev->private; int err =3D -EEXIST; @@ -2130,7 +2131,12 @@ static int raid10_add_disk(struct mddev *mddev, stru= ct md_rdev *rdev) continue; } =20 - err =3D mddev_stack_new_rdev(mddev, rdev); + if (lim =3D=3D MDDEV_STACK_SKIP) + err =3D 0; + else if (lim) + err =3D mddev_stack_rdev_into(mddev, rdev, lim); + else + err =3D mddev_stack_new_rdev(mddev, rdev); if (err) return err; p->head_position =3D 0; @@ -2147,7 +2153,12 @@ static int raid10_add_disk(struct mddev *mddev, stru= ct md_rdev *rdev) clear_bit(In_sync, &rdev->flags); set_bit(Replacement, &rdev->flags); rdev->raid_disk =3D repl_slot; - err =3D mddev_stack_new_rdev(mddev, rdev); + if (lim =3D=3D MDDEV_STACK_SKIP) + err =3D 0; + else if (lim) + err =3D mddev_stack_rdev_into(mddev, rdev, lim); + else + err =3D mddev_stack_new_rdev(mddev, rdev); if (err) return err; conf->fullsync =3D 1; @@ -4484,7 +4495,7 @@ static int raid10_start_reshape(struct mddev *mddev) rdev_for_each(rdev, mddev) if (rdev->raid_disk < 0 && !test_bit(Faulty, &rdev->flags)) { - if (raid10_add_disk(mddev, rdev) =3D=3D 0) { + if (raid10_add_disk(mddev, rdev, NULL) =3D=3D 0) { if (rdev->raid_disk >=3D conf->prev.raid_disks) set_bit(In_sync, &rdev->flags); diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index b91545ce090d..0ec555ada64a 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -8441,7 +8441,8 @@ static int raid5_remove_disk(struct mddev *mddev, str= uct md_rdev *rdev) return err; } =20 -static int raid5_add_disk(struct mddev *mddev, struct md_rdev *rdev) +static int raid5_add_disk(struct mddev *mddev, struct md_rdev *rdev, + struct queue_limits *lim) { struct r5conf *conf =3D mddev->private; int ret, err =3D -EEXIST; @@ -8728,7 +8729,7 @@ static int raid5_start_reshape(struct mddev *mddev) rdev_for_each(rdev, mddev) if (rdev->raid_disk < 0 && !test_bit(Faulty, &rdev->flags)) { - if (raid5_add_disk(mddev, rdev) =3D=3D 0) { + if (raid5_add_disk(mddev, rdev, NULL) =3D=3D 0) { if (rdev->raid_disk >=3D conf->previous_raid_disks) set_bit(In_sync, &rdev->flags); --=20 2.43.0 From nobody Fri Sep 25 16:51:18 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1882C41D4F1 for ; Thu, 10 Sep 2026 08:11:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027883; cv=none; b=oPxD5TqD1c0+UJXJZc24iIMRRzWLI9iI/Yacl0CKI1m9+njYM+OAVoRKZKNToW8n2Gb2SRgD/6wrf9+pMjXaYIiwFOf40QshYhUjNS4RfKsCZi0NeRVJqkuuDaZE8dsIzsAIDyAQHRdQSm5DZTVs5G6lEiJaOiPs2zOwsOJrxlw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027883; c=relaxed/simple; bh=gne6jQ1eDgyN6cYZe+ZWpX+VwTcHT+2IRfoXziUB8N4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=roYQhG/zIjS0fzjCptlJ1B37m4j46jowCgBcbSZk6nVK8NmOBqZ89sj85PxPEFnN2RNtLI8EeySgMrb953Y5WSlfuwWoJUa3aAOeBCmunApcWHEcXGKxiMTR1sDIH0mDkZhlMA7iy+YYawaKa9LYqjDnbJJ0FJr40EZsCIAtHJc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=Ik4lBZEL; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="Ik4lBZEL" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-499db1a5b5eso2307075e9.2 for ; Thu, 10 Sep 2026 01:11:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1789027879; x=1789632679; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=OA45OxGe2lbIuXbjnmED/p8Au3ms9csyQD/sDFCo9UY=; b=Ik4lBZEL9ox7iKIg74qfl6WWycLFAH5F4uUSDSXnCPH63C1xDRycJY0ngTK52uvxca AEFHmX0WKqmfJ5VHu6okFKdcFfW6FPoptd/fgaMOjT4tAXRaZGr72/tKSsgqav69idvU Int7BnnTvqlJVUNhuec1Vx4Ss9NCzmJgJRIrMogda/RN0XjckxdAjXmDx9lCjYV8c7lz 9atRoJLv+Z5APgV4meCzDXoTnsVK1Grn5dbs+YHX3YK3kLbpVJEQB0T4iHDaaIJBrVLf KKypE2AEWB6n1xI9BHaXB5UiyEzsjGX3HoWb9XJU4uSUb+ORoEgRWkqRRzaDII0mbWPO CiHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789027879; x=1789632679; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=OA45OxGe2lbIuXbjnmED/p8Au3ms9csyQD/sDFCo9UY=; b=XQX4eUXqDTr7kEKDW+DGQEPvwqGf1RWjHDTzpuMKbix9pD1d6KjDfpVDg+jryJeus+ 2po7swav67+NkEleByTkNA7vU+uHtiFBKmgbM2+x6wnHSaUNw41aFnaYJObrYYxJkN0L TCaUSVsBnzxMJbhdX0wO2HRfa3Yy/XtlQe83fgSzCo0y65eSQP6BmmnBs+ZWRDbY/slV 1HFSnK/nSUq7wrr84LcbUCwuGhkVovZ8/uUr5OJvjEW/xA0S06Jy5vPkUcJ5c/dYZ9oH dtu3IfKDO5p6SJKrxSThjexgGWEJdA2OPZc2Zeknp11WxfTGMHUoMqEvZflvkS+IZcV4 wOhw== X-Forwarded-Encrypted: i=1; AKwUvBzqopg4QWC8EzQOiFZpK3kbT7Ho5P1h/uWosBqDcXU+zhi5lGMAhpTm+bkJJZ3PgjzF0dcXpobaBrAP7HY=@vger.kernel.org X-Gm-Message-State: AFuF++mkBs7WDNciFUvZ2OKssRAgiz2d8pnBLMRAjg3Tg04VAaYtsnoz 1vh0u2bSvPw1onQOm5F0O1T8u1Uan+7nCEGmASHW8C2qtHHninUUc6Z4D4vFcbKbAiM= X-Gm-Gg: AYBFou0Bn9j9eL1A1CBsBuwLKTqjHJ821mX/ko3FPOv4AKv+DAqRRzJC1MKy2rt574G yGhZGkxOgCwurruwr8E0ooQBCYFYVnHwWNP+/llgL1MDmQeRlMfWdqw0m/oFzXy+Ofqiqg+rtUC jq4fHzr3oslKTLUFZZaVhr+0AlM6k6dWuxYfjjBVIixT/4dglYGgB+x1djAOmejGFb+bRqC8BWm VjHMYYdGDgqr6uhqGHvmJfGyQANJTAJkYuK/pjakHrB1qDsWt1h5OJDrwyuje+CwuDVMUnW+ebW QERC0vBnxHyERK94G1zGJHEV7HGIgQVb8XfXwAaZ+nnwXti0Os/GEHgipUsAc68mOfTGMZEdx5w 7qeA8axjC/A7Rj+XV7gA1As+wfTzePoG1FPtDh5s4qRyhstoAsWrGFZsY1rNqoQcj/7wVdlcnRN 25JMnYl0WSsemQIyPl/dmeqWkz2xLM9Kj3fE6FD6K1CSbqMwExHsPCacpSwPzUv6c4AnHNCrn9x p3aAk1knm6sMiM76FMwKwQhTPwSAWSw/aTyjaTp74gF X-Received: by 2002:a05:600c:c494:b0:49b:916a:e553 with SMTP id 5b1f17b1804b1-49d010c2a57mr296627915e9.0.1789027879077; Thu, 10 Sep 2026 01:11:19 -0700 (PDT) Received: from jwang-ThinkPad-T14-Gen-6.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d20fc1be3sm55261135e9.4.2026.09.10.01.11.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2026 01:11:18 -0700 (PDT) From: Jack Wang To: Song Liu , Yu Kuai , linux-raid@vger.kernel.org, Nilay Shroff , abd.masalkhi@gmail.com Cc: linux-block@vger.kernel.org, Jens Axboe , Christoph Hellwig , Damien Le Moal , Ming Lei , Xiao Ni , Li Nan , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, linux-kernel@vger.kernel.org, Jack Wang Subject: [PATCH v2 2/8] md: don't wait for q->limits_lock in check_sb_changes() Date: Thu, 10 Sep 2026 10:11:07 +0200 Message-ID: <20260910081114.1605746-3-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260910081114.1605746-1-jinpu.wang@ionos.com> References: <20260910081114.1605746-1-jinpu.wang@ionos.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jack Wang check_sb_changes() activates a spare another node added, reached from md_reload_sb() -> process_metadata_update() with reconfig_mutex held. Stacking the device's limits there waits for q->limits_lock under that mutex, which deadlocks: the lock's holder waits for the queue to drain, and that I/O can be waiting for a superblock update needing reconfig_mutex. The device is already a member, so its limits are stacked. Add it with MDDEV_STACK_SKIP and leave them alone. Assisted-by: LLM Signed-off-by: Jack Wang --- drivers/md/md.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 28fc903ffeea..e60dc2c7eb90 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -10885,7 +10885,14 @@ static void check_sb_changes(struct mddev *mddev, = struct md_rdev *rdev) rdev2->saved_raid_disk =3D -1; else rdev2->saved_raid_disk =3D role; - ret =3D remove_and_add_spares(mddev, rdev2, NULL); + /* + * reconfig_mutex is held, so q->limits_lock + * cannot be taken here. The device is + * already a member, its limits are stacked, + * so add it without touching them. + */ + ret =3D remove_and_add_spares(mddev, rdev2, + MDDEV_STACK_SKIP); pr_info("Activated spare: %pg\n", rdev2->bdev); /* wakeup mddev->thread here, so array could --=20 2.43.0 From nobody Fri Sep 25 16:51:18 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA5C83AB292 for ; Thu, 10 Sep 2026 08:11:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027884; cv=none; b=TGbM4WT6hK7ASfLxd+s1RupdMlfXVrL8bTzbnZVzA5jSjrayCbp5p/vkqXMGbJVoqvMrB5G7LmhZO2o/Qqmiq2ViOT8ExYG2bnGHxNJsYVgHlIY2A/nNxktbOSHTpWz1vRrIAx5vaawgeimILpSv75VbL0itjDzPMLRLpmgRwYo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027884; c=relaxed/simple; bh=K9IfShwLc/c5Al3z8c+Rt6NM850R3QuamoNvpA0sKPQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WrshCmnG9DWX+L9E9MeaPTCeLkRkm1gLSVT1mKCFW1OY86v9eAgibzOnsadTkA0aoectWdEiEEuHtBvltQYXfooLW5Mat0a2k4j0dSNn1h+AkU3Jax3/tHz4Iue7Eey6tNEnpspOFrKSA17k4T9pOhaWrRMFRDfjn+XKt8XQNJU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=VX3oo+i6; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="VX3oo+i6" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-499db1740e4so3233285e9.0 for ; Thu, 10 Sep 2026 01:11:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1789027880; x=1789632680; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=cgHwrkYbfZz0Oniv9vK94zADT2EQaqVt5Nm8OKIDVDc=; b=VX3oo+i6Z4fZYW5psAswbvCbxzhUG6RDy9AqWQDdZKltJvF+g1aGYC/sOFsUR76iLa +USCVZSfKPQWf/ceE/xpUx24XikmuzxvVRpFPrd+MYOqRIiFvIiGRVgpqTIeEV8cdk7h KzfbrB/+Rxe+J5mlhjw/pb+Nmz/WjrqxDJ6JTgHpj0cOM0cgVRwehs4Poi1/EiActOYU L7Uwu1Rq/Khu2XrIby/rGDDdGd8QiSmlnd/4z5zeAH0OUqGYbHWQFGndRuo89k19qm/4 Bm/AMpFU1r1elCR2sy48lfYV2mBH9FK3AwoFc9QquCgeWiNoYhylRzPR/s2/7G6E9jjn Spfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789027880; x=1789632680; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=cgHwrkYbfZz0Oniv9vK94zADT2EQaqVt5Nm8OKIDVDc=; b=DkztgbY9kJMcyCOaCestCJn9qbLRwtfZfX+WqSR/RUZx3eYLVcpu9/P3I2hBhg41qb bSHv2xnuLI2ayBmmIU8NzyfOa0uXSaZq2DY75hI+bHFl9JUuQupJYF53eYcw+mB0Whue obQ6zrC0a3gJTXrCBV1P/9TImYvesn1Nd+6eBu7/fO7vUcv3f0JBkTKYUOYP54nb7gIX rp5hnDonZkS4HrORRM22mJ+SGufs6P7bp/iV+Ael6dbZnvlgbqsDDgdezjGDpgx1kCjY RlJe44RXNT9GhBnRWq7QgIgLMfw1BBQijDQvK5hcqVLZIXFmQCFN8/lVzzU9/txmPTqZ 6WtQ== X-Forwarded-Encrypted: i=1; AKwUvByycKrehyGerE5OXV9UOTbKDlQtGadvV3PGBnbcxVkFcAkrJfcIcWchQV0jyNyPLt1G6u3aEnR79uHe3Y8=@vger.kernel.org X-Gm-Message-State: AFuF++mdsWGFkKVI+XwGSF36SHrqMHIrKN0jcBj3wOtN6Z72Wo5jNqnZ 5BY973eSe0oP6ftDvkqeT8tc622Hwea4OrYMccrZ4JbNqWj8G2xL3bXGH41B07o+T+U= X-Gm-Gg: AYBFou0WAw07jBEBKeFFCAyqbHj3kZ2/KidqLfAMvzWi3thZ55uBnZhGAZaAs4gOyFi mpjOxGP9dOgSCoPu7NacVLUx/yJu3CosXwv+cUe8KUgpjhXu5MIN0LgDL1NvBUlebTDZwKriSg1 crGqD8LEQJ0Vg/OWUXt0q9bEv58QHfbBwZIvAc2LJ/G4nWIN2Qm0ZJDJ20PJaZUD09dqrzXiOaq rf/dNrvcynoon5KUd3pSaExfWyfjDRLs8HOLnmt9yI5nWeR6Nqjfh+C4DRK+UFbSq7LBMpWw5ey YpsSJKjyPgFsJDaJIWn2QpOrzHmGXoo2e9lr6GAy+x5c/gMMG2/mcxrtKCQAMSlysQzXX2IhhNo rOKw0LhNLLSsum70N53eVn1u3aw46wSHox6J0w0weZmBczUoqgmjkL5f4wabZWfO2WrtBnhI7Mg LgWTGFScoPXzR33tvOcUtucBmuaikUceIG/J6mRt3uoQXaMtTyGxCuOfjiW6H5ZH4l6Ci5MUXQ4 WZk6H+JH0807fDNQWIBrZrAuAzADOOewKRigqyukEWpL0eIkW+3EK8= X-Received: by 2002:a05:600c:474a:b0:49c:ff81:e062 with SMTP id 5b1f17b1804b1-49d276628c8mr26060515e9.2.1789027880278; Thu, 10 Sep 2026 01:11:20 -0700 (PDT) Received: from jwang-ThinkPad-T14-Gen-6.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d20fc1be3sm55261135e9.4.2026.09.10.01.11.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2026 01:11:19 -0700 (PDT) From: Jack Wang To: Song Liu , Yu Kuai , linux-raid@vger.kernel.org, Nilay Shroff , abd.masalkhi@gmail.com Cc: linux-block@vger.kernel.org, Jens Axboe , Christoph Hellwig , Damien Le Moal , Ming Lei , Xiao Ni , Li Nan , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, linux-kernel@vger.kernel.org, Jack Wang Subject: [PATCH v2 3/8] md: pass a queue_limits through the rdev sysfs stores Date: Thu, 10 Sep 2026 10:11:08 +0200 Message-ID: <20260910081114.1605746-4-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260910081114.1605746-1-jinpu.wang@ionos.com> References: <20260910081114.1605746-1-jinpu.wang@ionos.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jack Wang state_store() and slot_store() can add a leg back to the array, which stacks its limits, and q->limits_lock has to be taken before the array is locked and suspended. Give the rdev sysfs store callback a struct queue_limits argument. rdev_attr_store() passes NULL, so there is no functional change; the user follows. Assisted-by: LLM Signed-off-by: Jack Wang --- drivers/md/md.c | 45 ++++++++++++++++++++++++++++++++------------- 1 file changed, 32 insertions(+), 13 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index e60dc2c7eb90..3067ea05ba27 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -3033,7 +3033,13 @@ static int cmd_match(const char *cmd, const char *st= r) struct rdev_sysfs_entry { struct attribute attr; ssize_t (*show)(struct md_rdev *, char *); - ssize_t (*store)(struct md_rdev *, const char *, size_t); + /* + * @lim: a queue limits update the caller owns, or NULL. Stores that + * can add a leg to the array must stack into it rather than take + * q->limits_lock themselves, see md_start_sync(). + */ + ssize_t (*store)(struct md_rdev *rdev, const char *page, size_t len, + struct queue_limits *lim); }; =20 static ssize_t @@ -3079,7 +3085,8 @@ state_show(struct md_rdev *rdev, char *page) } =20 static ssize_t -state_store(struct md_rdev *rdev, const char *buf, size_t len) +state_store(struct md_rdev *rdev, const char *buf, size_t len, + struct queue_limits *lim) { /* can write * faulty - simulates an error @@ -3257,7 +3264,8 @@ errors_show(struct md_rdev *rdev, char *page) } =20 static ssize_t -errors_store(struct md_rdev *rdev, const char *buf, size_t len) +errors_store(struct md_rdev *rdev, const char *buf, size_t len, + struct queue_limits *lim) { unsigned int n; int rv; @@ -3283,7 +3291,8 @@ slot_show(struct md_rdev *rdev, char *page) } =20 static ssize_t -slot_store(struct md_rdev *rdev, const char *buf, size_t len) +slot_store(struct md_rdev *rdev, const char *buf, size_t len, + struct queue_limits *lim) { int slot; int err; @@ -3378,7 +3387,8 @@ offset_show(struct md_rdev *rdev, char *page) } =20 static ssize_t -offset_store(struct md_rdev *rdev, const char *buf, size_t len) +offset_store(struct md_rdev *rdev, const char *buf, size_t len, + struct queue_limits *lim) { unsigned long long offset; if (kstrtoull(buf, 10, &offset) < 0) @@ -3404,7 +3414,8 @@ static ssize_t new_offset_show(struct md_rdev *rdev, = char *page) } =20 static ssize_t new_offset_store(struct md_rdev *rdev, - const char *buf, size_t len) + const char *buf, size_t len, + struct queue_limits *lim) { unsigned long long new_offset; struct mddev *mddev =3D rdev->mddev; @@ -3511,7 +3522,8 @@ static int strict_blocks_to_sectors(const char *buf, = sector_t *sectors) } =20 static ssize_t -rdev_size_store(struct md_rdev *rdev, const char *buf, size_t len) +rdev_size_store(struct md_rdev *rdev, const char *buf, size_t len, + struct queue_limits *lim) { struct mddev *my_mddev =3D rdev->mddev; sector_t oldsectors =3D rdev->sectors; @@ -3573,7 +3585,8 @@ static ssize_t recovery_start_show(struct md_rdev *rd= ev, char *page) return sprintf(page, "%llu\n", recovery_start); } =20 -static ssize_t recovery_start_store(struct md_rdev *rdev, const char *buf,= size_t len) +static ssize_t recovery_start_store(struct md_rdev *rdev, const char *buf,= size_t len, + struct queue_limits *lim) { unsigned long long recovery_start; =20 @@ -3612,7 +3625,9 @@ static ssize_t bb_show(struct md_rdev *rdev, char *pa= ge) { return badblocks_show(&rdev->badblocks, page, 0); } -static ssize_t bb_store(struct md_rdev *rdev, const char *page, size_t len) + +static ssize_t bb_store(struct md_rdev *rdev, const char *page, size_t len, + struct queue_limits *lim) { int rv =3D badblocks_store(&rdev->badblocks, page, len, 0); /* Maybe that ack was all we needed */ @@ -3627,7 +3642,9 @@ static ssize_t ubb_show(struct md_rdev *rdev, char *p= age) { return badblocks_show(&rdev->badblocks, page, 1); } -static ssize_t ubb_store(struct md_rdev *rdev, const char *page, size_t le= n) + +static ssize_t ubb_store(struct md_rdev *rdev, const char *page, size_t le= n, + struct queue_limits *lim) { return badblocks_store(&rdev->badblocks, page, len, 1); } @@ -3641,7 +3658,8 @@ ppl_sector_show(struct md_rdev *rdev, char *page) } =20 static ssize_t -ppl_sector_store(struct md_rdev *rdev, const char *buf, size_t len) +ppl_sector_store(struct md_rdev *rdev, const char *buf, size_t len, + struct queue_limits *lim) { unsigned long long sector; =20 @@ -3680,7 +3698,8 @@ ppl_size_show(struct md_rdev *rdev, char *page) } =20 static ssize_t -ppl_size_store(struct md_rdev *rdev, const char *buf, size_t len) +ppl_size_store(struct md_rdev *rdev, const char *buf, size_t len, + struct queue_limits *lim) { unsigned int size; =20 @@ -3766,7 +3785,7 @@ rdev_attr_store(struct kobject *kobj, struct attribut= e *attr, if (rdev->mddev =3D=3D NULL) rv =3D -ENODEV; else - rv =3D entry->store(rdev, page, length); + rv =3D entry->store(rdev, page, length, NULL); suspend ? mddev_unlock_and_resume(mddev) : mddev_unlock(mddev); } =20 --=20 2.43.0 From nobody Fri Sep 25 16:51:18 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DD0542589D for ; Thu, 10 Sep 2026 08:11:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027886; cv=none; b=RnjCC1paD5rWisFBmVYJaPkbtMqWVIdIdHjpIz4uB2yY41rcy3NGuibHoJ/KxkEIQ9pWRPseH3v1M8DThiS89o0xoiMy+tOQCLIL1iZgSdkZntMhAGiF5xsI1yu7saYEZKg3B13Vj+Ez9lLRVpuhij2cDlDk5dMUaAHU0nT79M8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027886; c=relaxed/simple; bh=SIwPKCSWNPT7kA5SO49gPlYWHhJgRQKntkekf3z0E/A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VfYo5mgYH8kfLrNxs4nwAavWZqAz0KhiUE8derTpCZsTgKyNv+RiTUOo4OkfEtlgEIel9Xt++/WxFdcntGoyjLobP2CoqQcNo7SQes3c5eN6D5cZvYadkeBcdiaOTY41rtSaizOayxG625ycgt2jiDXDFcvsK1sH/BLIJCmcG+0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=c8lOUu5/; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="c8lOUu5/" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-49cd5faa1cfso1716885e9.0 for ; Thu, 10 Sep 2026 01:11:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1789027881; x=1789632681; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=hIKqu2H5q6/v8mEseRX3ftASl1S+sA0hdu4/51URcsM=; b=c8lOUu5/1G67nuUz/+svYYor/KgjpMEzLrqJ/rNSwBk4Rb/pCBpX0tPq//iFs2Zd/m q4qXxCARcbjmvueVkmOhBuzWJzGbVisu81B3xB+sxExMlZac1xf2aPxms0VPQScEdGbP MjGt+yRUBEtoB9e1oZ+2g4veTM7Ey1zPCoT9Vy9MACByYlGVbWjERJMbftlnRtwOqPcT huT/nCaZ+ilzab+7lp7gFJNcz/nxi5CM3+vhZmv5wvkUKD8g6GR8KpmvAGWBO0KwYAx/ eoNexRZoa4lvtat58+py50TMHLUtlw3IQld2JgO9hs31M59uNCu0LN1cvNNvvDEqLyLa 7xCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789027881; x=1789632681; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=hIKqu2H5q6/v8mEseRX3ftASl1S+sA0hdu4/51URcsM=; b=fy4Q6yAaS5AbCLR0h4XZ7ZMRABLZYCy6ONb050i4dHXEQ0wCxMnuI0yrG1RVXyIOn9 /guaZcy/Nw5Xlh/uPDSp5+RU5Z+db8ySF14jHXGtHWDRwOQ5RTioIqXqPWfMhRwf1J/n +JSYVUOPRkXCEloJGrh14uGFGE9IzDX/k85FANasI7Jqt9UIC9mVgXZlXlpb0fIAExod 3VRh0EUxP7zFC2KnVuCtsQud+GNF5TZxRoLo3VSg5MF1OqItziZS6TtdhsWPApQtHrzK KgRn4SwqhsqmjiP+QXyI8RP5VYhEqdHCHP+yF8/eb1tVfEFR9CoUi1eOZJJI3+nnx8vO qCIw== X-Forwarded-Encrypted: i=1; AKwUvBzj5dcVGpegMs3LSCk2seI9EYNKMnUNkikhiQHFiWo0It2wJyb+sg9MSPaPgzdTXn6nlQf8IP+4UQLW5lE=@vger.kernel.org X-Gm-Message-State: AFuF++kkLLJH3vJIGGhURhI2Lnx+yKoOxMCproFtSONPEoMP36SKRccd vr2xugpUYwYSLrmXdgwaV5RbJd7OzCV9xeh1PJPt2iVc/cEsIyOU1OjxWP7FDrdAFtI= X-Gm-Gg: AYBFou1XO5nw+5BfJglWoXmjXnwHXEyCqB8p7aAiCS9bHx6urcDdh1JtpZThmsKoe+S upsyhRGHg+WNwp1+vb3NfLfqE+/YuKrg2hBhAFBrUDfN2blkoZIR6hKqxp+owUPaM0IRYkWBVF9 sJaqYKV+gsfclkViouqyGIQLH4NOeCPJChGhOHKGFszT/YNcYO5T8uJdgti16iaSP1dW1TGjJp6 hUMFAMJMQiyVIXbmOxZCofqtYlmpBTY+6sg6dmCJZpA5MHie7KjZl13Iyqcf4aRq5bIG7Yf0JeC G4aqFLJSgrS+PcW2AMH8U4IBBaRo3QjCduLSfm3Sz/tmxz6sxdX5nY+YqJU2Ef6nigoFcGLtQ24 bSDN5u8fpEheJ1H8e05/bV6D7yYNN5s1/6FxB12AZc+jcCiDHlCb7XU5H2Im/yDd7Gr8awErIMC CfK/cb3QFecMmIXKHZ5sqQxUFGvW5VFfdgoQgueSRXF1BHt+fKXX/gTiB2meL23oXVs8jpo3HQg I0Kr1yrAZCtZm/i36DAVTTAaVnv/+KHPOculmIqmDXR3+ucULwr3uU= X-Received: by 2002:a05:600c:8b2e:b0:49c:cbf4:572b with SMTP id 5b1f17b1804b1-49d01dcc1fdmr280205375e9.2.1789027881235; Thu, 10 Sep 2026 01:11:21 -0700 (PDT) Received: from jwang-ThinkPad-T14-Gen-6.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d20fc1be3sm55261135e9.4.2026.09.10.01.11.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2026 01:11:20 -0700 (PDT) From: Jack Wang To: Song Liu , Yu Kuai , linux-raid@vger.kernel.org, Nilay Shroff , abd.masalkhi@gmail.com Cc: linux-block@vger.kernel.org, Jens Axboe , Christoph Hellwig , Damien Le Moal , Ming Lei , Xiao Ni , Li Nan , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, linux-kernel@vger.kernel.org, Jack Wang Subject: [PATCH v2 4/8] md: defer the io_opt update out of the sync thread Date: Thu, 10 Sep 2026 10:11:09 +0200 Message-ID: <20260910081114.1605746-5-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260910081114.1605746-1-jinpu.wang@ionos.com> References: <20260910081114.1605746-1-jinpu.wang@ionos.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jack Wang mddev_update_io_opt() runs from end_reshape() in the sync thread, and md_reap_sync_thread() waits for that thread with reconfig_mutex held. Taking q->limits_lock there hangs a finishing reshape whenever the lock's holder waits for I/O that only md_check_recovery() can let complete, and no ordering avoids it: the sync thread is what lets that I/O finish. Hand the update to a work item, which holds neither reconfig_mutex nor the suspend and so takes q->limits_lock in the order the rest of md uses, before suspending. __md_stop() flushes it, as it suspends the array. Also give the function a queue_limits argument, so a caller that already owns an update has it changed in place; the users of that path follow. Assisted-by: LLM Signed-off-by: Jack Wang --- drivers/md/md.c | 49 ++++++++++++++++++++++++++++++++++++++------- drivers/md/md.h | 7 ++++++- drivers/md/raid10.c | 2 +- drivers/md/raid5.c | 2 +- 4 files changed, 50 insertions(+), 10 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 3067ea05ba27..87e17ba86d93 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -671,6 +671,7 @@ void mddev_put(struct mddev *mddev) =20 static void md_safemode_timeout(struct timer_list *t); static void md_start_sync(struct work_struct *ws); +static void md_io_opt_work(struct work_struct *ws); =20 static void active_io_release(struct percpu_ref *ref) { @@ -794,6 +795,7 @@ int mddev_init(struct mddev *mddev) mddev->level =3D LEVEL_NONE; =20 INIT_WORK(&mddev->sync_work, md_start_sync); + INIT_WORK(&mddev->io_opt_work, md_io_opt_work); INIT_WORK(&mddev->del_work, mddev_delayed_delete); =20 return 0; @@ -6330,20 +6332,51 @@ int mddev_stack_rdev_into(struct mddev *mddev, stru= ct md_rdev *rdev, EXPORT_SYMBOL_GPL(mddev_stack_rdev_into); =20 /* update the optimal I/O size after a reshape */ -void mddev_update_io_opt(struct mddev *mddev, unsigned int nr_stripes) +static void md_io_opt_work(struct work_struct *ws) { + struct mddev *mddev =3D container_of(ws, struct mddev, io_opt_work); + struct request_queue *q =3D mddev->gendisk->queue; struct queue_limits lim; =20 + /* + * Nothing is held here, so take q->limits_lock in the order the rest + * of md uses: before the suspend, see md_start_sync(). + */ + lim =3D queue_limits_start_update(q); + if (mddev_suspend(mddev, false) < 0) { + queue_limits_cancel_update(q); + return; + } + lim.io_opt =3D lim.io_min * READ_ONCE(mddev->io_opt_nr_stripes); + if (queue_limits_commit_update(q, &lim)) + pr_err("%s: could not apply queue limits\n", mdname(mddev)); + mddev_resume(mddev); +} + +void mddev_update_io_opt(struct mddev *mddev, unsigned int nr_stripes, + struct queue_limits *lim) +{ if (mddev_is_dm(mddev)) return; =20 - /* don't bother updating io_opt if we can't suspend the array */ - if (mddev_suspend(mddev, false) < 0) + /* + * With an update owned by the caller just change it in place; it is + * committed, and the array resumed, by whoever started it. Taking + * q->limits_lock here would nest it inside reconfig_mutex and the + * suspend, which deadlocks, see md_start_sync(). + */ + if (lim) { + lim->io_opt =3D lim->io_min * nr_stripes; return; - lim =3D queue_limits_start_update(mddev->gendisk->queue); - lim.io_opt =3D lim.io_min * nr_stripes; - queue_limits_commit_update(mddev->gendisk->queue, &lim); - mddev_resume(mddev); + } + + /* + * Called from the sync thread, which md_reap_sync_thread() waits for + * with reconfig_mutex held, so q->limits_lock cannot be taken here + * either. Hand it to a work item that holds neither. + */ + WRITE_ONCE(mddev->io_opt_nr_stripes, nr_stripes); + queue_work(md_misc_wq, &mddev->io_opt_work); } EXPORT_SYMBOL_GPL(mddev_update_io_opt); =20 @@ -7139,6 +7172,8 @@ static void __md_stop(struct mddev *mddev) { struct md_personality *pers =3D mddev->pers; =20 + /* the deferred io_opt update suspends the array, so let it finish */ + flush_work(&mddev->io_opt_work); mddev_detach(mddev); md_bitmap_destroy(mddev); spin_lock(&mddev->lock); diff --git a/drivers/md/md.h b/drivers/md/md.h index ebdd57677062..1b2e8720f0d1 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -554,6 +554,10 @@ struct mddev { /* used for register new sync thread */ struct work_struct sync_work; =20 + /* deferred io_opt update, see mddev_update_io_opt() */ + struct work_struct io_opt_work; + unsigned int io_opt_nr_stripes; + /* "lock" protects: * flush_bio transition from NULL to !NULL * rdev superblocks, events @@ -1056,7 +1060,8 @@ int mddev_stack_rdev_into(struct mddev *mddev, struct= md_rdev *rdev, * is added with the array's current limits. */ #define MDDEV_STACK_SKIP ((struct queue_limits *)ERR_PTR(-EAGAIN)) -void mddev_update_io_opt(struct mddev *mddev, unsigned int nr_stripes); +void mddev_update_io_opt(struct mddev *mddev, unsigned int nr_stripes, + struct queue_limits *lim); =20 extern const struct block_device_operations md_fops; =20 diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 222bd7badcff..5580ca77ef1e 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -4932,7 +4932,7 @@ static void end_reshape(struct r10conf *conf) conf->reshape_safe =3D MaxSector; spin_unlock_irq(&conf->device_lock); =20 - mddev_update_io_opt(conf->mddev, raid10_nr_stripes(conf)); + mddev_update_io_opt(conf->mddev, raid10_nr_stripes(conf), NULL); conf->fullsync =3D 0; } =20 diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 0ec555ada64a..3faa2a94c03b 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -8800,7 +8800,7 @@ static void end_reshape(struct r5conf *conf) wake_up(&conf->wait_for_reshape); =20 mddev_update_io_opt(conf->mddev, - conf->raid_disks - conf->max_degraded); + conf->raid_disks - conf->max_degraded, NULL); } } =20 --=20 2.43.0 From nobody Fri Sep 25 16:51:18 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2989B429803 for ; Thu, 10 Sep 2026 08:11:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027889; cv=none; b=bEuXq1tM1YB8Rd9gUJTDC6kCnxkEnGSdd27YjHkVApXgwmvfbWVfW4yPgJJ+hvHqpR3wmdDT1uVkBpx5BAXnBJC+qFN97KlPaWK1gohObS0212UfovmT6qRWer8G2BYNzwF1WlAM80m2+Yf+V+mpdIYEHhQFnIG1Km8Jh26duUY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027889; c=relaxed/simple; bh=ynYT4EzKnNUSkgWF8hXf0jG1Wiv30hWQLHY1SYDfJJk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dODCRQDSz83rILc/IyvC5mSlhRbpPORcqQm2ATRx44Ouen3HK2IM4ILRU9TdkpEr5UEEL0lMvK08dDb+RmlAIVq4uycLrkBhApKpeuB7brW9fGShH90L2AA35hEZi8rHbU6rp1XNOI8Qe23xdPlaAKaw7ybZyWbO+yahXp7M1jw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=S9sdwhW8; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="S9sdwhW8" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-49c5a927a1fso2979675e9.2 for ; Thu, 10 Sep 2026 01:11:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1789027883; x=1789632683; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=j936Hj+jQ6YcD/IvnzaDTxAfd7nwTnGHxjharHQeJpA=; b=S9sdwhW8kpkF/L+fHj00Jnlms4TrgNX7BTambGYToN61zlJ9tKQz5JwerPZs7bVtlN 5ZTGfLL67XWV6QgGw6uMmBsh+rQHBnxhftP7KULxxjCGb4H1B3X86lJWvG6EQs2wkHQL l0u1pexWZ0lGT8p0OV+yhvscmzu7wYC503gr+PM67MgB9CwBp3HKbsuwdIuy3ShoGuDu 30Jz6qk/Dum6Y0MsbK/GG00XSIim2zJkCO01beLBsdNwqwDp1eOsqMMDb/Yri5gFK99V dhay85x807BZFcYYoCYOpDCZeX1HTpQdTzuLFsO6j/TNjtx75/o58q1YR42xuMZKk8WE Yw6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789027883; x=1789632683; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=j936Hj+jQ6YcD/IvnzaDTxAfd7nwTnGHxjharHQeJpA=; b=KNw3+atvS2GPLbRHkZ2mA+KTktjLtjK1nP6h0YKks/ZrYpTGoGQaloCJzSyKeuT8c2 xLvP6NZQx5t80YWBVAFaC8Y/fmMEnPZeQ9Erugt5x6343dMyHafw87OQv984DFKJ6Otv nJQ9bl1BFkMfqi6iNEudTBGBfZ0BHA0i0C5uoRIdnejs1r61dZmNPlBQLRkU6esEtIo8 mdCN5f6xnEqueqpD6UX/L8rZfB/i/wFu64zgpipeuvJ628IVeQexh41RX6S06o1oB8GC wuGuWoI10MCrf6X7vplV6r8H0ZxT3ERxZLavlYHuTt2bMAV8OaT6OEGeO3bx4vSi4PZD D5qQ== X-Forwarded-Encrypted: i=1; AKwUvBwKNWUJqldYhhqdWcN4sRtPg4BZmbR4jn199bxXsKB+3ZkIyDYlWPm9LaStseSEj9EtGk30D0I6rh5gsIE=@vger.kernel.org X-Gm-Message-State: AFuF++liQtQzRGdC8BA/LiQHjHC3CrOXN9jQB2wjHIgcW1RJ6haCJvUR vjlVW7vYwOEpGsJ3QexbVWBV7wkJZAggL7FUp+oAjayvErVA0Jiel3o4rWSM2aHS480= X-Gm-Gg: AYBFou110w0cYZbP9PArAAuGBNUvUEtxfn3tgJx2mWXmvwLg2YvP1tMe7DbYRRNJ5IL lbkpeLyDHb/Mm0dJP23dReXveIM27PWfv16HHg76jVOQAl1bws9rJBne6nZ16HK5VQwjuvXwi0B J8qJcfm8XftoDnZpQskVrEooZRd6Z894DqtwHZyKqCm3IqMU4SDtBEtJEjQOE83cf09NaR0pZ7H P3TYUAqHiDhxqYABa0+OFnyk6RE3gCQUNA/2mtmoQrBapHExbeffnnCUUDxvT5xjznlNDKZE+TU BcsD+JVth8sbPBpUs2IHjuDv0e3L1f1RF0FNZOh5YDgCmWjqGOeswTbPbbLovARzlCX0u4IVd8O P9Ha53c2cgIvfAVNT1X3kftHca+LuRyXgFM9JB3pQ4y9HpsmlzMV0W1ESAcEoKp/g5I48+1nOQd nUxDc8GcWeeieK0E3XEG61SSSw80dmC16+cKsjtHlkSV5FaBDRmL1LKGq3Nx3Fx8XcFTYoupbJn uYQmTUD4r+ubDNxSTEu0fN2GA8YjzDnA3Bir/vfOEKj X-Received: by 2002:a05:600c:c494:b0:499:d95a:41f with SMTP id 5b1f17b1804b1-49d010c3969mr295917585e9.0.1789027882630; Thu, 10 Sep 2026 01:11:22 -0700 (PDT) Received: from jwang-ThinkPad-T14-Gen-6.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d20fc1be3sm55261135e9.4.2026.09.10.01.11.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2026 01:11:21 -0700 (PDT) From: Jack Wang To: Song Liu , Yu Kuai , linux-raid@vger.kernel.org, Nilay Shroff , abd.masalkhi@gmail.com Cc: linux-block@vger.kernel.org, Jens Axboe , Christoph Hellwig , Damien Le Moal , Ming Lei , Xiao Ni , Li Nan , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, linux-kernel@vger.kernel.org, Jack Wang Subject: [PATCH v2 5/8] md: take q->limits_lock before locking and suspending the array Date: Thu, 10 Sep 2026 10:11:10 +0200 Message-ID: <20260910081114.1605746-6-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260910081114.1605746-1-jinpu.wang@ionos.com> References: <20260910081114.1605746-1-jinpu.wang@ionos.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jack Wang Writing a queue limits attribute while a spare is re-added deadlocks the array: udev-worker queue_attr_store() holds q->limits_lock, waits in blk_mq_freeze_queue() for q_usage_counter to drain fio holds a q_usage_counter reference, parked in md_handle_request()'s is_suspended() loop mdadm suspended the array, waits for reconfig_mutex md_start_sync holds reconfig_mutex, waits for q->limits_lock Blocking on q->limits_lock while holding reconfig_mutex, or with the array suspended, is waiting for normal I/O, which mddev_suspend() already warns about with lockdep_assert_not_held(). So q->limits_lock has to nest outside both. Take the update before the array is locked and suspended, and pass it down so the personality stacks into it: - md_start_sync(), at both suspend points - md_ioctl() for ADD_NEW_DISK and HOT_REMOVE_DISK - rdev_attr_store(), for slot and for state "remove"/"re-add" - raid5 skip_copy_store(), which took the lock while suspended They are converted together because a mix of the two orders is an ABBA. All of them commit while the array is still quiesced. Two callers still take the lock inside reconfig_mutex with the array suspended: ->start_reshape() from action_store(), which suspends before flushing sync_work so the update cannot be held across it, and raid*_run() from level_store(), which a later patch converts. Verified with a raid1 of two ram devices, fio in flight and a loop writing queue/max_sectors_kb: 20 fail/remove/add cycles complete, where the same test wedges the array before the change. Fixes: c99f66e4084a ("block: fix queue freeze vs limits lock order in sysfs= store methods") Assisted-by: LLM Signed-off-by: Jack Wang --- drivers/md/md-autodetect.c | 2 +- drivers/md/md.c | 120 +++++++++++++++++++++++++++++++------ drivers/md/md.h | 3 +- drivers/md/raid5.c | 31 +++++++--- 4 files changed, 128 insertions(+), 28 deletions(-) diff --git a/drivers/md/md-autodetect.c b/drivers/md/md-autodetect.c index 4b80165afd23..929513109657 100644 --- a/drivers/md/md-autodetect.c +++ b/drivers/md/md-autodetect.c @@ -213,7 +213,7 @@ static void __init md_setup_drive(struct md_setup_args = *args) (1 << MD_DISK_ACTIVE) | (1 << MD_DISK_SYNC); } =20 - md_add_new_disk(mddev, &dinfo); + md_add_new_disk(mddev, &dinfo, NULL); } =20 if (!err) diff --git a/drivers/md/md.c b/drivers/md/md.c index 87e17ba86d93..0668a048db71 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -2983,7 +2983,7 @@ void md_update_sb(struct mddev *mddev, int force_chan= ge) } EXPORT_SYMBOL(md_update_sb); =20 -static int add_bound_rdev(struct md_rdev *rdev) +static int add_bound_rdev(struct md_rdev *rdev, struct queue_limits *lim) { struct mddev *mddev =3D rdev->mddev; int err =3D 0; @@ -2996,7 +2996,7 @@ static int add_bound_rdev(struct md_rdev *rdev) */ super_types[mddev->major_version]. validate_super(mddev, NULL/*freshest*/, rdev); - err =3D mddev->pers->hot_add_disk(mddev, rdev, NULL); + err =3D mddev->pers->hot_add_disk(mddev, rdev, lim); if (err) { md_kick_rdev_from_array(rdev); return err; @@ -3119,7 +3119,7 @@ state_store(struct md_rdev *rdev, const char *buf, si= ze_t len, } else if (cmd_match(buf, "remove")) { if (rdev->mddev->pers) { clear_bit(Blocked, &rdev->flags); - remove_and_add_spares(rdev->mddev, rdev, NULL); + remove_and_add_spares(rdev->mddev, rdev, lim); } if (rdev->raid_disk >=3D 0) err =3D -EBUSY; @@ -3238,7 +3238,7 @@ state_store(struct md_rdev *rdev, const char *buf, si= ze_t len, if (!mddev_is_clustered(rdev->mddev) || (err =3D mddev->cluster_ops->gather_bitmaps(rdev)) =3D=3D 0) { clear_bit(Faulty, &rdev->flags); - err =3D add_bound_rdev(rdev); + err =3D add_bound_rdev(rdev, lim); } } else err =3D -EBUSY; @@ -3325,7 +3325,7 @@ slot_store(struct md_rdev *rdev, const char *buf, siz= e_t len, if (rdev->mddev->pers->hot_remove_disk =3D=3D NULL) return -EINVAL; clear_bit(Blocked, &rdev->flags); - remove_and_add_spares(rdev->mddev, rdev, NULL); + remove_and_add_spares(rdev->mddev, rdev, lim); if (rdev->raid_disk >=3D 0) return -EBUSY; set_bit(MD_RECOVERY_NEEDED, &rdev->mddev->recovery); @@ -3356,7 +3356,7 @@ slot_store(struct md_rdev *rdev, const char *buf, siz= e_t len, clear_bit(In_sync, &rdev->flags); clear_bit(Bitmap_sync, &rdev->flags); err =3D rdev->mddev->pers->hot_add_disk(rdev->mddev, rdev, - NULL); + lim); if (err) { rdev->raid_disk =3D -1; return err; @@ -3762,6 +3762,9 @@ rdev_attr_store(struct kobject *kobj, struct attribut= e *attr, struct rdev_sysfs_entry *entry =3D container_of(attr, struct rdev_sysfs_e= ntry, attr); struct md_rdev *rdev =3D container_of(kobj, struct md_rdev, kobj); struct kernfs_node *kn =3D NULL; + struct request_queue *q =3D NULL; + struct queue_limits lim; + struct queue_limits *limp =3D NULL; bool suspend =3D false; ssize_t rv; struct mddev *mddev =3D READ_ONCE(rdev->mddev); @@ -3782,15 +3785,41 @@ rdev_attr_store(struct kobject *kobj, struct attrib= ute *attr, suspend =3D true; } =20 + /* + * These can add a leg back, which stacks its limits; the other + * state_store() values never reach ->hot_add_disk(). q->limits_lock + * nests outside the lock and the suspend, see md_start_sync(). + */ + if ((entry->store =3D=3D slot_store || + (entry->store =3D=3D state_store && + (cmd_match(page, "remove") || cmd_match(page, "re-add")))) && + !mddev_is_dm(mddev)) { + q =3D mddev->gendisk->queue; + lim =3D queue_limits_start_update(q); + limp =3D &lim; + } + rv =3D suspend ? mddev_suspend_and_lock(mddev) : mddev_lock(mddev); if (!rv) { if (rdev->mddev =3D=3D NULL) rv =3D -ENODEV; else - rv =3D entry->store(rdev, page, length, NULL); + rv =3D entry->store(rdev, page, length, limp); + /* apply the limits before the array takes I/O again */ + if (limp) { + int err =3D queue_limits_commit_update(q, limp); + + limp =3D NULL; + if (err && rv >=3D 0) + rv =3D err; + } suspend ? mddev_unlock_and_resume(mddev) : mddev_unlock(mddev); } =20 + /* only reached when the lock failed, so nothing was stacked */ + if (limp) + queue_limits_cancel_update(q); + if (kn) sysfs_unbreak_active_protection(kn); =20 @@ -7575,7 +7604,8 @@ static int get_disk_info(struct mddev *mddev, void __= user * arg) return 0; } =20 -int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info) +int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info, + struct queue_limits *lim) { struct md_rdev *rdev; dev_t dev =3D MKDEV(info->major,info->minor); @@ -7723,11 +7753,11 @@ int md_add_new_disk(struct mddev *mddev, struct mdu= _disk_info_s *info) if (err) mddev->cluster_ops->add_new_disk_cancel(mddev); else - err =3D add_bound_rdev(rdev); + err =3D add_bound_rdev(rdev, lim); } =20 } else if (!err) - err =3D add_bound_rdev(rdev); + err =3D add_bound_rdev(rdev, lim); =20 return err; } @@ -7780,7 +7810,8 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_d= isk_info_s *info) return 0; } =20 -static int hot_remove_disk(struct mddev *mddev, dev_t dev) +static int hot_remove_disk(struct mddev *mddev, dev_t dev, + struct queue_limits *lim) { struct md_rdev *rdev; =20 @@ -7795,7 +7826,7 @@ static int hot_remove_disk(struct mddev *mddev, dev_t= dev) goto kick_rdev; =20 clear_bit(Blocked, &rdev->flags); - remove_and_add_spares(mddev, rdev, NULL); + remove_and_add_spares(mddev, rdev, lim); =20 if (rdev->raid_disk >=3D 0) goto busy; @@ -8380,6 +8411,22 @@ static inline int md_ioctl_valid(unsigned int cmd) } } =20 +/* + * Commands that can reach ->hot_add_disk(). ADD_NEW_DISK only does so fo= r a + * journal device or a personality without ->hot_remove_disk, but that dep= ends + * on disk info still in user memory here, so it is included as a whole. + */ +static bool md_ioctl_may_add_disk(unsigned int cmd) +{ + switch (cmd) { + case ADD_NEW_DISK: + case HOT_REMOVE_DISK: + return true; + default: + return false; + } +} + static bool md_ioctl_need_suspend(unsigned int cmd) { switch (cmd) { @@ -8435,6 +8482,9 @@ static int md_ioctl(struct block_device *bdev, blk_mo= de_t mode, unsigned int noio_flags =3D 0; void __user *argp =3D (void __user *)arg; struct mddev *mddev =3D NULL; + struct request_queue *q =3D NULL; + struct queue_limits lim; + struct queue_limits *limp =3D NULL; bool suspend; =20 err =3D md_ioctl_valid(cmd); @@ -8485,11 +8535,20 @@ static int md_ioctl(struct block_device *bdev, blk_= mode_t mode, if (!md_is_rdwr(mddev)) flush_work(&mddev->sync_work); =20 + /* q->limits_lock nests outside both, see md_start_sync() */ + if (md_ioctl_may_add_disk(cmd) && !mddev_is_dm(mddev)) { + q =3D mddev->gendisk->queue; + lim =3D queue_limits_start_update(q); + limp =3D &lim; + } + suspend =3D md_ioctl_need_suspend(cmd); err =3D suspend ? mddev_suspend_and_lock(mddev) : mddev_lock(mddev); if (err) { pr_debug("md: ioctl lock interrupted, reason %d, cmd %d\n", err, cmd); + if (limp) + queue_limits_cancel_update(q); goto out; } if (suspend) @@ -8531,7 +8590,7 @@ static int md_ioctl(struct block_device *bdev, blk_mo= de_t mode, goto unlock; =20 case HOT_REMOVE_DISK: - err =3D hot_remove_disk(mddev, new_decode_dev(arg)); + err =3D hot_remove_disk(mddev, new_decode_dev(arg), limp); goto unlock; =20 case ADD_NEW_DISK: @@ -8547,7 +8606,7 @@ static int md_ioctl(struct block_device *bdev, blk_mo= de_t mode, /* Need to clear read-only for this */ break; else - err =3D md_add_new_disk(mddev, &info); + err =3D md_add_new_disk(mddev, &info, limp); goto unlock; } break; @@ -8585,7 +8644,7 @@ static int md_ioctl(struct block_device *bdev, blk_mo= de_t mode, if (copy_from_user(&info, argp, sizeof(info))) err =3D -EFAULT; else - err =3D md_add_new_disk(mddev, &info); + err =3D md_add_new_disk(mddev, &info, limp); goto unlock; } =20 @@ -8618,6 +8677,9 @@ static int md_ioctl(struct block_device *bdev, blk_mo= de_t mode, err !=3D -EINVAL) mddev->hold_active =3D 0; =20 + if (limp) + err =3D queue_limits_commit_update(q, limp) ?: err; + if (suspend) { memalloc_noio_restore(noio_flags); mddev_unlock_and_resume(mddev); @@ -10346,6 +10408,9 @@ static bool md_choose_sync_action(struct mddev *mdd= ev, int *spares, static void md_start_sync(struct work_struct *ws) { struct mddev *mddev =3D container_of(ws, struct mddev, sync_work); + struct request_queue *q =3D NULL; + struct queue_limits lim; + struct queue_limits *limp =3D NULL; int spares =3D 0; bool suspend =3D false; unsigned int noio_flags =3D 0; @@ -10357,6 +10422,17 @@ static void md_start_sync(struct work_struct *ws) */ if ((mddev->reshape_position =3D=3D MaxSector || !md_is_rdwr(mddev)) && md_spares_need_change(mddev)) { + /* + * Adding a spare below stacks its limits, which needs + * q->limits_lock. Take it before suspending: its holder + * waits in blk_mq_freeze_queue() for I/O that + * mddev->suspended holds back, so the other order deadlocks. + */ + if (!mddev_is_dm(mddev)) { + q =3D mddev->gendisk->queue; + lim =3D queue_limits_start_update(q); + limp =3D &lim; + } suspend =3D true; mddev_suspend(mddev, false); noio_flags =3D memalloc_noio_save(); @@ -10371,6 +10447,12 @@ static void md_start_sync(struct work_struct *ws) if (!suspend && (mddev->reshape_position =3D=3D MaxSector || !md_is_rdwr(= mddev)) && md_spares_need_change(mddev)) { mddev_unlock(mddev); + /* see above: q->limits_lock nests outside both */ + if (!mddev_is_dm(mddev)) { + q =3D mddev->gendisk->queue; + lim =3D queue_limits_start_update(q); + limp =3D &lim; + } mddev_suspend_and_lock_nointr(mddev); suspend =3D true; noio_flags =3D memalloc_noio_save(); @@ -10384,11 +10466,11 @@ static void md_start_sync(struct work_struct *ws) * As we only add devices that are already in-sync, we can * activate the spares immediately. */ - remove_and_add_spares(mddev, NULL, NULL); + remove_and_add_spares(mddev, NULL, limp); goto not_running; } =20 - if (!md_choose_sync_action(mddev, &spares, NULL)) + if (!md_choose_sync_action(mddev, &spares, limp)) goto not_running; =20 if (!mddev->pers->sync_request) @@ -10419,6 +10501,8 @@ static void md_start_sync(struct work_struct *ws) * https://bugzilla.kernel.org/show_bug.cgi?id=3D218200 * Therefore, use __mddev_resume(mddev, false). */ + if (limp && queue_limits_commit_update(q, limp)) + pr_err("%s: could not apply queue limits\n", mdname(mddev)); if (suspend) { memalloc_noio_restore(noio_flags); __mddev_resume(mddev, false); @@ -10441,6 +10525,8 @@ static void md_start_sync(struct work_struct *ws) * https://bugzilla.kernel.org/show_bug.cgi?id=3D218200 * Therefore, use __mddev_resume(mddev, false). */ + if (limp && queue_limits_commit_update(q, limp)) + pr_err("%s: could not apply queue limits\n", mdname(mddev)); if (suspend) { memalloc_noio_restore(noio_flags); __mddev_resume(mddev, false); diff --git a/drivers/md/md.h b/drivers/md/md.h index 1b2e8720f0d1..7f4e3ea8b826 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -1046,7 +1046,8 @@ struct mdu_disk_info_s; extern int mdp_major; void md_autostart_arrays(int part); int md_set_array_info(struct mddev *mddev, struct mdu_array_info_s *info); -int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info); +int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info, + struct queue_limits *lim); int do_md_run(struct mddev *mddev); #define MDDEV_STACK_INTEGRITY (1u << 0) int mddev_stack_rdev_limits(struct mddev *mddev, struct queue_limits *lim, diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 3faa2a94c03b..22759c631c4d 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -7288,6 +7288,9 @@ static ssize_t raid5_store_skip_copy(struct mddev *mddev, const char *page, size_t len) { struct r5conf *conf; + struct request_queue *q =3D NULL; + struct queue_limits lim; + struct queue_limits *limp =3D NULL; unsigned long new; int err; =20 @@ -7297,23 +7300,33 @@ raid5_store_skip_copy(struct mddev *mddev, const ch= ar *page, size_t len) return -EINVAL; new =3D !!new; =20 + /* q->limits_lock nests outside both, see md_start_sync() */ + if (!mddev_is_dm(mddev)) { + q =3D mddev->gendisk->queue; + lim =3D queue_limits_start_update(q); + limp =3D &lim; + } + err =3D mddev_suspend_and_lock(mddev); - if (err) + if (err) { + if (limp) + queue_limits_cancel_update(q); return err; + } conf =3D mddev->private; if (!conf) err =3D -ENODEV; else if (new !=3D conf->skip_copy) { - struct request_queue *q =3D mddev->gendisk->queue; - struct queue_limits lim =3D queue_limits_start_update(q); - conf->skip_copy =3D new; - if (new) - lim.features |=3D BLK_FEAT_STABLE_WRITES; - else - lim.features &=3D ~BLK_FEAT_STABLE_WRITES; - err =3D queue_limits_commit_update(q, &lim); + if (limp) { + if (new) + limp->features |=3D BLK_FEAT_STABLE_WRITES; + else + limp->features &=3D ~BLK_FEAT_STABLE_WRITES; + } } + if (limp) + err =3D queue_limits_commit_update(q, limp) ?: err; mddev_unlock_and_resume(mddev); return err ?: len; } --=20 2.43.0 From nobody Fri Sep 25 16:51:18 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE1B241D228 for ; Thu, 10 Sep 2026 08:11:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027892; cv=none; b=m9hF+oA2KJcLt2osmTHTxDbUAO80QQeALmfTK/5Z7LafYcunpP2ob16JPa15LAcxa6LqRZ1ZN2xzAi8H7201IAGmCK4/pxIYcBhX+TcCnAU88hi/oJ+H7FGkTvsW5b4HdoboP5q9P3FQELbvSnvdVJ3b02Uxkxj9UAfZSC/LkPE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027892; c=relaxed/simple; bh=nnUwbAGw66z8p9c5609SWShTLPj32yaDP8/9sn1ddGI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ON/oCQz8RHpVJJDtJtuBzFATBXLqDjOdJn+e0jl0NJOcD+W8ZyIV9YbOLJXK6IhehoSvBk2nmNux1EXgAFP4692CO4UCX2vbijE+Tz3A0kVOgQDt3zvd0jbLgU3aEtdPe18LiH1fQXDshImUnlHxC4B1cZkbJIIOigRMyaWFcgE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=RXtYQWaf; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="RXtYQWaf" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-49b4ba7fe26so2624285e9.2 for ; Thu, 10 Sep 2026 01:11:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1789027885; x=1789632685; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=MhyUVR5/3y9z/7b63ssenZZXMXcVxCwIZBzS9kva6Eg=; b=RXtYQWafYePgLxrGr+r+JiD9xLy7UKAp5pW/E2/+Kx/S+aAPPveCsgjO5bc8IXHuJR pXFjDM+PghMS+eyIIOOwvPo7O40VrYgvzDGH9aR2XauL4GW4HppkEBodoySkmLWmsqM9 lGYeZij3bx38HvnrhlinCfJVuNlYb6rYATZOCGTIXV/mH6l35CsoSSlQ+EFxMk531fM5 rA+5E7n81Fu8QwGyGeOVvGXln5CoYxoBHPsCWgkU9PGU2LMw1IrsjAmet9WC+yzHJC9l OsJ30r+e6+yDHV3Tohi42bVnIhbgWHJvHO+Iq5R5NVULcr6Dr3ykRZ80IJaY+T8MHIbl gLCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789027885; x=1789632685; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=MhyUVR5/3y9z/7b63ssenZZXMXcVxCwIZBzS9kva6Eg=; b=hgi5rPJ8j8f1NrgDP1rmeIZgtU9UJWVSVm/+bGx6NTVmrVrX660El8LDiTwAFF+4gV oxfZKuup3jFvfNvByWk6cHVAc4BvMUU5wRwCHh2oviZkzg9Z4zhf84IkIVl5WKSZ6RqR VqkBcmUA2EC0M8CTTapOqWoRWH219QBUu89wg7OC927v4WhrzOHWJq8PT54zqz6NGool 8MWf1CMp/1099EGtmfe9Ay1nskz7yBETwqtmHR9EvMNh0hhr/pWVLC0aqIBs3+B/shTW q3FcpAaNQbsf5LVrvEHb+L2uaqf7R7HNWHp4hT9VVoRqSSQfqW/Dw/DLEXAD9jfI3egB Dz6g== X-Forwarded-Encrypted: i=1; AKwUvBzRIZ9Zvx4FQsUod3LPRrXIY5BOgbEqc2ofPk0hJk/Na4iJwSpY5CkJwbIt6mAgDN96NuJq7yhkYVbpUMs=@vger.kernel.org X-Gm-Message-State: AFuF++nXqnA/Kz5h8sDbphZLzwF1vBf9v0tbT5JY09fCpvk6sO+UWyRF e2bPNU2m8141vuPqMzcGqUAHQ5rnPs8eBa9TopFYpcNUDaGRHPpru97eJLitfknF9VE= X-Gm-Gg: AYBFou0BkAO58AvGLsiUtvyOZrypNEuKxBhe4+tVMFDzENASVuAEk5oDXQw0g7Y0QVO BEZvQHdZsVxYu2JLvD3whafaM1Cg2fbxh6zuO2ElHpsDoVUhmPEGZuTKQZqQJ2C4rkoBkBwdzGk 7qOgE4oNnQe2hlDO2Qp/bOFTtKmDI8hRC0pou1Z+NQbCHKMrojOO7exFgtKBuF0AEPbjVjZHLJX l0T3y/DNMv4aKor3P6yrJSnWvPKKWfI7/3/SqNRkOkr7RrBJwZSJFDCf/D0qZNt9qQWIHWB+AQX xfkkhQuQ8Y7rLr9Pw/pyolOskg+ivf18e/lF0SgBTFKhhgt5+55tjMsV4TPf8qTOX6C2bz0JLqn 1Yr+SE4BwHT0lLhAeyp6vNc/J02yvXXl1wQU+BRt0vjTmeWA6B6H5L4LNJPHKi95ZNaD41U1lBz uz4/LOrmFmCEXPBbAOiY72ld8qOVceJPfGsvDHFUcMneucD2LYsxKMlOFLToie96tMI8MrwWv5T Pu5GfMnRaRZRJnMgQyH0jqMHTnKWVCMFeMQGQoPT6G7 X-Received: by 2002:a05:600c:6289:b0:49d:1da4:511b with SMTP id 5b1f17b1804b1-49d27670d04mr24484735e9.1.1789027884500; Thu, 10 Sep 2026 01:11:24 -0700 (PDT) Received: from jwang-ThinkPad-T14-Gen-6.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d20fc1be3sm55261135e9.4.2026.09.10.01.11.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2026 01:11:23 -0700 (PDT) From: Jack Wang To: Song Liu , Yu Kuai , linux-raid@vger.kernel.org, Nilay Shroff , abd.masalkhi@gmail.com Cc: linux-block@vger.kernel.org, Jens Axboe , Christoph Hellwig , Damien Le Moal , Ming Lei , Xiao Ni , Li Nan , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, linux-kernel@vger.kernel.org, Jack Wang Subject: [PATCH v2 6/8] md: pass a queue_limits through ->run() Date: Thu, 10 Sep 2026 10:11:11 +0200 Message-ID: <20260910081114.1605746-7-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260910081114.1605746-1-jinpu.wang@ionos.com> References: <20260910081114.1605746-1-jinpu.wang@ionos.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jack Wang raid*_run() -> queue_limits_set() takes q->limits_lock with reconfig_mutex held, the order the previous patches inverted. With lockdep on, creating an array and then adding a leg reports it: -> #1 (&q->limits_lock): -> #0 (&mddev->reconfig_mutex): queue_limits_set md_ioctl <- ADD_NEW_DISK raid1_run do_md_run md_ioctl <- RUN_ARRAY The earlier patch left this for level_store() alone; RUN_ARRAY reaches it too, so every array creation records the wrong order. Give ->run() a queue_limits argument and take the update at the entry points that start an array: md_ioctl() for RUN_ARRAY, level_store(), autorun_devices(), md_setup_drive(), and array_state_store() for readonly, read_auto and active -- but only while mddev->pers is NULL, as with the array running those states go to md_set_readonly(), which waits in stop_sync_thread() for the work that takes the same lock. dm-raid passes NULL: with no gendisk the personalities return before touching any limits. Assisted-by: LLM Signed-off-by: Jack Wang --- drivers/md/dm-raid.c | 2 +- drivers/md/md-autodetect.c | 21 ++++++- drivers/md/md-linear.c | 4 +- drivers/md/md.c | 116 +++++++++++++++++++++++++++++++------ drivers/md/md.h | 11 +++- drivers/md/raid0.c | 16 ++++- drivers/md/raid1.c | 16 ++++- drivers/md/raid10.c | 16 ++++- drivers/md/raid5.c | 16 ++++- 9 files changed, 182 insertions(+), 36 deletions(-) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index 21a1922bee4f..d043a5c49608 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -3258,7 +3258,7 @@ static int raid_ctr(struct dm_target *ti, unsigned in= t argc, char **argv) /* Keep array frozen until resume. */ md_frozen_sync_thread(&rs->md); =20 - r =3D md_run(&rs->md); + r =3D md_run(&rs->md, NULL); rs->md.in_sync =3D 0; /* Assume already marked dirty */ if (r) { ti->error =3D "Failed to run raid array"; diff --git a/drivers/md/md-autodetect.c b/drivers/md/md-autodetect.c index 929513109657..e15ae2fb58a2 100644 --- a/drivers/md/md-autodetect.c +++ b/drivers/md/md-autodetect.c @@ -126,6 +126,9 @@ static void __init md_setup_drive(struct md_setup_args = *args) dev_t devices[MD_SB_DISKS + 1], mdev; struct mdu_array_info_s ainfo =3D { }; struct mddev *mddev; + struct request_queue *q =3D NULL; + struct queue_limits lim; + struct queue_limits *limp =3D NULL; int err =3D 0, i; char name[16]; =20 @@ -216,11 +219,27 @@ static void __init md_setup_drive(struct md_setup_arg= s *args) md_add_new_disk(mddev, &dinfo, NULL); } =20 + /* + * do_md_run() restacks the array's limits, and q->limits_lock must + * not nest inside reconfig_mutex, so start the update with the array + * unlocked. This is __init and the array is not reachable yet. + */ + if (!err && !mddev_is_dm(mddev)) { + mddev_unlock(mddev); + q =3D mddev->gendisk->queue; + lim =3D queue_limits_start_update(q); + limp =3D &lim; + mddev_lock_nointr(mddev); + } + if (!err) - err =3D do_md_run(mddev); + err =3D do_md_run(mddev, limp); if (err) pr_warn("md: starting %s failed\n", name); out_unlock: + /* apply the limits before the array takes I/O */ + if (limp) + queue_limits_commit_update(q, limp); mddev_unlock_and_resume(mddev); out_mddev_put: mddev_put(mddev); diff --git a/drivers/md/md-linear.c b/drivers/md/md-linear.c index da82c313d459..5438c23a7242 100644 --- a/drivers/md/md-linear.c +++ b/drivers/md/md-linear.c @@ -178,7 +178,7 @@ static struct linear_conf *linear_conf(struct mddev *md= dev, int raid_disks, return ERR_PTR(ret); } =20 -static int linear_run(struct mddev *mddev) +static int linear_run(struct mddev *mddev, struct queue_limits *lim) { struct linear_conf *conf; int ret; @@ -186,7 +186,7 @@ static int linear_run(struct mddev *mddev) if (md_check_no_bitmap(mddev)) return -EINVAL; =20 - conf =3D linear_conf(mddev, mddev->raid_disks, NULL); + conf =3D linear_conf(mddev, mddev->raid_disks, lim); if (IS_ERR(conf)) return PTR_ERR(conf); =20 diff --git a/drivers/md/md.c b/drivers/md/md.c index 0668a048db71..5be956e80563 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -4105,13 +4105,30 @@ level_store(struct mddev *mddev, const char *buf, s= ize_t len) long level; void *priv, *oldpriv; struct md_rdev *rdev; + struct request_queue *q =3D NULL; + struct queue_limits lim; + struct queue_limits *limp =3D NULL; =20 if (slen =3D=3D 0 || slen >=3D sizeof(clevel)) return -EINVAL; =20 + /* + * The new personality restacks the array's queue limits in ->run(), + * and q->limits_lock has to be taken before the array is locked and + * suspended, see md_start_sync(). + */ + if (!mddev_is_dm(mddev)) { + q =3D mddev->gendisk->queue; + lim =3D queue_limits_start_update(q); + limp =3D &lim; + } + rv =3D mddev_suspend_and_lock(mddev); - if (rv) + if (rv) { + if (limp) + queue_limits_cancel_update(q); return rv; + } noio_flags =3D memalloc_noio_save(); =20 if (mddev->pers =3D=3D NULL) { @@ -4280,7 +4297,7 @@ level_store(struct mddev *mddev, const char *buf, siz= e_t len) mddev->in_sync =3D 1; timer_delete_sync(&mddev->safemode_timer); } - pers->run(mddev); + pers->run(mddev, limp); set_bit(MD_SB_CHANGE_DEVS, &mddev->sb_flags); if (!mddev->thread) md_update_sb(mddev, 1); @@ -4288,6 +4305,9 @@ level_store(struct mddev *mddev, const char *buf, siz= e_t len) md_new_event(); rv =3D len; out_unlock: + /* apply the limits before the array takes I/O again */ + if (limp) + rv =3D queue_limits_commit_update(q, limp) ?: rv; memalloc_noio_restore(noio_flags); mddev_unlock_and_resume(mddev); return rv; @@ -4708,6 +4728,10 @@ array_state_store(struct mddev *mddev, const char *b= uf, size_t len) { int err =3D 0; enum array_state st =3D match_word(buf, array_states); + bool starts_array, need_lim; + struct request_queue *q =3D NULL; + struct queue_limits lim; + struct queue_limits *limp =3D NULL; =20 /* No lock dependent actions */ switch (st) { @@ -4753,9 +4777,39 @@ array_state_store(struct mddev *mddev, const char *b= uf, size_t len) spin_unlock(&mddev->lock); return err ?: len; } + + /* + * These states start the array when it is not running, and ->run() + * restacks its limits, so take q->limits_lock first. Only then: + * with mddev->pers set they go to md_set_readonly(), which waits for + * the very work that takes the same lock. + */ + starts_array =3D (st =3D=3D readonly || st =3D=3D read_auto || st =3D=3D = active) && + !mddev_is_dm(mddev); +retry: + need_lim =3D starts_array && !READ_ONCE(mddev->pers); + if (need_lim) { + q =3D mddev->gendisk->queue; + lim =3D queue_limits_start_update(q); + limp =3D &lim; + } + err =3D mddev_lock(mddev); - if (err) + if (err) { + if (limp) + queue_limits_cancel_update(q); return err; + } + + /* mddev->pers was read without the lock, so redo it if it changed */ + if (need_lim !=3D (starts_array && !mddev->pers)) { + mddev_unlock(mddev); + if (limp) { + queue_limits_cancel_update(q); + limp =3D NULL; + } + goto retry; + } =20 switch (st) { case inactive: @@ -4772,7 +4826,7 @@ array_state_store(struct mddev *mddev, const char *bu= f, size_t len) else { mddev->ro =3D MD_RDONLY; set_disk_ro(mddev->gendisk, 1); - err =3D do_md_run(mddev); + err =3D do_md_run(mddev, limp); } break; case read_auto: @@ -4787,7 +4841,7 @@ array_state_store(struct mddev *mddev, const char *bu= f, size_t len) } } else { mddev->ro =3D MD_AUTO_READ; - err =3D do_md_run(mddev); + err =3D do_md_run(mddev, limp); } break; case clean: @@ -4813,7 +4867,7 @@ array_state_store(struct mddev *mddev, const char *bu= f, size_t len) } else { mddev->ro =3D MD_RDWR; set_disk_ro(mddev->gendisk, 0); - err =3D do_md_run(mddev); + err =3D do_md_run(mddev, limp); } break; default: @@ -4826,6 +4880,9 @@ array_state_store(struct mddev *mddev, const char *bu= f, size_t len) mddev->hold_active =3D 0; sysfs_notify_dirent_safe(mddev->sysfs_state); } + /* apply the limits before the array takes I/O */ + if (limp) + err =3D queue_limits_commit_update(q, limp) ?: err; mddev_unlock(mddev); =20 if (st =3D=3D readonly || st =3D=3D read_auto || st =3D=3D inactive || @@ -6763,7 +6820,7 @@ static void md_bitmap_set_none(struct mddev *mddev) md_bitmap_sysfs_add(mddev); } =20 -int md_run(struct mddev *mddev) +int md_run(struct mddev *mddev, struct queue_limits *lim) { int err; struct md_rdev *rdev; @@ -6892,7 +6949,7 @@ int md_run(struct mddev *mddev) if (start_readonly && md_is_rdwr(mddev)) mddev->ro =3D MD_AUTO_READ; /* read-only, but switch on first write */ =20 - err =3D pers->run(mddev); + err =3D pers->run(mddev, lim); if (err) pr_warn("md: pers->run() failed ...\n"); else if (pers->size(mddev, 0, 0) < mddev->array_sectors) { @@ -6988,12 +7045,12 @@ int md_run(struct mddev *mddev) } EXPORT_SYMBOL_GPL(md_run); =20 -int do_md_run(struct mddev *mddev) +int do_md_run(struct mddev *mddev, struct queue_limits *lim) { int err; =20 set_bit(MD_NOT_READY, &mddev->flags); - err =3D md_run(mddev); + err =3D md_run(mddev, lim); if (err) goto out; =20 @@ -7349,7 +7406,7 @@ static int do_md_stop(struct mddev *mddev, int mode) } =20 #ifndef MODULE -static void autorun_array(struct mddev *mddev) +static void autorun_array(struct mddev *mddev, struct queue_limits *lim) { struct md_rdev *rdev; int err; @@ -7364,7 +7421,7 @@ static void autorun_array(struct mddev *mddev) } pr_cont("\n"); =20 - err =3D do_md_run(mddev); + err =3D do_md_run(mddev, lim); if (err) { pr_warn("md: do_md_run() returned %d\n", err); do_md_stop(mddev, 0); @@ -7387,6 +7444,9 @@ static void autorun_devices(int part) { struct md_rdev *rdev0, *rdev, *tmp; struct mddev *mddev; + struct request_queue *q =3D NULL; + struct queue_limits lim; + struct queue_limits *limp =3D NULL; =20 pr_info("md: autorun ...\n"); while (!list_empty(&pending_raid_disks)) { @@ -7427,12 +7487,29 @@ static void autorun_devices(int part) if (IS_ERR(mddev)) break; =20 - if (mddev_suspend_and_lock(mddev)) + /* + * autorun_array() runs the array, which restacks its limits; + * q->limits_lock has to be taken before the array is locked + * and suspended, see md_start_sync(). + */ + if (!mddev_is_dm(mddev)) { + q =3D mddev->gendisk->queue; + lim =3D queue_limits_start_update(q); + limp =3D &lim; + } + + if (mddev_suspend_and_lock(mddev)) { pr_warn("md: %s locked, cannot run\n", mdname(mddev)); - else if (mddev->raid_disks || mddev->major_version + if (limp) { + queue_limits_cancel_update(q); + limp =3D NULL; + } + } else if (mddev->raid_disks || mddev->major_version || !list_empty(&mddev->disks)) { pr_warn("md: %s already running, cannot run %pg\n", mdname(mddev), rdev0->bdev); + if (limp) + queue_limits_cancel_update(q); mddev_unlock_and_resume(mddev); } else { pr_debug("md: created %s\n", mdname(mddev)); @@ -7442,9 +7519,13 @@ static void autorun_devices(int part) if (bind_rdev_to_array(rdev, mddev)) export_rdev(rdev); } - autorun_array(mddev); + autorun_array(mddev, limp); + if (limp && queue_limits_commit_update(q, limp)) + pr_warn("md: %s: could not apply queue limits\n", + mdname(mddev)); mddev_unlock_and_resume(mddev); } + limp =3D NULL; /* on success, candidates will be empty, on error * it won't... */ @@ -8536,7 +8617,8 @@ static int md_ioctl(struct block_device *bdev, blk_mo= de_t mode, flush_work(&mddev->sync_work); =20 /* q->limits_lock nests outside both, see md_start_sync() */ - if (md_ioctl_may_add_disk(cmd) && !mddev_is_dm(mddev)) { + if ((md_ioctl_may_add_disk(cmd) || cmd =3D=3D RUN_ARRAY) && + !mddev_is_dm(mddev)) { q =3D mddev->gendisk->queue; lim =3D queue_limits_start_update(q); limp =3D &lim; @@ -8660,7 +8742,7 @@ static int md_ioctl(struct block_device *bdev, blk_mo= de_t mode, goto unlock; =20 case RUN_ARRAY: - err =3D do_md_run(mddev); + err =3D do_md_run(mddev, limp); goto unlock; =20 case SET_BITMAP_FILE: diff --git a/drivers/md/md.h b/drivers/md/md.h index 7f4e3ea8b826..73f6ef20f266 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -760,7 +760,12 @@ struct md_personality * start up works that do NOT require md_thread. tasks that * requires md_thread should go into start() */ - int (*run)(struct mddev *mddev); + /* + * @lim: a queue limits update the caller owns, or NULL. Non-NULL + * means stack into it rather than take q->limits_lock, which has to + * nest outside reconfig_mutex, see md_start_sync(). + */ + int (*run)(struct mddev *mddev, struct queue_limits *lim); /* start up works that require md threads */ int (*start)(struct mddev *mddev); void (*free)(struct mddev *mddev, void *priv); @@ -959,7 +964,7 @@ extern void mddev_destroy(struct mddev *mddev); void md_init_stacking_limits(struct queue_limits *lim); struct mddev *md_alloc(dev_t dev, char *name); void mddev_put(struct mddev *mddev); -extern int md_run(struct mddev *mddev); +extern int md_run(struct mddev *mddev, struct queue_limits *lim); extern int md_start(struct mddev *mddev); extern void md_stop(struct mddev *mddev); extern void md_stop_writes(struct mddev *mddev); @@ -1048,7 +1053,7 @@ void md_autostart_arrays(int part); int md_set_array_info(struct mddev *mddev, struct mdu_array_info_s *info); int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info, struct queue_limits *lim); -int do_md_run(struct mddev *mddev); +int do_md_run(struct mddev *mddev, struct queue_limits *lim); #define MDDEV_STACK_INTEGRITY (1u << 0) int mddev_stack_rdev_limits(struct mddev *mddev, struct queue_limits *lim, unsigned int flags); diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c index 35e103f0c2c3..59141e4299a8 100644 --- a/drivers/md/raid0.c +++ b/drivers/md/raid0.c @@ -379,7 +379,8 @@ static void raid0_free(struct mddev *mddev, void *priv) kfree(conf); } =20 -static int raid0_set_limits(struct mddev *mddev) +static int raid0_set_limits(struct mddev *mddev, + struct queue_limits *caller_lim) { struct queue_limits lim; int err; @@ -398,10 +399,19 @@ static int raid0_set_limits(struct mddev *mddev) err =3D mddev_stack_rdev_limits(mddev, &lim, MDDEV_STACK_INTEGRITY); if (err) return err; + /* + * The caller owns an update and commits it itself; taking + * q->limits_lock here would take it a second time. + */ + if (caller_lim) { + *caller_lim =3D lim; + return 0; + } + return queue_limits_set(mddev->gendisk->queue, &lim); } =20 -static int raid0_run(struct mddev *mddev) +static int raid0_run(struct mddev *mddev, struct queue_limits *lim) { struct r0conf *conf; int ret; @@ -414,7 +424,7 @@ static int raid0_run(struct mddev *mddev) return -EINVAL; =20 if (!mddev_is_dm(mddev)) { - ret =3D raid0_set_limits(mddev); + ret =3D raid0_set_limits(mddev, lim); if (ret) return ret; } diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 78effcac138d..6713a53fd460 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -3170,7 +3170,8 @@ static struct r1conf *setup_conf(struct mddev *mddev) return ERR_PTR(err); } =20 -static int raid1_set_limits(struct mddev *mddev) +static int raid1_set_limits(struct mddev *mddev, + struct queue_limits *caller_lim) { struct queue_limits lim; int err; @@ -3185,10 +3186,19 @@ static int raid1_set_limits(struct mddev *mddev) err =3D mddev_stack_rdev_limits(mddev, &lim, MDDEV_STACK_INTEGRITY); if (err) return err; + /* + * The caller owns an update and commits it itself; taking + * q->limits_lock here would take it a second time. + */ + if (caller_lim) { + *caller_lim =3D lim; + return 0; + } + return queue_limits_set(mddev->gendisk->queue, &lim); } =20 -static int raid1_run(struct mddev *mddev) +static int raid1_run(struct mddev *mddev, struct queue_limits *lim) { struct r1conf *conf; int i; @@ -3219,7 +3229,7 @@ static int raid1_run(struct mddev *mddev) return PTR_ERR(conf); =20 if (!mddev_is_dm(mddev)) { - ret =3D raid1_set_limits(mddev); + ret =3D raid1_set_limits(mddev, lim); if (ret) { md_unregister_thread(mddev, &conf->thread); if (!mddev->private) diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 5580ca77ef1e..16143db6085b 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -3930,7 +3930,8 @@ static unsigned int raid10_nr_stripes(struct r10conf = *conf) return raid_disks / conf->geo.near_copies; } =20 -static int raid10_set_queue_limits(struct mddev *mddev) +static int raid10_set_queue_limits(struct mddev *mddev, + struct queue_limits *caller_lim) { struct r10conf *conf =3D mddev->private; struct queue_limits lim; @@ -3948,10 +3949,19 @@ static int raid10_set_queue_limits(struct mddev *md= dev) err =3D mddev_stack_rdev_limits(mddev, &lim, MDDEV_STACK_INTEGRITY); if (err) return err; + /* + * The caller owns an update and commits it itself; taking + * q->limits_lock here would take it a second time. + */ + if (caller_lim) { + *caller_lim =3D lim; + return 0; + } + return queue_limits_set(mddev->gendisk->queue, &lim); } =20 -static int raid10_run(struct mddev *mddev) +static int raid10_run(struct mddev *mddev, struct queue_limits *lim) { struct r10conf *conf; int i, disk_idx; @@ -4020,7 +4030,7 @@ static int raid10_run(struct mddev *mddev) } =20 if (!mddev_is_dm(conf->mddev)) { - int err =3D raid10_set_queue_limits(mddev); + int err =3D raid10_set_queue_limits(mddev, lim); =20 if (err) { ret =3D err; diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 22759c631c4d..28bd81de86c1 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -7944,7 +7944,8 @@ static int raid5_create_ctx_pool(struct r5conf *conf) return conf->ctx_pool ? 0 : -ENOMEM; } =20 -static int raid5_set_limits(struct mddev *mddev) +static int raid5_set_limits(struct mddev *mddev, + struct queue_limits *caller_lim) { struct r5conf *conf =3D mddev->private; struct queue_limits lim; @@ -7996,10 +7997,19 @@ static int raid5_set_limits(struct mddev *mddev) /* No restrictions on the number of segments in the request */ lim.max_segments =3D USHRT_MAX; =20 + /* + * The caller owns an update and commits it itself; taking + * q->limits_lock here would take it a second time. + */ + if (caller_lim) { + *caller_lim =3D lim; + return 0; + } + return queue_limits_set(mddev->gendisk->queue, &lim); } =20 -static int raid5_run(struct mddev *mddev) +static int raid5_run(struct mddev *mddev, struct queue_limits *lim) { struct r5conf *conf; int dirty_parity_disks =3D 0; @@ -8259,7 +8269,7 @@ static int raid5_run(struct mddev *mddev) md_set_array_sectors(mddev, raid5_size(mddev, 0, 0)); =20 if (!mddev_is_dm(mddev)) { - ret =3D raid5_set_limits(mddev); + ret =3D raid5_set_limits(mddev, lim); if (ret) goto abort; } --=20 2.43.0 From nobody Fri Sep 25 16:51:18 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0626A430CD5 for ; Thu, 10 Sep 2026 08:11:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027892; cv=none; b=AOjjLDIjQDd9YbCqNwmQx/YqahP2EB6frnouE6fxqdKYk0QSX2SjhX5Bb8fJtTjPpfhBkL9sTwSfkKCpDsu6NIJH+JMYMMTarvrzq1fVNil5sp81VSBEysILzHj2qmjed/w/xWje0cHfMaO5D0Wtiw0M/XJfo20MiH/jTA8O0aM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027892; c=relaxed/simple; bh=N4W4/1Nb4xxCQW/b/yh4LAIUt+gjMzmX7Ei/zLa6UWE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cp0sBuncFv7hd7uUwWfZ6YdW1SbhlFIsOUVoSUvH+k3GnMr+k/M1LJnVOVrD1LlLV1cF73uA6MWs5mQqcZYUYzVQi7CvKvMDDtdKqh52Hbwt5oWe2z8bS/FmQ2MVqUH6WWP9aDf2+tVsI/bBoYlBFjCK+Sz7wx5NvmGEVWLOo1k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=T0cUXcFR; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="T0cUXcFR" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-49d0726cdbcso3153355e9.0 for ; Thu, 10 Sep 2026 01:11:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1789027886; x=1789632686; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=+GHgzzQ8rfbRP2Put64Cq82EilqCh8jYa5rghgFsni8=; b=T0cUXcFRSTTCOZWzfmdw5HeKCYRLaYu12TxzP2/SIWE6PyPMHPyt89TkO5VQE+snSf tx9HUy0BVsGsBuWC1/WpnSEMPRS06BxJ/lEp+/+1cGEhmp9IoOus4x6O+MD7GvxOVJ+Q pdiZJpfEgsWdkODpzHVo4b9cj9pwbbrxAfGt5KNagkjAcD63Ri4Khqa40rVXueVjsNlt FI9x16Nzy7fhPNZsDYqef2jzxrrJb1oenLhvORzggAzha/aga88CuPzIoSdFQNAcK36K +NOjZZeR+vkm8H0MamnFkPIpW1svySdoLdnBcIN3lfkXS89CklcY9P06Oy/hNiu+Vtan ++vQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789027886; x=1789632686; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=+GHgzzQ8rfbRP2Put64Cq82EilqCh8jYa5rghgFsni8=; b=gRuHGcE0rSaqAZf5gHXJbrypFeSI+OcyMQNi3JO1qQodYsoiTxdR2rIK2+rBygSzn6 TFK3b9+j16PqySVTgiJldjSzWhQaYerfo3qH3ie+GrG9KNE1UEjp1EnRmQTailDFPXNp gWS2kuZlGybWfWxWWQJHlnMQXQclXGiOLa45SmVHmGF1CT/sLjGonPwmZoS5QCspH5Lp i9NfNJb8zIUSGKiWCJN8a/YKsRsGNlM8MLksvPWPRT10oAgsSwGMNz7YpbV8SLYSy2kt ZtDTMozq6PMCXLeWUnSdF4k8Dt8LPYbmENxkdTmoRba5N+ramFssgNgjpP5BeU8n3ILp A5uA== X-Forwarded-Encrypted: i=1; AKwUvBz85RsH9pDi1fMhUA8oDXBxmbPCO7eQmilcqQEDJ2t+PcRpmN2MwepOENprQE/ef7Mmxf9Pm6AFSdGEbvg=@vger.kernel.org X-Gm-Message-State: AFuF++mbFGyzSRgr3G01yRFhJsdFXJ+ahVbGr9vWz9RW455VgPyiIyZ8 mf4J29draAgQLK8/xn5G/ydeFS7R2CNF+u0BDr3TigtKXonX8qOs7yO783WEGl8bDnE= X-Gm-Gg: AYBFou0SYaoP81pgvA3XlN97m2RtwR94G+61LEV6WDPai+7HxvzYtEBIwaNfoDDeZED 3s8VlyCMaC+tdgTtE3wQHWf2FxJEpswhQG8EqYNb51iq3eZblyPoRTQX23R03ANagWo4oF9NZW9 En011cesUNxb/bwquEGLNAFm1jmHjyDtwo8OdagqZTPhM2vt8zQMARo58CHqtsK9T5HIdr4xOgV BGq0cRt4A3xqLB1us7pgMuB+oZGUXBfUco9mlK4Gzm4alqFWbQ8PNIlpW9+rSJiqpVsAYKgiv3E nVRARYC6bZfcpMOyb7ZY9MnKmkG21/Ec69KAxGbLfrmdVMjbVeDIBH3ACb9cITxha957ODYIger 77SnWoXKLQpK0bWjdYNCAUClrwWm72JyEXrfnSUj4k9PTmgwSjfrroNnZsyQi/PG+fcZNxOsvsC /T2wq2eIYMqrXDI4Q/Lp2n+T7/GmRO3WLjn/aCFp8p+Y8IdZgfdw86GNoCfM1QyGnVlzVf0OK24 IF99C+Odmuq1OjvixkgYxX7xNsDxnF75tQCWkPijIeu X-Received: by 2002:a05:600c:c48f:b0:49b:9241:7ff0 with SMTP id 5b1f17b1804b1-49d010c4f64mr288757255e9.0.1789027885618; Thu, 10 Sep 2026 01:11:25 -0700 (PDT) Received: from jwang-ThinkPad-T14-Gen-6.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d20fc1be3sm55261135e9.4.2026.09.10.01.11.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2026 01:11:25 -0700 (PDT) From: Jack Wang To: Song Liu , Yu Kuai , linux-raid@vger.kernel.org, Nilay Shroff , abd.masalkhi@gmail.com Cc: linux-block@vger.kernel.org, Jens Axboe , Christoph Hellwig , Damien Le Moal , Ming Lei , Xiao Ni , Li Nan , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, linux-kernel@vger.kernel.org, Jack Wang Subject: [PATCH v2 7/8] md: open new legs before locking the array Date: Thu, 10 Sep 2026 10:11:12 +0200 Message-ID: <20260910081114.1605746-8-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260910081114.1605746-1-jinpu.wang@ionos.com> References: <20260910081114.1605746-1-jinpu.wang@ionos.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jack Wang Opening a leg takes disk->open_mutex, and the scsi disk probe path nests q->limits_lock inside it (sd_open() -> sd_revalidate_disk()). md opens legs under reconfig_mutex, which this series makes q->limits_lock nest outside, closing a cycle. Booting with lockdep on an md root reports it during assembly: -> #2 (&q->limits_lock): sd_revalidate_disk / sd_open -> #1 (&disk->open_mutex): md_import_device md_add_new_disk md_ioctl <- ADD_NEW_DISK -> #0 (&mddev->reconfig_mutex): md_ioctl <- RUN_ARRAY Move every open out from under the lock. md_import_new_disk() mirrors md_add_new_disk()'s branch selection so all three of its branches take a pre-opened leg, and hot_add_disk(), new_dev_store() and md_setup_drive() open before they lock as well. The mddev fields the open depends on are read without reconfig_mutex, so each caller rechecks them once the array is locked and rejects the add with -EBUSY if the branch or the superblock format would have changed. md_autostart_arrays() needs no change: it opens under detected_devices_mutex. Assisted-by: LLM Signed-off-by: Jack Wang --- drivers/md/md-autodetect.c | 17 ++- drivers/md/md.c | 238 ++++++++++++++++++++++++++----------- drivers/md/md.h | 21 +++- 3 files changed, 203 insertions(+), 73 deletions(-) diff --git a/drivers/md/md-autodetect.c b/drivers/md/md-autodetect.c index e15ae2fb58a2..e592577356ad 100644 --- a/drivers/md/md-autodetect.c +++ b/drivers/md/md-autodetect.c @@ -208,6 +208,7 @@ static void __init md_setup_drive(struct md_setup_args = *args) .major =3D MAJOR(devices[i]), .minor =3D MINOR(devices[i]), }; + struct md_new_disk nd; =20 if (args->level !=3D LEVEL_NONE) { dinfo.number =3D i; @@ -216,7 +217,21 @@ static void __init md_setup_drive(struct md_setup_args= *args) (1 << MD_DISK_ACTIVE) | (1 << MD_DISK_SYNC); } =20 - md_add_new_disk(mddev, &dinfo, NULL); + /* + * Opening a leg takes disk->open_mutex, which must not nest + * inside reconfig_mutex, see md_import_new_disk(). Drop the + * array lock around it; this is __init and the array is not + * reachable yet, so nothing else can touch it in between. + */ + mddev_unlock(mddev); + if (md_import_new_disk(mddev, &dinfo, &nd)) { + mddev_lock_nointr(mddev); + continue; + } + mddev_lock_nointr(mddev); + + md_add_new_disk(mddev, &dinfo, &nd, NULL); + md_put_new_disk(&nd); } =20 /* diff --git a/drivers/md/md.c b/drivers/md/md.c index 5be956e80563..fa033d7d3831 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -4942,6 +4942,7 @@ new_dev_store(struct mddev *mddev, const char *buf, s= ize_t len) struct md_rdev *rdev; unsigned int noio_flags; int err; + int persistent, external, major_version, minor_version; =20 if (!*buf || *e !=3D ':' || !e[1] || e[1] =3D=3D '\n') return -EINVAL; @@ -4953,32 +4954,52 @@ new_dev_store(struct mddev *mddev, const char *buf,= size_t len) minor !=3D MINOR(dev)) return -EOVERFLOW; =20 - err =3D mddev_suspend_and_lock(mddev); - if (err) - return err; - noio_flags =3D memalloc_noio_save(); - if (mddev->persistent) { - rdev =3D md_import_device(dev, mddev->major_version, - mddev->minor_version); - if (!IS_ERR(rdev) && !list_empty(&mddev->disks)) { - struct md_rdev *rdev0 - =3D list_entry(mddev->disks.next, - struct md_rdev, same_set); - err =3D super_types[mddev->major_version] - .load_super(rdev, rdev0, mddev->minor_version); - if (err < 0) - goto out; - } - } else if (mddev->external) + /* + * Open before locking the array: bdev_open() takes disk->open_mutex, + * which must not nest inside reconfig_mutex, see md_import_new_disk(). + * The fields below are read without the lock and rechecked under it. + */ + persistent =3D READ_ONCE(mddev->persistent); + external =3D READ_ONCE(mddev->external); + major_version =3D READ_ONCE(mddev->major_version); + minor_version =3D READ_ONCE(mddev->minor_version); + + if (persistent) + rdev =3D md_import_device(dev, major_version, minor_version); + else if (external) rdev =3D md_import_device(dev, -2, -1); else rdev =3D md_import_device(dev, -1, -1); =20 - if (IS_ERR(rdev)) { - memalloc_noio_restore(noio_flags); - mddev_unlock_and_resume(mddev); + if (IS_ERR(rdev)) return PTR_ERR(rdev); + + err =3D mddev_suspend_and_lock(mddev); + if (err) { + export_rdev(rdev); + return err; } + noio_flags =3D memalloc_noio_save(); + + if (persistent !=3D mddev->persistent || external !=3D mddev->external || + major_version !=3D mddev->major_version || + minor_version !=3D mddev->minor_version) { + pr_warn("%s: array reconfigured while opening %pg\n", + mdname(mddev), rdev->bdev); + err =3D -EBUSY; + goto out; + } + + if (mddev->persistent && !list_empty(&mddev->disks)) { + struct md_rdev *rdev0 + =3D list_entry(mddev->disks.next, + struct md_rdev, same_set); + err =3D super_types[mddev->major_version] + .load_super(rdev, rdev0, mddev->minor_version); + if (err < 0) + goto out; + } + err =3D bind_rdev_to_array(rdev, mddev); out: if (err) @@ -7685,12 +7706,35 @@ static int get_disk_info(struct mddev *mddev, void = __user * arg) return 0; } =20 +/* + * @nd carries an rdev the caller opened before locking the array, for the + * branch its snapshot selected. Every caller must open first; doing it + * here would nest disk->open_mutex inside reconfig_mutex. + */ int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info, - struct queue_limits *lim) + struct md_new_disk *nd, struct queue_limits *lim) { struct md_rdev *rdev; dev_t dev =3D MKDEV(info->major,info->minor); =20 + /* + * The open ran unlocked, so anything that selects a different branch + * below, or a different superblock format, means it was done against + * an array that no longer looks like this one. + */ + if (nd && nd->rdev && + (nd->have_raid_disks !=3D (mddev->raid_disks !=3D 0) || + nd->have_pers !=3D !!mddev->pers || + nd->persistent !=3D mddev->persistent || + nd->major_version !=3D mddev->major_version || + nd->minor_version !=3D mddev->minor_version)) { + pr_warn("%s: array reconfigured while opening %pg\n", + mdname(mddev), nd->rdev->bdev); + export_rdev(nd->rdev); + nd->rdev =3D NULL; + return -EBUSY; + } + if (mddev_is_clustered(mddev) && !(info->state & ((1 << MD_DISK_CLUSTER_ADD) | (1 << MD_DISK_CANDIDATE)))= ) { pr_warn("%s: Cannot add to clustered mddev.\n", @@ -7703,13 +7747,12 @@ int md_add_new_disk(struct mddev *mddev, struct mdu= _disk_info_s *info, =20 if (!mddev->raid_disks) { int err; + /* expecting a device which has a superblock */ - rdev =3D md_import_device(dev, mddev->major_version, mddev->minor_versio= n); - if (IS_ERR(rdev)) { - pr_warn("md: md_import_device returned %ld\n", - PTR_ERR(rdev)); - return PTR_ERR(rdev); - } + if (WARN_ON_ONCE(!nd || !nd->rdev)) + return -EINVAL; + rdev =3D nd->rdev; + nd->rdev =3D NULL; if (!list_empty(&mddev->disks)) { struct md_rdev *rdev0 =3D list_entry(mddev->disks.next, @@ -7742,16 +7785,10 @@ int md_add_new_disk(struct mddev *mddev, struct mdu= _disk_info_s *info, mdname(mddev)); return -EINVAL; } - if (mddev->persistent) - rdev =3D md_import_device(dev, mddev->major_version, - mddev->minor_version); - else - rdev =3D md_import_device(dev, -1, -1); - if (IS_ERR(rdev)) { - pr_warn("md: md_import_device returned %ld\n", - PTR_ERR(rdev)); - return PTR_ERR(rdev); - } + if (WARN_ON_ONCE(!nd || !nd->rdev)) + return -EINVAL; + rdev =3D nd->rdev; + nd->rdev =3D NULL; /* set saved_raid_disk if appropriate */ if (!mddev->persistent) { if (info->state & (1<state & (1<rdev)) + return -EINVAL; + rdev =3D nd->rdev; + nd->rdev =3D NULL; rdev->desc_nr =3D info->number; if (info->raid_disk < mddev->raid_disks) rdev->raid_disk =3D info->raid_disk; @@ -7930,7 +7966,8 @@ static int hot_remove_disk(struct mddev *mddev, dev_t= dev, return -EBUSY; } =20 -static int hot_add_disk(struct mddev *mddev, dev_t dev) +/* @nd carries a leg the caller opened before the array was locked */ +static int hot_add_disk(struct mddev *mddev, struct md_new_disk *nd) { int err; struct md_rdev *rdev; @@ -7949,12 +7986,10 @@ static int hot_add_disk(struct mddev *mddev, dev_t = dev) return -EINVAL; } =20 - rdev =3D md_import_device(dev, -1, 0); - if (IS_ERR(rdev)) { - pr_warn("md: error, md_import_device() returned %ld\n", - PTR_ERR(rdev)); + if (WARN_ON_ONCE(!nd->rdev)) return -EINVAL; - } + rdev =3D nd->rdev; + nd->rdev =3D NULL; =20 if (mddev->persistent) rdev->sb_start =3D calc_dev_sboffset(rdev); @@ -8497,14 +8532,61 @@ static inline int md_ioctl_valid(unsigned int cmd) * journal device or a personality without ->hot_remove_disk, but that dep= ends * on disk info still in user memory here, so it is included as a whole. */ -static bool md_ioctl_may_add_disk(unsigned int cmd) + +/* + * Open the leg before the array is locked; bdev_open() takes + * disk->open_mutex, which must not nest inside reconfig_mutex. mddev is + * read unlocked on purpose, and md_add_new_disk() rechecks the snapshot. + */ +int md_import_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info, + struct md_new_disk *nd) { - switch (cmd) { - case ADD_NEW_DISK: - case HOT_REMOVE_DISK: - return true; - default: - return false; + dev_t dev =3D MKDEV(info->major, info->minor); + struct md_rdev *rdev; + + memset(nd, 0, sizeof(*nd)); + nd->have_raid_disks =3D READ_ONCE(mddev->raid_disks) !=3D 0; + nd->have_pers =3D !!READ_ONCE(mddev->pers); + nd->persistent =3D READ_ONCE(mddev->persistent); + nd->major_version =3D READ_ONCE(mddev->major_version); + nd->minor_version =3D READ_ONCE(mddev->minor_version); + + if (!nd->have_raid_disks) { + /* a device with a superblock, for an array being assembled */ + rdev =3D md_import_device(dev, nd->major_version, + nd->minor_version); + } else if (nd->have_pers) { + /* a hot spare; this is the branch that stacks limits */ + nd->stacks =3D true; + if (nd->persistent) + rdev =3D md_import_device(dev, nd->major_version, + nd->minor_version); + else + rdev =3D md_import_device(dev, -1, -1); + } else if (nd->major_version =3D=3D 0) { + rdev =3D md_import_device(dev, -1, 0); + } else { + /* md_add_new_disk() rejects this, nothing to open */ + return 0; + } + + if (IS_ERR(rdev)) { + int err =3D PTR_ERR(rdev); + + pr_warn("md: md_import_device returned %d\n", err); + return err; + } + + nd->rdev =3D rdev; + return 0; +} + +/* release a leg md_add_new_disk() did not take ownership of */ +void md_put_new_disk(struct md_new_disk *nd) +{ + if (nd->rdev) { + export_rdev(nd->rdev); + nd->rdev =3D NULL; } } =20 @@ -8566,6 +8648,8 @@ static int md_ioctl(struct block_device *bdev, blk_mo= de_t mode, struct request_queue *q =3D NULL; struct queue_limits lim; struct queue_limits *limp =3D NULL; + struct md_new_disk nd =3D { }; + mdu_disk_info_t info; bool suspend; =20 err =3D md_ioctl_valid(cmd); @@ -8616,8 +8700,27 @@ static int md_ioctl(struct block_device *bdev, blk_m= ode_t mode, if (!md_is_rdwr(mddev)) flush_work(&mddev->sync_work); =20 + if (cmd =3D=3D ADD_NEW_DISK) { + if (copy_from_user(&info, argp, sizeof(info))) { + err =3D -EFAULT; + goto out; + } + err =3D md_import_new_disk(mddev, &info, &nd); + if (err) + goto out; + } else if (cmd =3D=3D HOT_ADD_DISK) { + nd.rdev =3D md_import_device(new_decode_dev(arg), -1, 0); + if (IS_ERR(nd.rdev)) { + pr_warn("md: error, md_import_device() returned %ld\n", + PTR_ERR(nd.rdev)); + nd.rdev =3D NULL; + err =3D -EINVAL; + goto out; + } + } + /* q->limits_lock nests outside both, see md_start_sync() */ - if ((md_ioctl_may_add_disk(cmd) || cmd =3D=3D RUN_ARRAY) && + if ((nd.stacks || cmd =3D=3D HOT_REMOVE_DISK || cmd =3D=3D RUN_ARRAY) && !mddev_is_dm(mddev)) { q =3D mddev->gendisk->queue; lim =3D queue_limits_start_update(q); @@ -8681,14 +8784,10 @@ static int md_ioctl(struct block_device *bdev, blk_= mode_t mode, * So require mddev->pers and MD_DISK_SYNC. */ if (mddev->pers) { - mdu_disk_info_t info; - if (copy_from_user(&info, argp, sizeof(info))) - err =3D -EFAULT; - else if (!(info.state & (1<flags); return err; diff --git a/drivers/md/md.h b/drivers/md/md.h index 73f6ef20f266..73a27d83d65a 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -1051,8 +1051,27 @@ struct mdu_disk_info_s; extern int mdp_major; void md_autostart_arrays(int part); int md_set_array_info(struct mddev *mddev, struct mdu_array_info_s *info); +/* + * A leg opened before the array was locked, with the mddev fields that + * selected the branch and the superblock format. Opening takes + * disk->open_mutex, which must not nest inside reconfig_mutex; the fields + * are read unlocked and md_add_new_disk() rechecks them. + */ +struct md_new_disk { + struct md_rdev *rdev; + bool stacks; /* the add can reach ->hot_add_disk() */ + bool have_pers; + bool have_raid_disks; + int persistent; + int major_version; + int minor_version; +}; + +int md_import_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info, + struct md_new_disk *nd); +void md_put_new_disk(struct md_new_disk *nd); int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info, - struct queue_limits *lim); + struct md_new_disk *nd, struct queue_limits *lim); int do_md_run(struct mddev *mddev, struct queue_limits *lim); #define MDDEV_STACK_INTEGRITY (1u << 0) int mddev_stack_rdev_limits(struct mddev *mddev, struct queue_limits *lim, --=20 2.43.0 From nobody Fri Sep 25 16:51:18 2026 Received: from mail-wm2-f12.google.com (mail-wm2-f12.google.com [74.125.225.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 87AB54349A9 for ; Thu, 10 Sep 2026 08:11:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.225.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027895; cv=none; b=JxQXpBc4rR/l3wpuQwolrJXIm+u1b7CKyaz7ZVL2NRNFmJOAHYX6YRLx3qOfbK9J9th1g+LLt5bjiHZrGPZKx0xAELENO7pi5rukqaGhdzdtTff4NocsUIIFsw/jWW8TFNq2iO+bJMV8UvaTBAXbGT4eq5iB4BORV6oayp9Kqfg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789027895; c=relaxed/simple; bh=VWJq79Pbtf4IbOZyn7AR5EcaHFmtf9h2nH5b25XhR2E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VeaMUixw1lHJNSUtKmFeLTAxQrzBGHUIVyoGN6+nRlJeh5ZLETfgAsnv03bOXvnPvTqTsr/wHGZsofDlbU7bAUqpGI0ADRn0JpiLTnuHoaFmAzdi4k66dfzK3mnH4kWcWBtnsVYnfkibgo9Yvf0Guki2vOs6VBHQwzJbniO1MrA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com; spf=pass smtp.mailfrom=ionos.com; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b=TE/57EFV; arc=none smtp.client-ip=74.125.225.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=ionos.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ionos.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ionos.com header.i=@ionos.com header.b="TE/57EFV" Received: by mail-wm2-f12.google.com with SMTP id 5b1f17b1804b1-499db17b40eso3000725e9.2 for ; Thu, 10 Sep 2026 01:11:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; t=1789027888; x=1789632688; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=7WxxmD3yg0pg+i13gK8J2zlyLTSoxJMhIHGtHp0KYMk=; b=TE/57EFVuI+OZ/IiWW5Il/G1G+sP8z2qb3niY8ltt02d7SvWDiEv4y2+JsgB1uZpuZ NlkOcgVOmzdfLBo9dKE+TTnKthv2BzaS0uqpu8kS6oB1CO01zW5ZmrjEx7b7wncNv2az BkIAzeBogew5N2LQwO14t/UYLzSmCN/yHsEwHOn2pKe9N5SNTVNyK2W0O+vyF733wB6P wJzf7A4Vft/wXSE6lh4pKXtUDSPqqfK11SVkykpc4GiSfk6Cg1E56HPlrEzgIOCGiH96 ulWcvp0M9aF4WHHbDkxS/g1tI55KUguX0D/0aHgIrgkFHi5WnintFHxUT4o5R+F9qATX lrIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1789027888; x=1789632688; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=7WxxmD3yg0pg+i13gK8J2zlyLTSoxJMhIHGtHp0KYMk=; b=FphYOJnZJ9MZLNRsuJ36ulLwE5FRk3FmoN9fJj4MU+fN1aM3ezRLrIAGQgkMw/EoAZ 2a+kHocOEVO8f8d961YKQ1bVBnT94lRwaZ4N61rAOaXMyK2fpMubD8q97JhscFeW7wvL qBfS+j88L6mN6jJMa2UtnOabw5tpt8anXixtbgJMEKiD2N7IW5YHhugk5u3+f2rZTorF GDw+15zRBK9jSTqmbjIkPrOqaQ9kCzMvuPiHGKwy22gDHWyX+Jv35376tIbWA4mAOwOO 38/cm/7TyqCZjb6rJ427T10kkjv3w9wd1kBW95CqG27iuB2M/rBUnP2cwr5Opei+3LQ0 aqaQ== X-Forwarded-Encrypted: i=1; AKwUvBxkfU2RdwmoXY5lMrW3N26HtUv9yvOi7z40aey8hoTJFzMykbFdU12eHDqbesNIbCuEXvNXTsb5fxiBrkE=@vger.kernel.org X-Gm-Message-State: AFuF++mYipi4LUOzqemAqNiEphZcwaCb9pn5qxyq9hZR08EzPOE9Xm35 NX+yWcLmzFy9TimEHL9dn7eUb1QHVx1V1v5PJYVXezENMD+IshvdLpmwYTJ2WQt0QMc= X-Gm-Gg: AYBFou3Btgzg+IiV9xGUIBGGd3x898KHN7rbs4qNXc1GEbWcEaXFEXIUb7DqLY9pzxE yS+1D521SRSaMX4dhCGTpxJU6UydQvh86Qq3lUs/YkH4Oywn/sXu/pr3A9CTRME/Lmx1pr4/ebC 3BDkvIyV8Z7py2DMk/SEYxtgOOh/0loRrb/Sp+BE0qy5cGz4O6mHIlgjpbYeXQwfUWb80RpnO43 iHx/HhILghvqhtERDsGOr3Ir1lZWVKrK8pRxKmRYlr90Pn7D67oP8Y2wt6g3UEiNO4Dmk3rDDc2 9kRYnFWiTp5TTpNwLoF3svhCEyznz9zdwonzFtZImwmdLUsDB6vq2h5OE4zLhVJdGRXXsDp+bZg IqjCaN10ENwmDC5OqdR/wMivU0Hhme0piuKuKqP5okAdz7AJQOrzHtYUdqYJC9/cabj3ZKFuTqL rhd1vvFiIo6MY2LITtHwpbgnGYN2fN6RX5C1LYSJbyT7GuQgbk/JnVzRSy/mq38Aw+0lSXaKLaI yiXG2yRP/qxWjd5oDjMwVRqeW1iU/romutnkPWHfNsP X-Received: by 2002:a05:600c:3b25:b0:49b:910c:76fb with SMTP id 5b1f17b1804b1-49d01dd415bmr272852515e9.2.1789027887561; Thu, 10 Sep 2026 01:11:27 -0700 (PDT) Received: from jwang-ThinkPad-T14-Gen-6.fkb.profitbricks.net ([212.227.34.98]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49d20fc1be3sm55261135e9.4.2026.09.10.01.11.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2026 01:11:26 -0700 (PDT) From: Jack Wang To: Song Liu , Yu Kuai , linux-raid@vger.kernel.org, Nilay Shroff , abd.masalkhi@gmail.com Cc: linux-block@vger.kernel.org, Jens Axboe , Christoph Hellwig , Damien Le Moal , Ming Lei , Xiao Ni , Li Nan , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, linux-kernel@vger.kernel.org, Jack Wang Subject: [PATCH v2 8/8] md: link a new leg's holder before locking the array Date: Thu, 10 Sep 2026 10:11:13 +0200 Message-ID: <20260910081114.1605746-9-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260910081114.1605746-1-jinpu.wang@ionos.com> References: <20260910081114.1605746-1-jinpu.wang@ionos.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Jack Wang bd_link_disk_holder() takes the leg's disk->open_mutex, and bind_rdev_to_array() calls it with reconfig_mutex held, so the dependency the previous patch removed from md_import_device() is still there by another route: -> #2 (&q->limits_lock): sd_revalidate_disk / sd_open -> #1 (&disk->open_mutex): bd_link_disk_holder bind_rdev_to_array md_add_new_disk md_ioctl <- ADD_NEW_DISK -> #0 (&mddev->reconfig_mutex): md_ioctl <- RUN_ARRAY bd_unlink_disk_holder() only takes blk_holder_mutex, which is why the release side needs no change and made the link side easy to miss. Link the holder where the leg is opened, before the array is locked, and record it in a new HolderLinked flag so the release side knows whether there is a link to drop. A failed link is not fatal, as before. A leg that is linked but not yet bound is released through md_export_rdev(), which drops the link first. A leg is now linked before it is known to be acceptable, so a leg the array goes on to reject shows up in its slaves directory until the error path releases it. md then no longer takes disk->open_mutex under reconfig_mutex: of the functions that take it, md reaches bdev_open() and bd_link_disk_holder() from the paths above, bdev_release() and bdev_fput() only through fput(), which defers to task work, del_gendisk() only from mddev teardown, and never sync_bdevs(). Assisted-by: LLM Signed-off-by: Jack Wang --- drivers/md/md-autodetect.c | 2 +- drivers/md/md.c | 70 ++++++++++++++++++++++++++++---------- drivers/md/md.h | 7 +++- 3 files changed, 59 insertions(+), 20 deletions(-) diff --git a/drivers/md/md-autodetect.c b/drivers/md/md-autodetect.c index e592577356ad..b6f9fb36f1bb 100644 --- a/drivers/md/md-autodetect.c +++ b/drivers/md/md-autodetect.c @@ -231,7 +231,7 @@ static void __init md_setup_drive(struct md_setup_args = *args) mddev_lock_nointr(mddev); =20 md_add_new_disk(mddev, &dinfo, &nd, NULL); - md_put_new_disk(&nd); + md_put_new_disk(mddev, &nd); } =20 /* diff --git a/drivers/md/md.c b/drivers/md/md.c index fa033d7d3831..235f0d645cea 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -2632,7 +2632,7 @@ static int bind_rdev_to_array(struct md_rdev *rdev, s= truct mddev *mddev) sysfs_get_dirent_safe(rdev->kobj.sd, "bad_blocks"); =20 list_add_rcu(&rdev->same_set, &mddev->disks); - bd_link_disk_holder(rdev->bdev, mddev->gendisk); + /* the holder is linked with the open, see md_link_rdev_holder() */ =20 return 0; =20 @@ -2648,6 +2648,23 @@ void md_autodetect_dev(dev_t dev); /* just for claiming the bdev */ static struct md_rdev claim_rdev; =20 +/* + * bd_link_disk_holder() takes the leg's disk->open_mutex, so the link is + * made with the open, before the array is locked. bd_unlink_disk_holder() + * only takes blk_holder_mutex, so dropping it is safe under any lock. + */ +static void md_link_rdev_holder(struct md_rdev *rdev, struct mddev *mddev) +{ + if (!bd_link_disk_holder(rdev->bdev, mddev->gendisk)) + set_bit(HolderLinked, &rdev->flags); +} + +static void md_unlink_rdev_holder(struct md_rdev *rdev, struct mddev *mdde= v) +{ + if (test_and_clear_bit(HolderLinked, &rdev->flags)) + bd_unlink_disk_holder(rdev->bdev, mddev->gendisk); +} + static void export_rdev(struct md_rdev *rdev) { pr_debug("md: export_rdev(%pg)\n", rdev->bdev); @@ -2661,11 +2678,18 @@ static void export_rdev(struct md_rdev *rdev) kobject_put(&rdev->kobj); } =20 +/* release a leg that was linked before the array was locked */ +static void md_export_rdev(struct mddev *mddev, struct md_rdev *rdev) +{ + md_unlink_rdev_holder(rdev, mddev); + export_rdev(rdev); +} + static void md_kick_rdev_from_array(struct md_rdev *rdev) { struct mddev *mddev =3D rdev->mddev; =20 - bd_unlink_disk_holder(rdev->bdev, rdev->mddev->gendisk); + md_unlink_rdev_holder(rdev, rdev->mddev); list_del_rcu(&rdev->same_set); pr_debug("md: unbind<%pg>\n", rdev->bdev); mddev_destroy_serial_pool(rdev->mddev, rdev); @@ -4974,9 +4998,11 @@ new_dev_store(struct mddev *mddev, const char *buf, = size_t len) if (IS_ERR(rdev)) return PTR_ERR(rdev); =20 + md_link_rdev_holder(rdev, mddev); + err =3D mddev_suspend_and_lock(mddev); if (err) { - export_rdev(rdev); + md_export_rdev(mddev, rdev); return err; } noio_flags =3D memalloc_noio_save(); @@ -5003,7 +5029,7 @@ new_dev_store(struct mddev *mddev, const char *buf, s= ize_t len) err =3D bind_rdev_to_array(rdev, mddev); out: if (err) - export_rdev(rdev); + md_export_rdev(mddev, rdev); memalloc_noio_restore(noio_flags); mddev_unlock_and_resume(mddev); if (!err) @@ -7519,6 +7545,10 @@ static void autorun_devices(int part) limp =3D &lim; } =20 + /* link before locking, see md_link_rdev_holder() */ + rdev_for_each_list(rdev, tmp, &candidates) + md_link_rdev_holder(rdev, mddev); + if (mddev_suspend_and_lock(mddev)) { pr_warn("md: %s locked, cannot run\n", mdname(mddev)); if (limp) { @@ -7538,7 +7568,7 @@ static void autorun_devices(int part) rdev_for_each_list(rdev, tmp, &candidates) { list_del_init(&rdev->same_set); if (bind_rdev_to_array(rdev, mddev)) - export_rdev(rdev); + md_export_rdev(mddev, rdev); } autorun_array(mddev, limp); if (limp && queue_limits_commit_update(q, limp)) @@ -7552,7 +7582,7 @@ static void autorun_devices(int part) */ rdev_for_each_list(rdev, tmp, &candidates) { list_del_init(&rdev->same_set); - export_rdev(rdev); + md_export_rdev(mddev, rdev); } mddev_put(mddev); } @@ -7730,7 +7760,7 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_d= isk_info_s *info, nd->minor_version !=3D mddev->minor_version)) { pr_warn("%s: array reconfigured while opening %pg\n", mdname(mddev), nd->rdev->bdev); - export_rdev(nd->rdev); + md_export_rdev(mddev, nd->rdev); nd->rdev =3D NULL; return -EBUSY; } @@ -7763,13 +7793,13 @@ int md_add_new_disk(struct mddev *mddev, struct mdu= _disk_info_s *info, pr_warn("md: %pg has different UUID to %pg\n", rdev->bdev, rdev0->bdev); - export_rdev(rdev); + md_export_rdev(mddev, rdev); return -EINVAL; } } err =3D bind_rdev_to_array(rdev, mddev); if (err) - export_rdev(rdev); + md_export_rdev(mddev, rdev); return err; } =20 @@ -7806,7 +7836,7 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_d= isk_info_s *info, /* This was a hot-add request, but events doesn't * match, so reject it. */ - export_rdev(rdev); + md_export_rdev(mddev, rdev); return -EINVAL; } =20 @@ -7832,7 +7862,7 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_d= isk_info_s *info, } } if (has_journal || mddev->bitmap) { - export_rdev(rdev); + md_export_rdev(mddev, rdev); return -EBUSY; } set_bit(Journal, &rdev->flags); @@ -7847,7 +7877,7 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_d= isk_info_s *info, /* --add initiated by this node */ err =3D mddev->cluster_ops->add_new_disk(mddev, rdev); if (err) { - export_rdev(rdev); + md_export_rdev(mddev, rdev); return err; } } @@ -7857,7 +7887,7 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_d= isk_info_s *info, err =3D bind_rdev_to_array(rdev, mddev); =20 if (err) - export_rdev(rdev); + md_export_rdev(mddev, rdev); =20 if (mddev_is_clustered(mddev)) { if (info->state & (1 << MD_DISK_CANDIDATE)) { @@ -7919,7 +7949,7 @@ int md_add_new_disk(struct mddev *mddev, struct mdu_d= isk_info_s *info, =20 err =3D bind_rdev_to_array(rdev, mddev); if (err) { - export_rdev(rdev); + md_export_rdev(mddev, rdev); return err; } } @@ -8031,7 +8061,7 @@ static int hot_add_disk(struct mddev *mddev, struct m= d_new_disk *nd) return 0; =20 abort_export: - export_rdev(rdev); + md_export_rdev(mddev, rdev); return err; } =20 @@ -8577,15 +8607,18 @@ int md_import_new_disk(struct mddev *mddev, struct = mdu_disk_info_s *info, return err; } =20 + /* link the holder here too, for the same reason */ + md_link_rdev_holder(rdev, mddev); + nd->rdev =3D rdev; return 0; } =20 /* release a leg md_add_new_disk() did not take ownership of */ -void md_put_new_disk(struct md_new_disk *nd) +void md_put_new_disk(struct mddev *mddev, struct md_new_disk *nd) { if (nd->rdev) { - export_rdev(nd->rdev); + md_export_rdev(mddev, nd->rdev); nd->rdev =3D NULL; } } @@ -8717,6 +8750,7 @@ static int md_ioctl(struct block_device *bdev, blk_mo= de_t mode, err =3D -EINVAL; goto out; } + md_link_rdev_holder(nd.rdev, mddev); } =20 /* q->limits_lock nests outside both, see md_start_sync() */ @@ -8864,7 +8898,7 @@ static int md_ioctl(struct block_device *bdev, blk_mo= de_t mode, =20 out: /* a leg we opened but nothing took ownership of */ - md_put_new_disk(&nd); + md_put_new_disk(mddev, &nd); =20 if (cmd =3D=3D STOP_ARRAY_RO || (err && cmd =3D=3D STOP_ARRAY)) clear_bit(MD_CLOSING, &mddev->flags); diff --git a/drivers/md/md.h b/drivers/md/md.h index 73a27d83d65a..1a0d57d58ad1 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -294,6 +294,11 @@ enum flag_bits { * serial bios. */ Nonrot, /* non-rotational device (SSD) */ + HolderLinked, /* bd_link_disk_holder() succeeded for this + * leg. The link is made before the array is + * locked, as it takes disk->open_mutex, + * see md_import_new_disk(). + */ }; =20 static inline int is_badblock(struct md_rdev *rdev, sector_t s, sector_t s= ectors, @@ -1069,7 +1074,7 @@ struct md_new_disk { =20 int md_import_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info, struct md_new_disk *nd); -void md_put_new_disk(struct md_new_disk *nd); +void md_put_new_disk(struct mddev *mddev, struct md_new_disk *nd); int md_add_new_disk(struct mddev *mddev, struct mdu_disk_info_s *info, struct md_new_disk *nd, struct queue_limits *lim); int do_md_run(struct mddev *mddev, struct queue_limits *lim); --=20 2.43.0