From nobody Fri Oct 2 10:08:32 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7398E2459E1; Sun, 2 Aug 2026 11:25:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785669942; cv=none; b=o7mnZQfg3wS573Fm0VBShA2w1rRHs5igDMa/7LGQvLox3wyImt07bVnR4eMX7yNrOH3FF3x7Xht4kiTjZ+W9qdYFUK6xF+QtAAs7zRmnM3iG2mMgwKoU5o8cMTcMwvMsz4nYAywU6e9nxXRlnNnn04+DLrRj0k95XizG8d7Bkv4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785669942; c=relaxed/simple; bh=Djn9Wh91Tge1/LEJqxmAp63uERePRRuFeHogS2H/jq0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=udwZJnSNxS55vt20RdQQWlwaJQeEyoZx+NaVu2l9z2m0Y7NnU2O1+XReqCbO66Mv+eA6Cmtz30i1gNFb2ureYZopUgsiAH45DpkG1SIT/HFQJSU1eBIF7PAENf4EWmsywt/X/ulHaRZzpuMTBbeBEyttD5v6SKAYI/AIHl6XwBU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=idViBNig; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="idViBNig" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 221D91F000E9; Sun, 2 Aug 2026 11:25:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785669941; bh=+EElcQqQAaLhU9f5uqkcUEntAit7MrM4wismRiz6B4A=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=idViBNig88ItxvsXNGkgxuMReVHkcGcYRDkXoMBMraLZXfznQvQt90Z0su+0nfw0T K4ljVrItv6W0A+kE37/D1Y8S0ilg6RuHO+b76S3l2YE4Wc+VBEHBtsWLnraXBrt7Bz Zx2FlKleHAhqXr6BN1JVN8/pRRRqPYeeB5m7xHdPgItojR0dT4SacBnPAgBuWAYga2 C45V+aFYyWxfJKLT3Q4pMPzD0PoNnRIgAHYL9NloDe+KOQhdqG+rQuOLW2gBFmItCL 4AcrHNNkte8j01cQjeFJIuffav0GmCn4Lqexs66qk/KKX0DJh4H2mNUoepGeB0kPiO e88hTxNlLJaDw== From: Yu Kuai To: tj@kernel.org, axboe@kernel.dk Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, zhengqixing@huawei.com, hch@lst.de, yizhou.tang@shopee.com, yukuai@fygo.io, nilay@linux.ibm.com Subject: [PATCH v4 1/4] blk-cgroup: protect q->blkg_list iteration in blkg_destroy_all() with blkcg_mutex Date: Sun, 2 Aug 2026 19:25:17 +0800 Message-ID: <20260802112525.3933753-2-yukuai@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260802112525.3933753-1-yukuai@kernel.org> References: <20260802112525.3933753-1-yukuai@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Yu Kuai blkg_destroy_all() iterates q->blkg_list without holding blkcg_mutex, which can race with blkg_free_workfn() that removes blkgs from the list while holding blkcg_mutex. Add blkcg_mutex protection around the q->blkg_list iteration to prevent potential list corruption or use-after-free issues. Reviewed-by: Tang Yizhou Signed-off-by: Yu Kuai Reviewed-by: Nilay Shroff Reviewed-by: Tao Cui --- block/blk-cgroup.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index d9676126c5b5..eb0cfb10b859 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -569,6 +569,7 @@ static void blkg_destroy_all(struct gendisk *disk) int i; =20 restart: + mutex_lock(&q->blkcg_mutex); spin_lock_irq(&q->queue_lock); list_for_each_entry(blkg, &q->blkg_list, q_node) { struct blkcg *blkcg =3D blkg->blkcg; @@ -587,6 +588,7 @@ static void blkg_destroy_all(struct gendisk *disk) if (!(--count)) { count =3D BLKG_DESTROY_BATCH_SIZE; spin_unlock_irq(&q->queue_lock); + mutex_unlock(&q->blkcg_mutex); cond_resched(); goto restart; } @@ -606,6 +608,7 @@ static void blkg_destroy_all(struct gendisk *disk) =20 q->root_blkg =3D NULL; spin_unlock_irq(&q->queue_lock); + mutex_unlock(&q->blkcg_mutex); =20 wake_up_var(&q->root_blkg); } --=20 2.51.0 From nobody Fri Oct 2 10:08:32 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05BEA3655C7; Sun, 2 Aug 2026 11:25:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785669947; cv=none; b=AwZLACykDYDNaPvVuDh3zyDZZJnXqgwJnvkkshJ61rJsAga+tsm55Tl1hPmgB0wW7lDN44VxtN/9iOnK+K1KVo4twbC0uVJkZq8aOAYWZGqw8XIrOaboKCUmJx/IukQmasSJ4brksy4cOMHz81dtgIgKURRtfhwpNS5/FkRGYns= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785669947; c=relaxed/simple; bh=2UTzKqgDdno6vcTrIhtaC5W6GsAEgwiO/gdu5cNmrlY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KgFWYfhThCCm7jR3PSiXN0jc3nGLUNBijzpAvCvxQimRlH+xPhCtjcolVnPOxRO4N7PMe1OuyxjatKWwmb7vzdq/6ZVAElh+wng1P5/r2ssl3DvUlZTDmRpdwIFCQ/KcLdeQC4mpvLea2HMLBbjxo2WCjwIrWCwdSj3Jtucx9rA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=axpWoho+; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="axpWoho+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C520E1F00A3A; Sun, 2 Aug 2026 11:25:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785669945; bh=v0L+TiAxcy/6VWLByx/spXZdJvdVONE9HQS+Bjch3BU=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=axpWoho+EZLZlpnjQaVnqOhbFzfQ3qg94REMzMTvfBrJxftnXuZKyrnrBz42Z10o0 WwDg5iTCwAANupu7cPWSMrKVJKdtH+/bS5rXc9sxA5leTBWosqKOhOgxCEKBoqxAj7 Ydogc1jotqq0fL0+HV68TyuvQL/pdz+jqI5KrpnEwcNRgb+dxkF6+ZiA3QfvIuR8VJ FuxepamamQ4ounu6jJrI77jzjiZ6/rXKSdQ50ML0dx2uL/8vhthKfWLF9VsnvH0lVX xh/88230h7Nn67sarSAjwakv8ofwJQIHvBwsvi3qwv8QTc2bURv52PQtgqmn2Nu62d lWyW+ROvSK9gw== From: Yu Kuai To: tj@kernel.org, axboe@kernel.dk Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, zhengqixing@huawei.com, hch@lst.de, yizhou.tang@shopee.com, yukuai@fygo.io, nilay@linux.ibm.com Subject: [PATCH v4 2/4] blk-cgroup: fix race between policy activation and blkg destruction Date: Sun, 2 Aug 2026 19:25:18 +0800 Message-ID: <20260802112525.3933753-3-yukuai@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260802112525.3933753-1-yukuai@kernel.org> References: <20260802112525.3933753-1-yukuai@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zheng Qixing When switching an IO scheduler on a block device, blkcg_activate_policy() allocates blkg_policy_data (pd) for all blkgs attached to the queue. However, blkcg_activate_policy() may race with concurrent blkcg deletion, leading to use-after-free and memory leak issues. The use-after-free occurs in the following race: T1 (blkcg_activate_policy): - Successfully allocates pd for blkg1 (loop0->queue, blkcgA) - Fails to allocate pd for blkg2 (loop0->queue, blkcgB) - Enters the enomem rollback path to release blkg1 resources T2 (blkcg deletion): - blkcgA is deleted concurrently - blkg1 is freed via blkg_free_workfn() - blkg1->pd is freed T1 (continued): - Rollback path accesses blkg1->pd->online after pd is freed - Triggers use-after-free In addition, blkg_free_workfn() frees pd before removing the blkg from q->blkg_list. This allows blkcg_activate_policy() to allocate a new pd for a blkg that is being destroyed, leaving the newly allocated pd unreachable when the blkg is finally freed. Fix these races by extending blkcg_mutex coverage to serialize blkcg_activate_policy() rollback and blkg destruction, ensuring pd lifecycle is synchronized with blkg list visibility. Fixes: f1c006f1c685 ("blk-cgroup: synchronize pd_free_fn() from blkg_free_w= orkfn() and blkcg_deactivate_policy()") Signed-off-by: Zheng Qixing Reviewed-by: Tang Yizhou Signed-off-by: Yu Kuai Reviewed-by: Nilay Shroff Reviewed-by: Tao Cui --- block/blk-cgroup.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index eb0cfb10b859..047bb42c282b 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1566,6 +1566,8 @@ int blkcg_activate_policy(struct gendisk *disk, const= struct blkcg_policy *pol) =20 if (queue_is_mq(q)) memflags =3D blk_mq_freeze_queue(q); + + mutex_lock(&q->blkcg_mutex); retry: spin_lock_irq(&q->queue_lock); =20 @@ -1628,6 +1630,7 @@ int blkcg_activate_policy(struct gendisk *disk, const= struct blkcg_policy *pol) =20 spin_unlock_irq(&q->queue_lock); out: + mutex_unlock(&q->blkcg_mutex); if (queue_is_mq(q)) blk_mq_unfreeze_queue(q, memflags); if (pinned_blkg) --=20 2.51.0 From nobody Fri Oct 2 10:08:32 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 299FD381E94; Sun, 2 Aug 2026 11:25:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785669952; cv=none; b=fYdhUO71WTSp3R5BAAiAdvYoNvxjx6Q0zDeVIlYSwywlHelBleA0IBgtKR/Vz2Xs702WHIuSirM1hLvZ53tlWl3mcauHU1pDLq2SS+og9TLC67RCJsJxDbIOVoMA5FVJxFppVnfLkwFMZxh20Ung2WxsyU4q+0aSHet/C8Q4v20= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785669952; c=relaxed/simple; bh=CbIcUGaG40OgJzRfbvqKSYTHTJMk+WRH2xbtk4uUbps=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rX8YIrZoACsQpY3MM5CnRXx0WcuRlQC/2c/BhH9R4xndGjrCN7NFcdy3fS4yUJ7l1+jZCFG+l4IBsDT/jW0AG99xoq5rn1oDmDEolUDolo0VyYAfcpG0OtCZlp/cVMJ4qiPFy7wELtzYp/bkDzj4mjXiA8i5BisaI62N3YrBWnY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=C1u15YaA; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="C1u15YaA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 30F731F000E9; Sun, 2 Aug 2026 11:25:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785669950; bh=zU+TRKoCIS8QnsUNPujkhjRjzhengrzuAzjCZsXnVwM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=C1u15YaALUspMGEXBQUaDY3xQPADr/4inUFUJzqWP7Pe7OdwtmjG19Ki0RdSOsj2/ LKkdvlt3eeNdKc4ciloR46liyglNsn4mIPOVnUWJL1+1R2ftBB9yRtrmFEFOWTzWPW OOml2u6cRIYpCIfp0jJKW+1FJOMFmaECDFK5JsaWzCI/D7+KT9jckn1xOjAcxnR34Q rILNF0mbNgXkUJKXgcOqnsx0mMijmhfnqCnZZai4M2Kggn/n8Ld5u92jVSQJ1mUAyL sQyr3iHif6M3ZoN7v+synZteN9TtAw2yPmUfdRUim3Cv6T07zbjwv2+mCcp1mloAPX s59/eYjTd+bCg== From: Yu Kuai To: tj@kernel.org, axboe@kernel.dk Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, zhengqixing@huawei.com, hch@lst.de, yizhou.tang@shopee.com, yukuai@fygo.io, nilay@linux.ibm.com Subject: [PATCH v4 3/4] blk-cgroup: skip dying blkg in blkcg_activate_policy() Date: Sun, 2 Aug 2026 19:25:19 +0800 Message-ID: <20260802112525.3933753-4-yukuai@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260802112525.3933753-1-yukuai@kernel.org> References: <20260802112525.3933753-1-yukuai@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zheng Qixing When switching IO schedulers on a block device, blkcg_activate_policy() can race with concurrent blkcg deletion, leading to a use-after-free in rcu_accelerate_cbs. T1: T2: blkg_destroy kill(&blkg->refcnt) // blkg->refcnt=3D1->0 blkg_release // call_rcu(__blkg_release) ... blkg_free_workfn ->pd_free_fn(pd) elv_iosched_store elevator_switch ... iterate blkg list blkg_get(blkg) // blkg->refcnt=3D0->1 list_del_init(&blkg->q_node) blkg_put(pinned_blkg) // blkg->refcnt=3D1->0 blkg_release // call_rcu again rcu_accelerate_cbs // uaf Fix this by checking hlist_unhashed(&blkg->blkcg_node) before getting a reference to the blkg. This is the same check used in blkg_destroy() to detect if a blkg has already been destroyed. If the blkg is already unhashed, skip processing it since it's being destroyed. Fixes: f1c006f1c685 ("blk-cgroup: synchronize pd_free_fn() from blkg_free_w= orkfn() and blkcg_deactivate_policy()") Signed-off-by: Zheng Qixing Reviewed-by: Tang Yizhou Signed-off-by: Yu Kuai Reviewed-by: Nilay Shroff Reviewed-by: Tao Cui --- block/blk-cgroup.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 047bb42c282b..d1895bc60fcf 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1577,6 +1577,8 @@ int blkcg_activate_policy(struct gendisk *disk, const= struct blkcg_policy *pol) =20 if (blkg->pd[pol->plid]) continue; + if (hlist_unhashed(&blkg->blkcg_node)) + continue; =20 /* If prealloc matches, use it; otherwise try GFP_NOWAIT */ if (blkg =3D=3D pinned_blkg) { --=20 2.51.0 From nobody Fri Oct 2 10:08:32 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B2DF381E94; Sun, 2 Aug 2026 11:25:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785669958; cv=none; b=OBOSxnmlArgnPP0yHu+8nmaVTQqg0G4yrzXSGn1ldZxFBfgu1F0m6DCtu8G2DC/nEPwtuEOr0cRjTb3uay6So7nwY0q2sC6JYKv6FBsM/ywMmWIE4PqPE0sjZhITbHdGWDqXEIQw1LeCW8kvl4YrXkv+VeYo9+sV+00MxkF3NBs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785669958; c=relaxed/simple; bh=jel9Va4aEJ32X+R8Wb9KTKWlQY6onZovT4E5m9mtEe4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tttRc+XObJqSACJ0Udd+n2lbTnx4zgI5wuQNc+FzcObFTy86UdR25Ba+qBL2QckwLgMdpDLjp0I+PxvV7BkHHSVXaOsAUU3BHu6WpBGSh78ItBQA35p2JODXh2pX+TNZAwgrcNLzxL4lz7xX5y6PsUMBTXhp/sJTUDZvvWVoWTo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MHl2FXRa; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MHl2FXRa" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 646481F00A3A; Sun, 2 Aug 2026 11:25:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785669957; bh=whxTPtkSCFbK3RhJqyiIvmWdhRx7SscEg9SF0mVf0Rc=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=MHl2FXRaO35rSiqFTTP0HdmBDkJ2fUESZLuh+9W1GxqaoOiejE+7b+ak+ZVnhfhGi 5oDd5BAvH3eLfumLJqatVLBUcZPsTQ2OQ2UjAO67QvLMySDfF1dcpyoW14BbBSQWMV umIMeIWMlQOUNMaaFbVZeXkYPOyudNQfZ/GXRtxkaLowsUNLAue2SH/3wbV67vcli0 tqA+cr64cVJy6haH1UPHGquJe4HW+yP2KkkpfLgMue7xX2FWoIZpfcg8x+kzDBk9o5 WyCXHj7pAiPnp2u0ibq665kPhUZe4j1v7Tik85O8fSIu3ShDhBRrMEFiGM9fo5noUi bn8LqSBqHbVmQ== From: Yu Kuai To: tj@kernel.org, axboe@kernel.dk Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, zhengqixing@huawei.com, hch@lst.de, yizhou.tang@shopee.com, yukuai@fygo.io, nilay@linux.ibm.com Subject: [PATCH v4 4/4] blk-cgroup: factor policy pd teardown loop into helper Date: Sun, 2 Aug 2026 19:25:20 +0800 Message-ID: <20260802112525.3933753-5-yukuai@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260802112525.3933753-1-yukuai@kernel.org> References: <20260802112525.3933753-1-yukuai@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Zheng Qixing Move the teardown sequence which offlines and frees per-policy blkg_policy_data (pd) into a helper for readability. No functional change intended. Signed-off-by: Zheng Qixing Reviewed-by: Christoph Hellwig Reviewed-by: Tang Yizhou Signed-off-by: Yu Kuai Reviewed-by: Nilay Shroff --- block/blk-cgroup.c | 57 ++++++++++++++++++++++------------------------ 1 file changed, 27 insertions(+), 30 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index d1895bc60fcf..354637f3b158 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1529,6 +1529,31 @@ struct cgroup_subsys io_cgrp_subsys =3D { }; EXPORT_SYMBOL_GPL(io_cgrp_subsys); =20 +/* + * Tear down per-blkg policy data for @pol on @q. + */ +static void blkcg_policy_teardown_pds(struct request_queue *q, + const struct blkcg_policy *pol) +{ + struct blkcg_gq *blkg; + + list_for_each_entry(blkg, &q->blkg_list, q_node) { + struct blkcg *blkcg =3D blkg->blkcg; + struct blkg_policy_data *pd; + + spin_lock(&blkcg->lock); + pd =3D blkg->pd[pol->plid]; + if (pd) { + if (pd->online && pol->pd_offline_fn) + pol->pd_offline_fn(pd); + pd->online =3D false; + pol->pd_free_fn(pd); + WRITE_ONCE(blkg->pd[pol->plid], NULL); + } + spin_unlock(&blkcg->lock); + } +} + /** * blkcg_activate_policy - activate a blkcg policy on a gendisk * @disk: gendisk of interest @@ -1644,21 +1669,7 @@ int blkcg_activate_policy(struct gendisk *disk, cons= t struct blkcg_policy *pol) enomem: /* alloc failed, take down everything */ spin_lock_irq(&q->queue_lock); - list_for_each_entry(blkg, &q->blkg_list, q_node) { - struct blkcg *blkcg =3D blkg->blkcg; - struct blkg_policy_data *pd; - - spin_lock(&blkcg->lock); - pd =3D blkg->pd[pol->plid]; - if (pd) { - if (pd->online && pol->pd_offline_fn) - pol->pd_offline_fn(pd); - pd->online =3D false; - pol->pd_free_fn(pd); - WRITE_ONCE(blkg->pd[pol->plid], NULL); - } - spin_unlock(&blkcg->lock); - } + blkcg_policy_teardown_pds(q, pol); spin_unlock_irq(&q->queue_lock); ret =3D -ENOMEM; goto out; @@ -1677,7 +1688,6 @@ void blkcg_deactivate_policy(struct gendisk *disk, const struct blkcg_policy *pol) { struct request_queue *q =3D disk->queue; - struct blkcg_gq *blkg; unsigned int memflags; =20 if (!blkcg_policy_enabled(q, pol)) @@ -1690,20 +1700,7 @@ void blkcg_deactivate_policy(struct gendisk *disk, spin_lock_irq(&q->queue_lock); =20 __clear_bit(pol->plid, q->blkcg_pols); - - list_for_each_entry(blkg, &q->blkg_list, q_node) { - struct blkcg *blkcg =3D blkg->blkcg; - - spin_lock(&blkcg->lock); - if (blkg->pd[pol->plid]) { - if (blkg->pd[pol->plid]->online && pol->pd_offline_fn) - pol->pd_offline_fn(blkg->pd[pol->plid]); - pol->pd_free_fn(blkg->pd[pol->plid]); - blkg->pd[pol->plid] =3D NULL; - } - spin_unlock(&blkcg->lock); - } - + blkcg_policy_teardown_pds(q, pol); spin_unlock_irq(&q->queue_lock); mutex_unlock(&q->blkcg_mutex); =20 --=20 2.51.0