From nobody Thu Sep 24 13:37:12 2026 Received: from mta1.migadu.com (out-170.mta1.migadu.com [95.215.58.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E0FF3A71BE for ; Thu, 24 Sep 2026 05:46:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790228781; cv=none; b=rVH7Y/eeF4X8WIHPk+WEJTjAe0+k9CvhV6AahRpEZ88ZqgxNXMSANeeAGBnlvaqJJ/bNVFPOOeW/OreDYSDZgCKlk+jdG76zw2VGZfsNKzFWCe7rsOv+E0usIxUFzD5gejLaWlbhVN3dh6QJKkyEgYn9288LOlG20oXuDD8qyjM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790228781; c=relaxed/simple; bh=jJZ/JMXUL7+FEpA6VQwZfBgIi3TGWmFZKvUY2lUA1yg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EFx6sYHMvq0DImcVzV05JM7Acy24X2LiSAuQ8bG+L2LVHSxTiBMUbxf5GD2sXIQwjB+KiC8sQIu68ESTmFCdu2PTaZWLEXyfTFVK06mkWlgnL6bIJC8cfxsNqhNjJSc6rlZGywgcvRfSULUa9VR+k6MeC1OEnooKNogJpp7ECsM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=oWl0LEkW; arc=none smtp.client-ip=95.215.58.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="oWl0LEkW" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=jJZ/JMXUL7+FEpA6VQwZfBgIi3TGWmFZKvUY2lUA1yg=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1790228775; v=1; x=1790833575; b=oWl0LEkW3OCqeyfCEuJNproNJ7Z88wec8ctoJ3avuHu3hhYgYv5LARgDELPJk8841yB0ofDd /1IL8JoQZK8fIr6n8mzgQlhvgi7W2lXZm3D33GMGpJbRbkWS0P05BEE3Zg3nmG4xZ9NP2ikUzpP 0tB9M1tFCqugl44ETHBryVyY= X-Envelope-To: linux-kernel@vger.kernel.org Received: by smtp.migadu.com with ESMTPS id ee25ecbb27016b22; Thu, 24 Sep 2026 05:46:15 +0000 X-Mizu-Trace-ID: ee25ecbb27016b22 X-Migadu-Flow: FLOW_OUT From: Tao Cui To: tj@kernel.org, josef@toxicopanda.com, axboe@kernel.dk Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, andrii@kernel.org, eddyz87@gmail.com, ast@kernel.org, daniel@iogearbox.net, linux-kselftest@vger.kernel.org, cui.tao@linux.dev, cuitao@kylinos.cn, ameryhung@gmail.com, alexei.starovoitov@gmail.com Subject: [RFC PATCH v7 1/4] blk-iocost: add BPF struct_ops cost model support Date: Thu, 24 Sep 2026 13:45:46 +0800 Message-ID: <20260924054549.2271705-2-cui.tao@linux.dev> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924054549.2271705-1-cui.tao@linux.dev> References: <20260924054549.2271705-1-cui.tao@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Tao Cui Add the iocost_model_ops struct_ops. Attachment follows the hid_bpf_ops model: the struct_ops instance is per-device, the target device is set in the dev member from userspace before load, .reg attaches the model to that device and switches it away from the builtin linear model, and .unreg detaches it and restores the builtin model. The struct_ops core owns the program lifetime, so there is no name registry and no bound-state bookkeeping. calc_cost() receives the bio itself and reads whatever it needs from it (operation flags, size, sector, the issuing cgroup); the merge indicator stays in the separate flags argument as it is not a property of the bio. It is called from the bio charging path, so the model owns pricing for every IO on the device. The completion-time request sizing uses the transfer cost coefficients carried in the struct_ops (vtime per page for reads and writes) while a model is attached, so the builtin latency tracking and vrate adjustment follow the model's pricing; letting a model take over the QoS side is left for a later extension. Attaching to a partition's device number is rejected: the model prices the whole queue. Enabling iocost implicitly through the attach disables wbt, matching io.cost.qos. The cgroup callbacks are bound to the iocg policy init/free paths, one (cgroup, device) pair per invocation, matching the builtin cursor's lifetime, instead of the blkcg css lifecycle, which also drops the mutex from the cgroup online/offline paths. calc_cost() runs under RCU read lock; sleepable programs are rejected in .check_member. Signed-off-by: Tao Cui --- block/Kconfig | 10 ++ block/Makefile | 1 + block/blk-iocost-bpf.c | 171 +++++++++++++++++++++++ block/blk-iocost.c | 275 ++++++++++++++++++++++++++++++++++++- include/linux/blk-iocost.h | 103 ++++++++++++++ 5 files changed, 556 insertions(+), 4 deletions(-) create mode 100644 block/blk-iocost-bpf.c create mode 100644 include/linux/blk-iocost.h diff --git a/block/Kconfig b/block/Kconfig index 70e4a66d941f..1cafc1bd0dda 100644 --- a/block/Kconfig +++ b/block/Kconfig @@ -231,4 +231,14 @@ config BLK_ERROR_INJECTION =20 source "block/Kconfig.iosched" =20 +config BLK_CGROUP_IOCOST_BPF + bool "Enable BPF pluggable cost model support for the cost IO controller" + depends on BLK_CGROUP_IOCOST && BPF_SYSCALL && BPF_JIT && DEBUG_INFO_BTF + help + Enabling this option registers the "iocost_model_ops" BPF + struct_ops type, which allows a BPF program to fully replace + the builtin linear cost model on the device it is attached + to. The struct_ops is attached per device, following the + hid_bpf_ops model. + endif # BLOCK diff --git a/block/Makefile b/block/Makefile index e7bd320e3d69..ee5cebeea006 100644 --- a/block/Makefile +++ b/block/Makefile @@ -39,3 +39,4 @@ obj-$(CONFIG_BLK_INLINE_ENCRYPTION) +=3D blk-crypto.o blk= -crypto-profile.o \ blk-crypto-sysfs.o obj-$(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK) +=3D blk-crypto-fallback.o obj-$(CONFIG_BLOCK_HOLDER_DEPRECATED) +=3D holder.o +obj-$(CONFIG_BLK_CGROUP_IOCOST_BPF) +=3D blk-iocost-bpf.o diff --git a/block/blk-iocost-bpf.c b/block/blk-iocost-bpf.c new file mode 100644 index 000000000000..2306c5fd6d8d --- /dev/null +++ b/block/blk-iocost-bpf.c @@ -0,0 +1,171 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * blk-iocost: BPF struct_ops plumbing for pluggable cost models. + * + * Registers the "iocost_model_ops" struct_ops type. Attachment is + * per-device and follows the hid_bpf_ops model: the target device is + * set in the ops from userspace before load, .reg attaches the model + * to that device and switches it away from the builtin linear model, + * .unreg detaches it and restores the builtin model, and the struct_ops + * core owns the program lifetime. There is no name registry and no + * separate bound-state bookkeeping. + */ +#include +#include +#include +#include +#include +#include +#include + +static int bpf_iocost_model_init(struct btf *btf) +{ + s32 type_id; + + type_id =3D btf_find_by_name_kind(btf, "iocost_model_ops", BTF_KIND_STRUC= T); + if (type_id < 0) + return -EINVAL; + return 0; +} + +static bool bpf_iocost_is_valid_access(int off, int size, + enum bpf_access_type type, + const struct bpf_prog *prog, + struct bpf_insn_access_aux *info) +{ + return bpf_tracing_btf_ctx_access(off, size, type, prog, info); +} + +/* + * No iocost-specific helpers; bpf_base_func_proto already covers the + * cgroup storage helpers under CONFIG_CGROUPS. + */ +static const struct bpf_func_proto * +bpf_iocost_get_func_proto(enum bpf_func_id func_id, + const struct bpf_prog *prog) +{ + return bpf_base_func_proto(func_id, prog); +} + +static int bpf_iocost_check_member(const struct btf_type *t, + const struct btf_member *member, + const struct bpf_prog *prog) +{ + /* calc_cost() is called with RCU read lock held */ + if (prog->sleepable) + return -EINVAL; + return 0; +} + +static int bpf_iocost_init_member(const struct btf_type *t, + const struct btf_member *member, + void *kdata, const void *udata) +{ + struct iocost_model_ops *ops =3D kdata; + const struct iocost_model_ops *uops =3D udata; + u32 moff =3D __btf_member_bit_offset(t, member) / 8; + + switch (moff) { + case offsetof(struct iocost_model_ops, bdev_file): + /* + * kernel-private: the open bdev file pinning the queue; + * reject a userspace value instead of copying it + */ + if (uops->bdev_file) + return -EINVAL; + ops->bdev_file =3D NULL; + return 1; + case offsetof(struct iocost_model_ops, q): + /* kernel-private: the queue of the attached device */ + if (uops->q) + return -EINVAL; + ops->q =3D NULL; + return 1; + case offsetof(struct iocost_model_ops, dev): + /* + * copy it and return 1 to indicate that the member is + * handled here, or the verifier rejects the map if the + * userspace value is nonzero + */ + ops->dev =3D uops->dev; + return 1; + case offsetof(struct iocost_model_ops, read_vtime_per_page): + ops->read_vtime_per_page =3D uops->read_vtime_per_page; + return 1; + case offsetof(struct iocost_model_ops, write_vtime_per_page): + ops->write_vtime_per_page =3D uops->write_vtime_per_page; + return 1; + } + + return 0; +} + +/* + * kvalue is zeroed at map allocation and function members are only + * written when the BPF side provides a prog, so a model which did + * not implement calc_cost leaves it NULL. The dispatch would call + * it on every bio, so reject it here. + */ +static int bpf_iocost_validate(void *kdata) +{ + struct iocost_model_ops *ops =3D kdata; + + return ops->calc_cost ? 0 : -EINVAL; +} + +static int bpf_iocost_reg(void *kdata, struct bpf_link *link) +{ + struct iocost_model_ops *ops =3D kdata; + + if (!ops->dev) + return -EINVAL; + + return ioc_bpf_attach(ops); +} + +static void bpf_iocost_unreg(void *kdata, struct bpf_link *link) +{ + ioc_bpf_unreg(kdata); +} + +static const struct bpf_verifier_ops bpf_iocost_verifier_ops =3D { + .get_func_proto =3D bpf_iocost_get_func_proto, + .is_valid_access =3D bpf_iocost_is_valid_access, +}; + +static u64 bpf_iocost_calc_cost_stub(struct bio *bio, u64 flags) +{ + return 0; +} + +static void bpf_iocost_iocg_init_stub(struct blkcg *blkcg, + struct request_queue *q) +{ } +static void bpf_iocost_iocg_free_stub(struct blkcg *blkcg, + struct request_queue *q) +{ } + +static struct iocost_model_ops __bpf_ops_iocost_model_ops =3D { + .calc_cost =3D bpf_iocost_calc_cost_stub, + .iocg_init =3D bpf_iocost_iocg_init_stub, + .iocg_free =3D bpf_iocost_iocg_free_stub, +}; + +static struct bpf_struct_ops bpf_iocost_model_ops =3D { + .verifier_ops =3D &bpf_iocost_verifier_ops, + .init =3D bpf_iocost_model_init, + .check_member =3D bpf_iocost_check_member, + .init_member =3D bpf_iocost_init_member, + .validate =3D bpf_iocost_validate, + .reg =3D bpf_iocost_reg, + .unreg =3D bpf_iocost_unreg, + .name =3D "iocost_model_ops", + .cfi_stubs =3D &__bpf_ops_iocost_model_ops, + .owner =3D THIS_MODULE, +}; + +static int __init bpf_iocost_init(void) +{ + return register_bpf_struct_ops(&bpf_iocost_model_ops, iocost_model_ops); +} +late_initcall(bpf_iocost_init); diff --git a/block/blk-iocost.c b/block/blk-iocost.c index 2745bffcd5ee..21e4f8cbd9f2 100644 --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -177,6 +177,7 @@ #include #include #include +#include #include #include #include @@ -445,6 +446,11 @@ struct ioc { int autop_idx; bool user_qos_params:1; bool user_cost_model:1; + +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + /* attached BPF cost model, NULL =3D builtin linear model */ + const struct iocost_model_ops __rcu *model; +#endif }; =20 struct iocg_pcpu_stat { @@ -803,8 +809,16 @@ static int ioc_autop_idx(struct ioc *ioc, struct gendi= sk *disk) if (idx < AUTOP_SSD_DFL) return AUTOP_SSD_DFL; =20 - /* if user is overriding anything, maintain what was there */ - if (ioc->user_qos_params || ioc->user_cost_model) + /* if user is overriding anything, maintain what was there; the + * same while a BPF model is attached: the builtin coefficients + * are inert then, so stepping the profile is pointless + */ + if (ioc->user_qos_params || ioc->user_cost_model +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + || rcu_dereference_protected(ioc->model, + lockdep_is_held(&ioc->lock)) +#endif + ) return idx; =20 /* step up/down based on the vrate */ @@ -2572,7 +2586,19 @@ static void calc_vtime_cost_builtin(struct bio *bio,= struct ioc_gq *iocg, static u64 calc_vtime_cost(struct bio *bio, struct ioc_gq *iocg, bool is_m= erge) { u64 cost; - +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + const struct iocost_model_ops *model; + + rcu_read_lock(); + model =3D rcu_dereference(iocg->ioc->model); + if (model) { + cost =3D model->calc_cost(bio, + is_merge ? IOCOST_COST_F_MERGE : 0); + rcu_read_unlock(); + return min(cost, VTIME_PER_SEC); + } + rcu_read_unlock(); +#endif calc_vtime_cost_builtin(bio, iocg, is_merge, &cost); return cost; } @@ -2594,10 +2620,41 @@ static void calc_size_vtime_cost_builtin(struct req= uest *rq, struct ioc *ioc, } } =20 +/* + * Called from the request completion path, where no ioc->lock is + * held; the model pointer is read under RCU, matching the bio-side + * calc_vtime_cost(). + */ static u64 calc_size_vtime_cost(struct request *rq, struct ioc *ioc) { u64 cost; - +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + const struct iocost_model_ops *model; + + rcu_read_lock(); + model =3D rcu_dereference(ioc->model); + if (model && (req_op(rq) =3D=3D REQ_OP_READ || + req_op(rq) =3D=3D REQ_OP_WRITE)) { + unsigned int pages =3D + blk_rq_stats_sectors(rq) >> IOC_SECT_TO_PAGE_SHIFT; + u64 coeff =3D req_op(rq) =3D=3D REQ_OP_READ ? + model->read_vtime_per_page : + model->write_vtime_per_page; + + rcu_read_unlock(); + /* sub-page IO: nothing to transfer-price */ + if (!pages) + return 0; + /* zero transfer cost is a legal model; guard the division */ + if (!coeff) + return 0; + /* pages * coeff can wrap and dodge the clamp below */ + if (coeff > VTIME_PER_SEC || pages > VTIME_PER_SEC / coeff) + return VTIME_PER_SEC; + return min(pages * coeff, VTIME_PER_SEC); + } + rcu_read_unlock(); +#endif calc_size_vtime_cost_builtin(rq, ioc, &cost); return cost; } @@ -2891,6 +2948,9 @@ static void ioc_rqos_queue_depth_changed(struct rq_qo= s *rqos) static void ioc_rqos_exit(struct rq_qos *rqos) { struct ioc *ioc =3D rqos_to_ioc(rqos); +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + const struct iocost_model_ops *model; +#endif =20 blkcg_deactivate_policy(rqos->disk, &blkcg_policy_iocost); =20 @@ -2900,6 +2960,15 @@ static void ioc_rqos_exit(struct rq_qos *rqos) =20 timer_shutdown_sync(&ioc->timer); free_percpu(ioc->pcpu_stat); +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + spin_lock_irq(&ioc->lock); + model =3D rcu_dereference_protected(ioc->model, + lockdep_is_held(&ioc->lock)); + rcu_assign_pointer(ioc->model, NULL); + spin_unlock_irq(&ioc->lock); + if (model) + ioc_bpf_detach((struct iocost_model_ops *)model); +#endif kfree(ioc); } =20 @@ -3022,6 +3091,9 @@ static void ioc_pd_init(struct blkg_policy_data *pd) struct ioc_now now; struct blkcg_gq *tblkg; unsigned long flags; +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + const struct iocost_model_ops *model; +#endif =20 ioc_now(ioc, &now); =20 @@ -3048,6 +3120,19 @@ static void ioc_pd_init(struct blkg_policy_data *pd) spin_lock_irqsave(&ioc->lock, flags); weight_updated(iocg, &now); spin_unlock_irqrestore(&ioc->lock, flags); +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + /* + * the model pointer and the ops behind it are RCU-protected: + * a concurrent detach publishes NULL and the struct_ops image + * survives it by a grace period, so the callback is safe + * inside the read-side critical section + */ + rcu_read_lock(); + model =3D rcu_dereference(ioc->model); + if (model && model->iocg_init) + model->iocg_init(blkg->blkcg, ioc->rqos.disk->queue); + rcu_read_unlock(); +#endif } =20 static void iocg_release(struct rcu_head *rcu) @@ -3066,8 +3151,18 @@ static void ioc_pd_free(struct blkg_policy_data *pd) struct blkcg_gq *blkg =3D pd_to_blkg(pd); struct ioc *ioc =3D iocg->ioc; unsigned long flags; +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + const struct iocost_model_ops *model; +#endif =20 if (ioc) { +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + rcu_read_lock(); + model =3D rcu_dereference(ioc->model); + if (model && model->iocg_free) + model->iocg_free(blkg->blkcg, ioc->rqos.disk->queue); + rcu_read_unlock(); +#endif spin_lock_irqsave(&ioc->lock, flags); =20 if (!list_empty(&iocg->active_list)) { @@ -3433,17 +3528,34 @@ static u64 ioc_cost_model_prfill(struct seq_file *s= f, const char *dname =3D blkg_dev_name(pd->blkg); struct ioc *ioc =3D pd_to_iocg(pd)->ioc; u64 *u =3D ioc->params.i_lcoefs; +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + const struct iocost_model_ops *model; +#endif =20 if (!dname) return 0; =20 spin_lock_irq(&ioc->lock); +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + model =3D rcu_dereference_protected(ioc->model, + lockdep_is_held(&ioc->lock)); + + seq_printf(sf, "%s ctrl=3D%s model=3D%s " + "rbps=3D%llu rseqiops=3D%llu rrandiops=3D%llu " + "wbps=3D%llu wseqiops=3D%llu wrandiops=3D%llu\n", + dname, ioc->user_cost_model ? "user" : "auto", + model ? "bpf" : "linear", + u[I_LCOEF_RBPS], u[I_LCOEF_RSEQIOPS], + u[I_LCOEF_RRANDIOPS], u[I_LCOEF_WBPS], + u[I_LCOEF_WSEQIOPS], u[I_LCOEF_WRANDIOPS]); +#else seq_printf(sf, "%s ctrl=3D%s model=3Dlinear " "rbps=3D%llu rseqiops=3D%llu rrandiops=3D%llu " "wbps=3D%llu wseqiops=3D%llu wrandiops=3D%llu\n", dname, ioc->user_cost_model ? "user" : "auto", u[I_LCOEF_RBPS], u[I_LCOEF_RSEQIOPS], u[I_LCOEF_RRANDIOPS], u[I_LCOEF_WBPS], u[I_LCOEF_WSEQIOPS], u[I_LCOEF_WRANDIOPS]); +#endif spin_unlock_irq(&ioc->lock); return 0; } @@ -3457,6 +3569,141 @@ static int ioc_cost_model_show(struct seq_file *sf,= void *v) return 0; } =20 +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF +/* + * Attach a BPF cost model to the device named by ops->dev: resolve the + * queue, make sure iocost is on it, and publish the model. Attaching + * switches the device away from the builtin linear model; detaching + * restores it. The struct_ops core holds the program alive for the + * whole registered period, so no extra reference is taken on the ops. + */ +int ioc_bpf_attach(struct iocost_model_ops *ops) +{ + struct block_device *bdev; + struct request_queue *q; + struct ioc *ioc; + const struct iocost_model_ops *old; + struct file *bdevf; + int ret; + + bdevf =3D bdev_file_open_by_dev(new_decode_dev(ops->dev), + BLK_OPEN_READ, NULL, NULL); + if (IS_ERR(bdevf)) + return PTR_ERR(bdevf); + bdev =3D file_bdev(bdevf); + + if (bdev_is_partition(bdev)) { + fput(bdevf); + return -EINVAL; + } + + q =3D bdev->bd_queue; + if (!queue_is_mq(q)) { + fput(bdevf); + return -EOPNOTSUPP; + } + + mutex_lock(&q->rq_qos_mutex); + ioc =3D q_to_ioc(q); + if (!ioc) { + ret =3D blk_iocost_init(bdev->bd_disk); + if (ret) { + mutex_unlock(&q->rq_qos_mutex); + fput(bdevf); + return ret; + } + ioc =3D q_to_ioc(q); + } + + /* + * Stay under rq_qos_mutex until the model is published: + * ioc_rqos_exit() frees the ioc under this mutex, so holding + * it keeps the ioc alive through the publish below. The open + * bdev file pins the queue for as long as the model is + * attached; it is released by the .unreg side of the detach. + */ + spin_lock_irq(&ioc->lock); + old =3D rcu_dereference_protected(ioc->model, + lockdep_is_held(&ioc->lock)); + if (old) { + spin_unlock_irq(&ioc->lock); + mutex_unlock(&q->rq_qos_mutex); + fput(bdevf); + return -EBUSY; + } + if (!ioc->enabled) { + /* + * the controller must run for the model to be consulted: + * enable it like io.cost.qos enable=3D1 does + */ + blk_stat_enable_accounting(q); + blk_queue_flag_set(QUEUE_FLAG_RQ_ALLOC_TIME, q); + ioc->enabled =3D true; + ioc_refresh_params(ioc, true); + } + rcu_assign_pointer(ioc->model, ops); + ops->q =3D q; + ops->bdev_file =3D bdevf; + spin_unlock_irq(&ioc->lock); + /* match io.cost.qos: running iocost disables wbt */ + wbt_disable_default(bdev->bd_disk); + mutex_unlock(&q->rq_qos_mutex); + + return 0; +} + +/* + * Detach a model. The caller holds q->rq_qos_mutex, which serializes + * this against ioc_bpf_attach(), against ioc_rqos_exit() freeing the + * ioc, and against a concurrent .unreg, so the ops->q/model clearing + * is idempotent. The bdev file is not released here: it pins the + * queue for the .unreg side, which may still be about to lock it. + */ +void ioc_bpf_detach(struct iocost_model_ops *ops) +{ + struct request_queue *q =3D ops->q; + struct ioc *ioc; + + if (!q) + return; + + ioc =3D q_to_ioc(q); + /* pairs with the lockless READ_ONCE() in ioc_bpf_unreg() */ + WRITE_ONCE(ops->q, NULL); + + if (!ioc) + return; + + spin_lock_irq(&ioc->lock); + if (rcu_dereference_protected(ioc->model, + lockdep_is_held(&ioc->lock)) =3D=3D ops) + rcu_assign_pointer(ioc->model, NULL); + spin_unlock_irq(&ioc->lock); +} + +/* + * The .unreg side of the detach: serialize against the queue + * teardown, then drop the bdev file pinning the queue. ops->q is + * stable here: only ioc_bpf_detach() clears it, the file pin keeps + * the queue alive until it is dropped below, and .unreg runs once. + */ +void ioc_bpf_unreg(struct iocost_model_ops *ops) +{ + struct request_queue *q =3D READ_ONCE(ops->q); + struct file *bdevf =3D ops->bdev_file; + + ops->bdev_file =3D NULL; + if (q) + mutex_lock(&q->rq_qos_mutex); + ioc_bpf_detach(ops); + if (q) + mutex_unlock(&q->rq_qos_mutex); + if (bdevf) + fput(bdevf); +} + +#endif + static const match_table_t cost_ctrl_tokens =3D { { COST_CTRL, "ctrl=3D%s" }, { COST_MODEL, "model=3D%s" }, @@ -3531,11 +3778,31 @@ static ssize_t ioc_cost_model_write(struct kernfs_o= pen_file *of, char *input, user =3D false; else if (!strcmp(buf, "user")) user =3D true; +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + else if (!strcmp(buf, "bpf")) { + /* + * readback value while a BPF model is + * attached; attaching is done by loading + * the struct_ops, not through this file + */ + continue; + } +#endif else goto unlock; continue; case COST_MODEL: match_strlcpy(buf, &args[0], sizeof(buf)); +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + if (!strcmp(buf, "bpf")) { + /* + * readback value while a BPF model is + * attached; attaching is done by loading + * the struct_ops, not through this file + */ + continue; + } +#endif if (strcmp(buf, "linear")) goto unlock; continue; diff --git a/include/linux/blk-iocost.h b/include/linux/blk-iocost.h new file mode 100644 index 000000000000..ce120a8d6007 --- /dev/null +++ b/include/linux/blk-iocost.h @@ -0,0 +1,103 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_BLK_IOCOST_H +#define _LINUX_BLK_IOCOST_H + +#include +#include +#include + +#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF + +struct bio; +struct blkcg; +struct request_queue; + +/* + * Pluggable cost model interface for blk-iocost. + * + * A BPF struct_ops implementation is attached to one device, identified + * by the dev member set from userspace before load, following the + * hid_bpf_ops model: attaching the struct_ops switches the device to + * the BPF model, detaching it restores the builtin linear model, and + * the struct_ops core owns the lifetime of the program. The model + * then owns pricing for every charged IO on the device: it prices all + * operations, including flushes, from the bio charging path. + * + * calc_cost() is called from the IO submission path with RCU read lock + * held and must not sleep. It receives the bio itself so the model + * can read whatever it needs (operation flags, size, sector, the + * issuing cgroup through bio->bi_blkg). It returns the cost of the + * IO in vtime units, where 1 second of device time equals + * VTIME_PER_SEC (2^37, available to BPF programs through vmlinux.h). + * The returned value is clamped to 1 second of device time per IO. + * + * The struct_ops also carries the transfer cost coefficients, vtime + * per page for reads and writes: while a model is attached, the + * builtin latency tracking and vrate adjustment use these instead of + * the builtin linear coefficients for the completion-time request + * sizing, so the whole controller follows the model's pricing. Letting + * a model take over the QoS side entirely (latency tracking, vrate + * control) is left for a later extension. + * + * The cgroup callbacks are bound to the iocg policy lifetime, one + * (cgroup, device) pair per invocation, matching the builtin cursor: + * state created in init (or lazily on first use) must be released in + * free. + */ + +/* + * iocost-specific call metadata for calc_cost()'s model_flags + * argument; the merge indicator is not a property of the bio. + * An enum so the value is exported through BTF and BPF models can + * use it from vmlinux.h. + */ +enum { + IOCOST_COST_F_MERGE =3D 1 << 0, /* called from merge path */ +}; + +struct iocost_model_ops { + /* + * target device (major:minor), set from userspace before load; + * must stay the first member so userspace can write it through + * the struct_ops map's initial value + */ + dev_t dev; + /* kernel-private: the open bdev file pinning the queue */ + struct file *bdev_file; + + /* vtime per page, used by the builtin sizing and vrate logic */ + u64 read_vtime_per_page; + u64 write_vtime_per_page; + + u64 (*calc_cost)(struct bio *bio, u64 model_flags); + /* + * per-(cgroup, device) lifecycle: both callbacks run inside + * an RCU read-side critical section (see below) and must not + * sleep; IRQs may be enabled or disabled, so per-CPU state + * must not rely on the IRQs-off guarantee. iocg_init() is + * delivered for + * cgroups which appear on the device while the model is + * attached; cgroups which already exist when the model is + * attached never see an init, so iocg_free() must tolerate + * freeing state it never initialized. iocg_free() is only + * delivered while the model is attached: detaching does not + * flush state created by iocg_init(), so models must keep + * their per-cgroup state reclaimable by other means + */ + void (*iocg_init)(struct blkcg *blkcg, struct request_queue *q); + void (*iocg_free)(struct blkcg *blkcg, struct request_queue *q); + + /* private: */ + + /* queue of the attached device, NULL =3D not attached */ + struct request_queue *q; +}; + +int ioc_bpf_attach(struct iocost_model_ops *ops); +void ioc_bpf_detach(struct iocost_model_ops *ops); +void ioc_bpf_unreg(struct iocost_model_ops *ops); + +#else /* CONFIG_BLK_CGROUP_IOCOST_BPF */ + +#endif /* CONFIG_BLK_CGROUP_IOCOST_BPF */ +#endif /* _LINUX_BLK_IOCOST_H */ --=20 2.43.0 From nobody Thu Sep 24 13:37:12 2026 Received: from mta1.migadu.com (out-179.mta1.migadu.com [95.215.58.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4ADA5396D38 for ; Thu, 24 Sep 2026 05:46:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790228784; cv=none; b=JX7I3MTa0paUaa0raLvv2oI+2L3SHR2hyIRMumlZlQ0TdwQj5QPUTIEFk4+t7Eg92+6sXwvDfASEFT1OVTfrV6dRfjG5ASydEI8Dc//HxnQyTzrYj4s59HpA9T4g79EG4EGNAJE8V+iUdoLBytWFdRg+S2/HdHFV5vDYysh1dWo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790228784; c=relaxed/simple; bh=6JLNIAwtKTexkjNyhU377tDrFRNH0SRxqzkPHvqyq6c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hL+GDQRk/e6/rCthLzOa3QHNetI8BkW0nGvxIIWaJDKT+TE2rbyAhlf/G8PDte/JBRnWxYsuuZKbmnypVZnqsqJPPwXTyhYit5o343vxAGCtUpWzL2WcESX86bKXpcI0Naky2MygIfdSymKYLXw1Ga1cPBbIHeJZX5Ok6h6k9SI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=ms2XLGHc; arc=none smtp.client-ip=95.215.58.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="ms2XLGHc" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=6JLNIAwtKTexkjNyhU377tDrFRNH0SRxqzkPHvqyq6c=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1790228779; v=1; x=1790833579; b=ms2XLGHcmiVg1B/QyrV4Ib4/ExQBT3yQ4jYckL4c5WGOSRQ3XFNjKtNDA/MFCKeMszVf2mYF xOH4fE/zebeLMca+9kpzsfCMovv20SVDqE9SkNaLhTgRTgxN4N+QGR01H+6mVtcK+MjJv6NL44R +VI2dUgSUmXKmD75Jxn3Adfo= X-Envelope-To: linux-kernel@vger.kernel.org Received: by smtp.migadu.com with ESMTPS id 6306604b1a3171c4; Thu, 24 Sep 2026 05:46:18 +0000 X-Mizu-Trace-ID: 6306604b1a3171c4 X-Migadu-Flow: FLOW_OUT From: Tao Cui To: tj@kernel.org, josef@toxicopanda.com, axboe@kernel.dk Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, andrii@kernel.org, eddyz87@gmail.com, ast@kernel.org, daniel@iogearbox.net, linux-kselftest@vger.kernel.org, cui.tao@linux.dev, cuitao@kylinos.cn, ameryhung@gmail.com, alexei.starovoitov@gmail.com Subject: [RFC PATCH v7 2/4] selftests/bpf: add iocost cost model test Date: Thu, 24 Sep 2026 13:45:47 +0800 Message-ID: <20260924054549.2271705-3-cui.tao@linux.dev> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924054549.2271705-1-cui.tao@linux.dev> References: <20260924054549.2271705-1-cui.tao@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Tao Cui Add an example cost model implementing the full builtin linear HDD formula at double cost, and a test which attaches it to one device: the dev member of the struct_ops is written through the map's initial value before load, as hid_bpf tests do with hid_id, and attaching the struct_ops attaches the model to the device. The test verifies the model=3Dbpf readback while attached, that a second model on the same device fails with -EBUSY, and that detaching restores the builtin model. Under the same workload the doubled model charges 1.99x the builtin model (measured 2882us -> 5722us per IO, completed IO count halved). The example also sets the transfer cost coefficients so the builtin sizing follows the doubled pricing. Per-cgroup stream state uses a CGRP_STORAGE map keyed by the cgroup of the issuing bio, so the model inherits the cgroup lifetime. opf carries the full bio->bi_opf including REQ_* flag bits, so the operation must be extracted with a mask, not compared for equality. CONFIG_BLK_CGROUP_IOCOST and CONFIG_BLK_CGROUP_IOCOST_BPF are added to the selftest kernel config: without them vmlinux.h does not contain iocost_model_ops and the skeletons fail to build; the runtime skip cannot avoid a build dependency. Requires root, cgroup v2 and a device given as major:minor in $IOCOST_TEST_DEV. Signed-off-by: Tao Cui --- tools/testing/selftests/bpf/config | 2 + .../selftests/bpf/prog_tests/iocost_model.c | 182 ++++++++++++++++++ .../selftests/bpf/progs/iocost_model.c | 139 +++++++++++++ tools/testing/selftests/bpf/progs/iocost_ms.c | 159 +++++++++++++++ 4 files changed, 482 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/iocost_model.c create mode 100644 tools/testing/selftests/bpf/progs/iocost_model.c create mode 100644 tools/testing/selftests/bpf/progs/iocost_ms.c diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/b= pf/config index d292cb60a5a4..6e005145d3a8 100644 --- a/tools/testing/selftests/bpf/config +++ b/tools/testing/selftests/bpf/config @@ -140,3 +140,5 @@ CONFIG_SMC_HS_CTRL_BPF=3Dy CONFIG_DIBS=3Dy CONFIG_DIBS_LO=3Dy CONFIG_PM_WAKELOCKS=3Dy +CONFIG_BLK_CGROUP_IOCOST=3Dy +CONFIG_BLK_CGROUP_IOCOST_BPF=3Dy diff --git a/tools/testing/selftests/bpf/prog_tests/iocost_model.c b/tools/= testing/selftests/bpf/prog_tests/iocost_model.c new file mode 100644 index 000000000000..156c75367af0 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/iocost_model.c @@ -0,0 +1,182 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "iocost_model.skel.h" +#include "iocost_ms.skel.h" + +/* + * Read back the io.cost.model line of dev and copy the model=3D value + * into @model. Returns 0 on success. + */ +static int readback_model(const char *dev, char *model, size_t model_sz) +{ + char line[256], word[256], *m, *end; + FILE *fp; + int found =3D 0; + + fp =3D fopen("/sys/fs/cgroup/io.cost.model", "r"); + if (!fp) + return -1; + while (fgets(line, sizeof(line), fp)) { + if (sscanf(line, "%255s", word) =3D=3D 1 && !strcmp(word, dev)) { + found =3D 1; + break; + } + } + fclose(fp); + if (!found) + return -1; + + m =3D strstr(line, "model=3D"); + if (!m) + return -1; + m +=3D strlen("model=3D"); + end =3D m; + while (*end && *end !=3D ' ') + end++; + snprintf(model, model_sz, "%.*s", (int)(end - m), m); + return 0; +} + +/* + * Attach the example model to one device, given as major:minor in + * $IOCOST_TEST_DEV: the dev member is written through the struct_ops + * map's initial value before load, as hid_bpf_ops does with hid_id, + * and loading attaches the model to the device. Detaching the + * struct_ops restores the builtin model. + * + * Requires root, cgroup v2 and a device with iocost support. + */ +void serial_test_iocost_model(void) +{ + struct iocost_model *skel, *second; + unsigned int maj, min; + __u64 *ops_dev, *sdev; + int err; + char model[32], *dev; + + dev =3D getenv("IOCOST_TEST_DEV"); + if (!dev || geteuid() !=3D 0 || sscanf(dev, "%u:%u", &maj, &min) !=3D 2) { + test__skip(); + return; + } + + skel =3D iocost_model__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + /* dev is the first member of struct iocost_model_ops */ + ops_dev =3D bpf_map__initial_value(skel->maps.iocost_2x, NULL); + if (!ASSERT_OK_PTR(ops_dev, "initial_value")) { + iocost_model__destroy(skel); + return; + } + *ops_dev =3D makedev(maj, min); + + err =3D iocost_model__load(skel); + if (!ASSERT_OK(err, "skel_load")) { + iocost_model__destroy(skel); + return; + } + + err =3D iocost_model__attach(skel); + if (ASSERT_OK(err, "attach")) { + /* + * attached: the read path reports model=3Dbpf until the + * struct_ops is detached; ctrl keeps describing the + * builtin coefficients + */ + err =3D readback_model(dev, model, sizeof(model)); + if (ASSERT_OK(err, "readback")) + ASSERT_EQ(strcmp(model, "bpf"), 0, "model_bpf"); + + /* a second model on the same device fails with -EBUSY */ + second =3D iocost_model__open(); + if (ASSERT_OK_PTR(second, "second_open")) { + sdev =3D bpf_map__initial_value( + second->maps.iocost_2x, NULL); + if (!ASSERT_OK_PTR(sdev, "second_initial_value")) + goto out_destroy; + *sdev =3D makedev(maj, min); + err =3D iocost_model__load(second); + if (ASSERT_OK(err, "second_load")) { + struct bpf_link *l2; + + /* + * the kernel rejects attaching a second + * model to the device with EBUSY + */ + l2 =3D bpf_map__attach_struct_ops( + second->maps.iocost_2x); + if (!ASSERT_ERR_PTR(l2, "second_ebusy")) + bpf_link__destroy(l2); + else + ASSERT_EQ(libbpf_get_error(l2), -EBUSY, + "second_ebusy_errno"); + } +out_destroy: + iocost_model__destroy(second); + } + + iocost_model__detach(skel); + + err =3D readback_model(dev, model, sizeof(model)); + if (ASSERT_OK(err, "readback_after_detach")) + ASSERT_EQ(strcmp(model, "linear"), 0, "model_linear"); + } + + iocost_model__destroy(skel); +} +/* + * Same check for the multi-stream example model. Only one model can + * be attached to a device at a time; both tests attach and detach, so + * they are serial and independent. + */ +void serial_test_iocost_model_streams(void) +{ + struct iocost_ms *skel; + unsigned int maj, min; + __u64 *ops_dev; + int err; + char model[32], *dev; + + dev =3D getenv("IOCOST_TEST_DEV"); + if (!dev || geteuid() !=3D 0 || sscanf(dev, "%u:%u", &maj, &min) !=3D 2) { + test__skip(); + return; + } + + skel =3D iocost_ms__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + ops_dev =3D bpf_map__initial_value(skel->maps.iocost_ms, NULL); + if (!ASSERT_OK_PTR(ops_dev, "initial_value")) { + iocost_ms__destroy(skel); + return; + } + *ops_dev =3D makedev(maj, min); + + err =3D iocost_ms__load(skel); + if (!ASSERT_OK(err, "skel_load")) { + iocost_ms__destroy(skel); + return; + } + + err =3D iocost_ms__attach(skel); + if (ASSERT_OK(err, "attach")) { + err =3D readback_model(dev, model, sizeof(model)); + if (ASSERT_OK(err, "readback")) + ASSERT_EQ(strcmp(model, "bpf"), 0, "model_bpf"); + + iocost_ms__detach(skel); + + err =3D readback_model(dev, model, sizeof(model)); + if (ASSERT_OK(err, "readback_after_detach")) + ASSERT_EQ(strcmp(model, "linear"), 0, "model_linear"); + } + + iocost_ms__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/iocost_model.c b/tools/testi= ng/selftests/bpf/progs/iocost_model.c new file mode 100644 index 000000000000..369818bda1cf --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iocost_model.c @@ -0,0 +1,139 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Example iocost cost model: the builtin linear HDD formula with all + * costs doubled, for one device given by the dev member of the + * struct_ops. + * + * The constants mirror what calc_lcoefs() derives from the AUTOP_HDD + * defaults (rbps=3D174019176 rseqiops=3D41708 rrandiops=3D370, w-side + * analog) in vtime units where 1s =3D=3D 2^37. On a rotational device + * still on ctrl=3Dauto, a device with this model attached charges + * twice the builtin model under the same workload, so the + * doubled cost is a direct check that accounting goes through the + * BPF path. On a non-rotational device, or one with user-pinned + * coefficients, the ratio to the builtin model is arbitrary. + * + * A zero cursor means "no previous IO". The cursor advances for + * every priced bio with a non-zero size (READ/WRITE), merged ones + * included, truncating to whole sectors like the builtin, so flushes + * and discards leave it alone and merged streams do not drift past + * the 16MB seek threshold. + * + * The model implements the full linear formula itself, including + * flushes: there is no fallback to the builtin model, a dataless + * A dataless WRITE|REQ_PREFLUSH keeps the write base: the op is still + * WRITE, so it carries WSEQIO (or WRANDIO after a seek) plus one page. + */ + +#include "vmlinux.h" +#include +#include + +/* + * VTIME_PER_SEC, IOC_PAGE_SIZE/SHIFT, IOC_SECT_TO_PAGE_SHIFT and + * IOCOST_COST_F_MERGE come from vmlinux.h (BTF enum constants) + */ +#define LCOEF_RANDIO_PAGES 4096 /* 16MB seek threshold */ +#define IOCOST_REQ_OP_MASK 0xff /* REQ_OP_MASK, not in BTF */ + +/* + * DIV64_U64_ROUND_UP / DIV_ROUND_UP_ULL equivalents, folded at + * compile time + */ +#define RU(x, y) ((x) / (y) + (((x) % (y)) ? 1 : 0)) + +#define RBPS 174019176ULL +#define RSEQIOPS 41708ULL +#define RRANDIOPS 370ULL +#define WBPS 178075866ULL +#define WSEQIOPS 42705ULL +#define WRANDIOPS 378ULL + +#define RPAGE (RU(VTIME_PER_SEC, RU(RBPS, IOC_PAGE_SIZE))) +#define RSEQIO (RU(VTIME_PER_SEC, RSEQIOPS) - RPAGE) +#define RRANDIO (RU(VTIME_PER_SEC, RRANDIOPS) - RPAGE) +#define WPAGE (RU(VTIME_PER_SEC, RU(WBPS, IOC_PAGE_SIZE))) +#define WSEQIO (RU(VTIME_PER_SEC, WSEQIOPS) - WPAGE) +#define WRANDIO (RU(VTIME_PER_SEC, WRANDIOPS) - WPAGE) + +/* + * per-cgroup cursor storage: keyed by the cgroup, freed with it, so + * per-cgroup state follows the cgroup lifetime + */ +struct { + __uint(type, BPF_MAP_TYPE_CGRP_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, __u64); +} cursor_store SEC(".maps"); + +SEC("struct_ops") +u64 BPF_PROG(iocost_2x_calc_cost, struct bio *bio, u64 model_flags) +{ + u64 opf =3D bio->bi_opf, nbytes =3D bio->bi_iter.bi_size; + u64 sector =3D bio->bi_iter.bi_sector; + struct blkcg *blkcg =3D bio->bi_blkg->blkcg; + u64 pages, seek_pages =3D 0, base, coef_page, randio, cost; + __u64 *cursor, cur; + int priced; + + /* builtin truncates: max(sectors >> IOC_SECT_TO_PAGE_SHIFT, 1) */ + pages =3D nbytes >> IOC_PAGE_SHIFT; + if (!pages) + pages =3D 1; + + if ((opf & IOCOST_REQ_OP_MASK) =3D=3D REQ_OP_READ) { + base =3D RSEQIO; coef_page =3D RPAGE; randio =3D RRANDIO; + } else if ((opf & IOCOST_REQ_OP_MASK) =3D=3D REQ_OP_WRITE) { + base =3D WSEQIO; coef_page =3D WPAGE; randio =3D WRANDIO; + } else { + /* + * a fully owning model must price every op; unknown + * ops are priced as a single page write + */ + base =3D 0; coef_page =3D WPAGE; randio =3D 0; + } + + /* + * mirror the builtin cursor semantics: seek distance is only + * computed against a non-zero cursor, and the cursor is + * advanced for bios the builtin prices (READ/WRITE with a + * non-zero size), merged ones included, so flushes and + * discards leave it alone + */ + priced =3D (opf & IOCOST_REQ_OP_MASK) =3D=3D REQ_OP_READ || + (opf & IOCOST_REQ_OP_MASK) =3D=3D REQ_OP_WRITE; + cursor =3D bpf_cgrp_storage_get(&cursor_store, + blkcg->css.cgroup, NULL, + BPF_LOCAL_STORAGE_GET_F_CREATE); + if (!cursor) { + if (model_flags & IOCOST_COST_F_MERGE) + base =3D 0; + return 2 * (base + pages * coef_page); + } + cur =3D *cursor; + if (cur && priced) { + seek_pages =3D sector > cur ? sector - cur + : cur - sector; + seek_pages >>=3D IOC_SECT_TO_PAGE_SHIFT; + if (seek_pages > LCOEF_RANDIO_PAGES) + base =3D randio; + } + if (priced && nbytes) + *cursor =3D sector + (nbytes >> 9); + + if (model_flags & IOCOST_COST_F_MERGE) + base =3D 0; + + cost =3D 2 * (base + pages * coef_page); + return cost; +} + +SEC(".struct_ops") +struct iocost_model_ops iocost_2x =3D { + .read_vtime_per_page =3D 2 * RPAGE, + .write_vtime_per_page =3D 2 * WPAGE, + .calc_cost =3D (void *)iocost_2x_calc_cost, +}; + +char LICENSE[] SEC("license") =3D "GPL"; diff --git a/tools/testing/selftests/bpf/progs/iocost_ms.c b/tools/testing/= selftests/bpf/progs/iocost_ms.c new file mode 100644 index 000000000000..aaa2e4489c1b --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iocost_ms.c @@ -0,0 +1,159 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Example multi-stream sequentiality detection cost model. + * + * The builtin model keeps a single cursor per cgroup, so two + * interleaved sequential readers in one cgroup are all priced random + * (measured 89x overcharge, 12.9x throughput collapse), while random + * IO inside a hot window smaller than the 16MB seek threshold is + * priced sequential (measured 107x undercharge). This model replaces + * the single cursor with a per-cgroup table of stream slots: an IO is + * sequential iff its sector matches the expected next sector of any + * tracked stream. Interleaved streams keep their own slots, and + * windowed random IO rarely matches a moving expectation. + * + * Stream state lives in a CGRP_STORAGE map, so it is created and + * freed with the cgroup. The model implements the full builtin + * linear formula itself, including flush pricing. + */ +#include "vmlinux.h" +#include +#include + +/* + * VTIME_PER_SEC, IOC_PAGE_SIZE/SHIFT, IOC_SECT_TO_PAGE_SHIFT and + * IOCOST_COST_F_MERGE come from vmlinux.h (BTF enum constants) + */ +#define IOCOST_REQ_OP_MASK 0xff /* REQ_OP_MASK, not in BTF */ + +/* + * DIV64_U64_ROUND_UP / DIV_ROUND_UP_ULL equivalents, folded at + * compile time + */ +#define RU(x, y) ((x) / (y) + (((x) % (y)) ? 1 : 0)) + +#define RBPS 174019176ULL +#define RSEQIOPS 41708ULL +#define RRANDIOPS 370ULL +#define WBPS 178075866ULL +#define WSEQIOPS 42705ULL +#define WRANDIOPS 378ULL + +#define RPAGE (RU(VTIME_PER_SEC, RU(RBPS, IOC_PAGE_SIZE))) +#define RSEQIO (RU(VTIME_PER_SEC, RSEQIOPS) - RPAGE) +#define RRANDIO (RU(VTIME_PER_SEC, RRANDIOPS) - RPAGE) +#define WPAGE (RU(VTIME_PER_SEC, RU(WBPS, IOC_PAGE_SIZE))) +#define WSEQIO (RU(VTIME_PER_SEC, WSEQIOPS) - WPAGE) +#define WRANDIO (RU(VTIME_PER_SEC, WRANDIOPS) - WPAGE) + +#define NSLOTS 4 + +struct streams { + __u64 expected[NSLOTS]; /* next expected sector, per stream */ + __u64 stamp[NSLOTS]; /* LRU stamp, 0 =3D empty */ +}; + +/* + * per-cgroup stream table: keyed by the cgroup, freed with it + */ +struct { + __uint(type, BPF_MAP_TYPE_CGRP_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, struct streams); +} stream_tab SEC(".maps"); + +SEC("struct_ops") +u64 BPF_PROG(iocost_ms_calc_cost, struct bio *bio, u64 model_flags) +{ + u64 opf =3D bio->bi_opf, nbytes =3D bio->bi_iter.bi_size; + u64 sector =3D bio->bi_iter.bi_sector; + struct blkcg *blkcg =3D bio->bi_blkg->blkcg; + struct streams *s; + u64 pages, base, coef_page, randio, advance, now; + u32 i, victim =3D 0, found =3D 0xFFFFFFFF; + + if ((opf & IOCOST_REQ_OP_MASK) =3D=3D REQ_OP_READ) { + base =3D RSEQIO; coef_page =3D RPAGE; randio =3D RRANDIO; + } else if ((opf & IOCOST_REQ_OP_MASK) =3D=3D REQ_OP_WRITE) { + base =3D WSEQIO; coef_page =3D WPAGE; randio =3D WRANDIO; + } else { + /* + * a fully owning model must price every op; unknown + * ops are priced as per-page writes + */ + base =3D 0; coef_page =3D WPAGE; randio =3D 0; + } + advance =3D RU(nbytes, 512); /* sectors */ + + /* only bios the builtin prices participate in stream tracking */ + if (!(((opf & IOCOST_REQ_OP_MASK) =3D=3D REQ_OP_READ || + (opf & IOCOST_REQ_OP_MASK) =3D=3D REQ_OP_WRITE) && nbytes)) { + pages =3D nbytes >> IOC_PAGE_SHIFT; + if (!pages) + pages =3D 1; + return base + pages * coef_page; + } + + s =3D bpf_cgrp_storage_get(&stream_tab, blkcg->css.cgroup, NULL, + BPF_LOCAL_STORAGE_GET_F_CREATE); + if (!s) { + /* no storage: price per page, truncating like the builtin */ + pages =3D nbytes >> IOC_PAGE_SHIFT; + if (!pages) + pages =3D 1; + return base + pages * coef_page; + } + + /* + * Slot access is lockless, mirroring the builtin single-cursor + * update in ioc_rqos_throttle(): concurrent CPUs submitting for + * the same cgroup can race on slot updates; mispricing is + * bounded and acceptable for an example model. + */ + now =3D bpf_ktime_get_ns(); + for (i =3D 0; i < NSLOTS; i++) { + if (s->expected[i] =3D=3D sector && s->stamp[i]) { + found =3D i; + break; + } + } + if (found !=3D 0xFFFFFFFF) { + /* sequential: keep the seq base from the op branch */ + s->expected[found] =3D sector + advance; + s->stamp[found] =3D now; + } else { + base =3D randio; + for (i =3D 1; i < NSLOTS; i++) { + if (s->stamp[i] < s->stamp[victim]) + victim =3D i; + } + s->expected[victim] =3D sector + advance; + s->stamp[victim] =3D now; + } + + /* builtin truncates: max(sectors >> IOC_SECT_TO_PAGE_SHIFT, 1) */ + pages =3D nbytes >> IOC_PAGE_SHIFT; + if (!pages) + pages =3D 1; + if (model_flags & IOCOST_COST_F_MERGE) { + /* + * merged bios skip the base cost but still advance + * the stream position above, so a merge at the + * expected sector does not make the following new IO + * look random + */ + base =3D 0; + } + + return base + pages * coef_page; +} + +SEC(".struct_ops") +struct iocost_model_ops iocost_ms =3D { + .read_vtime_per_page =3D RPAGE, + .write_vtime_per_page =3D WPAGE, + .calc_cost =3D (void *)iocost_ms_calc_cost, +}; + +char LICENSE[] SEC("license") =3D "GPL"; --=20 2.43.0 From nobody Thu Sep 24 13:37:12 2026 Received: from mta0.migadu.com (out-177.mta0.migadu.com [91.218.175.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5EA073D3D14 for ; Thu, 24 Sep 2026 05:46:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790228788; cv=none; b=FAeDMWL/ijRVqgVyPhBS53faah3ZZePF0HF2PoDDjLHEoRph6vwRLL4Vt/m43JawSNzXAOUFgUxZ3RiI5c0JBuN2rqoaeb16ktzrCy4u8j15dd76Wk8DRyOXcG8gf9wyQMo/M4Rzzb2MaPEhViHHNVwGxk9BBTHiU5hpO8/PIpQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790228788; c=relaxed/simple; bh=RAOZKp8Q/Q0tA/184GLvyRwTPE2lSRMx6XcDAUCpv+k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oBGreFYYLM1OEpaG6n+yHrF8afCGx/XOe3eri3d5MXZDJlePHWe+n8gW77fXVtJMGG2GkgI3a/2+5KwVXmbuLtgEbPc0FHm4BaTyFgIvogjAabwHJbj5RYRCIfUSxIbNygPRft7/GIlbdiclLnr05b63XNR/i10EAWdk3S9+snQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=tGHLGWjo; arc=none smtp.client-ip=91.218.175.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="tGHLGWjo" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=RAOZKp8Q/Q0tA/184GLvyRwTPE2lSRMx6XcDAUCpv+k=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1790228783; v=1; x=1790833583; b=tGHLGWjoYkjJzLZNzidGKEnsYU8h7oi+IMNqAugjYvnmk9P19YeD22RCLbQs9g0J33KSfzFW CAazR4+LwU5q280aHNrszs+8jMelZE9/K+BRm/HYCiv0UmgSB/RB/RSbacAsNfFtcc8E08ZX2EJ Hp06OAiKLyMX3YQfRKrKfaE4= X-Envelope-To: linux-kernel@vger.kernel.org Received: by smtp.migadu.com with ESMTPS id 6eb70ab8b56be155; Thu, 24 Sep 2026 05:46:23 +0000 X-Mizu-Trace-ID: 6eb70ab8b56be155 X-Migadu-Flow: FLOW_OUT From: Tao Cui To: tj@kernel.org, josef@toxicopanda.com, axboe@kernel.dk Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, andrii@kernel.org, eddyz87@gmail.com, ast@kernel.org, daniel@iogearbox.net, linux-kselftest@vger.kernel.org, cui.tao@linux.dev, cuitao@kylinos.cn, ameryhung@gmail.com, alexei.starovoitov@gmail.com Subject: [RFC PATCH v7 3/4] blk-iocost: add iocost_ioc_tick tracepoint for per-period device summary Date: Thu, 24 Sep 2026 13:45:48 +0800 Message-ID: <20260924054549.2271705-4-cui.tao@linux.dev> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924054549.2271705-1-cui.tao@linux.dev> References: <20260924054549.2271705-1-cui.tao@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Tao Cui Add iocost_ioc_tick, emitted once per period from the tail of ioc_timer_fn() with the overall controller state: period_us, vrate, busy_level, active iocg count, usage percentage and running state. It fires every period the controller is running, including steady states, plus one final tick before the controller goes idle, which makes dormancy (e.g. a device saturated entirely by uncharged IO) directly visible. Unlike the existing iocost tracepoints, which are state-change driven and silent in steady state, this allows a bound cost model's behaviour to be evaluated without drgn. Depending on the autop profile this is 2-100 events per second per device; the added cost outside the tracepoint static key is one increment per active cgroup per period. Signed-off-by: Tao Cui --- block/blk-iocost.c | 15 ++++++++++++ include/trace/events/iocost.h | 45 +++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) diff --git a/block/blk-iocost.c b/block/blk-iocost.c index 21e4f8cbd9f2..a48751b6dbdd 100644 --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -2252,6 +2252,7 @@ static void ioc_timer_fn(struct timer_list *timer) struct ioc_now now; LIST_HEAD(surpluses); int nr_debtors, nr_shortages =3D 0, nr_lagging =3D 0; + int nr_active =3D 0; u64 usage_us_sum =3D 0; u32 ppm_rthr; u32 ppm_wthr; @@ -2288,6 +2289,8 @@ static void ioc_timer_fn(struct timer_list *timer) u64 vdone, vtime, usage_us; u32 hw_active, hw_inuse; =20 + nr_active++; + /* * Collect unused and wind vtime closer to vnow to prevent * iocgs from accumulating a large amount of budget. @@ -2449,6 +2452,18 @@ static void ioc_timer_fn(struct timer_list *timer) =20 ioc->busy_level =3D clamp(ioc->busy_level, -1000, 1000); =20 + /* + * Everything the tick reports is final here: busy_level was just + * computed, running and cur_period haven't changed, nr_active and + * usage_us_sum are complete, and vrate and period_us still hold + * the values this period ran in. Emit before the refresh below + * so the event reads the completed period directly. + */ + trace_iocost_ioc_tick(ioc, nr_active, usage_us_sum, + ioc->period_us, ioc->vtime_base_rate, + ioc->busy_level, ioc->running, + now.now - ioc->period_at); + ioc_adjust_base_vrate(ioc, rq_wait_pct, nr_lagging, nr_shortages, prev_busy_level, missed_ppm); =20 diff --git a/include/trace/events/iocost.h b/include/trace/events/iocost.h index e772b1bc60d6..32f19861a78f 100644 --- a/include/trace/events/iocost.h +++ b/include/trace/events/iocost.h @@ -178,6 +178,51 @@ TRACE_EVENT(iocost_ioc_vrate_adj, ) ); =20 +/* + * Periodic per-device summary, emitted once per period from the tail of + * ioc_timer_fn(). Unlike the state-change events above, this fires every + * period the controller is running, including steady states, and carries + * the overall controller state so basic monitoring doesn't require drgn. + */ +TRACE_EVENT(iocost_ioc_tick, + + TP_PROTO(struct ioc *ioc, int nr_active, u64 usage_us_sum, + u32 tick_period_us, u64 tick_vrate, + int tick_busy, int tick_running, u64 tick_dur), + + TP_ARGS(ioc, nr_active, usage_us_sum, tick_period_us, tick_vrate, + tick_busy, tick_running, tick_dur), + + TP_STRUCT__entry ( + __string(devname, ioc_name(ioc)) + __field(u64, cur_period) + __field(u32, period_us) + __field(u64, vrate) + __field(int, busy_level) + __field(int, nr_active) + __field(u32, usage_pct) + __field(int, running) + ), + + TP_fast_assign( + __assign_str(devname); + __entry->cur_period =3D atomic64_read(&ioc->cur_period); + __entry->period_us =3D tick_period_us; + __entry->vrate =3D tick_vrate; + __entry->busy_level =3D tick_busy; + __entry->nr_active =3D nr_active; + __entry->usage_pct =3D tick_dur ? + div64_u64(usage_us_sum * 100, tick_dur) : 0; + __entry->running =3D tick_running; + ), + + TP_printk("[%s] period=3D%llu:%uus vrate=3D%llu busy=3D%d active=3D%d usa= ge=3D%u%% running=3D%d", + __get_str(devname), __entry->cur_period, __entry->period_us, + __entry->vrate, __entry->busy_level, __entry->nr_active, + __entry->usage_pct, __entry->running + ) +); + TRACE_EVENT(iocost_iocg_forgive_debt, =20 TP_PROTO(struct ioc_gq *iocg, const char *path, struct ioc_now *now, --=20 2.43.0 From nobody Thu Sep 24 13:37:12 2026 Received: from mta1.migadu.com (out-198.mta1.migadu.com [95.215.58.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F6693FA5D0 for ; Thu, 24 Sep 2026 05:46:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.198 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790228791; cv=none; b=AQlNz0S8CchnjPYLM/TfkoXVfbrT+nhcb1w5BSSuHGpFm967te0DCthWvYUq4H+4PeC0FjXwSXlMu8A9N4ZN5JEyEaZRWp2pgknsg4UvX/7V4yrHxpD6ThTqOhWMfQ5DlvorwiX3nA9+ynF2n0uzOono7tdYLapBP/FtDc4dwdE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790228791; c=relaxed/simple; bh=gq5YaNX1tq+qtMZoYlZYtZqfDgp0LgXDTh2kYD4sEXA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IVK3Nn4hbQXxY1lymsQsWQOhXS4IsZpb8hA2mV+gkNV0aUwP568tweVRQmUcf5I+1jxWHNRfMBbewgw0C7lsnbqym5vNFQp0Cqpe4V3JEE/Wgl91gw/Wa6oy0d9dVJYBJSv7g6Qantor0C5CmWuDT17rxEi+KE42GVAKADnkoxw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=WGsxiJMd; arc=none smtp.client-ip=95.215.58.198 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="WGsxiJMd" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=gq5YaNX1tq+qtMZoYlZYtZqfDgp0LgXDTh2kYD4sEXA=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1790228787; v=1; x=1790833587; b=WGsxiJMdZ6QjlZ0hegC3GJ1huPIpq9/xhC3m50cDR1vzGLBvHHgpzedSrQVd6xkhzFBtsEut jeqhKubPrIOqGf3WeZDvXz3DkO6lmiNm1aE5R3LnpsKhctKFJaN6T6RbgFmaRleaityXCdcEsIj w/Er8thaBzxfsus71VownKU8= X-Envelope-To: linux-kernel@vger.kernel.org Received: by smtp.migadu.com with ESMTPS id 1003cfcb96295a8f; Thu, 24 Sep 2026 05:46:27 +0000 X-Mizu-Trace-ID: 1003cfcb96295a8f X-Migadu-Flow: FLOW_OUT From: Tao Cui To: tj@kernel.org, josef@toxicopanda.com, axboe@kernel.dk Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, andrii@kernel.org, eddyz87@gmail.com, ast@kernel.org, daniel@iogearbox.net, linux-kselftest@vger.kernel.org, cui.tao@linux.dev, cuitao@kylinos.cn, ameryhung@gmail.com, alexei.starovoitov@gmail.com Subject: [RFC PATCH v7 4/4] docs: cgroup-v2: document the iocost BPF cost model attachment Date: Thu, 24 Sep 2026 13:45:49 +0800 Message-ID: <20260924054549.2271705-5-cui.tao@linux.dev> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924054549.2271705-1-cui.tao@linux.dev> References: <20260924054549.2271705-1-cui.tao@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Tao Cui Document the BPF cost model attachment in the io.cost.model section of the cgroup v2 documentation: attaching an iocost_model_ops struct_ops to a device by its major:minor, the model=3Dbpf readback while attached (ctrl keeps describing the coefficients), that detaching restores the builtin model, and that writes never select a model. Signed-off-by: Tao Cui --- Documentation/admin-guide/cgroup-v2.rst | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-= guide/cgroup-v2.rst index 8d2603751c51..306cc929c88d 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2117,8 +2117,30 @@ IO Interface Files =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D ctrl "auto" or "user" model The cost model in use - "linear" + or "bpf" while a BPF model is + attached =3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 + When CONFIG_BLK_CGROUP_IOCOST_BPF is enabled, a BPF cost model + can be attached to a device by loading an "iocost_model_ops" + struct_ops with the whole disk's major:minor in its "dev" member + (a partition's major:minor is rejected); + attaching switches the device's pricing to the model, detaching + the struct_ops restores the builtin linear model. While a model + is attached, "model" reads back "bpf"; "ctrl" keeps describing + the builtin coefficients, which are inert while the model is + attached: coefficient writes are stored and take effect again + after the struct_ops is detached, and the automatic profile + stepping does not switch profiles. Writing "ctrl=3Dbpf" or + "model=3Dbpf" is accepted as a no-op so a saved configuration + still parses, but re-attaching the model requires loading the + struct_ops again, not writing to this file. Attaching + implicitly enables the controller if needed (disabling wbt + like io.cost.qos does); detaching does not disable it again, + and writing "enable=3D0" to io.cost.qos suspends the model's + pricing until the controller is re-enabled, while it stays + attached. + When "ctrl" is "auto", the kernel may change all parameters dynamically. When "ctrl" is set to "user" or any other parameters are written to, "ctrl" become "user" and the --=20 2.43.0