From nobody Sat Jul 25 02:43:19 2026 Received: from out-185.mta0.migadu.com (out-185.mta0.migadu.com [91.218.175.185]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23F473BD64A for ; Mon, 20 Jul 2026 09:37:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.185 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784540270; cv=none; b=hXgTmz8rBBA7wKpWnd6ZCGvElpcpNKBMWrzV1hLcSb5nY4j61Jm7sZxL3Eqt7QfuZBVrP9pjnMO9oY0KR/rS6PqA3Tw6UOSC+TnLz1+Em4o7TC79B1ny7XSodWCEMmlrM8r9BnW2qgogHNhJ6xx9BypLF4GNVKFUpQqHCVUfMcY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784540270; c=relaxed/simple; bh=HHHR2fCkjaL7OZ4yudrjiMX2vqSjKQ3OEyHXi0EHKog=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hYwriQ27VVqOlK2Hpv8ni4df+2tc3kKFSkfG5sugAOlwtATQFjiSBWEfIuIHu0HNxlwpepCXca8PtTRVhCkzQB7QXOYczhqjPRNHC09h4Kkvs6yOYjSMNrcZeH5Qv57fbU19/dmZbZ8ixAycG/fGQ+GGC1MYz6hsZNrNwyhhKig= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=OnToIPNA; arc=none smtp.client-ip=91.218.175.185 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="OnToIPNA" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1784540266; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/PquVwhdyrMNc13NQQZsLRGO9rX2J1W8dm5bTzHHsNo=; b=OnToIPNAngCHRLWHE3PNKSwylR/6+6y/D46OdAv1WSyjg/oTl0xf+tHLT+6x7/M721DZ92 6aOD0M4pX0VA1pK8dWMBrfiXsiuiZ+5xtU7Sy1MP1MUrAsZLDnHzK3YMGFVF+/12Drbp+Y WlsaObqhBL2h1eb8PeqpB61Ion17HDA= From: Tao Cui To: Jens Axboe Cc: Tejun Heo , Josef Bacik , Omar Sandoval , Bart Van Assche , Yu Kuai , Chaitanya Kulkarni , Hannes Reinecke , Ming Lei , Damien Le Moal , Nilay Shroff , linux-block@vger.kernel.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, cui.tao@linux.dev, cuitao@kylinos.cn Subject: [PATCH 1/4] block/blk-stat: drain per-cpu callback stats over possible CPUs Date: Mon, 20 Jul 2026 17:37:23 +0800 Message-ID: <20260720093726.28965-2-cui.tao@linux.dev> In-Reply-To: <20260720093726.28965-1-cui.tao@linux.dev> References: <20260720093726.28965-1-cui.tao@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" From: Tao Cui blk_stat_timer_fn() sums and resets a callback's per-cpu buckets using for_each_online_cpu(). A CPU that goes offline with pending samples is skipped, so its samples are neither accumulated into the window nor cleared; they sit in the bucket until the CPU comes back online, at which point the stale values are flushed into whatever window is then running. This silently corrupts the latency picture that consumers (notably writeback throttling via wbt, and blk-mq latency tracking) base decisions on around CPU hotplug: under-counting while the CPU is offline, then a burst of stale data on re-online. Fixes: 34dbad5d26e2 ("blk-stat: convert to callback-based statistics report= ing") Signed-off-by: Tao Cui --- block/blk-stat.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blk-stat.c b/block/blk-stat.c index de126e1ea5ac..d57c2fc6bf06 100644 --- a/block/blk-stat.c +++ b/block/blk-stat.c @@ -83,7 +83,7 @@ static void blk_stat_timer_fn(struct timer_list *t) for (bucket =3D 0; bucket < cb->buckets; bucket++) blk_rq_stat_init(&cb->stat[bucket]); =20 - for_each_online_cpu(cpu) { + for_each_possible_cpu(cpu) { struct blk_rq_stat *cpu_stat; =20 cpu_stat =3D per_cpu_ptr(cb->cpu_stat, cpu); --=20 2.43.0 From nobody Sat Jul 25 02:43:19 2026 Received: from out-188.mta0.migadu.com (out-188.mta0.migadu.com [91.218.175.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1FF453BD64A for ; Mon, 20 Jul 2026 09:37:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.188 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784540277; cv=none; b=njGoKWHG506A6KRJdugPQ8CopGY8Ej/qw/Ujyk10zYhNKRaSAf2xw+87Ch7BVSlMV23ehUUMnrnP9C3K5Hj3KmJX9rzHNb/zgSH4hmkew17psS+o0pgLF8SF5c51gfSy1EW2DAu2vXSGkMrTNbGjV95DDm1tQwOVsJStGML5TLE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784540277; c=relaxed/simple; bh=SLoaC+4K/Aj0nLpdx60Po9LvLU4QZwA8uDgtvsM4m94=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UaPRnlQqsqpSykthyiW2g0PdkixBYOk1d5efPKO4Fw0ExiV53Eg26zN7QqISmnl6sMtogjeEXA5P0R1lLn8P8VZqSyuo8G9hFUCmpH2xUdWuOME7b/mL/C6ZZ9blcRdpKtkiBLcu8HwRmY3sFPyhPDiG4x2kk11YhEibGhDaIRw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=ZmH+rWf5; arc=none smtp.client-ip=91.218.175.188 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="ZmH+rWf5" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1784540273; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t10s5HeuWcsqSvUkdc4XvpT/zR0qAKN1Grpg4N+I0L4=; b=ZmH+rWf5ydqM9qKt5JqfN0DLMMnvdABO0vEbGn1FVRFLkGNTE7ZFGSsnGPGAXofijvZIRx jzXtkVxRM22BA2yp0s89ffOvmAyyuVTJXco6pC4nT9C4PXmDyzYqMxFRIvv7e1PQ7tQwh9 t133sYgURD3M+9bxFxlf5Tu+0HhSUkc= From: Tao Cui To: Jens Axboe Cc: Tejun Heo , Josef Bacik , Omar Sandoval , Bart Van Assche , Yu Kuai , Chaitanya Kulkarni , Hannes Reinecke , Ming Lei , Damien Le Moal , Nilay Shroff , linux-block@vger.kernel.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, cui.tao@linux.dev, cuitao@kylinos.cn Subject: [PATCH 2/4] block/blk-iolatency: account per-cpu latency stats over possible CPUs Date: Mon, 20 Jul 2026 17:37:24 +0800 Message-ID: <20260720093726.28965-3-cui.tao@linux.dev> In-Reply-To: <20260720093726.28965-1-cui.tao@linux.dev> References: <20260720093726.28965-1-cui.tao@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" From: Tao Cui iolatency_check_latencies() and iolatency_ssd_stat() iterate a blkg's per-cpu latency stats with for_each_online_cpu(). When a CPU that has accumulated io.latency samples goes offline, its bucket is skipped: the check loop (which also resets) neither sums nor clears it, and the show path under-reports. On re-online the stranded samples are flushed into a later check window, which can trigger a spurious throttle/scale adjustment. Fixes: d70675121546 ("block: introduce blk-iolatency io controller") Fixes: 1fa2840e56f9 ("blk-iolatency: use a percentile approache for ssd's") Signed-off-by: Tao Cui --- block/blk-iolatency.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/blk-iolatency.c b/block/blk-iolatency.c index 1cc33aa0b669..d6c401d6174c 100644 --- a/block/blk-iolatency.c +++ b/block/blk-iolatency.c @@ -523,7 +523,7 @@ static void iolatency_check_latencies(struct iolatency_= grp *iolat, u64 now) =20 latency_stat_init(iolat, &stat); preempt_disable(); - for_each_online_cpu(cpu) { + for_each_possible_cpu(cpu) { struct latency_stat *s; s =3D per_cpu_ptr(iolat->stats, cpu); latency_stat_sum(iolat, &stat, s); @@ -924,7 +924,7 @@ static void iolatency_ssd_stat(struct iolatency_grp *io= lat, struct seq_file *s) =20 latency_stat_init(iolat, &stat); preempt_disable(); - for_each_online_cpu(cpu) { + for_each_possible_cpu(cpu) { struct latency_stat *s; s =3D per_cpu_ptr(iolat->stats, cpu); latency_stat_sum(iolat, &stat, s); --=20 2.43.0 From nobody Sat Jul 25 02:43:19 2026 Received: from out-184.mta0.migadu.com (out-184.mta0.migadu.com [91.218.175.184]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 26BD43D1CAC for ; Mon, 20 Jul 2026 09:38:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.184 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784540282; cv=none; b=r/D8h33ULLREK2ODskD++xwx3ckjDzhPjptm1gd8xdmsFLcymMM+597nozQRB1hWJBKR3cVS7y+EJuZUGRn4Uiu0BC9kKkxDXnMl/2iGUfwzEF+NZgbyR4Z2zTx91bBHKo8QfwF1CNAS2IAnyYoCOkNI7KltEzcVloDLqXv7xcU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784540282; c=relaxed/simple; bh=lRK9/1L4NjYyQwus96ziJ6sziwXOvt9QhqWN+ul9A8Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Uz1VCXBKHvrckypTHpSB0HoTYKOwcrpaSL9gclD/H9SWptAMYH6wsL00xW9CdR3c6/jHf5i1E0Vb7axUBQTqR2mTS/BCjytWAECRMK81xZI2SUGm4IVdS3IJAU1mJ90W5BKG2WMh0hZvTq1ycY8Ziq0aH3Z7eVlk7Yg12Co+czs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=wIMEz7HT; arc=none smtp.client-ip=91.218.175.184 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="wIMEz7HT" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1784540279; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XYcX65iGs0S1tfl7tVIHoz6s0aK6dAZv4VVFdKfUgcM=; b=wIMEz7HTeFD0sssquAStjhvKa3/ZwUSxtYL3nTUnU/SyHeaxzcHiD/0rAr69jgtn+sm4xR xazaZtHKmaRr8+Ox81ciVztNpxgfAMA2eNGhwNIMi+oA2RerzLt/VOrCCdpTb3HAh1mpOD BoyRkYwgMaDPmSZTMych53Bgdcq1mzU= From: Tao Cui To: Jens Axboe Cc: Tejun Heo , Josef Bacik , Omar Sandoval , Bart Van Assche , Yu Kuai , Chaitanya Kulkarni , Hannes Reinecke , Ming Lei , Damien Le Moal , Nilay Shroff , linux-block@vger.kernel.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, cui.tao@linux.dev, cuitao@kylinos.cn Subject: [PATCH 3/4] block/blk-iocost: collect per-cpu latency stats over possible CPUs Date: Mon, 20 Jul 2026 17:37:25 +0800 Message-ID: <20260720093726.28965-4-cui.tao@linux.dev> In-Reply-To: <20260720093726.28965-1-cui.tao@linux.dev> References: <20260720093726.28965-1-cui.tao@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" From: Tao Cui ioc_lat_stat() walks ioc->pcpu_stat with for_each_online_cpu() to compute missed-ppm and rq_wait deltas. An offlined CPU is skipped, so its delta is dropped from the period and its last_* watermark is not advanced; on re-online the next collection sees a delta spanning the whole offline interval, corrupting the latency/vrate picture. Fixes: 7caa47151ab2 ("blkcg: implement blk-iocost") Signed-off-by: Tao Cui --- block/blk-iocost.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/blk-iocost.c b/block/blk-iocost.c index 8b2aeba2e1e3..b60625613e09 100644 --- a/block/blk-iocost.c +++ b/block/blk-iocost.c @@ -1592,7 +1592,7 @@ static void ioc_lat_stat(struct ioc *ioc, u32 *missed= _ppm_ar, u32 *rq_wait_pct_p u64 rq_wait_ns =3D 0; int cpu, rw; =20 - for_each_online_cpu(cpu) { + for_each_possible_cpu(cpu) { struct ioc_pcpu_stat *stat =3D per_cpu_ptr(ioc->pcpu_stat, cpu); u64 this_rq_wait_ns; =20 --=20 2.43.0 From nobody Sat Jul 25 02:43:19 2026 Received: from out-188.mta0.migadu.com (out-188.mta0.migadu.com [91.218.175.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2FFBE3D668C for ; Mon, 20 Jul 2026 09:38:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.188 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784540287; cv=none; b=VIKUJdfPBh2oXMgo9MNeJNjJ57daqMUUAgDo5fkFWR2uyCL3I6+sDNIUVh01XigvTYP575MaQ9sfU3ICGJOLjI1yJtz/TttntqORut8LbwNGER2+wtSeQzTJP0Eac23hms6pwvGV6vpzbxeg3IeIxl9XmM7hmXdxalpNEB7M98Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784540287; c=relaxed/simple; bh=CwcS1rpDH9/zVMo1p/kk3iMOwfOYQpuysnAULIMw0Pc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=at8EismuPaKw8l5iHWpTPbX1Rt1mpWCtOKpZ97Gp60Vt3pUm3IQp5YyemqfI3m1e6xWUVaIemwo3/7JaOfI8XwXNdvooVa2H8HzNq86JCgxzhsMdMInzujgP0v3SlwYRxnWoskDKMOL3oWkbGjRsItu+p8VOH935XjVnqsSEBss= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=oFe+Y33Q; arc=none smtp.client-ip=91.218.175.188 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="oFe+Y33Q" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1784540284; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iu062VwjFJmKb1NYwQO4WuL9MAqm4vzNE/XIfYy4xek=; b=oFe+Y33QSRXyYWUXCA7Oc6uCw1WlmHDohQcWnNoGA/iV2IJN243vtB/Lo49nx5vTumTfBr Kov2ZXHu5FsqFC6x6+865QYbCOfWTDW8x/bHBE3SfSq3jZAlgHJHvx4GmKZPSgoC8jDa6i 8O6GTMeyN6qOZJga5dBgG/R9B6gLbn4= From: Tao Cui To: Jens Axboe Cc: Tejun Heo , Josef Bacik , Omar Sandoval , Bart Van Assche , Yu Kuai , Chaitanya Kulkarni , Hannes Reinecke , Ming Lei , Damien Le Moal , Nilay Shroff , linux-block@vger.kernel.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, cui.tao@linux.dev, cuitao@kylinos.cn Subject: [PATCH 4/4] block/kyber-iosched: flush per-cpu latency buckets over possible CPUs Date: Mon, 20 Jul 2026 17:37:26 +0800 Message-ID: <20260720093726.28965-5-cui.tao@linux.dev> In-Reply-To: <20260720093726.28965-1-cui.tao@linux.dev> References: <20260720093726.28965-1-cui.tao@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" From: Tao Cui kyber_timer_fn() sums the per-cpu latency histograms with for_each_online_cpu(). A CPU that goes offline mid-interval leaves its bucket un-flushed; the samples are lost from the current decision and re-appear (stale) when the CPU is onlined again. Fixes: 6e25cb01ea20 ("kyber: implement improved heuristics") Signed-off-by: Tao Cui --- block/kyber-iosched.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/kyber-iosched.c b/block/kyber-iosched.c index 971818bcdc9d..2ee552ab8135 100644 --- a/block/kyber-iosched.c +++ b/block/kyber-iosched.c @@ -275,7 +275,7 @@ static void kyber_timer_fn(struct timer_list *t) bool bad =3D false; =20 /* Sum all of the per-cpu latency histograms. */ - for_each_online_cpu(cpu) { + for_each_possible_cpu(cpu) { struct kyber_cpu_latency *cpu_latency; =20 cpu_latency =3D per_cpu_ptr(kqd->cpu_latency, cpu); --=20 2.43.0