From nobody Sun Feb 8 14:44:58 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75F2912C486 for ; Mon, 1 Jul 2024 10:19:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829143; cv=none; b=lZiQPNXI01pFsT9+x5p+UvnrLevX6wA7x9gCWU8PcTiDXHoC6xRxhWiSM2ZYGsIigxAjJz8qio13bVKTNGDqsbCo80odF/rWbPUjPJDwSr8cp7Taa28GD6PGOzWD9U8jns/2kT6C3nNktVEjqDBK1CRocnPRvH1TmneqRZOFTc0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829143; c=relaxed/simple; bh=z+6QJK2NRXZ1G8UVShVCdiJwma9rji51m5mPHCZ6/Fc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=r8NK/EmsCs3PipqvM6va7liN6IYOuwKtQglPDPZJ/rhbActRQn2f++CqCINgPt/Of6lEguE55AoJ+1vY6lN2iOcQy5VWotWBUeXPblliEzYEoia48OvSNOuSLl2sSPyB5iduje+00hKcEO/S32kcPQWS5DwaiSTud1YykZJ+B7A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=YM/PJcxH; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=VTo8gdsr; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="YM/PJcxH"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="VTo8gdsr" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1719829136; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zQjbgYi559EvGTCmhM3yem27rQqPiOzkClsVHFeLpfs=; b=YM/PJcxHwSehEOlQfkaTF4BnY07Zq32iP3ZcBIWPLKQiVcYEFsxkBRx2EijNDgdCsx+atb hXiJee9A0JmMl2Z8SwJXdxCf0HatONsdmnd0ijXZ/7+WaAw6DF/WHKEUxwo4goXmpoFlLd vIpwxiMxhNUdA4yyESWcoJUUNvb/Qlqh4X4SP9f1BpISOZhMaT+P18FJwTj4i2nwG52Ia8 JRjIIqLCd6uwWc/Xw9D2M54vv1XE+4kQ74c4tA8tpapQPbl2zymRMaJSZXjlQvL/Oa2lTJ xZZbkkvXj5LXd7eav4Ffq9l8W3/fI435XLULsKv2sg1/nzSQ2mgve2AwrH3Yew== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1719829136; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zQjbgYi559EvGTCmhM3yem27rQqPiOzkClsVHFeLpfs=; b=VTo8gdsrUvQUNvQNnvwyvXcLW7Vk8z1W8VUieFN6/kEP7BytOiRdSwdDiAw2NE7wkIupcN rY565oIGhtBEs9Bw== Date: Mon, 01 Jul 2024 12:18:37 +0200 Subject: [PATCH v3 1/8] timers/migration: Do not rely always on group->parent Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240701-tmigr-fixes-v3-1-25cd5de318fb@linutronix.de> References: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> In-Reply-To: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> To: Frederic Weisbecker , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Borislav Petkov , Anna-Maria Behnsen When reading group->parent without holding the group lock it is racy against CPUs coming online the first time and thereby creating another level of the hierarchy. This is not a problem when this value is read once to decide whether to abort a propagation or not. The worst outcome is an unnecessary/early CPU wake up. But it is racy when reading it several times during a single 'action' (like activation, deactivation, checking for remote timer expiry,...) and relying on the consitency of this value without holding the lock. This happens at the moment e.g. in tmigr_inactive_up() which is also calling tmigr_udpate_events(). Code relys on group->parent not to change during this 'action'. Update parent struct member description to explain the above only once. Remove parent pointer checks when they are not mandatory (like update of data->childmask). Remove a warning, which would be nice but the trigger of this warning is not reliable and add expand the data structure member description instead. Expand a comment, why it is safe to rely on parent pointer here (inside hierarchy update). Fixes: 7ee988770326 ("timers: Implement the hierarchical pull model") Reported-by: Borislav Petkov Signed-off-by: Anna-Maria Behnsen Reviewed-by: Frederic Weisbecker --- kernel/time/timer_migration.c | 33 +++++++++++++++------------------ kernel/time/timer_migration.h | 12 +++++++++++- 2 files changed, 26 insertions(+), 19 deletions(-) diff --git a/kernel/time/timer_migration.c b/kernel/time/timer_migration.c index 84413114db5c..d91efe1dc3bf 100644 --- a/kernel/time/timer_migration.c +++ b/kernel/time/timer_migration.c @@ -507,7 +507,14 @@ static void walk_groups(up_f up, void *data, struct tm= igr_cpu *tmc) * (get_next_timer_interrupt()) * @firstexp: Contains the first event expiry information when last * active CPU of hierarchy is on the way to idle to make - * sure CPU will be back in time. + * sure CPU will be back in time. It is updated in top + * level group only. Be aware, there could occur a new top + * level of the hierarchy between the 'top level call' in + * tmigr_update_events() and the check for the parent group + * in walk_groups(). Then @firstexp might contain a value + * !=3D KTIME_MAX even if it was not the final top + * level. This is not a problem, as the worst outcome is a + * CPU which might wake up a little early. * @evt: Pointer to tmigr_event which needs to be queued (of idle * child group) * @childmask: childmask of child group @@ -649,7 +656,7 @@ static bool tmigr_active_up(struct tmigr_group *group, =20 } while (!atomic_try_cmpxchg(&group->migr_state, &curstate.state, newstat= e.state)); =20 - if ((walk_done =3D=3D false) && group->parent) + if (walk_done =3D=3D false) data->childmask =3D group->childmask; =20 /* @@ -1317,20 +1324,9 @@ static bool tmigr_inactive_up(struct tmigr_group *gr= oup, /* Event Handling */ tmigr_update_events(group, child, data); =20 - if (group->parent && (walk_done =3D=3D false)) + if (walk_done =3D=3D false) data->childmask =3D group->childmask; =20 - /* - * data->firstexp was set by tmigr_update_events() and contains the - * expiry of the first global event which needs to be handled. It - * differs from KTIME_MAX if: - * - group is the top level group and - * - group is idle (which means CPU was the last active CPU in the - * hierarchy) and - * - there is a pending event in the hierarchy - */ - WARN_ON_ONCE(data->firstexp !=3D KTIME_MAX && group->parent); - trace_tmigr_group_set_cpu_inactive(group, newstate, childmask); =20 return walk_done; @@ -1552,10 +1548,11 @@ static void tmigr_connect_child_parent(struct tmigr= _group *child, data.childmask =3D child->childmask; =20 /* - * There is only one new level per time. When connecting the - * child and the parent and set the child active when the parent - * is inactive, the parent needs to be the uppermost - * level. Otherwise there went something wrong! + * There is only one new level per time (which is protected by + * tmigr_mutex). When connecting the child and the parent and + * set the child active when the parent is inactive, the parent + * needs to be the uppermost level. Otherwise there went + * something wrong! */ WARN_ON(!tmigr_active_up(parent, child, &data) && parent->parent); } diff --git a/kernel/time/timer_migration.h b/kernel/time/timer_migration.h index 6c37d94a37d9..494f68cc13f4 100644 --- a/kernel/time/timer_migration.h +++ b/kernel/time/timer_migration.h @@ -22,7 +22,17 @@ struct tmigr_event { * struct tmigr_group - timer migration hierarchy group * @lock: Lock protecting the event information and group hierarchy * information during setup - * @parent: Pointer to the parent group + * @parent: Pointer to the parent group. Pointer is updated when a + * new hierarchy level is added because of a CPU coming + * online the first time. Once it is set, the pointer will + * not be removed or updated. When accessing parent pointer + * lock less to decide whether to abort a propagation or + * not, it is not a problem. The worst outcome is an + * unnecessary/early CPU wake up. But do not access parent + * pointer several times in the same 'action' (like + * activation, deactivation, check for remote expiry,...) + * without holding the lock as it is not ensured that value + * will not change. * @groupevt: Next event of the group which is only used when the * group is !active. The group event is then queued into * the parent timer queue. --=20 2.39.2 From nobody Sun Feb 8 14:44:58 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75EED12B169 for ; Mon, 1 Jul 2024 10:19:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829143; cv=none; b=E5bs//51G1jCO14QGFTmB9/3g0aV9l13v9WR5CYmsa5Za6xBmEeE/4g0ua0I9JDbcBeLOf88uEOGXvvxubtuYwmOa06lui4zVjlWbaYwrAcOpNjvngy1DP2QLZoVGVbRAr4342VLS2HlXzuYipOsLw9n6/0P/ozEqeem6OVlWkk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829143; c=relaxed/simple; bh=73/+32Pt5ZgVf66k/oBMpzVXw9NiBnAKwBXnk+mI6/4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=QyZoe5gzFPk75L4+1E4VMsWOhMAFaPxYwFdmnc0JhjgZ0YM3e38AaNKqXdwBfsgQge0D9sl2MA+s6uhmo/DC6/uqGGMfCC0QIyFQtzMeB0HyWz2IB2jVxhXaRunw4CYVXlWPfRLiKB8JoITPSSAsh62FRxODLM4P9erkrFBPSoc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Rptgbgne; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=3dBffno7; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Rptgbgne"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="3dBffno7" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1719829136; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BrHQGMIRp/aS/Wc6JReRN1VVZ8fudzi2uwwPAILPq7k=; b=RptgbgneYsnvdhlovEbJGWQBOW+z7hNUUojmahvotLdN/IpFf9x2v586DpvBtwuXulrH0m sUwpVJqYdvC2CrpFEovcGYAT3S73kPvENBxMwede81BhYquxDRuMB3srxIUo8QwZD45Aoe /dqoTRsxOiofL2dIk/2JWSte6CP8T5F/WdUoqCkC40kJnZlaIS+4TlcvUx6l47buqNpKrX vwmVXP51vVJIeFtHi5aEn0tBZWBJesU2k+dAzUMTeyIxdSErzC7dfJee/wlgFOPHX/uWpP u1PgOqG8ALol8RopdmY1yCkAMllCIl0AkNWik3RQZS9Q0m0DCNcNhaDuPs7n0A== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1719829136; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BrHQGMIRp/aS/Wc6JReRN1VVZ8fudzi2uwwPAILPq7k=; b=3dBffno7v1KiHBceS+W4Ro91TWO3FFEGMHDzulQnBeZ/PrWt0vLitub87T3mKtzKhXyZK7 ztooF2Y6owg1k+Dw== Date: Mon, 01 Jul 2024 12:18:38 +0200 Subject: [PATCH v3 2/8] timers/migration: Move hierarchy setup into cpuhotplug prepare callback Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240701-tmigr-fixes-v3-2-25cd5de318fb@linutronix.de> References: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> In-Reply-To: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> To: Frederic Weisbecker , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Anna-Maria Behnsen When a CPU comes online the first time, it is possible that a new top level group will be created. In general all propagation is done from the bottom to top. This minimizes complexity and prevents possible races. But when a new top level group is created, the formely top level group needs to be connected to the new level. This is the only time, when the direction to propagate changes is changed: the changes are propagated from top (new top level group) to bottom (formerly top level group). This introduces two races (see (A) and (B)) as reported by Frederic: (A) This race happens, when marking the formely top level group as active, but the last active CPU of the formerly top level group goes idle. Then it's likely that formerly group is no longer active, but marked nevertheless as active in new top level group: [GRP0:0] migrator =3D 0 active =3D 0 nextevt =3D KTIME_MAX / \ 0 1 .. 7 active idle 0) Hierarchy has for now only 8 CPUs and CPU 0 is the only active CPU. [GRP1:0] migrator =3D TMIGR_NONE active =3D NONE nextevt =3D KTIME_MAX \ [GRP0:0] [GRP0:1] migrator =3D 0 migrator =3D TMIGR_NONE active =3D 0 active =3D NONE nextevt =3D KTIME_MAX nextevt =3D KTIME_MAX / \ 0 1 .. 7 8 active idle !online 1) CPU 8 is booting and creates a new group in first level GRP0:1 and therefore also a new top group GRP1:0. For now the setup code proceeded only until the connected between GRP0:1 to the new top group. The connection between CPU8 and GRP0:1 is not yet established and CPU 8 is still !online. [GRP1:0] migrator =3D TMIGR_NONE active =3D NONE nextevt =3D KTIME_MAX / \ [GRP0:0] [GRP0:1] migrator =3D 0 migrator =3D TMIGR_NONE active =3D 0 active =3D NONE nextevt =3D KTIME_MAX nextevt =3D KTIME_MAX / \ 0 1 .. 7 8 active idle !online 2) Setup code now connects GRP0:0 to GRP1:0 and observes while in tmigr_connect_child_parent() that GRP0:0 is not TMIGR_NONE. So it prepares to call tmigr_active_up() on it. It hasn't done it yet. [GRP1:0] migrator =3D TMIGR_NONE active =3D NONE nextevt =3D KTIME_MAX / \ [GRP0:0] [GRP0:1] migrator =3D TMIGR_NONE migrator =3D TMIGR_NONE active =3D NONE active =3D NONE nextevt =3D KTIME_MAX nextevt =3D KTIME_MAX / \ 0 1 .. 7 8 idle idle !online 3) CPU 0 goes idle. Since GRP0:0->parent has been updated by CPU 8 with GRP0:0->lock held, CPU 0 observes GRP1:0 after calling tmigr_update_events() and it propagates the change to the top (no change there and no wakeup programmed since there is no timer). [GRP1:0] migrator =3D GRP0:0 active =3D GRP0:0 nextevt =3D KTIME_MAX / \ [GRP0:0] [GRP0:1] migrator =3D TMIGR_NONE migrator =3D TMIGR_NONE active =3D NONE active =3D NONE nextevt =3D KTIME_MAX nextevt =3D KTIME_MAX / \ 0 1 .. 7 8 idle idle !online 4) Now the setup code finally calls tmigr_active_up() to and sets GRP0:0 active in GRP1:0 [GRP1:0] migrator =3D GRP0:0 active =3D GRP0:0, GRP0:1 nextevt =3D KTIME_MAX / \ [GRP0:0] [GRP0:1] migrator =3D TMIGR_NONE migrator =3D 8 active =3D NONE active =3D 8 nextevt =3D KTIME_MAX nextevt =3D KTIME_MAX / \ | 0 1 .. 7 8 idle idle active 5) Now CPU 8 is connected with GRP0:1 and CPU 8 calls tmigr_active_up() out of tmigr_cpu_online(). [GRP1:0] migrator =3D GRP0:0 active =3D GRP0:0 nextevt =3D T8 / \ [GRP0:0] [GRP0:1] migrator =3D TMIGR_NONE migrator =3D TMIGR_NONE active =3D NONE active =3D NONE nextevt =3D KTIME_MAX nextevt =3D T8 / \ | 0 1 .. 7 8 idle idle idle 5) CPU 8 goes idle with a timer T8 and relies on GRP0:0 as the migrator. But it's not really active, so T8 gets ignored. --> The update which is done in third step is not noticed by setup code. So a wrong migrator is set to top level group and a timer could get ignored. (B) Reading group->parent and group->childmask when an hierarchy update is ongoing and reaches the formerly top level group is racy as those values could be inconsistent. (The notation of migrator and active now slightly changes in contrast to the above example, as now the childmasks are used.) [GRP1:0] migrator =3D TMIGR_NONE active =3D 0x00 nextevt =3D KTIME_MAX \ [GRP0:0] [GRP0:1] migrator =3D TMIGR_NONE migrator =3D TMIGR_NONE active =3D 0x00 active =3D 0x00 nextevt =3D KTIME_MAX nextevt =3D KTIME_MAX childmask=3D 0 childmask=3D 1 parent =3D NULL parent =3D GRP1:0 / \ 0 1 .. 7 8 idle idle !online childmask=3D1 1) Hierarchy has 8 CPUs. CPU 8 is at the moment in the process of onlining but did not yet connect GRP0:0 to GRP1:0. [GRP1:0] migrator =3D TMIGR_NONE active =3D 0x00 nextevt =3D KTIME_MAX / \ [GRP0:0] [GRP0:1] migrator =3D TMIGR_NONE migrator =3D TMIGR_NONE active =3D 0x00 active =3D 0x00 nextevt =3D KTIME_MAX nextevt =3D KTIME_MAX childmask=3D 0 childmask=3D 1 parent =3D GRP1:0 parent =3D GRP1:0 / \ 0 1 .. 7 8 idle idle !online childmask=3D1 2) Setup code (running on CPU 8) now connects GRP0:0 to GRP1:0, updates parent pointer of GRP0:0 and ... [GRP1:0] migrator =3D TMIGR_NONE active =3D 0x00 nextevt =3D KTIME_MAX / \ [GRP0:0] [GRP0:1] migrator =3D 0x01 migrator =3D TMIGR_NONE active =3D 0x01 active =3D 0x00 nextevt =3D KTIME_MAX nextevt =3D KTIME_MAX childmask=3D 0 childmask=3D 1 parent =3D GRP1:0 parent =3D GRP1:0 / \ 0 1 .. 7 8 active idle !online childmask=3D1 tmigr_walk.childmask =3D 0 3) ... CPU 0 comes active in the same time. As migrator in GRP0:0 was TMIGR_NONE, childmask of GRP0:0 is stored in update propagation data structure tmigr_walk (as update of childmask is not yet visible/updated). And now ... [GRP1:0] migrator =3D TMIGR_NONE active =3D 0x00 nextevt =3D KTIME_MAX / \ [GRP0:0] [GRP0:1] migrator =3D 0x01 migrator =3D TMIGR_NONE active =3D 0x01 active =3D 0x00 nextevt =3D KTIME_MAX nextevt =3D KTIME_MAX childmask=3D 2 childmask=3D 1 parent =3D GRP1:0 parent =3D GRP1:0 / \ 0 1 .. 7 8 active idle !online childmask=3D1 tmigr_walk.childmask =3D 0 4) ... childmask of GRP0:0 is updated by CPU 8 (still part of setup code). [GRP1:0] migrator =3D 0x00 active =3D 0x00 nextevt =3D KTIME_MAX / \ [GRP0:0] [GRP0:1] migrator =3D 0x01 migrator =3D TMIGR_NONE active =3D 0x01 active =3D 0x00 nextevt =3D KTIME_MAX nextevt =3D KTIME_MAX childmask=3D 2 childmask=3D 1 parent =3D GRP1:0 parent =3D GRP1:0 / \ 0 1 .. 7 8 active idle !online childmask=3D1 tmigr_walk.childmask =3D 0 5) CPU 0 sees the connection to GRP1:0 and now propagates active state to GRP1:0 but with childmask =3D 0 as stored in propagation data structure. --> Now GRP1:0 always has a migrator as 0x00 !=3D TMIGR_NONE and for all CP= Us it looks like GRP1:0 is always active. To prevent those races, the setup of the hierarchy is moved into the cpuhotplug prepare callback. The prepare callback is not executed by the CPU which will come online, it is executed by the CPU which prepares onlining of the other CPU. This CPU will not go idle and so it is ensured, that the formerly top level group cannot go idle and change top level group state and the propagation could be done without a risk. The direction for the updates is now forced to look like "from bottom to top". Split setup functionality of online callback into the cpuhotplug prepare callback and setup hotplug state. Reorder the code, that all prepare related functions are close to each other and online and offline callbacks are also close together. Fixes: 7ee988770326 ("timers: Implement the hierarchical pull model") Signed-off-by: Anna-Maria Behnsen Reviewed-by: Frederic Weisbecker --- include/linux/cpuhotplug.h | 1 + kernel/time/timer_migration.c | 179 +++++++++++++++++++++++---------------= ---- 2 files changed, 99 insertions(+), 81 deletions(-) diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 7a5785f405b6..df59666a2a66 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -122,6 +122,7 @@ enum cpuhp_state { CPUHP_KVM_PPC_BOOK3S_PREPARE, CPUHP_ZCOMP_PREPARE, CPUHP_TIMERS_PREPARE, + CPUHP_TMIGR_PREPARE, CPUHP_MIPS_SOC_PREPARE, CPUHP_BP_PREPARE_DYN, CPUHP_BP_PREPARE_DYN_END =3D CPUHP_BP_PREPARE_DYN + 20, diff --git a/kernel/time/timer_migration.c b/kernel/time/timer_migration.c index d91efe1dc3bf..5c030b30ed0a 100644 --- a/kernel/time/timer_migration.c +++ b/kernel/time/timer_migration.c @@ -1438,6 +1438,66 @@ u64 tmigr_quick_check(u64 nextevt) return KTIME_MAX; } =20 +/* + * tmigr_trigger_active() - trigger a CPU to become active again + * + * This function is executed on a CPU which is part of cpu_online_mask, wh= en the + * last active CPU in the hierarchy is offlining. With this, it is ensured= that + * the other CPU is active and takes over the migrator duty. + */ +static long tmigr_trigger_active(void *unused) +{ + struct tmigr_cpu *tmc =3D this_cpu_ptr(&tmigr_cpu); + + WARN_ON_ONCE(!tmc->online || tmc->idle); + + return 0; +} + +static int tmigr_cpu_offline(unsigned int cpu) +{ + struct tmigr_cpu *tmc =3D this_cpu_ptr(&tmigr_cpu); + int migrator; + u64 firstexp; + + raw_spin_lock_irq(&tmc->lock); + tmc->online =3D false; + WRITE_ONCE(tmc->wakeup, KTIME_MAX); + + /* + * CPU has to handle the local events on his own, when on the way to + * offline; Therefore nextevt value is set to KTIME_MAX + */ + firstexp =3D __tmigr_cpu_deactivate(tmc, KTIME_MAX); + trace_tmigr_cpu_offline(tmc); + raw_spin_unlock_irq(&tmc->lock); + + if (firstexp !=3D KTIME_MAX) { + migrator =3D cpumask_any_but(cpu_online_mask, cpu); + work_on_cpu(migrator, tmigr_trigger_active, NULL); + } + + return 0; +} + +static int tmigr_cpu_online(unsigned int cpu) +{ + struct tmigr_cpu *tmc =3D this_cpu_ptr(&tmigr_cpu); + + /* Check whether CPU data was successfully initialized */ + if (WARN_ON_ONCE(!tmc->tmgroup)) + return -EINVAL; + + raw_spin_lock_irq(&tmc->lock); + trace_tmigr_cpu_online(tmc); + tmc->idle =3D timer_base_is_idle(); + if (!tmc->idle) + __tmigr_cpu_activate(tmc); + tmc->online =3D true; + raw_spin_unlock_irq(&tmc->lock); + return 0; +} + static void tmigr_init_group(struct tmigr_group *group, unsigned int lvl, int node) { @@ -1512,7 +1572,7 @@ static struct tmigr_group *tmigr_get_group(unsigned i= nt cpu, int node, static void tmigr_connect_child_parent(struct tmigr_group *child, struct tmigr_group *parent) { - union tmigr_state childstate; + struct tmigr_walk data; =20 raw_spin_lock_irq(&child->lock); raw_spin_lock_nested(&parent->lock, SINGLE_DEPTH_NESTING); @@ -1540,22 +1600,22 @@ static void tmigr_connect_child_parent(struct tmigr= _group *child, * child to the new parent. So tmigr_connect_child_parent() is * executed with the formerly top level group (child) and the newly * created group (parent). + * + * * It is ensured, that the child is active, as this setup path is + * executed in hotplug prepare callback. This is exectued by an + * already connected and !idle CPU. Even if all other CPUs go idle, + * the CPU executing the setup will be responsible up to current top + * level group. Therefore propagate active state unconditionally. */ - childstate.state =3D atomic_read(&child->migr_state); - if (childstate.migrator !=3D TMIGR_NONE) { - struct tmigr_walk data; - - data.childmask =3D child->childmask; + data.childmask =3D child->childmask; =20 - /* - * There is only one new level per time (which is protected by - * tmigr_mutex). When connecting the child and the parent and - * set the child active when the parent is inactive, the parent - * needs to be the uppermost level. Otherwise there went - * something wrong! - */ - WARN_ON(!tmigr_active_up(parent, child, &data) && parent->parent); - } + /* + * There is only one new level per time (which is protected by + * tmigr_mutex). When connecting the child and the parent and set the + * child active when the parent is inactive, the parent needs to be the + * uppermost level. Otherwise there went something wrong! + */ + WARN_ON(!tmigr_active_up(parent, child, &data) && parent->parent); } =20 static int tmigr_setup_groups(unsigned int cpu, unsigned int node) @@ -1661,80 +1721,32 @@ static int tmigr_add_cpu(unsigned int cpu) return ret; } =20 -static int tmigr_cpu_online(unsigned int cpu) +static int tmigr_cpu_prepare(unsigned int cpu) { - struct tmigr_cpu *tmc =3D this_cpu_ptr(&tmigr_cpu); - int ret; - - /* First online attempt? Initialize CPU data */ - if (!tmc->tmgroup) { - raw_spin_lock_init(&tmc->lock); - - ret =3D tmigr_add_cpu(cpu); - if (ret < 0) - return ret; - - if (tmc->childmask =3D=3D 0) - return -EINVAL; + struct tmigr_cpu *tmc =3D per_cpu_ptr(&tmigr_cpu, cpu); + int ret =3D 0; =20 - timerqueue_init(&tmc->cpuevt.nextevt); - tmc->cpuevt.nextevt.expires =3D KTIME_MAX; - tmc->cpuevt.ignore =3D true; - tmc->cpuevt.cpu =3D cpu; - - tmc->remote =3D false; - WRITE_ONCE(tmc->wakeup, KTIME_MAX); - } - raw_spin_lock_irq(&tmc->lock); - trace_tmigr_cpu_online(tmc); - tmc->idle =3D timer_base_is_idle(); - if (!tmc->idle) - __tmigr_cpu_activate(tmc); - tmc->online =3D true; - raw_spin_unlock_irq(&tmc->lock); - return 0; -} - -/* - * tmigr_trigger_active() - trigger a CPU to become active again - * - * This function is executed on a CPU which is part of cpu_online_mask, wh= en the - * last active CPU in the hierarchy is offlining. With this, it is ensured= that - * the other CPU is active and takes over the migrator duty. - */ -static long tmigr_trigger_active(void *unused) -{ - struct tmigr_cpu *tmc =3D this_cpu_ptr(&tmigr_cpu); + /* Not first online attempt? */ + if (tmc->tmgroup) + return ret; =20 - WARN_ON_ONCE(!tmc->online || tmc->idle); + raw_spin_lock_init(&tmc->lock); =20 - return 0; -} + ret =3D tmigr_add_cpu(cpu); + if (ret < 0) + return ret; =20 -static int tmigr_cpu_offline(unsigned int cpu) -{ - struct tmigr_cpu *tmc =3D this_cpu_ptr(&tmigr_cpu); - int migrator; - u64 firstexp; + if (tmc->childmask =3D=3D 0) + return -EINVAL; =20 - raw_spin_lock_irq(&tmc->lock); - tmc->online =3D false; + timerqueue_init(&tmc->cpuevt.nextevt); + tmc->cpuevt.nextevt.expires =3D KTIME_MAX; + tmc->cpuevt.ignore =3D true; + tmc->cpuevt.cpu =3D cpu; + tmc->remote =3D false; WRITE_ONCE(tmc->wakeup, KTIME_MAX); =20 - /* - * CPU has to handle the local events on his own, when on the way to - * offline; Therefore nextevt value is set to KTIME_MAX - */ - firstexp =3D __tmigr_cpu_deactivate(tmc, KTIME_MAX); - trace_tmigr_cpu_offline(tmc); - raw_spin_unlock_irq(&tmc->lock); - - if (firstexp !=3D KTIME_MAX) { - migrator =3D cpumask_any_but(cpu_online_mask, cpu); - work_on_cpu(migrator, tmigr_trigger_active, NULL); - } - - return 0; + return ret; } =20 static int __init tmigr_init(void) @@ -1793,6 +1805,11 @@ static int __init tmigr_init(void) tmigr_hierarchy_levels, TMIGR_CHILDREN_PER_GROUP, tmigr_crossnode_level); =20 + ret =3D cpuhp_setup_state(CPUHP_AP_TMIGR_ONLINE, "tmigr:prepare", + tmigr_cpu_prepare, NULL); + if (ret) + goto err; + ret =3D cpuhp_setup_state(CPUHP_AP_TMIGR_ONLINE, "tmigr:online", tmigr_cpu_online, tmigr_cpu_offline); if (ret) --=20 2.39.2 From nobody Sun Feb 8 14:44:58 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75E9B85934 for ; Mon, 1 Jul 2024 10:19:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829142; cv=none; b=VXsD6z3GpuRMJCNxxaWITeHpAay+KKLc5cH0eq9GPFr8TSr7FAJsgekliD32e3NDtlGFstLJ38VGOVWw1bq07tdYfFTxOPC++sAoKA8/namw16YQxAlpjDJ99squB+9zd1jiWOWTxBaxOKbqHUIqw2NS89MX6w0Ml/LSVRBoJU0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829142; c=relaxed/simple; bh=Zo7jte0xJA7gqL2z3VTBSa/ju6aCpBt01nulJ8aS9Lo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=g0wPWvqHhJ0f7WHDzYqRTPnFfIfJt8Yg+M/62ZsQ3atR4k+LSG5Ds/nWzgyHL7Q0iSOni6xKhDT0QDeDnWQCOIRF565SSAubMjmWqI9LOQ7YJhah8QKj5+JmKtNRhHTceY/BJXvR0z5fNDKTlIlwe7dsK3Fu/dseqIDHSDQNsH0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=13OzOF0G; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=zts6jgZO; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="13OzOF0G"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="zts6jgZO" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1719829136; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=brPKlDhy+dXUY1lRR5BGqIYBAri98xW3VtWbYMv4jjE=; b=13OzOF0GVc+vXFXC8ElvTJaXxNuhpsUmKF1H3PXu9smdMmRVWMU8RuGOevP53MiE+AhWQ5 wGvmEPiNgGbG1WkJl/Z+yELtdWOzzSgZ//rdKdZay273NOy2+IiP50+naIT7eTpIDDMG5z AabNntOooIWKk2IRTYkX/zRrptDm7p6QtO/EtjFU7GWlCrXz1boNk5hI+GGzl6Qq1dJv7A cRbPgXRMOMrtAMNTd00tIK0gavhaFS4GIuAs9fv2ymbP99Tfluci47AXCII7/QVJw52yZb yPudKzlBvXH+b7b7GsX8+1aEiSEzTUVddWUc8yrN+X8xzJ6MjKFzd7/ySNbG3w== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1719829136; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=brPKlDhy+dXUY1lRR5BGqIYBAri98xW3VtWbYMv4jjE=; b=zts6jgZOQo/Csqe+TFdgS4ghpwewtGyNxfbKoGVKjUZj8WtOCA1SVU99qShAEjN7BgWkql A0VVr+8kgGrx0eCg== Date: Mon, 01 Jul 2024 12:18:39 +0200 Subject: [PATCH v3 3/8] timers/migration: Improve tracing Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240701-tmigr-fixes-v3-3-25cd5de318fb@linutronix.de> References: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> In-Reply-To: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> To: Frederic Weisbecker , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Anna-Maria Behnsen Trace points of inactive and active propagation are located at the end of the related functions. The interesting information of those trace points is the updated group state. When trace points are not located directly at the place where group state changed, order of trace points in traces could be confusing. Move inactive and active propagation trace points directly after update of group state values. Signed-off-by: Anna-Maria Behnsen Reviewed-by: Frederic Weisbecker --- kernel/time/timer_migration.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/kernel/time/timer_migration.c b/kernel/time/timer_migration.c index 5c030b30ed0a..0ae7f2084d27 100644 --- a/kernel/time/timer_migration.c +++ b/kernel/time/timer_migration.c @@ -656,6 +656,8 @@ static bool tmigr_active_up(struct tmigr_group *group, =20 } while (!atomic_try_cmpxchg(&group->migr_state, &curstate.state, newstat= e.state)); =20 + trace_tmigr_group_set_cpu_active(group, newstate, childmask); + if (walk_done =3D=3D false) data->childmask =3D group->childmask; =20 @@ -673,8 +675,6 @@ static bool tmigr_active_up(struct tmigr_group *group, */ group->groupevt.ignore =3D true; =20 - trace_tmigr_group_set_cpu_active(group, newstate, childmask); - return walk_done; } =20 @@ -1306,9 +1306,10 @@ static bool tmigr_inactive_up(struct tmigr_group *gr= oup, =20 WARN_ON_ONCE((newstate.migrator !=3D TMIGR_NONE) && !(newstate.active)); =20 - if (atomic_try_cmpxchg(&group->migr_state, &curstate.state, - newstate.state)) + if (atomic_try_cmpxchg(&group->migr_state, &curstate.state, newstate.sta= te)) { + trace_tmigr_group_set_cpu_inactive(group, newstate, childmask); break; + } =20 /* * The memory barrier is paired with the cmpxchg() in @@ -1327,8 +1328,6 @@ static bool tmigr_inactive_up(struct tmigr_group *gro= up, if (walk_done =3D=3D false) data->childmask =3D group->childmask; =20 - trace_tmigr_group_set_cpu_inactive(group, newstate, childmask); - return walk_done; } =20 --=20 2.39.2 From nobody Sun Feb 8 14:44:58 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 784FE12C491 for ; Mon, 1 Jul 2024 10:19:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829143; cv=none; b=R0L9NwdfVZEaxpqCOv7tG3yyhvkJ3KqddYjBsS+3r0jAfHESpVGfLuz3BJQtGXjiEiKT2ulqQzeDNGQ57PY808lO3GjoXWtwXwoS4rpWm6muBgxcbVSp4rHzscVgKuMMREv025yGcStDFqjGx3zL2m/cRMpcTAnY1RX+Ml8qpdA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829143; c=relaxed/simple; bh=5ikh8IH4woSIgTgstlLlThJYOjyMKptYoGYDMRliWG0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dA8udmjin0GVGy3QWXUz5Lq7xfPAcDs8qUrhD1qNu4w71YluceyFw3RymfyqTlDC8fKFgdx+tfENkuprevJIRMZkD0jUy5M3jipbd/sCNC9UvEGz/b1zzEUCRBZVJ5nxxkQzACFM1nbNxsV99mB/L/gfN85SrStYcEUqb3oaq3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=sjV1lRCJ; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=h0QEqtk1; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="sjV1lRCJ"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="h0QEqtk1" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1719829137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PyBCuRiWGrZQ9/Lh9FP2UfloCMkS+RDbkIpMrfk9kbM=; b=sjV1lRCJGFwnMEMYB/ub0cV4ddS1PhkEtBwppkz2j6wYd49+Kx8vB6WMQvXCzQWjOVm7Gq XAhlY3JOcfROPd9TbaVun3QoSrOvEJMnpWqm+MCRVS4fC+OEF43lC8RlvYK6WWi9l1xJuG k7pAGe/mPkHjqM1XkU3IS/dWe3jE2eekHX4tOVQg071pr+eQp7pKTMZQMKZWGCCqnOAfbX nP/1AZ100dWC9CZrzewdZeeoEE+Mw4nwQc5agTRV0GQAnz67+XXGtnLGgIoVRsdTnyY/pX wmq641hTHoWZwglQuPhtYsj3bBneM1w/X/GmupyPbDkedPDe+QEtCx0/CtSt0Q== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1719829137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PyBCuRiWGrZQ9/Lh9FP2UfloCMkS+RDbkIpMrfk9kbM=; b=h0QEqtk1J57vth3R48m5OdqUWldakxItRCBuOuphA71liVOZgAaTTOKC+HMavdfJO0AD9u 9ynQMeTrPfWkFKCg== Date: Mon, 01 Jul 2024 12:18:40 +0200 Subject: [PATCH v3 4/8] timers/migration: Use a single struct for hierarchy walk data Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240701-tmigr-fixes-v3-4-25cd5de318fb@linutronix.de> References: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> In-Reply-To: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> To: Frederic Weisbecker , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Anna-Maria Behnsen Two different structs are defined for propagating data from one to another level when walking the hierarchy. Several struct members exist in both structs which makes generalization harder. Merge those two structs into a single one and use it directly in walk_groups() and the corresponding function pointers instead of introducing pointer casting all over the place. Signed-off-by: Anna-Maria Behnsen Reviewed-by: Frederic Weisbecker --- kernel/time/timer_migration.c | 126 ++++++++++++++++++--------------------= ---- 1 file changed, 55 insertions(+), 71 deletions(-) diff --git a/kernel/time/timer_migration.c b/kernel/time/timer_migration.c index 0ae7f2084d27..b4391abfb4a9 100644 --- a/kernel/time/timer_migration.c +++ b/kernel/time/timer_migration.c @@ -475,69 +475,31 @@ static bool tmigr_check_lonely(struct tmigr_group *gr= oup) return bitmap_weight(&active, BIT_CNT) <=3D 1; } =20 -typedef bool (*up_f)(struct tmigr_group *, struct tmigr_group *, void *); - -static void __walk_groups(up_f up, void *data, - struct tmigr_cpu *tmc) -{ - struct tmigr_group *child =3D NULL, *group =3D tmc->tmgroup; - - do { - WARN_ON_ONCE(group->level >=3D tmigr_hierarchy_levels); - - if (up(group, child, data)) - break; - - child =3D group; - group =3D group->parent; - } while (group); -} - -static void walk_groups(up_f up, void *data, struct tmigr_cpu *tmc) -{ - lockdep_assert_held(&tmc->lock); - - __walk_groups(up, data, tmc); -} - /** * struct tmigr_walk - data required for walking the hierarchy * @nextexp: Next CPU event expiry information which is handed into * the timer migration code by the timer code * (get_next_timer_interrupt()) - * @firstexp: Contains the first event expiry information when last - * active CPU of hierarchy is on the way to idle to make - * sure CPU will be back in time. It is updated in top - * level group only. Be aware, there could occur a new top - * level of the hierarchy between the 'top level call' in - * tmigr_update_events() and the check for the parent group - * in walk_groups(). Then @firstexp might contain a value - * !=3D KTIME_MAX even if it was not the final top - * level. This is not a problem, as the worst outcome is a - * CPU which might wake up a little early. + * @firstexp: Contains the first event expiry information when + * hierarchy is completely idle. When CPU itself was the + * last going idle, information makes sure, that CPU will + * be back in time. When using this value in the remote + * expiry case, firstexp is stored in the per CPU tmigr_cpu + * struct of CPU which expires remote timers. It is updated + * in top level group only. Be aware, there could occur a + * new top level of the hierarchy between the 'top level + * call' in tmigr_update_events() and the check for the + * parent group in walk_groups(). Then @firstexp might + * contain a value !=3D KTIME_MAX even if it was not the + * final top level. This is not a problem, as the worst + * outcome is a CPU which might wake up a little early. * @evt: Pointer to tmigr_event which needs to be queued (of idle * child group) * @childmask: childmask of child group * @remote: Is set, when the new timer path is executed in * tmigr_handle_remote_cpu() - */ -struct tmigr_walk { - u64 nextexp; - u64 firstexp; - struct tmigr_event *evt; - u8 childmask; - bool remote; -}; - -/** - * struct tmigr_remote_data - data required for remote expiry hierarchy wa= lk * @basej: timer base in jiffies * @now: timer base monotonic - * @firstexp: returns expiry of the first timer in the idle timer - * migration hierarchy to make sure the timer is handled in - * time; it is stored in the per CPU tmigr_cpu struct of - * CPU which expires remote timers - * @childmask: childmask of child group * @check: is set if there is the need to handle remote timers; * required in tmigr_requires_handle_remote() only * @tmc_active: this flag indicates, whether the CPU which triggers @@ -546,15 +508,43 @@ struct tmigr_walk { * idle, only the first event of the top level has to be * considered. */ -struct tmigr_remote_data { - unsigned long basej; - u64 now; - u64 firstexp; - u8 childmask; - bool check; - bool tmc_active; +struct tmigr_walk { + u64 nextexp; + u64 firstexp; + struct tmigr_event *evt; + u8 childmask; + bool remote; + unsigned long basej; + u64 now; + bool check; + bool tmc_active; }; =20 +typedef bool (*up_f)(struct tmigr_group *, struct tmigr_group *, struct tm= igr_walk *); + +static void __walk_groups(up_f up, struct tmigr_walk *data, + struct tmigr_cpu *tmc) +{ + struct tmigr_group *child =3D NULL, *group =3D tmc->tmgroup; + + do { + WARN_ON_ONCE(group->level >=3D tmigr_hierarchy_levels); + + if (up(group, child, data)) + break; + + child =3D group; + group =3D group->parent; + } while (group); +} + +static void walk_groups(up_f up, struct tmigr_walk *data, struct tmigr_cpu= *tmc) +{ + lockdep_assert_held(&tmc->lock); + + __walk_groups(up, data, tmc); +} + /* * Returns the next event of the timerqueue @group->events * @@ -625,10 +615,9 @@ static u64 tmigr_next_groupevt_expires(struct tmigr_gr= oup *group) =20 static bool tmigr_active_up(struct tmigr_group *group, struct tmigr_group *child, - void *ptr) + struct tmigr_walk *data) { union tmigr_state curstate, newstate; - struct tmigr_walk *data =3D ptr; bool walk_done; u8 childmask; =20 @@ -867,10 +856,8 @@ bool tmigr_update_events(struct tmigr_group *group, st= ruct tmigr_group *child, =20 static bool tmigr_new_timer_up(struct tmigr_group *group, struct tmigr_group *child, - void *ptr) + struct tmigr_walk *data) { - struct tmigr_walk *data =3D ptr; - return tmigr_update_events(group, child, data); } =20 @@ -1002,9 +989,8 @@ static void tmigr_handle_remote_cpu(unsigned int cpu, = u64 now, =20 static bool tmigr_handle_remote_up(struct tmigr_group *group, struct tmigr_group *child, - void *ptr) + struct tmigr_walk *data) { - struct tmigr_remote_data *data =3D ptr; struct tmigr_event *evt; unsigned long jif; u8 childmask; @@ -1062,7 +1048,7 @@ static bool tmigr_handle_remote_up(struct tmigr_group= *group, void tmigr_handle_remote(void) { struct tmigr_cpu *tmc =3D this_cpu_ptr(&tmigr_cpu); - struct tmigr_remote_data data; + struct tmigr_walk data; =20 if (tmigr_is_not_available(tmc)) return; @@ -1104,9 +1090,8 @@ void tmigr_handle_remote(void) =20 static bool tmigr_requires_handle_remote_up(struct tmigr_group *group, struct tmigr_group *child, - void *ptr) + struct tmigr_walk *data) { - struct tmigr_remote_data *data =3D ptr; u8 childmask; =20 childmask =3D data->childmask; @@ -1164,7 +1149,7 @@ static bool tmigr_requires_handle_remote_up(struct tm= igr_group *group, bool tmigr_requires_handle_remote(void) { struct tmigr_cpu *tmc =3D this_cpu_ptr(&tmigr_cpu); - struct tmigr_remote_data data; + struct tmigr_walk data; unsigned long jif; bool ret =3D false; =20 @@ -1252,10 +1237,9 @@ u64 tmigr_cpu_new_timer(u64 nextexp) =20 static bool tmigr_inactive_up(struct tmigr_group *group, struct tmigr_group *child, - void *ptr) + struct tmigr_walk *data) { union tmigr_state curstate, newstate, childstate; - struct tmigr_walk *data =3D ptr; bool walk_done; u8 childmask; =20 --=20 2.39.2 From nobody Sun Feb 8 14:44:58 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59E5F13C9A4 for ; Mon, 1 Jul 2024 10:19:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829145; cv=none; b=bnxjUS78w+KcAdi+1oPYdqB6NL0Nj4odCIhDlSOLpscArkab1J7eeCv2AYCJCc0Lsct0vtaC/5usb7aoILNfJg44ucLZwIfnY0sjvOiR/Ue5x0cz/HXdbr6CNpnDE8FR2rcRZP+rMfUsMhnUyjIqH0Gvzd7JQ9cMnOy4ES2Y+E0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829145; c=relaxed/simple; bh=AfaZt+iIB+Mu6UbReX4cOUZDpnT5VxlA7zY/U0lFktQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=u7iGjuH1BNZHcDrBI8mbXZctFAdCkY7iyo9quPty0cyd1gT5KvLLgGZTRi+x5s5aNTsuawBRF7DrVjSvDJtfRi3eD81fYZQnMRctQwCEaaStFFewQseSMj9y42vaj79IaKYaOwyzUtevHwmlSMO8YdzrNJR9+VnqXMR6S0iwie0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=JhQeYHSp; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=CJnMT9hc; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="JhQeYHSp"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="CJnMT9hc" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1719829137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oON15loQKefft5FbGHUbwgp/Rc/h5lGwlddmep9FW6I=; b=JhQeYHSp6pdFOEtt9VtWNr3OO15+JYzIxLWiXexsgWUGZ/4NRzLm4hFuCzTtiy3lLXl3db eH5f7zXIi1+aoO1yy9WK3fCI2qQeMJuHGZhCy9pwSe7ms0IPezh37LOzf+Ipf6aGrq4fj9 3s9194nBAJs+7SKddWxek2t0z+z2K5bU1NkxGHz9DlJy9huTGTJc1MymAeF4BNqJGL0bgz 58bycqIqX5YQqlSt+EnuxOOMRM7gP6MqIsybZhtLbm10qIfnvBfTNTHeSgIO34sjPItdkl aX1HfzwoEOr1yzGBV2ZzkvlphI7fed5fIs3iTEWqGEjXOUXTGOGTy061jX1ung== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1719829137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oON15loQKefft5FbGHUbwgp/Rc/h5lGwlddmep9FW6I=; b=CJnMT9hcjRfXeM9lCPmV8kHPLsVNfFIv8wJrQuAIUcdW9XLsCG9TBigG1fiuzKejzbGw/C kPAs6PUc2wgPTBBg== Date: Mon, 01 Jul 2024 12:18:41 +0200 Subject: [PATCH v3 5/8] timers/migration: Read childmask and parent pointer in a single place Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240701-tmigr-fixes-v3-5-25cd5de318fb@linutronix.de> References: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> In-Reply-To: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> To: Frederic Weisbecker , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Anna-Maria Behnsen Reading the childmask and parent pointer is required when propagating changes through the hierarchy. At the moment this reads are spread all over the place which makes it harder to follow. Move those reads to a single place to keep code clean. Signed-off-by: Anna-Maria Behnsen Reviewed-by: Frederic Weisbecker --- kernel/time/timer_migration.c | 20 +++++--------------- 1 file changed, 5 insertions(+), 15 deletions(-) diff --git a/kernel/time/timer_migration.c b/kernel/time/timer_migration.c index b4391abfb4a9..a681cf89910e 100644 --- a/kernel/time/timer_migration.c +++ b/kernel/time/timer_migration.c @@ -535,6 +535,7 @@ static void __walk_groups(up_f up, struct tmigr_walk *d= ata, =20 child =3D group; group =3D group->parent; + data->childmask =3D group->childmask; } while (group); } =20 @@ -647,9 +648,6 @@ static bool tmigr_active_up(struct tmigr_group *group, =20 trace_tmigr_group_set_cpu_active(group, newstate, childmask); =20 - if (walk_done =3D=3D false) - data->childmask =3D group->childmask; - /* * The group is active (again). The group event might be still queued * into the parent group's timerqueue but can now be handled by the @@ -1027,12 +1025,10 @@ static bool tmigr_handle_remote_up(struct tmigr_gro= up *group, } =20 /* - * Update of childmask for the next level and keep track of the expiry - * of the first event that needs to be handled (group->next_expiry was - * updated by tmigr_next_expired_groupevt(), next was set by - * tmigr_handle_remote_cpu()). + * Keep track of the expiry of the first event that needs to be handled + * (group->next_expiry was updated by tmigr_next_expired_groupevt(), + * next was set by tmigr_handle_remote_cpu()). */ - data->childmask =3D group->childmask; data->firstexp =3D group->next_expiry; =20 raw_spin_unlock_irq(&group->lock); @@ -1110,7 +1106,7 @@ static bool tmigr_requires_handle_remote_up(struct tm= igr_group *group, * group before reading the next_expiry value. */ if (group->parent && !data->tmc_active) - goto out; + return false; =20 /* * The lock is required on 32bit architectures to read the variable @@ -1135,9 +1131,6 @@ static bool tmigr_requires_handle_remote_up(struct tm= igr_group *group, raw_spin_unlock(&group->lock); } =20 -out: - /* Update of childmask for the next level */ - data->childmask =3D group->childmask; return false; } =20 @@ -1309,9 +1302,6 @@ static bool tmigr_inactive_up(struct tmigr_group *gro= up, /* Event Handling */ tmigr_update_events(group, child, data); =20 - if (walk_done =3D=3D false) - data->childmask =3D group->childmask; - return walk_done; } =20 --=20 2.39.2 From nobody Sun Feb 8 14:44:58 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59E1513C908 for ; Mon, 1 Jul 2024 10:19:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829146; cv=none; b=b25BGWFkyeLcwatZYi21BZJSN4mVI0RZ92EnuM1brah9CXDEYLZwYPU3ffymiR6BkDTaFJMU05fq33OXjGjieS9e9x4XKZTL8z9qQdwUXsBH7A8mXldcjPum05TV1e3W1ZMX2akz8Qp511/Du8WdmFWhneK8QpxxPIrd0xF+X14= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829146; c=relaxed/simple; bh=dT3YF+l6qUA0t/dwgPfof5qptqbcVdqreu6ctldk+0U=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=CjTKgbsDs9ZvfD+N2R5+wd3q/0095n7MheHSQUnE/N65mJql6kdzMS1Fyo4eeL0QTxsjIlDUgg6omEA98Ju2gKLno7iJhGaOa278aM9bVC3arYsRv88G6MaRGS3na04XtytcoW3hzJoCW/8QtJcZxlP3hvc6Jz9VecyaDspE5So= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=qoiFSwGw; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=5MQ/654C; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="qoiFSwGw"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="5MQ/654C" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1719829137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S1XK08QEW0DGmeu0xkDB8INwdLhY21pOj2tuCloT6F0=; b=qoiFSwGwjDQkkjWfGHiPdat8O0Wm0lWnr/4CCFJhV5X+bQXARPSEcHuPIWz8hMSHjD2XJG 9c0IsqIGwA749Mcnf78jx6lfOEuZ8JQPILZKP0V0GkIYASCcxzChua5vfMAO85MIALW9A0 f05uO4tFwAjexjOR52SurW7++dx3ewWr0GV8++4fk2tkjGuCa2FjmlDz3z4mL6m0YR+2IY KL28g2oYS55Wu8B7StiMwyYXFenVEH4gk2tNMe5SJ8KslQLbRFYKimBP2fbM9Joaq84r4i fA9e8yFTBlTIprEinSpAvMCSlNSYLI/9HeZMGd7/F4Izrr/j4D/cOpFfVQQS6w== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1719829137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S1XK08QEW0DGmeu0xkDB8INwdLhY21pOj2tuCloT6F0=; b=5MQ/654CXDQ8ejFGhq2LjpxZPf+uDBvcmwVCuoRR18MbE61O/arpTfbc3/1+dtuCTp1WRP DRHfexQau4ONsqCw== Date: Mon, 01 Jul 2024 12:18:42 +0200 Subject: [PATCH v3 6/8] timers/migration: Rename childmask by parentmask to make naming more obvious Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240701-tmigr-fixes-v3-6-25cd5de318fb@linutronix.de> References: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> In-Reply-To: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> To: Frederic Weisbecker , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Anna-Maria Behnsen childmask in the group reflects the mask that is required to 'reference' this group in the parent. When reading childmask, this might be confusing, as this suggests, that this is the mask of the child of the group. Clarify this by renaming childmask in the tmigr_group and tmc_group by parentmask. Signed-off-by: Anna-Maria Behnsen --- include/trace/events/timer_migration.h | 4 ++-- kernel/time/timer_migration.c | 24 ++++++++++++------------ kernel/time/timer_migration.h | 15 +++++++-------- 3 files changed, 21 insertions(+), 22 deletions(-) diff --git a/include/trace/events/timer_migration.h b/include/trace/events/= timer_migration.h index 79f19e76a80b..e3c5afd1638d 100644 --- a/include/trace/events/timer_migration.h +++ b/include/trace/events/timer_migration.h @@ -52,7 +52,7 @@ TRACE_EVENT(tmigr_connect_child_parent, __entry->lvl =3D child->parent->level; __entry->numa_node =3D child->parent->numa_node; __entry->num_children =3D child->parent->num_children; - __entry->childmask =3D child->childmask; + __entry->childmask =3D child->parentmask; ), =20 TP_printk("group=3D%p childmask=3D%0x parent=3D%p lvl=3D%d numa=3D%d num_= children=3D%d", @@ -81,7 +81,7 @@ TRACE_EVENT(tmigr_connect_cpu_parent, __entry->lvl =3D tmc->tmgroup->level; __entry->numa_node =3D tmc->tmgroup->numa_node; __entry->num_children =3D tmc->tmgroup->num_children; - __entry->childmask =3D tmc->childmask; + __entry->childmask =3D tmc->parentmask; ), =20 TP_printk("cpu=3D%d childmask=3D%0x parent=3D%p lvl=3D%d numa=3D%d num_ch= ildren=3D%d", diff --git a/kernel/time/timer_migration.c b/kernel/time/timer_migration.c index a681cf89910e..b73d89e78163 100644 --- a/kernel/time/timer_migration.c +++ b/kernel/time/timer_migration.c @@ -495,7 +495,7 @@ static bool tmigr_check_lonely(struct tmigr_group *grou= p) * outcome is a CPU which might wake up a little early. * @evt: Pointer to tmigr_event which needs to be queued (of idle * child group) - * @childmask: childmask of child group + * @childmask: parentmask of child group * @remote: Is set, when the new timer path is executed in * tmigr_handle_remote_cpu() * @basej: timer base in jiffies @@ -535,7 +535,7 @@ static void __walk_groups(up_f up, struct tmigr_walk *d= ata, =20 child =3D group; group =3D group->parent; - data->childmask =3D group->childmask; + data->childmask =3D group->parentmask; } while (group); } =20 @@ -669,7 +669,7 @@ static void __tmigr_cpu_activate(struct tmigr_cpu *tmc) { struct tmigr_walk data; =20 - data.childmask =3D tmc->childmask; + data.childmask =3D tmc->parentmask; =20 trace_tmigr_cpu_active(tmc); =20 @@ -1049,7 +1049,7 @@ void tmigr_handle_remote(void) if (tmigr_is_not_available(tmc)) return; =20 - data.childmask =3D tmc->childmask; + data.childmask =3D tmc->parentmask; data.firstexp =3D KTIME_MAX; =20 /* @@ -1057,7 +1057,7 @@ void tmigr_handle_remote(void) * in tmigr_handle_remote_up() anyway. Keep this check to speed up the * return when nothing has to be done. */ - if (!tmigr_check_migrator(tmc->tmgroup, tmc->childmask)) { + if (!tmigr_check_migrator(tmc->tmgroup, tmc->parentmask)) { /* * If this CPU was an idle migrator, make sure to clear its wakeup * value so it won't chase timers that have already expired elsewhere. @@ -1150,7 +1150,7 @@ bool tmigr_requires_handle_remote(void) return ret; =20 data.now =3D get_jiffies_update(&jif); - data.childmask =3D tmc->childmask; + data.childmask =3D tmc->parentmask; data.firstexp =3D KTIME_MAX; data.tmc_active =3D !tmc->idle; data.check =3D false; @@ -1310,7 +1310,7 @@ static u64 __tmigr_cpu_deactivate(struct tmigr_cpu *t= mc, u64 nextexp) struct tmigr_walk data =3D { .nextexp =3D nextexp, .firstexp =3D KTIME_MAX, .evt =3D &tmc->cpuevt, - .childmask =3D tmc->childmask }; + .childmask =3D tmc->parentmask }; =20 /* * If nextexp is KTIME_MAX, the CPU event will be ignored because the @@ -1388,7 +1388,7 @@ u64 tmigr_quick_check(u64 nextevt) if (WARN_ON_ONCE(tmc->idle)) return nextevt; =20 - if (!tmigr_check_migrator_and_lonely(tmc->tmgroup, tmc->childmask)) + if (!tmigr_check_migrator_and_lonely(tmc->tmgroup, tmc->parentmask)) return KTIME_MAX; =20 do { @@ -1551,7 +1551,7 @@ static void tmigr_connect_child_parent(struct tmigr_g= roup *child, raw_spin_lock_nested(&parent->lock, SINGLE_DEPTH_NESTING); =20 child->parent =3D parent; - child->childmask =3D BIT(parent->num_children++); + child->parentmask =3D BIT(parent->num_children++); =20 raw_spin_unlock(&parent->lock); raw_spin_unlock_irq(&child->lock); @@ -1580,7 +1580,7 @@ static void tmigr_connect_child_parent(struct tmigr_g= roup *child, * the CPU executing the setup will be responsible up to current top * level group. Therefore propagate active state unconditionally. */ - data.childmask =3D child->childmask; + data.childmask =3D child->parentmask; =20 /* * There is only one new level per time (which is protected by @@ -1646,7 +1646,7 @@ static int tmigr_setup_groups(unsigned int cpu, unsig= ned int node) raw_spin_lock_irq(&group->lock); =20 tmc->tmgroup =3D group; - tmc->childmask =3D BIT(group->num_children++); + tmc->parentmask =3D BIT(group->num_children++); =20 raw_spin_unlock_irq(&group->lock); =20 @@ -1709,7 +1709,7 @@ static int tmigr_cpu_prepare(unsigned int cpu) if (ret < 0) return ret; =20 - if (tmc->childmask =3D=3D 0) + if (tmc->parentmask =3D=3D 0) return -EINVAL; =20 timerqueue_init(&tmc->cpuevt.nextevt); diff --git a/kernel/time/timer_migration.h b/kernel/time/timer_migration.h index 494f68cc13f4..3d08e611c3d6 100644 --- a/kernel/time/timer_migration.h +++ b/kernel/time/timer_migration.h @@ -51,9 +51,8 @@ struct tmigr_event { * @num_children: Counter of group children to make sure the group is only * filled with TMIGR_CHILDREN_PER_GROUP; Required for setup * only - * @childmask: childmask of the group in the parent group; is set - * during setup and will never change; can be read - * lockless + * @parentmask: mask of the group in the parent group; is set during + * setup and will never change; can be read lockless * @list: List head that is added to the per level * tmigr_level_list; is required during setup when a * new group needs to be connected to the existing @@ -69,7 +68,7 @@ struct tmigr_group { unsigned int level; int numa_node; unsigned int num_children; - u8 childmask; + u8 parentmask; struct list_head list; }; =20 @@ -89,7 +88,7 @@ struct tmigr_group { * hierarchy * @remote: Is set when timers of the CPU are expired remotely * @tmgroup: Pointer to the parent group - * @childmask: childmask of tmigr_cpu in the parent group + * @parentmask: mask of tmigr_cpu in the parent group * @wakeup: Stores the first timer when the timer migration * hierarchy is completely idle and remote expiry was done; * is returned to timer code in the idle path and is only @@ -102,7 +101,7 @@ struct tmigr_cpu { bool idle; bool remote; struct tmigr_group *tmgroup; - u8 childmask; + u8 parentmask; u64 wakeup; struct tmigr_event cpuevt; }; @@ -118,8 +117,8 @@ union tmigr_state { u32 state; /** * struct - split state of tmigr_group - * @active: Contains each childmask bit of the active children - * @migrator: Contains childmask of the child which is migrator + * @active: Contains each mask bit of the active children + * @migrator: Contains mask of the child which is migrator * @seq: Sequence counter needs to be increased when an update * to the tmigr_state is done. It prevents a race when * updates in the child groups are propagated in changed --=20 2.39.2 From nobody Sun Feb 8 14:44:58 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 977B213D625 for ; Mon, 1 Jul 2024 10:19:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829146; cv=none; b=GmKWKb7y996q6AFFrsKRdNHykpo9eiKqdGqEfB653tv65w9rDVb1AAq6y353QYuVF/TBQxCWuxoU52gaWYnyKrac9jNVx3jMK2rrV5Y6I62NfgZsE8IBdA3HvCP+SRoCzHNWQrDBU/MGc7syWGDY9cwbjuv1dbHOrCwYEsp1NqI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829146; c=relaxed/simple; bh=d1bdnk8WhuesMoDYlsyVW3NQHYSNR0skAu9tJwrhzmA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=JHwV34M7G8Fj6a5QmuMwfqAp8nLZpM6d1mNwnsNijj0Ti4IdxdcqDVd5ukCTm3Xb0U2Pay3yW2RTdQgxvf6FFxsT8R4m7PPmm3Vws6TF7guOfRfEPn9tglCwTkmKQEJgkLGdHXq41bAsN3mjDK4fFfz2iztHbAdKd4Y+XXlQpqI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=BrOJa88j; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=h3uMs5kf; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="BrOJa88j"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="h3uMs5kf" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1719829137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bCPPgf214+6N8Oo0G5UoJkaX1kFyTqM3hFGlx6gP1Jw=; b=BrOJa88jQEpUtulkmC1RA6HxGXwMttxhH14n7i0W7MbJ0XYzi1kI5zvU0fY2X8aB7TsIF5 nAaxoISn9a+xcavVRXwAhAl/sKK8AMp/jGOayD7Ft7JUSQq6WSz4L+0GaniZ40kLh2fP4X gW3jppH5QB+tSpjgWqGkLi7DOgquyxAHmtgjDOWKFRmrOk+5R7k7IG+pdhxEP0TKPmC1ea q2+/V2cY/YayhGykG5SSWRYXRkHOtMz3AABKoVsP738TszdsLnUtJkxVeO6TDhLkzMUD1R OoJRhVqMH6UcaxxzpdWjm9tuwAnUiX1Ugv2F5j3PPim4ACjVuJ2lndr/8t9+SQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1719829137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bCPPgf214+6N8Oo0G5UoJkaX1kFyTqM3hFGlx6gP1Jw=; b=h3uMs5kfde4IwJ6RI1rwC5dBo55Zc1xklpIyLvHDmcsjUWXi9qdasMVD8MvdPkXx4VYGuH wCYzqqn4czKJ0rBg== Date: Mon, 01 Jul 2024 12:18:43 +0200 Subject: [PATCH v3 7/8] timers/migration: Spare write when nothing changed Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240701-tmigr-fixes-v3-7-25cd5de318fb@linutronix.de> References: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> In-Reply-To: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> To: Frederic Weisbecker , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Anna-Maria Behnsen The wakeup value is written unconditionally in tmigr_cpu_new_timer(). When there was no new next timer expiry that needs to be propagated, then the value that was read before is written. This is not required. Move write to the place where wakeup value could have changed. Signed-off-by: Anna-Maria Behnsen Reviewed-by: Frederic Weisbecker --- kernel/time/timer_migration.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/kernel/time/timer_migration.c b/kernel/time/timer_migration.c index b73d89e78163..bbc849539dd2 100644 --- a/kernel/time/timer_migration.c +++ b/kernel/time/timer_migration.c @@ -1215,14 +1215,13 @@ u64 tmigr_cpu_new_timer(u64 nextexp) if (nextexp !=3D tmc->cpuevt.nextevt.expires || tmc->cpuevt.ignore) { ret =3D tmigr_new_timer(tmc, nextexp); + /* + * Make sure the reevaluation of timers in idle path + * will not miss an event. + */ + WRITE_ONCE(tmc->wakeup, ret); } } - /* - * Make sure the reevaluation of timers in idle path will not miss an - * event. - */ - WRITE_ONCE(tmc->wakeup, ret); - trace_tmigr_cpu_new_timer_idle(tmc, nextexp); raw_spin_unlock(&tmc->lock); return ret; --=20 2.39.2 From nobody Sun Feb 8 14:44:58 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 95C6B13D61A for ; Mon, 1 Jul 2024 10:19:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829146; cv=none; b=Jzhrcuinj8amj5v3bFtc6mds73mBcTtqn8HP+RFjNygHjt0mxamdCbRN2vQA4vHsesaCzDNQYqTqJZeuJyQ7YuKPXg1hMrGBJy4p/YyhKzfCVoH2uL0XerstktBWcLwatBDlSS0lvzTM9IYel8Z1AsuGGjVfCkRvHqgvsuuL0f4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719829146; c=relaxed/simple; bh=benwhNThWSYC3tZONYtpR+lOK7wXsAZRQexO5Qwp3pM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FiB6Cy6UhAZC2g5HUGoknESFKFDsuURuYwidJOq0tBjsWCaQYRfOqxBO1vfO5QEKYGsDMCY2azOOGs/h21W4l7S8DTYup07QS9I/dLmzzW83K+EEtx3UFFeWX1QSHQKd5vB7+cVCfHpnNTzCoB67AdCO1AN/rrN++68dDvYJlRc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=n0FfDH+q; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=jXcjzKep; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="n0FfDH+q"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="jXcjzKep" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1719829137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+aUY3MI7mRJT+MnfqFQU16kFIx+aJUbq9+kmQSfCc/Q=; b=n0FfDH+qnXmOGLBz9flxw9KTp6KZuUYB6lRlLus+iPkhSYTC+hozmuXsLcA5hz5tj4gflV eZWr5jU7DnJ8p1MBKmRq5/jYbh+uVXKekbBZqCfCFtLy0jOQ0DnTgXsDpS7jZPs+OiZ3Am 3/sZMz5nrHM5ys9FgwlxZjyRlm40JnPinoBbTgVE/3MedsYpFBQ1jWZcEreS9qGnFUxh66 7PuS7iyXUQ0/noYGnTfQk9Az3ZxNXIVyCQ1CGl0AL3Ic3FqYcUSgW0d1S6C3dXto+PjOe9 EBv8+bTfalVTfvSeV0vIANmvXJB3fJ+mJOQpKdHSnTY/Wp4/+h/v5H5KtjJNVQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1719829137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+aUY3MI7mRJT+MnfqFQU16kFIx+aJUbq9+kmQSfCc/Q=; b=jXcjzKepguMzvb1fuMqxggg4uBwrm1Al6qo120abjShX/WYR0kvMleBJuKwfXp0y77/gkZ YPo+rY0v70u8fRDw== Date: Mon, 01 Jul 2024 12:18:44 +0200 Subject: [PATCH v3 8/8] timers/migration: Fix grammar in comment Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240701-tmigr-fixes-v3-8-25cd5de318fb@linutronix.de> References: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> In-Reply-To: <20240701-tmigr-fixes-v3-0-25cd5de318fb@linutronix.de> To: Frederic Weisbecker , Thomas Gleixner , linux-kernel@vger.kernel.org Cc: Anna-Maria Behnsen Signed-off-by: Anna-Maria Behnsen Reviewed-by: Frederic Weisbecker --- kernel/time/timer_migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/time/timer_migration.c b/kernel/time/timer_migration.c index bbc849539dd2..bc40cac6b58d 100644 --- a/kernel/time/timer_migration.c +++ b/kernel/time/timer_migration.c @@ -1368,7 +1368,7 @@ u64 tmigr_cpu_deactivate(u64 nextexp) * the only one in the level 0 group; and if it is the * only one in level 0 group, but there are more than a * single group active on the way to top level) - * * nextevt - when CPU is offline and has to handle timer on his own + * * nextevt - when CPU is offline and has to handle timer on its own * or when on the way to top in every group only a single * child is active but @nextevt is before the lowest * next_expiry encountered while walking up to top level. --=20 2.39.2