From nobody Sat Sep 26 11:47:52 2026 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 61B8A29B78F for ; Wed, 2 Sep 2026 03:39:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788320346; cv=none; b=W9FOJ9ZtGil++JaKKxejlv/5wDWoWufU+gfqSOG7hhn86SOdGubN7znU+b4yQv4BQa+pVv/3nYkcEF2vhhHqyS+wkVxkVmmdIfQMxyuP3ywJtVwXzjv0igHlYgd9UWE5OOBJ4G31QtdaZ54J7VVaSN/mpZGsz/rmmMpFvEqQKu0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788320346; c=relaxed/simple; bh=ymXB/kQ0oxAbSey+LXIW1PCx8yu1EEfH7MZyjtlc8ZQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IW4tco8eGH5n3e3JTa7h4iH9oGUwwsccU2HqmCnXJsccz446+pOCghXkV2Ot6VduGVyQSKVI/i5KsskgyAda/EqqL28aa6KU1G5WiL27jmw6st1WqK+sbT4M4qUMFHEgprbuttuySPeZ98zsvfzoFhiOQVvlbXiGWm/ZeF9asv4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BxaE9lxj; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BxaE9lxj" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2d94c868ea5so5498215ad.3 for ; Tue, 01 Sep 2026 20:39:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788320345; x=1788925145; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=d/F8fpEp4iGHT0skiRxv7WJG5rGIQnzqOxwAlDHs2fo=; b=BxaE9lxjvUSU/1ZvsSGgwu1Lpb5xoHGZn2p1UzqeTMtUnQb+yJycBNoniGjgkAtRYW E0XCdYd+0PtxF8/pI9e/af7+eIhO21GraYDaDM2vAVwCoNHao8xb+yNci7bcq2gupiSc YZUCwgGqeT8qQcQprLa8iqQrhTAH+FkuPwESMsFgwsatrGec2Ny5dSw4uvpJNGgHUap3 SjH7YahJCJUk2wVEtvlH8E9Tg8bm6/e6r/PaDNOxKDlrCYa3/at2WpaXR46ZYkclwpdE Xk+62tVnOhFvzu6mfmCPBw2NXlCVs12EYNdN2XW2gDqXk+ZCVz35ccCRGi1jfodUsPyC 1uvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788320345; x=1788925145; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=d/F8fpEp4iGHT0skiRxv7WJG5rGIQnzqOxwAlDHs2fo=; b=F2fTOIKCDlTB/7CluaqPcvqiRnuieMJ5o5drc61HwzxXWWGds4XQK0P+J57iPkayg2 QLyEAubfnO/V+MBqc2+ORg/zGoNTMRNsSHEohj39tgazS7pJTsAc8BPeb6QDd4S9t+kR 825Gq3c0NjFqRf5etz47KVsmszmswpP6YZyoM6FWyjJrHIcxSlEUHkpK9NPawg4AB7+A Cf+zJvDuvy0EDpvTheNAHwyFEboBK7R6wCZ0A7wOM7cIuzXoIcya4YkUudhJkKu5qshP TTEcmIy4Ggm+5FaFfXBJIhoHGpzdb5DaDDASyDdsm32ja0D8cEbNjxT5OcDDr0Ya6FgX 2Dyw== X-Forwarded-Encrypted: i=1; AHgh+RqN+IZlqD8y3ws2/Cb87UoxDLsFZ4xZLSKH119v/hrbwD7FNM6xYasxhQtH1TqFk1foMlylnewSjq06sso=@vger.kernel.org X-Gm-Message-State: AFuF++nqbLgwfy+7kwsVOOJf4gU25zk6ytM5DSHY2rSSeQuqOG2cHshG as4JshtcD5aMs6aXXHlQIbtoNwGa7LUf+e0gkYQjDvHwp4pVlrD7s+OF X-Gm-Gg: AYBFou32kskTKIoquE8aa1gC/HcGdqBatVebqJOvMFlef0hyiEGxi0X5bOsPoWzouBO sFPIEcfTOpL2hFyzJql3IiKSNK85HREtIuw7/bmGAYY7ZsbOmzOrFN8awdLt4q1YXV9Xxjpeibx aR2vKFOn+umsbMo2gLDApA77bGG4aik747t9SxpsxidQLAhbva8h7JCUGVS+lvOJh432fK4Y/Qs TcUCQl+Jk8fs+ClYpE7xY8KwKANGb9a0uQGk1v0VEgJTrg3OXbSb7gDQLPVGV6AAKH90kgb2a8B hzySblAHlwV2+8aWG4Xcbu0fJKhs5my03Uz+JJ5X7sDdOLZqvsEVF8E6SBN/ymXLL1DqdD9ksDK wYr2rdaPvC//onLjWLToQZw4ymGVKMf5Hj21yiFQPKwARs2ckVEL+EYN3BDbh3fvKg0YcLyEqQH QppRkJj/NHx4gBEpNQ+QL7OXKxhhmzIE98LtzhCMk0YgqM5Qs2Qe1dZPugGVD8ScVRlXLU8uu/ X-Received: by 2002:a17:903:380d:b0:2d9:51bd:534f with SMTP id d9443c01a7336-2daec6fe7c7mr29796515ad.10.1788320344611; Tue, 01 Sep 2026 20:39:04 -0700 (PDT) Received: from osman.mioffice.cn ([43.224.245.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2dadd386eefsm5715715ad.28.2026.09.01.20.39.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Sep 2026 20:39:04 -0700 (PDT) From: Zhan Xusheng X-Google-Original-From: Zhan Xusheng To: thomas.weissschuh@linutronix.de Cc: tglx@kernel.org, luto@kernel.org, vincenzo.frascino@arm.com, david.laight.linux@gmail.com, zhanxusheng@xiaomi.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 1/4] vdso/math64: Use OPTIMIZER_HIDE_VAR() in __iter_div_u64_rem() Date: Wed, 2 Sep 2026 11:37:58 +0800 Message-ID: <20260902033801.2912699-2-zhanxusheng@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260902033801.2912699-1-zhanxusheng@xiaomi.com> References: <20260902033801.2912699-1-zhanxusheng@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The loop subtracts because the quotient is tiny at every caller, and the asm() keeps the compiler from turning it into a division instead. It offers a memory alternative for the value: asm("" : "+rm"(dividend)); clang takes that alternative and spills the value inside the loop, on 64-bit as well as 32-bit. OPTIMIZER_HIDE_VAR() is the register-only form of the same barrier and is what the rest of the kernel uses for this. The helper sits on the clock_gettime() path through vdso_set_timespec(), so this is measurable: x86 vDSO built with clang 18 loses 64 bytes of vdso64 text and 112 bytes of vdso32. gcc 13 emits the same code either way. Neither compiler turns the loop into a division with the new form, checked at both widths. Signed-off-by: Zhan Xusheng --- include/vdso/math64.h | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/include/vdso/math64.h b/include/vdso/math64.h index 22ae212f8b28..83ebac2e5c1b 100644 --- a/include/vdso/math64.h +++ b/include/vdso/math64.h @@ -8,9 +8,11 @@ __iter_div_u64_rem(u64 dividend, u32 divisor, u64 *remaind= er) u32 ret =3D 0; =20 while (dividend >=3D divisor) { - /* The following asm() prevents the compiler from - optimising this loop into a modulo operation. */ - asm("" : "+rm"(dividend)); + /* + * Prevent the compiler from optimising this loop into a + * modulo operation. + */ + OPTIMIZER_HIDE_VAR(dividend); =20 dividend -=3D divisor; ret++; --=20 2.43.0 From nobody Sat Sep 26 11:47:52 2026 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E817D1F5821 for ; Wed, 2 Sep 2026 03:39:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788320357; cv=none; b=DPgpyyoG3k8FLzkyFh3y9jyFkjgZpi3SpX0Zvk/4TavehkhDHku9hECZatZc9RqUMmTWzSt1lSL4QC2u9Sh5fiM9gqgHBxvEuiW7eaQc/8DzoA8pSevytiUOKox7zVemP5kbTOBxaHj42Dl9FSAGtT6NJszPqeP65J9Eby0vpmw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788320357; c=relaxed/simple; bh=pNDQYWahXAcj04IY+6y1XqOX+1du2ZcEmNJpACTrvps=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qSAGQm90dL9PdozKW/X/ZwpGcSCZk8mqjYt7jh1RwQeymrUH9pw+EQNIYkjfGiLHtYaqVaItvRZhWKo080+ynZJl+I8lHJ0BLcJZvEMlFtYc13oKivR67CbO0K4qxAvz2qZTS0EEStGWd09VHmdyNr3ZijH4mU/1hcj8O4h1iZI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=pLMC+gii; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="pLMC+gii" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-2ce98cb8165so5978865ad.1 for ; Tue, 01 Sep 2026 20:39:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788320355; x=1788925155; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=yNqa9yT4Y/nUgLdm9d2X8y4bUC7AsEowg0WP+KpNCNE=; b=pLMC+gii99FNDsA6DkwpO+P/S97O+i7rx6+jPsYSrc8QWJ+278QMtKeGmmGyZ5Wkw+ 1mvS0JJw/d/eJi3eir/KiadlqH2QrIG7gocxvKZVkcBOl2hm/8URCKqB9SX+/uhhJkne 7cB6pv1zRdo5994BuDc4OZKpvNF4qIk3Y3TJ27j3hAdMOvh4pP3AatnP1n7CYr3ImvyS Kl+m2PlW040cKO1i4XIh94cEI9RyVZEYdZRHD4YSD2bkPqEMeNnuEdtSORjByurrq9GI zILRtGy95p52O3by66lfOxRyw+cfdG1PSlDNb1/fmExqC7uw+8Zccp4eQVXzn1ub1lNK rJIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788320355; x=1788925155; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=yNqa9yT4Y/nUgLdm9d2X8y4bUC7AsEowg0WP+KpNCNE=; b=PAX/mYoJ5XPITqgFiAmpITmHaAoGLyFAMcNGPfhmMbvSBWgZcO3GWIbDY6IVWmj+NF x5rS2Cn7aSdu4xtwZ9iTbWchFt/aLgNkRnlD7cre0JDHErPA92hujbYlKTym1T2KvlxT hATSx9ggPWrMTveP260f8LrlPBvLQaJ65qsfbiRFvs1ndG1DFsv7bRZa8rtdKIGhAjGk 4gIGqKY9cShbv+9sXvv/tnkvauidfRMaiUVnNiut2VAqt3wWQ9MfVRnTyTcZosfHx2q4 +LSBPg8LhAQN9ceWbmNUu6r3LtFNGG7j/9tt7ufS45edfVreUqGqIcQmBh83MgpyxVqX kUKA== X-Forwarded-Encrypted: i=1; AKwUvBzP5Rapt2LId7UXfGGlPaNI/5ev3drsR3BS0BOWi6jBf6M59YPYmbCxOICoejG6vZGmZ74RZbCeSrEDCns=@vger.kernel.org X-Gm-Message-State: AFuF++lmx0mhX0mGh6Yq7BbiWLeVjMKdiD9M3628fA/P5U0uun+Vsf8N lU+9en5csN1+gLAGWJ8cAjUdJOxdPLvafBoyTeKynDkjVOuWirgiyQd8 X-Gm-Gg: AYBFou0t5VyKto8ll1C9CGKkzW0pHj44E1WPQbLTqgx25RwvWzY4qMenyhqCmxnHLE9 mV61//9wzuYkdcYYclmZiXTwS+3lDBmuTPYt22qS52G9jsCzC+f5fdVxXyGShFsEb7i31DTC9/I i686zL9Z9ckrxqYrvG/+TwqsdUurtx5ldrS4BqLWKlPBfDR2riOwOZnwPRQa2yligNkpRws/j/C XkIiiq6kJrOgYWbpMp/SinYUJUXp2xLX1Ag8kZrdtSiuax/cuvT65ovFE+W6o58ro6l93OaM7ox bwAnTEJ+GQwi1tURcom46qyDej8jl0i1XT1RBld5WYYJED8HZv7wWzIB28uLitQgyaWYGHzP6/R j3biYp8r1OC6ZN5K0B7cPn/eG10WJT3kSTkdZJ2UyoqetAP+tPuHrV/EchgXNUr7kF8C5VXdPJf 68mJX0IAgVLtl/dqkdRwzIvmgYcbTlpbJOCaU5rn5ISJSpwLJVFaZgCSo4CVlmahpgoF7ETcRb X-Received: by 2002:a17:902:f687:b0:2d9:14e8:bcb7 with SMTP id d9443c01a7336-2daf125a7d4mr1779085ad.3.1788320355118; Tue, 01 Sep 2026 20:39:15 -0700 (PDT) Received: from osman.mioffice.cn ([43.224.245.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2dadd386eefsm5715715ad.28.2026.09.01.20.39.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Sep 2026 20:39:14 -0700 (PDT) From: Zhan Xusheng X-Google-Original-From: Zhan Xusheng To: thomas.weissschuh@linutronix.de Cc: tglx@kernel.org, luto@kernel.org, vincenzo.frascino@arm.com, david.laight.linux@gmail.com, zhanxusheng@xiaomi.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 2/4] vdso/math64: Add and use __iter_div64_u64_rem() Date: Wed, 2 Sep 2026 11:37:59 +0800 Message-ID: <20260902033801.2912699-3-zhanxusheng@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260902033801.2912699-1-zhanxusheng@xiaomi.com> References: <20260902033801.2912699-1-zhanxusheng@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The vDSO basetimes for CLOCK_MONOTONIC and CLOCK_BOOTTIME are kept in the scaled nanoseconds of tkr_mono, so normalising them means dividing by NSEC_PER_SEC << shift, which does not fit the u32 divisor of __iter_div_u64_rem(). update_vdso_time_data() therefore open-codes the iterative division twice. Subtracting is the right shape here rather than dividing, because the quotient is never more than one: xtime_nsec is kept below one scaled second by accumulate_nsecs_to_secs(), and the offset added to it is a normalised timespec64 fraction, so the dividend stays below twice the divisor. That holds on every architecture, which matters more than what any one of them charges for a division. Add __iter_div64_u64_rem(), the u64-divisor counterpart of __iter_div_u64_rem(), and use it for both. The remainder goes straight into the basetime as the coarse clocks already do, which also makes the copy of the CLOCK_MONOTONIC values for CLOCK_BOOTTIME take both of them from the same place. The quotient is a u32 like the u32-divisor version returns. It keeps 32-bit from carrying the counter in a register pair: vsyscall.o loses 16 bytes of text on x86-64 and the loop drops from 36 to 28 instructions on 32-bit gcc. No functional change. Suggested-by: David Laight Signed-off-by: Zhan Xusheng Reviewed-by: Thomas Wei=C3=9Fschuh --- include/vdso/math64.h | 21 +++++++++++++++++++++ kernel/time/vsyscall.c | 16 +++++----------- 2 files changed, 26 insertions(+), 11 deletions(-) diff --git a/include/vdso/math64.h b/include/vdso/math64.h index 83ebac2e5c1b..0c91410e5f69 100644 --- a/include/vdso/math64.h +++ b/include/vdso/math64.h @@ -23,6 +23,27 @@ __iter_div_u64_rem(u64 dividend, u32 divisor, u64 *remai= nder) return ret; } =20 +static __always_inline u32 +__iter_div64_u64_rem(u64 dividend, u64 divisor, u64 *remainder) +{ + u32 ret =3D 0; + + while (dividend >=3D divisor) { + /* + * Prevent the compiler from optimising this loop into a + * modulo operation. + */ + OPTIMIZER_HIDE_VAR(dividend); + + dividend -=3D divisor; + ret++; + } + + *remainder =3D dividend; + + return ret; +} + #if defined(CONFIG_ARCH_SUPPORTS_INT128) && defined(__SIZEOF_INT128__) =20 #ifndef mul_u64_u32_add_u64_shr diff --git a/kernel/time/vsyscall.c b/kernel/time/vsyscall.c index aa59919b8f2c..f43dd3f4744b 100644 --- a/kernel/time/vsyscall.c +++ b/kernel/time/vsyscall.c @@ -41,14 +41,12 @@ static inline void update_vdso_time_data(struct vdso_ti= me_data *vdata, struct ti =20 nsec =3D tk->tkr_mono.xtime_nsec; nsec +=3D ((u64)tk->wall_to_monotonic.tv_nsec << tk->tkr_mono.shift); - while (nsec >=3D (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) { - nsec -=3D (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift); - vdso_ts->sec++; - } - vdso_ts->nsec =3D nsec; + vdso_ts->sec +=3D __iter_div64_u64_rem(nsec, (u64)NSEC_PER_SEC << tk->tkr= _mono.shift, + &vdso_ts->nsec); =20 /* Copy MONOTONIC time for BOOTTIME */ sec =3D vdso_ts->sec; + nsec =3D vdso_ts->nsec; /* Add the boot offset */ sec +=3D tk->monotonic_to_boot.tv_sec; nsec +=3D (u64)tk->monotonic_to_boot.tv_nsec << tk->tkr_mono.shift; @@ -56,12 +54,8 @@ static inline void update_vdso_time_data(struct vdso_tim= e_data *vdata, struct ti /* CLOCK_BOOTTIME */ vdso_ts =3D &vc[CS_HRES_COARSE].basetime[CLOCK_BOOTTIME]; vdso_ts->sec =3D sec; - - while (nsec >=3D (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) { - nsec -=3D (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift); - vdso_ts->sec++; - } - vdso_ts->nsec =3D nsec; + vdso_ts->sec +=3D __iter_div64_u64_rem(nsec, (u64)NSEC_PER_SEC << tk->tkr= _mono.shift, + &vdso_ts->nsec); =20 /* CLOCK_MONOTONIC_RAW */ vdso_ts =3D &vc[CS_RAW].basetime[CLOCK_MONOTONIC_RAW]; --=20 2.43.0 From nobody Sat Sep 26 11:47:52 2026 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EE732EFDA4 for ; Wed, 2 Sep 2026 03:39:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788320362; cv=none; b=FGO7FXHd3ETgZR6JzyEXYaQRjHRp7Gj7EzouqRJh7eUY+FVpEq18xQ2oHdOanrDSiOCeFuv4Y9fjWIt4PsR1uWfRE3TuKfFEM5YfneEPNtICAIP21gRHci5OI2soqBDfsIRhJGpL86F4OJkwMvIDT4kq/xezrjk+mWCzAGx5PEs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788320362; c=relaxed/simple; bh=A4/1ri5HuIDttueCUT0PCIq2gr18ZPmxmc1yUhOG+m4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=VZ/MKP/yk/LDPH8mYky0iIJXMxB/Qaq1GTdbdn0ZjqUaO7Bxl3NUx0NlJSla+lolKibRt7rDdOXCTzzW49WuRqga9MnaHoZ05885d1L60R4sVMJFVLq2W9RcqCLONNHeZUC39H0xd2684pyabG925Ro9CdNLBwzswUOnS1oTulI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ipK9o7Zx; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ipK9o7Zx" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2d8fd3b729dso4761505ad.1 for ; Tue, 01 Sep 2026 20:39:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788320360; x=1788925160; darn=vger.kernel.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=Hy/ujTz3CUWmNrrscj1BmpB2cqE+O9iKHNdWqpEvyjc=; b=ipK9o7ZxcTeKk/ABGmBzYX/G9K/eHH51lcjfGBn/XEL84jpJczs3vV7HeyTZzah564 DqWtSRvab5Z85A5tldT9Izl1BjkQP1kFMWIxB/6js2QxxrwFCaUYqHGVLxZdj5uTNNet vDyWKZBUrytrMvGrlKf+PyyYOUIdX1UPGLGargyLez88j7y1BRajrevwoD/4vy96XO/v kerOEL08I9FwwDuBCDscdgozoCdElNIb0GrWa12z5G1qJGiR5doCgAvCxH0xqKotJfjc sSdEIl41QiX6i/ZhUPyDm3s/lre9vPnemJuYnsd/tmFCjdKlGVWeQaHMQZgQ9rL98dvM ou1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788320360; x=1788925160; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=Hy/ujTz3CUWmNrrscj1BmpB2cqE+O9iKHNdWqpEvyjc=; b=es67zVLIwVj9I+y2Et46/YdVTg0kX+sO2NeJ6/yalkkz2qCfFhjBCY63QSpGKdVQ8p AIUSC9XuzsiW9c/2S3V0GyFiPhaRms8jw8Ci8vOgRaINF6LgEDDVetRILGv+Eaa49YSU crdd6nBhLI5SN7euLOwlug9l8AHGihpFfxsEgUGXEtnw2NFlLjKRHivbXQ01atbUPhdX XJEEnJAk9PmVKXsxm4fR6HhH8+AeHbR/d0O9/zXU5pXbVUHkggoeFzJnlL2tXc9dGUtg a2i5Tszc8UHA9Uh9Ks6ARugfeo2Vq8KtH9vYe0sgvjAosP3zJMRkQgpn72NWZOxChV+v mUlA== X-Forwarded-Encrypted: i=1; AHgh+RpJa4owg8Flol5Nmv7vgNRI5YYKi0bshnIip9QwGT5C83lg1qmjPVV+3osNZhFXemH0vrUV96xT+vtpmt8=@vger.kernel.org X-Gm-Message-State: AFuF++n+Y3fdXhdYqxKC+a+t/iHEojRSdb47PhoCzDt8TTKLee41kVSI lGgnnQm4J4+U3YbhbnOrA9hiXVfMZ7c/sSt4hTOP0LgQTO15WrQV8FOB X-Gm-Gg: AYBFou094Wd5z+bCa/Xq8EnA7ccfnGe4RZxo6FUVLz6pI1ShilzKo/UrmYPQUEGqy5j kXRgiHET8O2tkjuuU0JtU5r8PJTTOR9iZBvX2af1ImUIyDRJRnnETuOkDck3JSxdVGvVLF23nHn 2Poy3oZbS7/OICyX8yTMdTUp6cofeDpC2YSbFjhATBRO14iIgJLxpA8EDd87usH9fbgSMT+5Gnx NUpDs+jd0QGEtZmnLTKKkO0m9iLj7bm+Dmkw/qPRJSL00bsmemx6Tw50Lie/nbgvb+Ir3Nrxw19 0ESb3aB8l2smnLYbGdt7PAE7c+4Vd+U4eo2LX/Q5O4BaK/t6OvI2WVvg9raBttfFjBApVDzUMT2 K86eYfU897K+5rjN5Srg2AIuBpOxcrwZlni2KEyUR5DIm+JxJdxviqzA7ZubADnluZxGYUIMnnY tSFB7QD2h1irXSUGjRLTIq5NhGRiUYTvR/4SpQvZ8ND57Qcz8NvD7EUZA/oxPeRTUXDCATKPVPV pl8+hEvbMg= X-Received: by 2002:a17:903:3850:b0:2d8:d4d3:da4e with SMTP id d9443c01a7336-2daec772204mr29255265ad.18.1788320360398; Tue, 01 Sep 2026 20:39:20 -0700 (PDT) Received: from osman.mioffice.cn ([43.224.245.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2dadd386eefsm5715715ad.28.2026.09.01.20.39.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Sep 2026 20:39:20 -0700 (PDT) From: Zhan Xusheng X-Google-Original-From: Zhan Xusheng To: thomas.weissschuh@linutronix.de Cc: tglx@kernel.org, luto@kernel.org, vincenzo.frascino@arm.com, david.laight.linux@gmail.com, zhanxusheng@xiaomi.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 3/4] vdso/vsyscall: Keep the CLOCK_AUX base scaled Date: Wed, 2 Sep 2026 11:38:00 +0800 Message-ID: <20260902033801.2912699-4-zhanxusheng@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260902033801.2912699-1-zhanxusheng@xiaomi.com> References: <20260902033801.2912699-1-zhanxusheng@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The vDSO basetime of a clock is stored in the scaled nanoseconds of tkr_mono, so that the reader can floor the base and the cycle delta together in vdso_calc_ns(). vdso_time_update_aux() instead shifts the base down to nanoseconds, adds the offset, and shifts it back up, which zeroes the fractional nanoseconds of xtime_nsec. The reader then floors the base and the delta separately: ktime_get_aux(): base + ((delta * mult + xtime_nsec) >> shift) vdso: base + (xtime_nsec >> shift) + ((delta * mult) >> shift) Since floor(a) + floor(b) <=3D floor(a + b), the vDSO reports 0 or 1 ns below the syscall for the same clock. It is not a monotonicity problem: across an update the step is floor(a + d) - floor(a) - floor(d), which is 0 or 1, never negative. Add the offset in scaled nanoseconds as the other high resolution clocks do, and normalise with __iter_div64_u64_rem() so that the stored base stays below one second and the userspace fast-path does not iterate more in __iter_div_u64_rem(). Only the sub-second field changes. (a + (b << shift)) >> shift is exactly (a >> shift) + b, so the seconds carried out of the normalisation are the same as before; what the old form dropped was the low shift bits of the remainder. monotonic_to_aux.tv_nsec is a normalised timespec64 fraction, so it stays below NSEC_PER_SEC even for a negative offset, and the sum stays below 2 * (NSEC_PER_SEC << shift). The largest shift clocks_calc_mult_shift() can pick is 32, which makes that 8.6e18 against a u64 limit of 1.8e19. Fixes: 380b84e168e5 ("vdso/vsyscall: Update auxiliary clock data in the dat= apage") Signed-off-by: Zhan Xusheng Reviewed-by: Thomas Wei=C3=9Fschuh --- kernel/time/vsyscall.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/kernel/time/vsyscall.c b/kernel/time/vsyscall.c index f43dd3f4744b..0e4b499328c0 100644 --- a/kernel/time/vsyscall.c +++ b/kernel/time/vsyscall.c @@ -155,11 +155,11 @@ void vdso_time_update_aux(struct timekeeper *tk) =20 vdso_ts->sec =3D tk->xtime_sec + tk->monotonic_to_aux.tv_sec; =20 - nsec =3D tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift; - nsec +=3D tk->monotonic_to_aux.tv_nsec; - vdso_ts->sec +=3D __iter_div_u64_rem(nsec, NSEC_PER_SEC, &nsec); - nsec =3D nsec << tk->tkr_mono.shift; - vdso_ts->nsec =3D nsec; + nsec =3D tk->tkr_mono.xtime_nsec; + nsec +=3D (u64)tk->monotonic_to_aux.tv_nsec << tk->tkr_mono.shift; + vdso_ts->sec +=3D __iter_div64_u64_rem(nsec, + (u64)NSEC_PER_SEC << tk->tkr_mono.shift, + &vdso_ts->nsec); } =20 __arch_update_vdso_clock(vc); --=20 2.43.0 From nobody Sat Sep 26 11:47:52 2026 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3ADD82DC792 for ; Wed, 2 Sep 2026 03:39:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788320380; cv=none; b=FEAzCd1sHudBnXUssGrlmsNNNKEZPSzgI2k/taD6eUSjoqREgUt5MhnEEF8A/HT2t4cqTjt+0R6s69yhYyCCcM+MQBKzGtEoLyEO30Rs3lfD1dYNjDKc+XDzAyPWqsQX5mq4kA6ycdukTILKTQpkXX4DRmd18JR8ya7vhhFxjyk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788320380; c=relaxed/simple; bh=bHwVtyoXVu2TVi4pOCw17jGirueig4po4LiRmrpAKas=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QmigFvHLCt2GU+VI12AKDosmmsjNTTHj0CQidFlbERnSiJaRwGO/lx0v/345wIjPx8GRfJGMzwVIg5U/c2TPwLjbDh29F4+h+AlGDiaf+8zI3fTPiTlIfE1K0FzhTiBZ55E8eRr+9L2E4P1+1PuBVb6bl355B0wuyRm4HJKlKZw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=obbS/Zcu; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="obbS/Zcu" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2d91518a63fso4696815ad.0 for ; Tue, 01 Sep 2026 20:39:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788320378; x=1788925178; darn=vger.kernel.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=fqWDWsxv8wlgt756/7Sn//VhWsyTD+w5jVSYYxwonOo=; b=obbS/ZcumqZis68mMXFl5VYXpDX5sffA5FJUifrA+PxMeGVM/MdBppGFiTvBxkggeP DoFyJfCU8zhmZVN7YZQwSMNoK2repkv/0lPB+lwd2ejdhry8af4UBgyKJAAp9YstN8F/ shafxlrEvN0MRLcj5oDqkxP52ylUcWmI9SxST4N/6GYeSZguQOhH8di+MMqW2JPgVadS vELGwzjMw7wuhsVtuJ7Vo7iV+PqXLZUr9ItGBqNXeuJfvGzuBKvZrpMi6blAtT9asB0K JuPa5PL/pVfS7RrhROQxOWbpyf3HPYFD/sZyY+r/qDCIb+YkpW75DBWn0qgRVqk4+Of5 hGsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788320378; x=1788925178; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=fqWDWsxv8wlgt756/7Sn//VhWsyTD+w5jVSYYxwonOo=; b=LB9iT0ZRQM++NURPW4URLoAAKqc7jDj2y6ryU8fT8USZdTvXBj/uAUBLcpY9OQiQj7 rRdplMOcBBIppvAhTnEckE+tqBq5cf+aMkHEUd6V7iBTfK7Yugsd9E3p9zbczstE6KjF i8zjLCGnmzY8qAV+BVoaUMmkPdiNiFBAN9Nbf/x1p5KVnbbgqf6lXYVW6b26b1D4cZ6d n7StoLQrOfZ0eUuuDzkaz2OIYRICr8wp0+LmHEzJf4n7H/Y4WsTMZtr13IqcDKoUq3Mi 3gwot9qoVrj55UxePjO6iAMkCJVM708EZbYRRnbelPiuqdhbdqRZei0DcdgFydEeMSo9 BwDw== X-Forwarded-Encrypted: i=1; AKwUvBwPMO0zuxjBSg/4ZRhRUu0qPBrFZS14Gd5V8xl9EwI4i+NeeCddcQcO7rlDdn7NN9vULDH0nvOZo8/6um0=@vger.kernel.org X-Gm-Message-State: AFuF++kmrLIC12oupcBpxhszs29NW2Ma946Vi4Zn8dkpIw6dwI+F1rlD zHxE28EWPij0Am4FJBVQPAMGeIrvjnLQUrwMYWoJ9ItA1AdMOWChy14m X-Gm-Gg: AYBFou1d01D9gaaB66urLl0HF2/5NQhYPbxX+A20Ccsn4Fe/E8hWarf00xQAdTNYaLe 12sMLCmEoKgEeQX41ZkRd0RidY/hcBJM3FtjPr0HeYKDkfuePJwlowNRMRGIs2YCFZq0Et1FSG5 nIGTHc8rv/et97hOS7LpNHicLXLyYnF7hnmRpHIi40D7evYxiuClZ9nYp1bf74yWG2XghIjXsZQ 8ao5JGhmXQC1PLK1RpA6EzzzqFBqjduKtNr5tq0+PQg+foN56rKncI87t1FJS2mfl9bZUPmdb0W RKyBLFekK9BkEw+UAAB5rD4abwW4764C0Ocf/O7xjhvWH1FJG/yvA1TnbB1xf+IKCFH+tiYtMNp ldKJhfx8GNyZJUNyxkSPfhexc2ie/HkyyeIxGoexAJ/tDn5se7bKXPVPaDOQo1rNGUuX2JtFSH5 BmWoDRj7zW0ALOJ4btyys9yb6kEnVDKZ9jfP/yL/AcnlgI1hNehM4Ve3nk1gU3Fb2SQ6g4s0R+ X-Received: by 2002:a17:903:1a24:b0:2c9:e846:a57e with SMTP id d9443c01a7336-2daec457723mr29222385ad.0.1788320378515; Tue, 01 Sep 2026 20:39:38 -0700 (PDT) Received: from osman.mioffice.cn ([43.224.245.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2dadd386eefsm5715715ad.28.2026.09.01.20.39.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Sep 2026 20:39:38 -0700 (PDT) From: Zhan Xusheng X-Google-Original-From: Zhan Xusheng To: thomas.weissschuh@linutronix.de Cc: tglx@kernel.org, luto@kernel.org, vincenzo.frascino@arm.com, david.laight.linux@gmail.com, zhanxusheng@xiaomi.com, linux-kernel@vger.kernel.org, Zhan Xusheng Subject: [PATCH v4 4/4] vdso/gettimeofday: Assert that the clock id fits the dispatch mask Date: Wed, 2 Sep 2026 11:38:01 +0800 Message-ID: <20260902033801.2912699-5-zhanxusheng@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260902033801.2912699-1-zhanxusheng@xiaomi.com> References: <20260902033801.2912699-1-zhanxusheng@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: Zhan Xusheng From: Zhan Xusheng The clock id dispatch turns the id into a bit in a u32: if (!vdso_clockid_valid(clock)) return false; msk =3D 1U << clock; vdso_clockid_valid() admits everything up to CLOCK_AUX_LAST, which is 23, so the shift is in range. Nothing states the dependency though, and raising MAX_AUX_CLOCKS past 16 would take CLOCK_AUX_LAST to 32 or beyond and make the shift undefined. Assert it at both dispatch sites. The condition is on a parameter rather than a constant, so it relies on the compiler deriving the range from the vdso_clockid_valid() bail-out above it. gcc 13 and clang 18 both do: x86 vdso64 and vdso32 build clean, and raising MAX_AUX_CLOCKS to 17 fails the assert as intended. Suggested-by: Thomas Wei=C3=9Fschuh Signed-off-by: Zhan Xusheng Reviewed-by: Thomas Wei=C3=9Fschuh --- No change since v1. lib/vdso/gettimeofday.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c index f7a591aba59f..ef4dcc614489 100644 --- a/lib/vdso/gettimeofday.c +++ b/lib/vdso/gettimeofday.c @@ -285,6 +285,7 @@ __cvdso_clock_gettime_common(const struct vdso_time_dat= a *vd, clockid_t clock, * Convert the clockid to a bitmask and use it to check which * clocks are handled in the VDSO directly. */ + BUILD_BUG_ON(clock >=3D BITS_PER_TYPE(msk)); msk =3D 1U << clock; if (likely(msk & VDSO_HRES)) vc =3D &vc[CS_HRES_COARSE]; @@ -438,6 +439,7 @@ bool __cvdso_clock_getres_common(const struct vdso_time= _data *vd, clockid_t cloc * Convert the clockid to a bitmask and use it to check which * clocks are handled in the VDSO directly. */ + BUILD_BUG_ON(clock >=3D BITS_PER_TYPE(msk)); msk =3D 1U << clock; if (msk & (VDSO_HRES | VDSO_RAW)) { /* --=20 2.43.0