From nobody Fri Sep 25 06:47:06 2026 Received: from mail-pz2-f42.google.com (mail-pz2-f42.google.com [74.125.228.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C8BC1E9B37 for ; Wed, 16 Sep 2026 02:33:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.228.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789525994; cv=none; b=k5mVmdN04uYutSVBXrsNa+ZV8+dwZ9cbvvu4SGFIwJT2z5jDNK6yYNVSj0BsIopNiU/Ka3QA7iwErvyQO6oP0Z8v9zeONRhVSzQXUxHvR3+D8tlHCil1OAihprI1XSaGswN9eYfn71lDoLv5A6dPKM3aCJrGI2+qlxhCI0KS/8g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789525994; c=relaxed/simple; bh=GQbBa6CHnNfUb9DKjXxRdQQLXR1WhYhhE4Qswr/Ui+o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kYuGrcj0v9y3CzF1f/uD9zzwwEUW2Q8/PQZT0P8uwOZh8b8+0WgYoV4D75ktcekjv7Imr4busQYsEZbQPkgCleg1wGb+T4b8Ce0ZrKwxbH9GvI6JZQ61QvBAQgd3kXw+AJS2uqoUQ6GP42E3nTnWWgbKyvlZ+eGz2d5fpU2QrfU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=sAf6Bz1n; arc=none smtp.client-ip=74.125.228.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="sAf6Bz1n" Received: by mail-pz2-f42.google.com with SMTP id d2e1a72fcca58-86dd69a1b15so96651b3a.0 for ; Tue, 15 Sep 2026 19:33:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789525986; x=1790130786; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=xza0m8WSgHtBFmr77jh/0SIGm5QzN/lBTrXqXqcFKCY=; b=sAf6Bz1n9fg7IovQQ4DpS1KmMTzJyOJRDOepKys7rHh0eAPAVmxULsMimoaNQ+5Y5V FJ5xD95oHqbb+zV5u8GO3YVS+6vT1Zy+H2TB/K76X+6goCXraoJvt7QJvFOVazlYIb01 7GsbAxdSz2zKM8Lc3peqxXIcIfp9b/9hROU+A6m8FbQl5DgxKyEgPnTiNR+UZrlk/lbO 3+4/jrufCTUFxhc+Iufmy34af+smlWzk/Oa3FH2LZJXwVU0z+5aDJlvPhm9jB/1iPqmg OGCpIYZDcgVrUccYNUjPPmzONc1gIQ76K5OuX1fXudZdhVv4HQ3U8TW8M85Ec4tZaAhP jg8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789525986; x=1790130786; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=xza0m8WSgHtBFmr77jh/0SIGm5QzN/lBTrXqXqcFKCY=; b=DS2htlbH/60dvAIgaoI546bWIasbunLHtPdB65lECYyZrz5WsLX2aYrFcbM7VLpn0q nJH1+FVxeFxCF8AZ5UduDbxY/B9kGz4jyNTmv6sXLP8kgxTh2jh35ImA9BHTdf8wjXNL CnUrMSCsSy+1rUE4QxDHRGv3RRnGwhxuG9rX8LmiKUtxqWS902U6aQS3PZmfqpDW+6IE kGQU3BG6v6a2B6RI2X8LOnGZj1+YS7en6UWGbZj8pgGqBFj9J2KHoSbtw5svz/CQGaAS pPc2dGwiLNuikdsjN0xL0iFfd0eUZvcIGPfFrhz/b3ixHBul5Og/Z9aC+fhTJnypRDuG uWpw== X-Forwarded-Encrypted: i=1; AKwUvBx0DqKZPOKnLeQk6CKmEQyOYU1uoBPtfcb+FTGSUrgWjDX/YnBf7xs70/StEjhMX/R0G6zqJYUU3HvFRyU=@vger.kernel.org X-Gm-Message-State: AFuF++kqvi1HQpxmpMOdy9ZU/LgV6LIrIG3a5A3awRLepw+B5JU6uh8k /syC5bGdDFlHcGENl1LQ6Ww5Q1FaUZ+3aTexJe6Go/UBBOhkCs5UN7K1 X-Gm-Gg: AYBFou0WLv132ivQqSjCJY/w8LC78v5kpBKGosAHvBE8pdGC8tRjsImhZH/UuG3agPt 0l9wdFyhfasa5M695gZc5BExWtI6mtX+ztWCxpW4IrphcJSfOo6LjuNS8Q0dY6/5oyuCaMFP1xV 1C3gOi3eKwVvWMKQgaW4T9k0vyCx+3YC2mkr4FjwbFO28L0oBdHuPAq5p1abkF5ev/KMFBTOrTg c3DDv3IYs5FBfyko+UNqzBvJbXDBa2EL63Iiu+3ygwmbFnoVsOfHu4Sx2OOga6mTM5W+inGgqt/ kitS46gUZRLhlWE3IKk1JAqvJZ6gQpOL16kSJiGIdsu8qDQVKmTYw414MSc0qLWzJVRcRL4BY4K yaDoTmtjlfNnjhOoST171b0Cel1mL2aqvuz0eONlhRwI9MJKBfHe3i6LHMragbSC3BVgykIs8/V ftM6GtgSPqgtV8E0KElJ1+3MNhZ2EIFiGaLvRmJVS4GpKAiJ4D7/Ry6FAuVoCC2Hm5Lw/wY+xvS /cIR/HFdXrE X-Received: by 2002:a05:6a00:1308:b0:869:86ae:e94f with SMTP id d2e1a72fcca58-87232a6fe6amr942116b3a.4.1789525986447; Tue, 15 Sep 2026 19:33:06 -0700 (PDT) Received: from osman.mioffice.cn ([43.224.245.178]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-8720123e374sm381370b3a.24.2026.09.15.19.33.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Sep 2026 19:33:05 -0700 (PDT) From: Zhan Xusheng X-Google-Original-From: Zhan Xusheng To: tglx@kernel.org, thomas.weissschuh@linutronix.de Cc: luto@kernel.org, vincenzo.frascino@arm.com, david.laight.linux@gmail.com, linux-kernel@vger.kernel.org, zhanxusheng@xiaomi.com Subject: [PATCH v5 1/4] vdso/math64: Use OPTIMIZER_HIDE_VAR() in __iter_div_u64_rem() Date: Wed, 16 Sep 2026 10:32:49 +0800 Message-ID: <20260916023252.418473-2-zhanxusheng@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260916023252.418473-1-zhanxusheng@xiaomi.com> References: <20260916023252.418473-1-zhanxusheng@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" __iter_div_u64_rem() divides by repeated subtraction, because its callers only ever produce a small quotient. A barrier inside the subtraction loop keeps the compiler from replacing the loop with a division: asm("" : "+rm"(dividend)); The "rm" constraint permits a memory operand. clang picks it and spills the dividend inside the loop, at 32-bit and 64-bit alike. The helper sits on the clock_gettime() fast path through vdso_set_timespec(), so the spill is not free: with clang 18 the x86 vDSO is 64 bytes larger in vdso64 text and 112 bytes larger in vdso32 text than with a register-only barrier. Use OPTIMIZER_HIDE_VAR(), which is the register-only form of the same barrier and the form the rest of the kernel uses. gcc 13 generates identical code either way, and neither compiler turns the loop into a division at either width. Put the function signature on one line while touching it, and reformat the comment, which described the asm() that is now gone. Signed-off-by: Zhan Xusheng --- include/vdso/math64.h | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/include/vdso/math64.h b/include/vdso/math64.h index 22ae212f8b28..c628d6cf447c 100644 --- a/include/vdso/math64.h +++ b/include/vdso/math64.h @@ -2,15 +2,16 @@ #ifndef __VDSO_MATH64_H #define __VDSO_MATH64_H =20 -static __always_inline u32 -__iter_div_u64_rem(u64 dividend, u32 divisor, u64 *remainder) +static __always_inline u32 __iter_div_u64_rem(u64 dividend, u32 divisor, u= 64 *remainder) { u32 ret =3D 0; =20 while (dividend >=3D divisor) { - /* The following asm() prevents the compiler from - optimising this loop into a modulo operation. */ - asm("" : "+rm"(dividend)); + /* + * Prevent the compiler from optimising this loop into a + * modulo operation. + */ + OPTIMIZER_HIDE_VAR(dividend); =20 dividend -=3D divisor; ret++; --=20 2.43.0 From nobody Fri Sep 25 06:47:06 2026 Received: from mail-pz2-f41.google.com (mail-pz2-f41.google.com [74.125.228.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF85539B4A5 for ; Wed, 16 Sep 2026 02:33:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.228.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789526000; cv=none; b=m46JiVRSV3B0h/p93Hb/wP/cOHRZ80vOjm2VFK6RLhqDRxgsGUPweFF4WaonTzRvuxpUEKzpbRNR0Y1nZlN78REA4FOmTf+2kF9YV9g/al5K3j72F6yH/TSMC0XUWAxAGBmIB+Sg/ghmWviMlwgEpGJKV3FYQtyjWEdWVltXAAc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789526000; c=relaxed/simple; bh=NAqWj+ZTOvNmujxsklsafUndubiCMRUwPApWoeVVm38=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=cbjQh6c3SmsR/Lhu12XxkkPuQVMvqS7RfvcUtW9jP7cnlo+xiCaSSB2uGnvp64beADOk8QygPAg7ym8L02WeT0/LwtLhstvnd2sFuWzk1LhbccsX/mkV7iQKO6uPg94kAezX77v6kLlicP2t0XDaJnLw50RK/HtW2Onm4Q6Chv8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=I9yw/Tq+; arc=none smtp.client-ip=74.125.228.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="I9yw/Tq+" Received: by mail-pz2-f41.google.com with SMTP id d2e1a72fcca58-8692a856865so325717b3a.2 for ; Tue, 15 Sep 2026 19:33:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789525994; x=1790130794; darn=vger.kernel.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=q5QrDeVRwtO7tHo5nqKxHXoOWqHMvo8ILiu8uEBK/AE=; b=I9yw/Tq+PhvepGdCBPzzSvxw1IhTqlTi+6KpZMDxLCrBFhjeAIIhV8cjPazpjA7hc+ eZq9F3qbaL+zTXVHzBHz/w9xhG8HBZPCQcn8TVGg7FwPYJjYMX/KrBfl/eTI7UwXfE5b x4us1KTqvnsYNaCPGZoCXBKewc1PrXgdcjIHSwn2RLKODunyyoQH4w2AXxTTdXEF8DcM 7c2LXlhbpsksXHtXZUMqoKoxRmq1nGCPJZHHsUIgsosTgS7JdrqtmPN7o0AvZnPzK8to JggGkRJc+VZQ893YnAXmRsa1qjrXnp0u/JOaWPRCeEGdcNRy92EdiXAAdm/muR8Y7Rfk BHqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789525994; x=1790130794; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=q5QrDeVRwtO7tHo5nqKxHXoOWqHMvo8ILiu8uEBK/AE=; b=U5RV/5K0U5gQiDhl8VJL7lIy5E6HLjDynZTIa/AflIfpE1+AxOcxM/fuHJohtCp89M uDqyNzORTyjPSHlNwVtVIXwdX0HrGsrHCZUHGIDrlBogNB/E3eMWQN9wuy6+4EIN64YL N68rGqG+YlLxy8lYQpAT2D3HCJ6zxycAhj93XZnMuYpjtxp2mjJUVQMQYCDIlmks5kU6 nsdCnISm2ECipWp5RoMGtgC/x0HHJPDWD6s/LP2YRwGcRJH60sLXdFu0lalGcjLgMBAb 7027yqctNP0dWRO3ULK3oEXYxl9Tc5hMNCcRO4WBWcJHhpN3JZrXFr7nbKMAeQcyAup3 Xy6Q== X-Forwarded-Encrypted: i=1; AKwUvByTkzKkMLFF8zdspxnWaf+0CvmF3+hwx9GJ4ZInmdHEfvne3cqmu7lI41T3td6VcoM1q4nF4NakKHyCzu4=@vger.kernel.org X-Gm-Message-State: AFuF++niP+2GvInlOHWzBoQNGpUTUqkqxMmFS3Pyr+4KKx4BD9UUNQei yJ6NHUcPGpX/cDgoagji/HPnTmJwNXSy0ZU3SwDGW7j5QzpNb095F+7p X-Gm-Gg: AYBFou1TGpYeaIu1N0s5SXByDFBSLThrku5d1jzE2GA1yysG5zDZ4BRXB20BoII9NDK axSJ+AfJe2Nd2fI7oIPb0eGF7I/KCBiGBcgGR0t50efcJgLa6m2NL3XjEcVSc0MH9S81VDtffgy e2jv/ylduPNva72GBX/z4tYW2z51WrK7SjjbmA3z3JlTeVcZrBIPTIgh072dvCgSo61fxPYi7YY GJdKUt2VnGWDkjeLI8NvhfJjDa4rmaZE6caaHONaL5DNEfXs+7WwnIylTh+PFPV+QtrDlPJIWx7 xmNCC4H+8NVhFco3WE+ioKmetjTlpYGpdxTKkGKC36pGQ4EfCSV9vwEBjPFVYsfwzBnwNWZadHp wy4l52kR3xHItRysfq7HH/TPWWzSFPBiX4N4TylFzAafk+deZtQjS+rGjigk443yYzobihfKqwl /dSs8J+VMBQWbAzjO66vVtLwEjDPdJwJRtIFQJ8JkY4brkxjU+53eeFs5qXYFnyt2Gn88aIn0pm mRWR1w7i+U= X-Received: by 2002:a05:6a00:bd8a:b0:857:72ba:ff0e with SMTP id d2e1a72fcca58-8723998c23bmr1471536b3a.22.1789525993907; Tue, 15 Sep 2026 19:33:13 -0700 (PDT) Received: from osman.mioffice.cn ([43.224.245.178]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-8720123e374sm381370b3a.24.2026.09.15.19.33.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Sep 2026 19:33:13 -0700 (PDT) From: Zhan Xusheng X-Google-Original-From: Zhan Xusheng To: tglx@kernel.org, thomas.weissschuh@linutronix.de Cc: luto@kernel.org, vincenzo.frascino@arm.com, david.laight.linux@gmail.com, linux-kernel@vger.kernel.org, zhanxusheng@xiaomi.com Subject: [PATCH v5 2/4] vdso/math64: Add and use __iter_div64_u64_rem() Date: Wed, 16 Sep 2026 10:32:50 +0800 Message-ID: <20260916023252.418473-3-zhanxusheng@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260916023252.418473-1-zhanxusheng@xiaomi.com> References: <20260916023252.418473-1-zhanxusheng@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The vDSO basetimes for CLOCK_MONOTONIC and CLOCK_BOOTTIME are stored in the scaled nanoseconds of tkr_mono, so normalising them requires a division by NSEC_PER_SEC << shift. That divisor does not fit the u32 parameter of __iter_div_u64_rem(), so update_vdso_time_data() open-codes the same iterative division twice. Repeated subtraction is the appropriate form at these two sites because the quotient never exceeds one. accumulate_nsecs_to_secs() keeps xtime_nsec below one scaled second, and the offset added to it is a normalised timespec64 fraction, so the dividend stays below twice the divisor. That bound holds on every architecture, which matters more here than the cost of a division on any particular one. Add __iter_div64_u64_rem(), the u64-divisor counterpart of __iter_div_u64_rem(), and use it at both sites. Return the quotient as a u32 for consistency with the u32-divisor variant; the bound above leaves no use for a wider type. Store the remainder directly into the basetime, as the coarse clocks already do. The CLOCK_BOOTTIME copy of the CLOCK_MONOTONIC values then takes both fields from the same place. Folding the two open-coded loops into one inlined helper removes 16 bytes of vsyscall.o text on x86-64 with gcc 13. No functional change. Suggested-by: David Laight Signed-off-by: Zhan Xusheng Reviewed-by: Thomas Wei=C3=9Fschuh --- include/vdso/math64.h | 20 ++++++++++++++++++++ kernel/time/vsyscall.c | 16 +++++----------- 2 files changed, 25 insertions(+), 11 deletions(-) diff --git a/include/vdso/math64.h b/include/vdso/math64.h index c628d6cf447c..55b45f5cf615 100644 --- a/include/vdso/math64.h +++ b/include/vdso/math64.h @@ -22,6 +22,26 @@ static __always_inline u32 __iter_div_u64_rem(u64 divide= nd, u32 divisor, u64 *re return ret; } =20 +static __always_inline u32 __iter_div64_u64_rem(u64 dividend, u64 divisor,= u64 *remainder) +{ + u32 ret =3D 0; + + while (dividend >=3D divisor) { + /* + * Prevent the compiler from optimising this loop into a + * modulo operation. + */ + OPTIMIZER_HIDE_VAR(dividend); + + dividend -=3D divisor; + ret++; + } + + *remainder =3D dividend; + + return ret; +} + #if defined(CONFIG_ARCH_SUPPORTS_INT128) && defined(__SIZEOF_INT128__) =20 #ifndef mul_u64_u32_add_u64_shr diff --git a/kernel/time/vsyscall.c b/kernel/time/vsyscall.c index aa59919b8f2c..f43dd3f4744b 100644 --- a/kernel/time/vsyscall.c +++ b/kernel/time/vsyscall.c @@ -41,14 +41,12 @@ static inline void update_vdso_time_data(struct vdso_ti= me_data *vdata, struct ti =20 nsec =3D tk->tkr_mono.xtime_nsec; nsec +=3D ((u64)tk->wall_to_monotonic.tv_nsec << tk->tkr_mono.shift); - while (nsec >=3D (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) { - nsec -=3D (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift); - vdso_ts->sec++; - } - vdso_ts->nsec =3D nsec; + vdso_ts->sec +=3D __iter_div64_u64_rem(nsec, (u64)NSEC_PER_SEC << tk->tkr= _mono.shift, + &vdso_ts->nsec); =20 /* Copy MONOTONIC time for BOOTTIME */ sec =3D vdso_ts->sec; + nsec =3D vdso_ts->nsec; /* Add the boot offset */ sec +=3D tk->monotonic_to_boot.tv_sec; nsec +=3D (u64)tk->monotonic_to_boot.tv_nsec << tk->tkr_mono.shift; @@ -56,12 +54,8 @@ static inline void update_vdso_time_data(struct vdso_tim= e_data *vdata, struct ti /* CLOCK_BOOTTIME */ vdso_ts =3D &vc[CS_HRES_COARSE].basetime[CLOCK_BOOTTIME]; vdso_ts->sec =3D sec; - - while (nsec >=3D (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) { - nsec -=3D (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift); - vdso_ts->sec++; - } - vdso_ts->nsec =3D nsec; + vdso_ts->sec +=3D __iter_div64_u64_rem(nsec, (u64)NSEC_PER_SEC << tk->tkr= _mono.shift, + &vdso_ts->nsec); =20 /* CLOCK_MONOTONIC_RAW */ vdso_ts =3D &vc[CS_RAW].basetime[CLOCK_MONOTONIC_RAW]; --=20 2.43.0 From nobody Fri Sep 25 06:47:06 2026 Received: from mail-pz2-f41.google.com (mail-pz2-f41.google.com [74.125.228.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D5BE3793D0 for ; Wed, 16 Sep 2026 02:33:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.228.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789526007; cv=none; b=oj4XO9jdpCMoBAwpNoGb5joZq2q17qggv3kLqnijYr+Lf+Q3naoEwlIGjidjR+80LZ1n+Oq58EPV7IxECFTVKplfGA7bTnW7ab51NYpRuxlrxm1gwuBAlag8tGrglRGj5Lb4+vwoOEMXgYi4R1y9QHHHhtTta3hoSbQxpXKhdpQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789526007; c=relaxed/simple; bh=A4/1ri5HuIDttueCUT0PCIq2gr18ZPmxmc1yUhOG+m4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UfSNUTHJqxo7s+3AkQEWaCg8haJbWchsqfc934+8gVCUshOI0yEqfHWvQCA3tbHBZRcGc7jG9DqRkeUbwaXOZMDAWCi3XMPxvvCUfBK6l8fgfLlihZPoWNlJCzTn6mUdC8Hyr14TpwucAVP/IVjt5UFtoe5HFVU0kzHBbrHGVEw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GhILKFaO; arc=none smtp.client-ip=74.125.228.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GhILKFaO" Received: by mail-pz2-f41.google.com with SMTP id d2e1a72fcca58-868a9c48f9eso465805b3a.3 for ; Tue, 15 Sep 2026 19:33:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789525999; x=1790130799; darn=vger.kernel.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=Hy/ujTz3CUWmNrrscj1BmpB2cqE+O9iKHNdWqpEvyjc=; b=GhILKFaOwy7m4lMN7/Y+MdvbuAMCmjjooo0UB9lSdESekAW0HCpimwQdWhprJyxoxK 2pLLy/pxOVEPCt7TTxPOZEhHWixsKGW9G0aT8uLtZFcZYUaHGA5ePREwaNh5FrdBhVbe PLWJL86A2yaUzlNjP/uop6sKbrak9cm96UmhZpmkJkRwdpuO2NCtUhy5C7NXwk1kAnsI p4p8COfLCe3+qXKB2j3XKmXMR796qqfKKheUchb5EKs7b8sSL5xAsvW7uvJl8GGzngsj pvqDaqwN3c0CHO8HGnqZgqGha6XXvJVyBjes2k/RbxEepRX7Laai4RYal+DL7UsCDogO 5uFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789525999; x=1790130799; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=Hy/ujTz3CUWmNrrscj1BmpB2cqE+O9iKHNdWqpEvyjc=; b=2Vmfe48u4qbWWuLGrj2CXi5XjbgW3ZbA4cmJ5W0gFy8MX0Kj+pvPrXjJKFD0Y1x/fv GkhFHrbtsSDOJEr6fbL07g2uq2Mm+C3IWL6mgflzO+jNrm5LQiY9FTeGaH/xXLqAawRa o2yyzhi4OthrEKJrkwcZMaiDTLxFXsg1uXBA5dzs5xruujDOpVa8JhMYUbeNVzlUjiC7 MSCPGKDW5XLE+/bS1pAZ24z7y7AvcleLFdecZFGJnHfGlzxRh/5Kef9bLU3LnaXAbIxT SYv1DMrC3r585pZmDhsflvJEgxNwW/Hkwu/TJPixOM8XNpj5N9M+mNXKHDryyLH62XLR Legw== X-Forwarded-Encrypted: i=1; AKwUvBxws3CXCddMaSj9am04Sv+UH5HOrWQd3J/NgIxlqtwFaRCoBsprzLehCPqkMET+Y6mUJYSLQHlQ9UaRJV0=@vger.kernel.org X-Gm-Message-State: AFuF++kMcuSZqVDcSLozDgY7VFuSHlQaZehXD7WTQnbJ01nJpbBmZo73 MJwoQIp6xJnA/7hZ0b8ruHflsVpUgAuEFrVHxe2laphxbsS+76kE4IZD X-Gm-Gg: AYBFou2qowZUffkr7iM6etlCE6NwAMVROpq6wBD5BGzlhgtjmAC7yEvEgCRrbpEUcO2 B2G/bOSX5zdS0yTAjCBq7HLQvq3uD5U47tiQ5WsQPS3vb9i5du2cE/DfTbULnPJsvHWfm97aytB WPfLiuJ1xcemTv4+MSX7n16tl5fWCFtJTqq/D/ELHmE1fUhMp8kWEajelbneImVThRZmZyHVIpM +LXFZBHp8877Z1m8GdD0FUmrNFzFZSNkrRwW9IMMgkfrQD6REWoOGYaor4a9zcw6YMqXVdyq5Gj XLrFWrjAsZJRlV6YMqjT0vJxMYUr/JVb63V76O4GvxQEkdK6qOo4FTbva7KAKFK7+tYVlm4fVYA PrBTJkyRZaaD6oMIMnivErI1ak5JkLS6ZXI5J6rxTlTmyptp5j4j4yq8cetxaLYbxDRrFxQahiR gvwZ7XmFKK/DzFxDHo23Nu0SeQAbmssAJX0onycX+axLWwQzWgccLFJ2VObdZ3Wu1IMbG6lvJVd BKMKXpao/k= X-Received: by 2002:a05:6a00:9296:b0:871:41fa:d1d6 with SMTP id d2e1a72fcca58-8723cf31f63mr1313742b3a.14.1789525999470; Tue, 15 Sep 2026 19:33:19 -0700 (PDT) Received: from osman.mioffice.cn ([43.224.245.178]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-8720123e374sm381370b3a.24.2026.09.15.19.33.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Sep 2026 19:33:19 -0700 (PDT) From: Zhan Xusheng X-Google-Original-From: Zhan Xusheng To: tglx@kernel.org, thomas.weissschuh@linutronix.de Cc: luto@kernel.org, vincenzo.frascino@arm.com, david.laight.linux@gmail.com, linux-kernel@vger.kernel.org, zhanxusheng@xiaomi.com Subject: [PATCH v5 3/4] vdso/vsyscall: Keep the CLOCK_AUX base scaled Date: Wed, 16 Sep 2026 10:32:51 +0800 Message-ID: <20260916023252.418473-4-zhanxusheng@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260916023252.418473-1-zhanxusheng@xiaomi.com> References: <20260916023252.418473-1-zhanxusheng@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The vDSO basetime of a clock is stored in the scaled nanoseconds of tkr_mono, so that the reader can floor the base and the cycle delta together in vdso_calc_ns(). vdso_time_update_aux() instead shifts the base down to nanoseconds, adds the offset, and shifts it back up, which zeroes the fractional nanoseconds of xtime_nsec. The reader then floors the base and the delta separately: ktime_get_aux(): base + ((delta * mult + xtime_nsec) >> shift) vdso: base + (xtime_nsec >> shift) + ((delta * mult) >> shift) Since floor(a) + floor(b) <=3D floor(a + b), the vDSO reports 0 or 1 ns below the syscall for the same clock. It is not a monotonicity problem: across an update the step is floor(a + d) - floor(a) - floor(d), which is 0 or 1, never negative. Add the offset in scaled nanoseconds as the other high resolution clocks do, and normalise with __iter_div64_u64_rem() so that the stored base stays below one second and the userspace fast-path does not iterate more in __iter_div_u64_rem(). Only the sub-second field changes. (a + (b << shift)) >> shift is exactly (a >> shift) + b, so the seconds carried out of the normalisation are the same as before; what the old form dropped was the low shift bits of the remainder. monotonic_to_aux.tv_nsec is a normalised timespec64 fraction, so it stays below NSEC_PER_SEC even for a negative offset, and the sum stays below 2 * (NSEC_PER_SEC << shift). The largest shift clocks_calc_mult_shift() can pick is 32, which makes that 8.6e18 against a u64 limit of 1.8e19. Fixes: 380b84e168e5 ("vdso/vsyscall: Update auxiliary clock data in the dat= apage") Signed-off-by: Zhan Xusheng Reviewed-by: Thomas Wei=C3=9Fschuh --- kernel/time/vsyscall.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/kernel/time/vsyscall.c b/kernel/time/vsyscall.c index f43dd3f4744b..0e4b499328c0 100644 --- a/kernel/time/vsyscall.c +++ b/kernel/time/vsyscall.c @@ -155,11 +155,11 @@ void vdso_time_update_aux(struct timekeeper *tk) =20 vdso_ts->sec =3D tk->xtime_sec + tk->monotonic_to_aux.tv_sec; =20 - nsec =3D tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift; - nsec +=3D tk->monotonic_to_aux.tv_nsec; - vdso_ts->sec +=3D __iter_div_u64_rem(nsec, NSEC_PER_SEC, &nsec); - nsec =3D nsec << tk->tkr_mono.shift; - vdso_ts->nsec =3D nsec; + nsec =3D tk->tkr_mono.xtime_nsec; + nsec +=3D (u64)tk->monotonic_to_aux.tv_nsec << tk->tkr_mono.shift; + vdso_ts->sec +=3D __iter_div64_u64_rem(nsec, + (u64)NSEC_PER_SEC << tk->tkr_mono.shift, + &vdso_ts->nsec); } =20 __arch_update_vdso_clock(vc); --=20 2.43.0 From nobody Fri Sep 25 06:47:06 2026 Received: from mail-pz2-f43.google.com (mail-pz2-f43.google.com [74.125.228.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C62235677D for ; Wed, 16 Sep 2026 02:33:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.228.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789526012; cv=none; b=WzvOf7y+VPVJqL6JrDkRNvoptZvgJQzRO0+ijx5Oq7mEIv+4cvwHVG8Sy2VCpyIKlbFg5Ieah/1+eRDZQ5D+YRNTb49oyN2XfV0RHRqNe3GL35dmVx1Rxu4GRQV0/bjkfVLAYByBgbnRXH9YTwOk6NzBHvoq7RxptdG2OKOro3Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789526012; c=relaxed/simple; bh=aKuxJWlKl+EhB5MpSS714HjjKEL4RuX7c5AfRZCYPnY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=tEbUYEgXllGEsahGEdWem5YZA6/8oOT3n7Hqq+sXeFNPxts/xYM0GyN18FKDHeicBs2XrT2G1IjlF8YTzXiKzFcEIm1yqNGfWwtm1rUf7/M9xIjpFuGunykqzqgBKP6KJREtTqdqLhgGc8UBLrKUCxwILeJ6Q7rBknp0BjQYE/w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IWYCnC6r; arc=none smtp.client-ip=74.125.228.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IWYCnC6r" Received: by mail-pz2-f43.google.com with SMTP id d2e1a72fcca58-86212a185dcso408509b3a.1 for ; Tue, 15 Sep 2026 19:33:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789526004; x=1790130804; darn=vger.kernel.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=2pl4XdnoNke2GAPJ4SfLM15DICEb57hSAl0sGeGEkfs=; b=IWYCnC6rsze8RKj+n+weYU6RuYTlBaYnt33cuwm+YUVQJG9Moofbl90+n96iYI2KHv /UCleOUUyThEQq6BSIvgMgYJqitteEBcdVWKgthf+MEkxhAAT/xBAmRUwHKSIufl2PTm 1+DrfzFUNCpJw060rUd+wcD2LYWdf2O+2shyZV/sFE3cfdRpxTKJmVClOfnCwXjNkq8y jRexlp0YChoKeUz3FY6ux7KIfmbJQ3nryKbRXTb9guvFCMuQ3upLz4SSiTLsGDOuHp1i aRQ+BuvzsiW4NorZB5eTjrKaZnodG9EuIq49SqhlIojqd43oSJCS4ytZRNNSTrPXr05q DZqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789526004; x=1790130804; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=2pl4XdnoNke2GAPJ4SfLM15DICEb57hSAl0sGeGEkfs=; b=LE8tCs+q45MR0wgLN3SkoBqYShCKQocWGSsoiVTlPtufZFnxFoGvaWFr+nSJjSn35k cAH7QWQAscfLqw6Sx3fETH2c1lv8RifU8Ply95qDuUSVZlzajJSq6SNaIRdq+p3uT9uc EJeCpkdqRFCmI2QTLgR+xQDTY03/yaCfoelXFv43kFZNg5HAWtFhq5oA/EV7l+6SfljM YNqqk41TW0oEiH1ekN1rdCkUGqY+/bWBJ+IZUBcS0VXcjMxparpFqo8iCsH13vMTv/ee VW5FILx1PdiEMHe9v+sWAKRoKCJnv56eDLTgB2aAUuBFz+5NCOdpGNlTgn0JezpAEoh5 9Pag== X-Forwarded-Encrypted: i=1; AKwUvBwD+nhTz/XnuoD8wqk/qABgLbe6tcsehEwHbnZ1CejigFlx3oAaaiZm6N6JeDPDPsuPUNbE38yc/NVlNTE=@vger.kernel.org X-Gm-Message-State: AFuF++liivNyrm9Diz2cydcb2BEOjw+/BSFhklZU/w4NyYF4JiG758gd TSo2KjLXwSrzyQgkR+FlyVQ5owi+3N8Cjp/+BAa9DjEyaf/9U8XQyZoX X-Gm-Gg: AYBFou36+x2zI7zMae6fORc3vY0wbJxaMSv6o/niZCB7ayxIQ1Ayl2NNXVhZJSigK0m XoPCNc6JcnDfprCQUmMAD+lDJT7/A/T16izJ6MDoL2IFtuOtOseQnVdPb4O4dyuuOOx5+2m0Jrt 061daA8i46x83QjrR7uxepeWr6+FD6Wp1pIyNOviTK+3Vu8wNnO9wkS24pMhgWUUbGJJTfuhD7Y XogYJguPdxRLhdt+QBs/SHGYkj6NW+MM4JF+m9wwxFR0TOkBjyQy8qLabZqaGXy1ina2aagQYer ePMeaB5IKZaxH5Lvd5txu1xNlB89G+1nujzrshqJESWXexobQ8tedAm+CvKY8fXCH0VxhRKvNET tS+hNO4JDvYrMVgwMXNawvl1ua85upHQOZVr3oaJS7lIn8n/+8btB4c/FGAgHEALGrocOS3JgSn ngNB6Kn+Jg+9V5FUT1rAI10swDzmuIzETuN4GRRrE9vk0V2mgDGJ6oBIc409DELaSWc3IOZ6PHn KHTaQMUYxQW X-Received: by 2002:a05:6a00:4613:b0:85f:3d78:79cc with SMTP id d2e1a72fcca58-8723b87e98fmr1325730b3a.9.1789526004220; Tue, 15 Sep 2026 19:33:24 -0700 (PDT) Received: from osman.mioffice.cn ([43.224.245.178]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-8720123e374sm381370b3a.24.2026.09.15.19.33.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Sep 2026 19:33:23 -0700 (PDT) From: Zhan Xusheng X-Google-Original-From: Zhan Xusheng To: tglx@kernel.org, thomas.weissschuh@linutronix.de Cc: luto@kernel.org, vincenzo.frascino@arm.com, david.laight.linux@gmail.com, linux-kernel@vger.kernel.org, zhanxusheng@xiaomi.com Subject: [PATCH v5 4/4] vdso/gettimeofday: Assert that the clockid fits into the u32 bitmask Date: Wed, 16 Sep 2026 10:32:52 +0800 Message-ID: <20260916023252.418473-5-zhanxusheng@xiaomi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260916023252.418473-1-zhanxusheng@xiaomi.com> References: <20260916023252.418473-1-zhanxusheng@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable __cvdso_clock_gettime_common() and __cvdso_clock_getres_common() convert the clockid into a bitmask and match it against VDSO_HRES, VDSO_COARSE, VDSO_RAW and VDSO_AUX: msk =3D 1U << clock; vdso_clockid_valid() rejects anything above CLOCK_AUX_LAST beforehand, and CLOCK_AUX_LAST is 23, so the shift count is in range. Nothing records that dependency though. Raising MAX_AUX_CLOCKS beyond 16 moves CLOCK_AUX_LAST to 32 and makes the shift undefined. Add a BUILD_BUG_ON() at both conversion sites. The condition is on a function parameter rather than a constant, so it relies on the compiler deriving the range from the vdso_clockid_valid() bail-out above it. gcc 13 and clang 18 both do: the x86 vdso64 and vdso32 builds stay clean, and raising MAX_AUX_CLOCKS to 17 trips the assert. Suggested-by: Thomas Wei=C3=9Fschuh Signed-off-by: Zhan Xusheng Reviewed-by: Thomas Wei=C3=9Fschuh --- lib/vdso/gettimeofday.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c index f7a591aba59f..ef4dcc614489 100644 --- a/lib/vdso/gettimeofday.c +++ b/lib/vdso/gettimeofday.c @@ -285,6 +285,7 @@ __cvdso_clock_gettime_common(const struct vdso_time_dat= a *vd, clockid_t clock, * Convert the clockid to a bitmask and use it to check which * clocks are handled in the VDSO directly. */ + BUILD_BUG_ON(clock >=3D BITS_PER_TYPE(msk)); msk =3D 1U << clock; if (likely(msk & VDSO_HRES)) vc =3D &vc[CS_HRES_COARSE]; @@ -438,6 +439,7 @@ bool __cvdso_clock_getres_common(const struct vdso_time= _data *vd, clockid_t cloc * Convert the clockid to a bitmask and use it to check which * clocks are handled in the VDSO directly. */ + BUILD_BUG_ON(clock >=3D BITS_PER_TYPE(msk)); msk =3D 1U << clock; if (msk & (VDSO_HRES | VDSO_RAW)) { /* --=20 2.43.0