From nobody Sat Feb 7 16:26:21 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 508B43019D6 for ; Fri, 30 Jan 2026 13:30:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769779810; cv=none; b=fxOnr3SYnab/BDDRaTnTozigjzwj9FFHhRatvDS214KMIHWV4fok2edF4+OlthD/cJdKeOzETXBfC/BlqepoO78TjeMaCIk8cK+6JiZG3kGHTs6BKavD4lerqVz9RLKdvt1evx0j/H3QK/+KbKmOO1pw5bMHzkrFvqUPBaShnAo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769779810; c=relaxed/simple; bh=26nK5ZuTSH07Hc66Z0TGP4MV/KjUVkYvGOV4qaYkpHE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZDrlSpcsU+WeMM64tdTF5gaVK9/RpXzVTlCXl1DPc012MusZXQ3FOsG/qMW4MkhKcL9+kjDuzohCUaW+uTH/ebyRAiNCFengJ12unhot0d4JJzzjneI9B7BGpYA/r3E3Jxrt0/lDNhskfs+JYslvswXfipGpQF0/oS1rLxgg984= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--elver.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=UA/LVt28; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--elver.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="UA/LVt28" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-48025e12b5bso35329785e9.2 for ; Fri, 30 Jan 2026 05:30:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1769779808; x=1770384608; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ah0eS4SkK+p5W7jb3OOf6k8LtqY2IjJqo0gqvKUDvkY=; b=UA/LVt28VdvUGx7LrOwb++LDTM5EKu+uZH2uIkzO9tyHque+VS3R6XSwL4Gi/ORfwv uoB4VCmcBI1bh4PxzvutsKJIWSqTGawLtYm5kSQl3tfb9Y9ydokw4dHPbVZLEa85pMk2 qYTfEdbV5mSFV7HDANrpf49gGzMl3/NTSf1WPv/sEVefwrbhB0DVr+Qe5AFtYWeg2TEV L2JT8KDd0A7CkOAfClsxQl+kI7UpcZOosI8E0D/UncUkVqW2DVxq8ZonHCDUQXwTWGJJ 6Oq2iWMXIkAzgmEWySvdTyWreHsTub9NFEcI2uzQyddDDn7+mwO0jkgnA6I3T0vGYxEe fGiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769779808; x=1770384608; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ah0eS4SkK+p5W7jb3OOf6k8LtqY2IjJqo0gqvKUDvkY=; b=dn9QvBObHcPcVRWOp4cpaiQDoXczU/eyungiIdxMdyv/EZEHNvOAMB/CxD5ooGriG5 6vFs69S8H7HELHm6+8l97hlU9pZ+Fhi0LsWO5CDMmx03/e7yrFf51gk04hQfHvyBiqNC YP2+gFM9gh3eeK+sgcDFWpkbDddiKVDH9V7bWfxV/h9ciDts4CxRhp7VbT0/+0gJokBA QL2FjL2cjZAMED5VFeKkrDP4OtOW/MGygMEoAzdGJYZ1ktuILdPeJThDTngyu3eYB81u 3/UeoA0rh8eiCBO9ttQp61j5S4JldhBM7mBABfWsealMsPsvZB8IfXu5uSi1pbp8JplL He/g== X-Forwarded-Encrypted: i=1; AJvYcCXaeKPHqY7NRlO6QZlLQZArHN25SjdZq/hKOYYJnMyjGcVxWlwAnhbwzCQTo//nqykIyErbyoux5Yajpbw=@vger.kernel.org X-Gm-Message-State: AOJu0YyJ8YtElCg6rje3e3bFZH5MKiXjEcnqPgMSQ1n+WseSCZOh3aJr +2thI5cnL9PAAOGglVVJqyb68Qs8TNhQ7h+HHmWJduFaNGkS8zX4Dm9FQGZmm9UhxJKD7y+VrE3 YSw== X-Received: from wmby7.prod.google.com ([2002:a05:600c:c047:b0:477:a678:a39a]) (user=elver job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3b1d:b0:477:9cdb:e32e with SMTP id 5b1f17b1804b1-482db46014dmr37732115e9.9.1769779807742; Fri, 30 Jan 2026 05:30:07 -0800 (PST) Date: Fri, 30 Jan 2026 14:28:24 +0100 In-Reply-To: <20260130132951.2714396-1-elver@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260130132951.2714396-1-elver@google.com> X-Mailer: git-send-email 2.53.0.rc1.225.gd81095ad13-goog Message-ID: <20260130132951.2714396-2-elver@google.com> Subject: [PATCH v3 1/3] arm64: Fix non-atomic __READ_ONCE() with CONFIG_LTO=y From: Marco Elver To: elver@google.com, Peter Zijlstra , Will Deacon Cc: Ingo Molnar , Thomas Gleixner , Boqun Feng , Waiman Long , Bart Van Assche , llvm@lists.linux.dev, David Laight , Catalin Marinas , Arnd Bergmann , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, Boqun Feng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The implementation of __READ_ONCE() under CONFIG_LTO=3Dy incorrectly qualified the fallback "once" access for types larger than 8 bytes, which are not atomic but should still happen "once" and suppress common compiler optimizations. The cast `volatile typeof(__x)` applied the volatile qualifier to the pointer type itself rather than the pointee. This created a volatile pointer to a non-volatile type, which violated __READ_ONCE() semantics. Fix this by casting to `volatile typeof(*__x) *`. With a defconfig + LTO + debug options build, we see the following functions to be affected: xen_manage_runstate_time (884 -> 944 bytes) xen_steal_clock (248 -> 340 bytes) ^-- use __READ_ONCE() to load vcpu_runstate_info structs Fixes: e35123d83ee3 ("arm64: lto: Strengthen READ_ONCE() to acquire when CO= NFIG_LTO=3Dy") Cc: Reviewed-by: Boqun Feng Signed-off-by: Marco Elver Tested-by: David Laight --- arch/arm64/include/asm/rwonce.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/rwonce.h b/arch/arm64/include/asm/rwonc= e.h index 78beceec10cd..fc0fb42b0b64 100644 --- a/arch/arm64/include/asm/rwonce.h +++ b/arch/arm64/include/asm/rwonce.h @@ -58,7 +58,7 @@ default: \ atomic =3D 0; \ } \ - atomic ? (typeof(*__x))__u.__val : (*(volatile typeof(__x))__x);\ + atomic ? (typeof(*__x))__u.__val : (*(volatile typeof(*__x) *)__x);\ }) =20 #endif /* !BUILD_VDSO */ --=20 2.53.0.rc1.225.gd81095ad13-goog From nobody Sat Feb 7 16:26:21 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E4D0F2FD68A for ; Fri, 30 Jan 2026 13:30:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769779813; cv=none; b=hGTNoMMaLnoi8VUGz64HSgqB2MCfNCH9kFYG2Zmnj5YR19eBjsSTC8Hzs+at4+PfxtGojMuOOnkDelDKZhu5p3ofe+G3uKqaM2o35YbcVKZF/FfjsWROXguMLbkGzf6aVhKyhnM13rvBg2vem3qz1QmKjKRTUzi2wAursZfadps= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769779813; c=relaxed/simple; bh=N+0RvBUJ08uz7gY4C8XhNnF6MHXEg5OROSx72CVegc8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Y5x0Kz6PfkIaSVHjIVhR4wiiaee2sGUQk2yQmgs6JNopUHQN/BfkVfw7h0iZvl234tsMgqrx24eghc148Bi+/UQCI6x2Ft1tWnjDmeXLAk9sysfMF703C/Gm3GSa8nqwnYG+u3chCkRIXDZIVEEqTWNw9o0SLLdQTu+tv9pSUOo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--elver.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CrWts64V; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--elver.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CrWts64V" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-4806fce5ab1so12721475e9.2 for ; Fri, 30 Jan 2026 05:30:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1769779810; x=1770384610; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=scSVMH9or2TMfZesMIFkbkxALtorvxefZYfnvMnEqKA=; b=CrWts64VZJgm0kzxO2nxJv9/1VtuOxgxvZ4808HOro7Fy4o8qGcs1kDkMkkgU7gAkT AxCjj1QMjnMtIFSYSYSs44Woqfk7YSgVEzYa2dPTh8NlJTOeas6UeeWZia1mPRLIW0aX Itj/UG9twZj7mdj4xUjWB7iPAmzHH4NYt47aXgpMREQKmEYKGqKhF8wh7BNDLJhIwA4+ Xb9rNzf9D4yqLyLX88LWKGqZwdd+JoGfpKAxcPauuPAB9HJvM7CHofPBALDMtrzxupXG ZYkTpwgU1YgNi5HNOHos/Bf4FOOIzKT90t7RKvgKCnQZXzJKRJuaovoXpqVxGqOTncVF 8iEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769779810; x=1770384610; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=scSVMH9or2TMfZesMIFkbkxALtorvxefZYfnvMnEqKA=; b=EqoIt2F97rxOT3Cc96tDLSJvyZGqbx0G8X+VWJOCCsuDNxa17poOjc1cMFgnernTMU F0g2rg4Ep/IxavA5Gl8DCkJvUsI+fnAq59Pw8kV41kNW6Oiu0/yu6Ja4Th5bIWapPAC8 8HE8S1TZluC0MeLx0qhZQ8UDY1vs0tyNOCkYP3z1l5uK9Rv0uf0NSVZEHCDHPxr/nly/ 1DRdSR1a1HZY+FsXH6CoPMW0aZI5lADx8tmJiLNpwGsJZg1tqsOD+hCSMBdCaFRytwCP IqWAvYwCaR4nITFFyb8Wrhk+YMZjRDabA6rLf83Kzi/68frt/QBMksA5yQoq7IuWDx+G 416A== X-Forwarded-Encrypted: i=1; AJvYcCUKS0KGLobJCn0U9DAfoWzeVS/gKLR33n/PenvgQYQaj4489yg3Q+3I+qpuwUKGRcbbxs792GUqa8l9V2c=@vger.kernel.org X-Gm-Message-State: AOJu0Yzl62yFWYnR9ZD1SKjLfYKTs/KhIiy49ZLcGTVAykfcpo84tAHG EVnAYBdYoenBW5n4fJJdkmykbT8A/SaQe6+4cCcu6/pA1eR32sT2XVC7CGR5YKmS7Px+Xq0ecF7 YOQ== X-Received: from wmpx5.prod.google.com ([2002:a05:600c:1885:b0:480:4a03:7b7f]) (user=elver job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4507:b0:475:dcbb:7903 with SMTP id 5b1f17b1804b1-482db45ea8emr33261705e9.9.1769779810315; Fri, 30 Jan 2026 05:30:10 -0800 (PST) Date: Fri, 30 Jan 2026 14:28:25 +0100 In-Reply-To: <20260130132951.2714396-1-elver@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260130132951.2714396-1-elver@google.com> X-Mailer: git-send-email 2.53.0.rc1.225.gd81095ad13-goog Message-ID: <20260130132951.2714396-3-elver@google.com> Subject: [PATCH v3 2/3] arm64: Optimize __READ_ONCE() with CONFIG_LTO=y From: Marco Elver To: elver@google.com, Peter Zijlstra , Will Deacon Cc: Ingo Molnar , Thomas Gleixner , Boqun Feng , Waiman Long , Bart Van Assche , llvm@lists.linux.dev, David Laight , Catalin Marinas , Arnd Bergmann , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rework arm64 LTO __READ_ONCE() to improve code generation as follows: 1. Replace _Generic-based __unqual_scalar_typeof() with more complete __rwonce_typeof_unqual(). This strips qualifiers from all types, not just integer types, which is required to be able to assign (must be non-const) to __u.__val in the non-atomic case (required for #2). Once our minimum compiler versions are bumped, this just becomes TYPEOF_UNQUAL() (or typeof_unqual() should we decide to adopt C23 naming). Sadly the fallback version of __rwonce_typeof_unqual() cannot be used as a general TYPEOF_UNQUAL() fallback (see code comments). One subtle point here is that non-integer types of __val could be const or volatile within the union with the old __unqual_scalar_typeof(), if the passed variable is const or volatile. This would then result in a forced load from the stack if __u.__val is volatile; in the case of const, it does look odd if the underlying storage changes, but the compiler is told said member is "const" -- it smells like UB. 2. Eliminate the atomic flag and ternary conditional expression. Move the fallback volatile load into the default case of the switch, ensuring __u is unconditionally initialized across all paths. The statement expression now unconditionally returns __u.__val. This refactoring appears to help the compiler improve (or fix) code generation. With a defconfig + LTO + debug options builds, we observe different codegen for the following functions: btrfs_reclaim_sweep (708 -> 1032 bytes) btrfs_sinfo_bg_reclaim_threshold_store (200 -> 204 bytes) check_mem_access (3652 -> 3692 bytes) [inlined bpf_map_is_rdonly] console_flush_all (1268 -> 1264 bytes) console_lock_spinning_disable_and_check (180 -> 176 bytes) igb_add_filter (640 -> 636 bytes) igb_config_tx_modes (2404 -> 2400 bytes) kvm_vcpu_on_spin (480 -> 476 bytes) map_freeze (376 -> 380 bytes) netlink_bind (1664 -> 1656 bytes) nmi_cpu_backtrace (404 -> 400 bytes) set_rps_cpu (516 -> 520 bytes) swap_cluster_readahead (944 -> 932 bytes) tcp_accecn_third_ack (328 -> 336 bytes) tcp_create_openreq_child (1764 -> 1772 bytes) tcp_data_queue (5784 -> 5892 bytes) tcp_ecn_rcv_synack (620 -> 628 bytes) xen_manage_runstate_time (944 -> 896 bytes) xen_steal_clock (340 -> 296 bytes) Increase of some functions are due to more aggressive inlining due to better codegen (in this build, e.g. bpf_map_is_rdonly is no longer present due to being inlined completely). Signed-off-by: Marco Elver Reviewed-by: David Laight @gmail.com --- v3: * Comment. v2: * Add __rwonce_typeof_unqual() as fallback for old compilers. --- arch/arm64/include/asm/rwonce.h | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/rwonce.h b/arch/arm64/include/asm/rwonc= e.h index fc0fb42b0b64..42c9e8429274 100644 --- a/arch/arm64/include/asm/rwonce.h +++ b/arch/arm64/include/asm/rwonce.h @@ -19,6 +19,20 @@ "ldapr" #sfx "\t" #regs, \ ARM64_HAS_LDAPR) =20 +#ifdef USE_TYPEOF_UNQUAL +#define __rwonce_typeof_unqual(x) TYPEOF_UNQUAL(x) +#else +/* + * Fallback for older compilers (Clang < 19). + * + * Uses the fact that, for all supported Clang versions, 'auto' correctly = drops + * qualifiers. Unlike typeof_unqual(), the type must be completely defined= , i.e. + * no forward-declared struct pointer dereferences. The array-to-pointer = decay + * case does not matter for usage in READ_ONCE() either. + */ +#define __rwonce_typeof_unqual(x) typeof(({ auto ____t =3D (x); ____t; })) +#endif + /* * When building with LTO, there is an increased risk of the compiler * converting an address dependency headed by a READ_ONCE() invocation @@ -32,8 +46,7 @@ #define __READ_ONCE(x) \ ({ \ typeof(&(x)) __x =3D &(x); \ - int atomic =3D 1; \ - union { __unqual_scalar_typeof(*__x) __val; char __c[1]; } __u; \ + union { __rwonce_typeof_unqual(*__x) __val; char __c[1]; } __u; \ switch (sizeof(x)) { \ case 1: \ asm volatile(__LOAD_RCPC(b, %w0, %1) \ @@ -56,9 +69,9 @@ : "Q" (*__x) : "memory"); \ break; \ default: \ - atomic =3D 0; \ + __u.__val =3D *(volatile typeof(*__x) *)__x; \ } \ - atomic ? (typeof(*__x))__u.__val : (*(volatile typeof(*__x) *)__x);\ + __u.__val; \ }) =20 #endif /* !BUILD_VDSO */ --=20 2.53.0.rc1.225.gd81095ad13-goog From nobody Sat Feb 7 16:26:21 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ADCCC30EF9A for ; Fri, 30 Jan 2026 13:30:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769779816; cv=none; b=a9vBIdMcWMFcxnTvyc1HgChEK8YEsLrpkI3EG3D0YSQpY2CBemp9FwkVwUzLcs7QGnkskzdqCSn3Bo1O3R69PZ02MPNXOcehw0NT672/df9N9wAAdUzWRX0QmlgL+gKw9QTK8CP5ysSlwkLxqtJqqvNF1IwY4+0gghnzF5Y60sA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769779816; c=relaxed/simple; bh=T4dPZtG1oh8ppeo/D7aOCiu7eQficsG0EdVSGiJeJNI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=MTvFA2NzPLi4zczyqTqROGxZcmD6K7P0E1NpGmqRtJcmyuKKjbBOcDs0ujGT4eK+a5O7+TP81JMQqC6YGh+ZTYXOQAWtG8hs0HCuT/zg8QM0dfWpTDSZjK0j2zln4chs0/kM+Ez2huuFQT9ojlb5JK1c1d7c4JhyuUcVfJMbzxQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--elver.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ZT+oddGn; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--elver.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ZT+oddGn" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-480711d09f1so34382875e9.0 for ; Fri, 30 Jan 2026 05:30:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1769779813; x=1770384613; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=EIZZgTWN+WVkGCe2DzZUi2LmQSShjbtn6Lr33NLpRdc=; b=ZT+oddGn6nO6KfS3mkRwqNfPmD3kf6tfALKXEEX6Dq4j238QromjIPEUeFn+sfCkcq V6GGwxS0UPLI+Ucd+ejYe4shR8VZSzibzCin/ChG5AjM9PeJAUuG+fgaQVcGdqh3az5D rQgAdw4pRRtoM6tu9DhsJlvEI15UbuoUWEflI5fo0QFzDwpeXGjjGTZvY6jI8b0vR/DK O6Xrm2fSo9CX3g1UrzP/SDuWowGekHZ3+1Clxlhr2P6Ie7SbBWuJlroDCOPSfys44v1U IzP+pnrRsD0fG6OtfWi6Vl3UHTjkTVTwg1kwRG9Vpn5tebkxcHZx1hiLP55QLj5xwAp8 pvgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769779813; x=1770384613; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=EIZZgTWN+WVkGCe2DzZUi2LmQSShjbtn6Lr33NLpRdc=; b=Y5MlEZb8CHj+SFbNLdYcj11m0+nW9aVXrQx2UsDWsEdx0Fv0nUatTMBD6/a7IEFzuG bonTVbBcxdLyGeAoY0xEbp5k6eyOWVUGugIOhWGkEdDDybki1CyWJRiQNJh/smLwDd0S k+9ibjpWVaG0PBmjeQQk7R8Gq3MY3KnSyp4qz+66ccarL8UBdzTzN6qkI7WmEhiAHmuG MVB9LTe2s4vTyOeoOmfS190DHLm51fna8VVme9BuRC2DzFECFXwMRFB1HZfguW4piLk5 XsYP2YQ767OvKcAB9oL8SHU3yWqrfLVj0XJEPInlOxZrGxyttnalo1vXaJ+3D5vYvOIL DyOg== X-Forwarded-Encrypted: i=1; AJvYcCU7QAUYizxAUl8+jqgCQueJT6AoilAexaEuUEY9UngA4zrj1jE4+WIjoiNl3irzfIsL/WEe4wi9+IOS6Ic=@vger.kernel.org X-Gm-Message-State: AOJu0YwVHV2GDqfqtkr1f584bq43QilsZPyXNtkacsGeDD6gbiTXTkxV BAqWzpywHzONQ8LuaROsgf6noWoMWDb7zBoMvCgU6RkollgHXPnnmJBA7HouWvw6gDYNEPNQ8lk /Sg== X-Received: from wmot9.prod.google.com ([2002:a05:600c:4509:b0:477:7aa2:99cb]) (user=elver job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:c4a3:b0:480:6c75:ddce with SMTP id 5b1f17b1804b1-482db49da16mr40851245e9.33.1769779813044; Fri, 30 Jan 2026 05:30:13 -0800 (PST) Date: Fri, 30 Jan 2026 14:28:26 +0100 In-Reply-To: <20260130132951.2714396-1-elver@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260130132951.2714396-1-elver@google.com> X-Mailer: git-send-email 2.53.0.rc1.225.gd81095ad13-goog Message-ID: <20260130132951.2714396-4-elver@google.com> Subject: [PATCH v3 3/3] arm64, compiler-context-analysis: Permit alias analysis through __READ_ONCE() with CONFIG_LTO=y From: Marco Elver To: elver@google.com, Peter Zijlstra , Will Deacon Cc: Ingo Molnar , Thomas Gleixner , Boqun Feng , Waiman Long , Bart Van Assche , llvm@lists.linux.dev, David Laight , Catalin Marinas , Arnd Bergmann , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kernel test robot , Boqun Feng Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When enabling Clang's Context Analysis (aka. Thread Safety Analysis) on kernel/futex/core.o (see Peter's changes at [1]), in arm64 LTO builds we could see: | kernel/futex/core.c:982:1: warning: spinlock 'atomic ? __u.__val : q->loc= k_ptr' is still held at the end of function [-Wthread-safety-analysis] | 982 | } | | ^ | kernel/futex/core.c:976:2: note: spinlock acquired here | 976 | spin_lock(lock_ptr); | | ^ | kernel/futex/core.c:982:1: warning: expecting spinlock 'q->lock_ptr' to b= e held at the end of function [-Wthread-safety-analysis] | 982 | } | | ^ | kernel/futex/core.c:966:6: note: spinlock acquired here | 966 | void futex_q_lockptr_lock(struct futex_q *q) | | ^ | 2 warnings generated. Where we have: extern void futex_q_lockptr_lock(struct futex_q *q) __acquires(q->lock_ptr= ); .. void futex_q_lockptr_lock(struct futex_q *q) { spinlock_t *lock_ptr; /* * See futex_unqueue() why lock_ptr can change. */ guard(rcu)(); retry: >> lock_ptr =3D READ_ONCE(q->lock_ptr); spin_lock(lock_ptr); ... } At the time of the above report (prior to removal of the 'atomic' flag), Clang Thread Safety Analysis's alias analysis resolved 'lock_ptr' to 'atomic ? __u.__val : q->lock_ptr' (now just '__u.__val'), and used this as the identity of the context lock given it cannot "see through" the inline assembly; however, we want 'q->lock_ptr' as the canonical context lock. While for code generation the compiler simplified to '__u.__val' for pointers (8 byte case -> 'atomic' was set), TSA's analysis (a) happens much earlier on the AST, and (b) would be the wrong deduction. Now that we've gotten rid of the 'atomic' ternary comparison, we can return '__u.__val' through a pointer that we initialize with '&x', but then update via a pointer-to-pointer. When READ_ONCE()'ing a context lock pointer, TSA's alias analysis does not invalidate the initial alias when updated through the pointer-to-pointer, and we make it effectively "see through" the __READ_ONCE(). Code generation is unchanged. Link: https://lkml.kernel.org/r/20260121110704.221498346@infradead.org [1] Reported-by: kernel test robot Closes: https://lore.kernel.org/oe-kbuild-all/202601221040.TeM0ihff-lkp@int= el.com/ Cc: Peter Zijlstra Tested-by: Boqun Feng Signed-off-by: Marco Elver Reviewed-by: David Laight --- v3: * Use 'typeof(*__ret)'. * Commit message. v2: * Rebase. --- arch/arm64/include/asm/rwonce.h | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/rwonce.h b/arch/arm64/include/asm/rwonc= e.h index 42c9e8429274..b7de74d4bf07 100644 --- a/arch/arm64/include/asm/rwonce.h +++ b/arch/arm64/include/asm/rwonce.h @@ -45,8 +45,12 @@ */ #define __READ_ONCE(x) \ ({ \ - typeof(&(x)) __x =3D &(x); \ - union { __rwonce_typeof_unqual(*__x) __val; char __c[1]; } __u; \ + auto __x =3D &(x); \ + auto __ret =3D (__rwonce_typeof_unqual(*__x) *)__x; \ + /* Hides alias reassignment from Clang's -Wthread-safety. */ \ + auto __retp =3D &__ret; \ + union { typeof(*__ret) __val; char __c[1]; } __u; \ + *__retp =3D &__u.__val; \ switch (sizeof(x)) { \ case 1: \ asm volatile(__LOAD_RCPC(b, %w0, %1) \ @@ -71,7 +75,7 @@ default: \ __u.__val =3D *(volatile typeof(*__x) *)__x; \ } \ - __u.__val; \ + *__ret; \ }) =20 #endif /* !BUILD_VDSO */ --=20 2.53.0.rc1.225.gd81095ad13-goog