From nobody Tue Apr 7 07:29:18 2026 Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9CC3F286A9 for ; Sun, 15 Mar 2026 00:50:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773535807; cv=none; b=GQxfXoj2l9IN7SEL+jLdIZBvdqS9DdEvYeFkhyOxHnq1MYw640c1KBSXnccD+P3BdZ6DlR99CxYOArp7sV0lnXIXGKB4pa9c7d3ZlVQeyw6RtL2AHUl12UpI6+tXjM6Bl65RG8AvSv0ItBcIW22o2KNWUL3zedISKIrCP0esAzk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773535807; c=relaxed/simple; bh=0C7kqFzQL1mCdVnf0+mlCBvN4N36107hmCJ3qd5gXlM=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition; b=UmkPmIneYN16NLjDPdYlxp3PY40oWDo9+Dr7ss6rgHypMCco/bCO/nJCuYhEeZBwIcW99VC2KTm5TwtQFq9ZxHH+av9mh2ic/wWeTF7FhRijHYqdfvr+LfaOvlPW6qiOGiHUezX2/ILF6dCMQkVo/pdbtXLFqZlZJIMYEWWh4O8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Eqry9V1X; arc=none smtp.client-ip=209.85.215.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Eqry9V1X" Received: by mail-pg1-f172.google.com with SMTP id 41be03b00d2f7-c73aabd620bso2297106a12.1 for ; Sat, 14 Mar 2026 17:50:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1773535806; x=1774140606; darn=vger.kernel.org; h=user-agent:content-transfer-encoding:content-disposition :mime-version:mail-followup-to:message-id:subject:cc:to:from:date :from:to:cc:subject:date:message-id:reply-to; bh=XpbOikVku5YR5vNSA6pCrMtta1WFwEavqnarNyqEQ2k=; b=Eqry9V1X3aiQ+GzLT9sqSof1Uwy7jKI/TViJmc37Wy9WGiS+CAmL5Xseo8BLnDfAYv o5J4BLgnWDA4Tyzuz7ALl8x1E5YFm3bk8pfaOKr+jRSASst6qatnhgKdUsVXx9tvFX1w 0Gys4QNyoxiDOLKZLDzpJAcouq2l9c1IK3zV5SAkjl4GbZQU/jq1fon+VzZHzeLAYPaM 0tpm5xdWDSUGUsOX1MfNrwNEa+oQiItorcyl43Ah4Ld//93rg/+htDq0hqjoZOyLvdQo qbce6ZlGzWPS9w4nDMoAHyY03RWA9JgO9aG0MytUnVIpdLgzhB1yXaOxT56ZxLE/7db/ HyqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773535806; x=1774140606; h=user-agent:content-transfer-encoding:content-disposition :mime-version:mail-followup-to:message-id:subject:cc:to:from:date :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XpbOikVku5YR5vNSA6pCrMtta1WFwEavqnarNyqEQ2k=; b=Jvny67n4eFXkUe9c85/KVjrFUSZ5lbJeu881LubyusPnRRUcSFn8JoxVVx51IYCr/N xZFWJW2xVdlsX72lxoXw0eO2otA7tWM0E76LNtK1PDQq8ONON8JIRP52Z85ZGOTIgv5I JmkVlyzA2EfT9grKY5k6r7cZMdaxyarydudA23zu9qFwWiuDKx35bpph+LBwZEOXXlcp 7zv+CSZYFrHeLsDT1yo/uA4kkHF7OX1eWcrKgK1nFDU8D2JZu1AjZloOq+xCCt8wCgl4 B/qe4ak5pW9Ve+gsrtuB7irQDoi5mY9M/TtyxmWTFquVM3hUdY7IuHsNJS0EnxJh7B5O PaYg== X-Forwarded-Encrypted: i=1; AJvYcCVGT1uxnJ8Xmf63Oj5UIyTNRBZPDKP5JT/wbz/Jhko3WeOG3VmhzTx306dZU92ShQ/ddfBgql23UaRRcT0=@vger.kernel.org X-Gm-Message-State: AOJu0Yyk04HMms0qtqFOXfJATEpUyGiCE85kZuVD3qfeJcrAZonwlXyG h0J2ikbAQKkspGsXqn3o1S6aXAJOR9XvhrdSZ3Ry8ssR1xw/UwjB+hkPvGCr++A1Tu8= X-Gm-Gg: ATEYQzx6YEkPilwohvHXcrAtsrzxRDMY7dw6/fetOze2QU/7bD0KmT+2hjQbzpPZTRz 1/S0XAnkWVJjSMgxtjvY3TyeXMK39S9ttPlgJ5CK26pMkU+euPwIKHfE5BilbT+RvWo5gNWZPTV hGJMdlznI1lsltL9cHDL68cmvivWuFT6w5CVRgCT/W9INjEGZqlonTZMl9irUAESY3hgWat/QuR u4fpgeJO1NSdlVTXbFpIUFl+dgHHsLy/T/F5LdMSTQv8tjL1Laca5i1aHPd4w5bsY5HehDFyfnq h5ZLcYqJbmwSLF0F4H73N0R04wgnZG7mKgfKABmbPexIoPLIddGcWpkEc5ak2WgBBQVPhh7Ui0Z 6hemebayG6dNc3fE91J/bnHgqtcze4UykS0ow5cqHt7NIcSQXXvhUZv5UhHni/qA8x1rDYNpdc8 kffY6HIZGlMy7xiF52MS5ykIsGHmFp0dXe6Z233Ww4 X-Received: by 2002:a05:6a21:3212:b0:398:a6d4:dbf7 with SMTP id adf61e73a8af0-398ecd98afbmr7285057637.55.1773535805842; Sat, 14 Mar 2026 17:50:05 -0700 (PDT) Received: from udknight.localhost ([120.36.202.188]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c7402214e1csm66228a12.33.2026.03.14.17.50.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 14 Mar 2026 17:50:05 -0700 (PDT) Received: from udknight.localhost (localhost [127.0.0.1]) by udknight.localhost (8.14.9/8.14.4) with ESMTP id 62F0lmet032279; Sun, 15 Mar 2026 08:47:48 +0800 Received: (from root@localhost) by udknight.localhost (8.14.9/8.14.9/Submit) id 62F0lkjo032278; Sun, 15 Mar 2026 08:47:46 +0800 Date: Sun, 15 Mar 2026 08:47:46 +0800 From: Wang YanQing To: linux@armlinux.org.uk Cc: akpm@linux-foundation.org, willy@infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v2] arm: lpae: fix non-atomic page table entry update issue Message-ID: <20260315004746.GA32062@udknight> Mail-Followup-To: Wang YanQing , linux@armlinux.org.uk, akpm@linux-foundation.org, willy@infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Mutt/1.7.1 (2016-10-04) The ARM Architecture Reference Manual explicitly dictates that writes of 64= -bit translation table descriptors must be single-copy atomic: ARM Architecture Reference Manual ARMv7-A and ARMv7-R edition (https://deve= loper.arm.com/documentation/ddi0406/latest) " ... A3.5.3 Atomicity in the ARM architecture ... In an implementation that includes the Large Physical Address Extension, LD= RD, and STRD accesses to 64-bit aligned locations are 64-bit single-copy atomic as seen by translation table walks = and accesses to translation tables. Note The Large Physical Address Extension adds this requirement to avoid the nee= d for complex measures to avoid atomicity issues when changing translation table entries, without creating = a requirement that all locations in the memory system are 64-bit single-copy atomic. This addition means: =E2=80=A2The system designer must ensure that all writable memory locations= that might be used to hold translations, such as bulk SDRAM, can be accessed with 64-bit single-copy atomicity. =E2=80=A2Software must ensure that translation tables are not held in memor= y locations that cannot meet this atomicity requirement, such as peripherals that are typically accessed using a narrow= bus. This requirement places no burden on read-only memory locations for which r= eads have no side effects, since it is impossible to detect the size of memory accesses to such locations. ... " ARM Architecture Reference Manual for A-profile architecture (https://devel= oper.arm.com/documentation/ddi0487/latest) " ... E2.2.1 Requirements for single-copy atomicity In AArch32 state, the single-copy atomic PE accesses are: =E2=80=A2All byte accesses. =E2=80=A2All halfword accesses to halfword-aligned locations. =E2=80=A2All word accesses to word-aligned locations. =E2=80=A2Memory accesses caused by LDREXD and STREXD instructions to double= word-aligned locations. LDM, LDC, LDRD, STM, STC, STRD, PUSH, POP, RFE, SRS, VLDM, VLDR, VSTM, and = VSTR instructions are executed as a sequence of word-aligned word accesses. Each 32-bit word access is guarante= ed to be single-copy atomic. The architecture does not require subsequences of two or more word accesses fro= m the sequence to be single-copy atomic. LDRD and STRD accesses to 64-bit aligned locations are 64-bit single-copy a= tomic as seen by translation table walks and accesses to translation tables. ... " To archieve this 64-bit atomicity on a 32-bit architecture, the linux kerne= l relies on the STRD (Store Register Dual) instruction, but the copy_pmd() in pgtable-3level.h is C code, then compile= r could do very crazy optimization for it and generate code that break the atomicity, for example, we get below copy_= pmd() assembly code with gcc 12.4.0 (Using CC_OPTIMIZE_FOR_PERFORMANCE, it is the default compile option): " gdb vmlinux gdb disassemble do_translation_fault gdb ... gdb 0xc020e544 <+136>: ldr.w r4, [r0, r1, lsl #3] @load low 32-bit of= pmdps gdb 0xc020e548 <+140>: ldr r0, [r6, #4] @load high 32-bit o= f pmdps gdb 0xc020e54a <+142>: orrs.w r6, r4, r0 @ pmd_none(pmd_k[in= dex]) gdb 0xc020e54e <+146>: beq.n 0xc020e586 gdb ... gdb 0xc020e562 <+166>: str.w r4, [r5, r1, lsl #3] @store low 32-bit t= o pmdpd gdb 0xc020e566 <+170>: str r0, [r2, #4] @store hight 32-bit= to pmdpd The code breaks the atomicity and valid bit is in the low 32-bit, page tabl= e walker could see and cache the partial write entry, this will cause very strange translation-related issues when next page table (level3 PTE table) physical= address is larger than 32-bits. So let's use WRITE_ONCE() to protect the page table entry update functions = from crazy optimization. Signed-off-by: Wang YanQing --- Changes v1-v2: 1: Add documentation reference in changelog, suggested by Russell King arch/arm/include/asm/pgtable-3level.h | 28 +++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/arm/include/asm/pgtable-3level.h b/arch/arm/include/asm/p= gtable-3level.h index 7b71a3d414b7..b077174a4231 100644 --- a/arch/arm/include/asm/pgtable-3level.h +++ b/arch/arm/include/asm/pgtable-3level.h @@ -120,15 +120,15 @@ PMD_TYPE_SECT) #define pmd_leaf(pmd) pmd_sect(pmd) =20 -#define pud_clear(pudp) \ - do { \ - *pudp =3D __pud(0); \ - clean_pmd_entry(pudp); \ +#define pud_clear(pudp) \ + do { \ + WRITE_ONCE(*pudp, __pud(0)); \ + clean_pmd_entry(pudp); \ } while (0) =20 #define set_pud(pudp, pud) \ do { \ - *pudp =3D pud; \ + WRITE_ONCE(*pudp, pud); \ flush_pmd_entry(pudp); \ } while (0) =20 @@ -139,16 +139,16 @@ static inline pmd_t *pud_pgtable(pud_t pud) =20 #define pmd_bad(pmd) (!(pmd_val(pmd) & PMD_TABLE_BIT)) =20 -#define copy_pmd(pmdpd,pmdps) \ - do { \ - *pmdpd =3D *pmdps; \ - flush_pmd_entry(pmdpd); \ +#define copy_pmd(pmdpd, pmdps) \ + do { \ + WRITE_ONCE(*pmdpd, READ_ONCE(*pmdps)); \ + flush_pmd_entry(pmdpd); \ } while (0) =20 -#define pmd_clear(pmdp) \ - do { \ - *pmdp =3D __pmd(0); \ - clean_pmd_entry(pmdp); \ +#define pmd_clear(pmdp) \ + do { \ + WRITE_ONCE(*pmdp, __pmd(0)); \ + clean_pmd_entry(pmdp); \ } while (0) =20 /* @@ -241,7 +241,7 @@ static inline void set_pmd_at(struct mm_struct *mm, uns= igned long addr, else pmd_val(pmd) |=3D PMD_SECT_AP2; =20 - *pmdp =3D __pmd(pmd_val(pmd) | PMD_SECT_nG); + WRITE_ONCE(*pmdp, __pmd(pmd_val(pmd) | PMD_SECT_nG)); flush_pmd_entry(pmdp); } =20 --=20 2.34.1