From nobody Fri Sep 25 00:42:09 2026 Received: from fanzine2.igalia.com (fanzine2.igalia.com [213.97.179.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 441A434402B; Fri, 18 Sep 2026 00:27:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.97.179.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789691256; cv=none; b=GHlvAXYpwDEfAWVGYPbcHtYbTDUisXow8JJhih+q2NQnfB4AZS72zrQcbm2Ds9es70fbmP+ijTd4h8SvooKbQLOhFxqtE3xMDSkTDHq1ObCSnpHfC6DkiG/eGFSBDv+A07ZwNSvOa/IG2J0R1OxZ9rofSd0BSlaUU7OOBySaY0I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789691256; c=relaxed/simple; bh=XilMisU5+p6oEp17zSrE2tM3KClck2J9FqG4mWzxr9k=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LzJd1CX3S/8AzVkckPOFh1/D/sS/BeG8UymeYlHIuWVectjsWI2oZo1wtmgRckMameLf0zz54mZ6PNZv8MktTPyzgL3xrRwULiq96kc4E5G5Yt/p5ZvSnBcOyFVFdSgr6C67a/IbgYDSRLiFP5KySh3eXhF8Bj3Lm33LYlAuYrw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=DGgTvue0; arc=none smtp.client-ip=213.97.179.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="DGgTvue0" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Cc:To:Message-Id:Content-Transfer-Encoding:Content-Type: MIME-Version:Subject:Date:From:From:Reply-To; bh=AAJVlL7V3Svw6ebZPQuFtG6AvhLc+t5u5Nl4UBUE390=; b=DGgTvue0enzvlX3BmKWcCiF8LK RRQ+UBBCuqUVMaFZsWn9M8ob8JJbON6YWl9/kR7+lmO6Rtu/2VHdHVODbQih6o0TRj2PmalaldDuh bkxnPqdwr393dG7G3z0dxw4b5KA5sOLkMKBDCuq3l7BQsGkHbgR1zyPUjipyThRx42yuQFS2WvLPl saSP3K5cBabKp2xztD0RKw1kwTcknJKqwE2CseeOO4rV58LsPpekCP9u00kSbgckEo8zoV8tINNGr H+tyBzofp+10Ww0uXhey4KwixnZi5WmhziYlKuSiA2B1N335dyY5NqAgW73+f6WsQo6ba/IDm+A4J tv4XIa6g==; Received: from [177.172.123.214] (helo=[192.168.15.100]) by fanzine2.igalia.com with esmtpsa (Cipher TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1x7MRl-003iDv-9y; Fri, 18 Sep 2026 02:27:21 +0200 From: =?utf-8?q?Andr=C3=A9_Almeida?= Date: Thu, 17 Sep 2026 21:26:59 -0300 Subject: [PATCH v9 1/3] arm64: vdso: Split native/compat mremap callbacks Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260917-tonyk-robust_arm-v9-1-ab082b1b4c4b@igalia.com> References: <20260917-tonyk-robust_arm-v9-0-ab082b1b4c4b@igalia.com> In-Reply-To: <20260917-tonyk-robust_arm-v9-0-ab082b1b4c4b@igalia.com> To: Catalin Marinas , Will Deacon , Thomas Gleixner , Mark Rutland , Mathieu Desnoyers , Sebastian Andrzej Siewior , Peter Zijlstra , Florian Weimer , Darren Hart , Ingo Molnar , Davidlohr Bueso , Arnd Bergmann , Uros Bizjak , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, kernel-dev@igalia.com, LKML , =?utf-8?q?Andr=C3=A9_Almeida?= X-Mailer: b4 0.15.2 From: Mark Rutland Currently the native and compat VDSOs share a common vdso_mremap() function which is used as their vm_special_mapping::mremap callback. In subsequent patches the native and compat VDSOs will need distinct mremap logic, which will be easier to manage with separate functions. Give the compat VDSO its own aarch32_vdso_mremap() function. For now this is identical to vdso_mremap(). At the same time, fix the odd whitespace in the vdso_mremap() prototype. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Signed-off-by: Andr=C3=A9 Almeida --- arch/arm64/kernel/vdso.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c index 592dd8668de4..089a70d96220 100644 --- a/arch/arm64/kernel/vdso.c +++ b/arch/arm64/kernel/vdso.c @@ -58,7 +58,7 @@ static struct vdso_abi_info vdso_info[] __ro_after_init = =3D { }; =20 static int vdso_mremap(const struct vm_special_mapping *sm, - struct vm_area_struct *new_vma) + struct vm_area_struct *new_vma) { current->mm->context.vdso =3D (void *)new_vma->vm_start; =20 @@ -162,6 +162,14 @@ static int aarch32_sigpage_mremap(const struct vm_spec= ial_mapping *sm, return 0; } =20 +static int aarch32_vdso_mremap(const struct vm_special_mapping *sm, + struct vm_area_struct *new_vma) +{ + current->mm->context.vdso =3D (void *)new_vma->vm_start; + + return 0; +} + static struct vm_special_mapping aarch32_vdso_maps[] =3D { [AA32_MAP_VECTORS] =3D { .name =3D "[vectors]", /* ABI */ @@ -174,7 +182,7 @@ static struct vm_special_mapping aarch32_vdso_maps[] = =3D { }, [AA32_MAP_VDSO] =3D { .name =3D "[vdso]", - .mremap =3D vdso_mremap, + .mremap =3D aarch32_vdso_mremap, }, }; =20 --=20 2.55.0 From nobody Fri Sep 25 00:42:09 2026 Received: from fanzine2.igalia.com (fanzine2.igalia.com [213.97.179.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB9781D5AD4; Fri, 18 Sep 2026 00:27:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.97.179.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789691258; cv=none; b=kHlurRl9hmlv6/TixIqUZ6SLcVY5Fiw1kZ1QjaWPPHMY3k/97OUNnjqvAEs99TMghM4HBE+md/d1Wm7OrtBl+4h+s9x7ElcT49MGmi91WNlxGU7Rjcc1B2N/jwVGkKYg5H9IHhQOxDBdsZnMx/hvDNrutZZWsXZM3oReQqEe1C0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789691258; c=relaxed/simple; bh=R3gZqfgcCnb3nnmEfbBX3M/oTrdXI8HFAZlXc6hU8C8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=fyjM7rXbI+0cVk4o617SHtaFJvOqiNhMTWhhvbCC1wqGEmmpJtdw1JOYMsWgJEnlz1rkGiGMeoPsAt53Jbq/ZKM/lO5ijdEpPNZbpphr+iEQm76d0+nC+bPIAVln4khchrX2DdjXxDmSwh3RJvYcFgKGTa9rijzlr9EUEx7ffHs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=dX+xbrzH; arc=none smtp.client-ip=213.97.179.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="dX+xbrzH" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Cc:To:Message-Id:Content-Transfer-Encoding:Content-Type: MIME-Version:Subject:Date:From:From:Reply-To; bh=L0EVnDrLrlZsk6u+X20+pzKDeqqgQ00+8S0/mdXWL1c=; b=dX+xbrzHq4IOyXYKAgVtgh4Vef yD9lpi9TGtBhOP3mYXSJBL+W6zq8YFvkxfwCuJOT2dy5mYY2MffvF42LVamvquCobXvTxWgqiiYfG BjsjGam9lZFysp/blaw8BKB9Afy9ngQ9Tot/h5nqUfBQrs7sT4Bg4ltKaXg8ttXXvNUGERa8WiOVP wr/JwT6l5DL6hEtKJ8sfyFDN16dD8FNpoTI0CE8rxPMKgFtQGWjAFi9dtgGuMI9q9VK6WaFZYYNNv wlTxCESxHmu1FeqrC3kTXZBOx5g0lz84FuGar5iO9XiMlboegl/VXCDMIVsvEN8/iOQPO8z1etFOb c4QlA9XQ==; Received: from [177.172.123.214] (helo=[192.168.15.100]) by fanzine2.igalia.com with esmtpsa (Cipher TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1x7MRp-003iDv-EY; Fri, 18 Sep 2026 02:27:25 +0200 From: =?utf-8?q?Andr=C3=A9_Almeida?= Date: Thu, 17 Sep 2026 21:27:00 -0300 Subject: [PATCH v9 2/3] arm64: vdso: Prepare for robust futex unlock support Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260917-tonyk-robust_arm-v9-2-ab082b1b4c4b@igalia.com> References: <20260917-tonyk-robust_arm-v9-0-ab082b1b4c4b@igalia.com> In-Reply-To: <20260917-tonyk-robust_arm-v9-0-ab082b1b4c4b@igalia.com> To: Catalin Marinas , Will Deacon , Thomas Gleixner , Mark Rutland , Mathieu Desnoyers , Sebastian Andrzej Siewior , Peter Zijlstra , Florian Weimer , Darren Hart , Ingo Molnar , Davidlohr Bueso , Arnd Bergmann , Uros Bizjak , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, kernel-dev@igalia.com, LKML , =?utf-8?q?Andr=C3=A9_Almeida?= X-Mailer: b4 0.15.2 To solve the robust futex's list_pending_op clearing race condition, prepare for implement __vdso_futex_robust_try_unlock() for arm64 with the following steps: - Create a helper function that sets the struct futex_mm_data with the VDSO's labels addresses. The robust futex fixup mechanism needs to compare the current instruction pointer to the VDSO instructions range. - Split vdso_mremap() in vdso_mremap() and aarch32_mremap(), this allows the VDSO to be setup correctly regarding the instructions addresses for both ABIs when a mremap happens. - Implement arch_futex_robust_unlock_get_pop() for arm64, checking for r2 and r3 registers values for the fixup function. The role of this registers is explained in the commit that implement the assembly portion of the VDSO. Signed-off-by: Andr=C3=A9 Almeida --- v9: - Simplify arm64_futex_robust_unlock_get_pop by using regs->regs[] and if instead of ternary operation - Split mremap changes into other patch v6: - Restructured this commit. Move the arch bits away, kept just the generic/helper functions. v4: - Guard symbols from vdso.lds.S with ifdef - drop update_ips() from sigpage remap function v3: - Fix adding vdso base addr twice - Call vdso_futex_robust_unlock_update_ips() on remap as well v2: - Fixed linker not finding VDSO symbols --- --- arch/arm64/include/asm/futex_robust.h | 24 ++++++++++++++++++++++++ arch/arm64/kernel/vdso.c | 17 +++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/arch/arm64/include/asm/futex_robust.h b/arch/arm64/include/asm= /futex_robust.h new file mode 100644 index 000000000000..389ce2a9484d --- /dev/null +++ b/arch/arm64/include/asm/futex_robust.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_ARM64_FUTEX_ROBUST_H +#define _ASM_ARM64_FUTEX_ROBUST_H + +#include + +static __always_inline void __user *arm64_futex_robust_unlock_get_pop(stru= ct pt_regs *regs) +{ + /* + * In the asm for __vdso_futex_robust_list{64,32}_try_unlock(), w3 + * stores the result of the stlxr instruction. If it's zero, the then + * the ll/sc cmpxchg succeeded and the pending op pointer needs to be + * cleared. + */ + if (regs->regs[3]) + return NULL; + + return (void __user *)regs->regs[2]; +} + +#define arch_futex_robust_unlock_get_pop(regs) \ + arm64_futex_robust_unlock_get_pop(regs) + +#endif /* _ASM_ARM64_FUTEX_ROBUST_H */ diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c index 089a70d96220..95162061cd82 100644 --- a/arch/arm64/kernel/vdso.c +++ b/arch/arm64/kernel/vdso.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -57,6 +58,22 @@ static struct vdso_abi_info vdso_info[] __ro_after_init = =3D { #endif /* CONFIG_COMPAT_VDSO */ }; =20 +#ifdef CONFIG_FUTEX_ROBUST_UNLOCK +static inline void __vdso_futex_update_ips(struct mm_struct *mm, bool is_3= 2bit, void *startp, + void *endp) +{ + unsigned long start =3D (unsigned long)startp; + unsigned long end =3D (unsigned long)endp; + struct futex_mm_data *fd =3D &mm->futex; + + futex_set_vdso_cs_range(fd, 0, start, end, is_32bit); +} + +#else +static inline void __vdso_futex_update_ips(struct mm_struct *mm, bool is_3= 2bit, void *startp, + void *endp) +#endif /* CONFIG_FUTEX_ROBUST_UNLOCK */ + static int vdso_mremap(const struct vm_special_mapping *sm, struct vm_area_struct *new_vma) { --=20 2.55.0 From nobody Fri Sep 25 00:42:09 2026 Received: from fanzine2.igalia.com (fanzine2.igalia.com [213.97.179.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CAC6F349CD1; Fri, 18 Sep 2026 00:27:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.97.179.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789691262; cv=none; b=NgvxaE5d6N2e48sx7EfeWNHV8KjkPldSxtjjAFY/McIjLSQ2+A+OuxJu1cf427v50wb9r5VxApY8bVDNl52/s3ThJ49/T1jZEi/8fY8w/VmUTfUhKdcxC5pehkbd9SAQsr+HLLzCSxoCVhcfG9GFZXnilSDG+aJwtCW0W2oSc0c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789691262; c=relaxed/simple; bh=ZGSipOxDaEzZPZIFkQqfILvqlIIOm+3j79tLnKuFVf4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=gfJz549brT/XvvwfIIABD9+LsVPLn0nOhvx6Pot5vgLOJJHcEsH4rtePwWIDXnqyjZlv/WoB/6sjl5wxrOjnui9xZfdghlK64/VN4Uuai7kIYjA+kfjCU6yx3Oz6wha79MO5DEqrjKf0IRxignDlioByqKY3FWDDAsSm5UGY6mk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=XodOcefM; arc=none smtp.client-ip=213.97.179.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="XodOcefM" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Cc:To:Message-Id:Content-Transfer-Encoding:Content-Type: MIME-Version:Subject:Date:From:From:Reply-To; bh=naEoqeYe8X/Jw2DES4clEmxwgKBwecpzN9gXVazxB2c=; b=XodOcefMOcWjo30lLhyeafEgnY lkG5gqfSOJfXXJ5QQtAqadwAFQEt7Eo31AzUj3UDkGIT1CglyBfycgr/i9RqiVlfvJmBCgTjbIYLj teh6yAEbomupgAWSWsvtY/ZnfmG1cVhwI3H3T9t5JcznyqaEk4KbYzEdr09JF1xAp6jqDF4ynp6OC oe1+mArvhPj07ZyE66ZtRi+7Y1x5ZWTJAvqOM3Ps0idvsTU0TMHRhcvR0d1AP4bJiUrN9F5MoSYXJ HBeVkKWa6gaeZSxXhv8zaOZum5EUMHqbMmdpUersL4o7ICCCe8nYSmVFTVepaMLLam8Ov25R+H8Z3 kiLeyigA==; Received: from [177.172.123.214] (helo=[192.168.15.100]) by fanzine2.igalia.com with esmtpsa (Cipher TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1x7MRt-003iDv-JQ; Fri, 18 Sep 2026 02:27:29 +0200 From: =?utf-8?q?Andr=C3=A9_Almeida?= Date: Thu, 17 Sep 2026 21:27:01 -0300 Subject: [PATCH v9 3/3] arm64: vdso: Implement __vdso_futex_robust_try_unlock() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260917-tonyk-robust_arm-v9-3-ab082b1b4c4b@igalia.com> References: <20260917-tonyk-robust_arm-v9-0-ab082b1b4c4b@igalia.com> In-Reply-To: <20260917-tonyk-robust_arm-v9-0-ab082b1b4c4b@igalia.com> To: Catalin Marinas , Will Deacon , Thomas Gleixner , Mark Rutland , Mathieu Desnoyers , Sebastian Andrzej Siewior , Peter Zijlstra , Florian Weimer , Darren Hart , Ingo Molnar , Davidlohr Bueso , Arnd Bergmann , Uros Bizjak , =?utf-8?q?Thomas_Wei=C3=9Fschuh?= Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, kernel-dev@igalia.com, LKML , =?utf-8?q?Andr=C3=A9_Almeida?= X-Mailer: b4 0.15.2 The futex's robust list uAPI has a struct robust_list_head::list_op_pending pointer used by userspace as a temporary variable while the mutex unlock is happening. User sets it to the futex address that's about to be released and removed from the robust list, and list_op_pending is cleared after. After a thread dies, the kernel checks it's list_op_pending and wakes the mutex in that address, to prevent starvation, and flip a bit in the mutex word (FUTEX_OWNER_DIED). However, there's a critical section where the user thread dies after the mutex is released but before list_op_pending is cleared. If that happens, another thread can wake up, use the lock, release it, and free its memory. Now, if the robust list cleanup happens after this, the killed thread's list_op_pending becomes a dandling pointer. The kernel wrongly treats this address as a mutex, calls a futex_wake() on it and flips a bit, causing a memory corruption. To avoid using the dangling pointer, implement __vdso_futex_robust_try_unlock() for arm64. Make the VDSO release the mutex and clear the list_op_pending fields, just as is done in userspace right now. But having it in a VDSO means that, in the case of a killed user thread, the kernel can know exactly in which part of the release process the thread was interrupt, check the registers for the operation success and clears the list_op_pending on behalf of the user thread to prevent the use-after-free bug. The need for checking the instructions addresses and the register makes this mechanism arch-dependent. Implement it using LL/SC semantics. If the user instruction pointer is between the labels __futex_list64_try_unlock_cs_start and __futex_list64_try_unlock_cs_end, the critical section was interrupted. The kernel checks for the result register (always w3) of the stlxr instruction used for atomically releasing the mutex. If it's 0, the release happened and the kernel should clear the list_op_pending field (always stored at x2). Signed-off-by: Andr=C3=A9 Almeida --- v9: - Refactor assembly code style - Make sure [pop_reg] is allocated to a register, defining it as a "r" input - Make [lock] a read-write param using +Q v7: - Typo in message: success result for stlex is 0, not 1 - pop_reg is read afterwards so define it as an output parameter "+Q" v6: - Complete reword of commit message to make it clear - Better commit split, only the specific aarch64 things here - Use explicity labels instead of macros v4: - Guard makefile for vfutex.o with ifdef - Moved _start label one instruction above - Use results register (w3) to check for store success instead of using ze= ro flag v3: - Managed to get pop to always be stored at x2 --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/vdso.c | 17 ++++++++++++++-- arch/arm64/kernel/vdso/Makefile | 10 ++++++++++ arch/arm64/kernel/vdso/vdso.lds.S | 9 +++++++++ arch/arm64/kernel/vdso/vfutex.c | 41 +++++++++++++++++++++++++++++++++++= ++++ 5 files changed, 76 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b5a51b0ef944..5a3705b9d2e1 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -222,6 +222,7 @@ config ARM64 select HAVE_RELIABLE_STACKTRACE select HAVE_POSIX_CPU_TIMERS_TASK_WORK select HAVE_FUNCTION_ARG_ACCESS_API + select HAVE_FUTEX_ROBUST_UNLOCK select MMU_GATHER_RCU_TABLE_FREE select HAVE_RSEQ select HAVE_RUST if RUSTC_SUPPORTS_ARM64 diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c index 95162061cd82..052667ba6cde 100644 --- a/arch/arm64/kernel/vdso.c +++ b/arch/arm64/kernel/vdso.c @@ -69,16 +69,27 @@ static inline void __vdso_futex_update_ips(struct mm_st= ruct *mm, bool is_32bit, futex_set_vdso_cs_range(fd, 0, start, end, is_32bit); } =20 +static inline void vdso_futex_update_ips(struct mm_struct *mm) +{ + unsigned long vdso =3D (unsigned long)mm->context.vdso; + + __vdso_futex_update_ips(mm, false, + VDSO_SYMBOL(vdso, futex_list64_try_unlock_cs_start), + VDSO_SYMBOL(vdso, futex_list64_try_unlock_cs_end)); +} + #else -static inline void __vdso_futex_update_ips(struct mm_struct *mm, bool is_3= 2bit, void *startp, - void *endp) +static inline void vdso_futex_update_ips(struct mm_struct *mm) {} #endif /* CONFIG_FUTEX_ROBUST_UNLOCK */ =20 + static int vdso_mremap(const struct vm_special_mapping *sm, struct vm_area_struct *new_vma) { current->mm->context.vdso =3D (void *)new_vma->vm_start; =20 + vdso_futex_update_ips(current->mm); + return 0; } =20 @@ -366,5 +377,7 @@ int arch_setup_additional_pages(struct linux_binprm *bp= rm, int uses_interp) ret =3D __setup_additional_pages(VDSO_ABI_AA64, mm, bprm, uses_interp); mmap_write_unlock(mm); =20 + vdso_futex_update_ips(mm); + return ret; } diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makef= ile index 7dec05dd33b7..985346c7a0bb 100644 --- a/arch/arm64/kernel/vdso/Makefile +++ b/arch/arm64/kernel/vdso/Makefile @@ -11,6 +11,10 @@ include $(srctree)/lib/vdso/Makefile.include =20 obj-vdso :=3D vgettimeofday.o note.o sigreturn.o vgetrandom.o vgetrandom-c= hacha.o =20 +ifdef CONFIG_FUTEX_ROBUST_UNLOCK + obj-vdso +=3D vfutex.o +endif + # Build rules targets :=3D $(obj-vdso) vdso.so vdso.so.dbg obj-vdso :=3D $(addprefix $(obj)/, $(obj-vdso)) @@ -45,9 +49,11 @@ CC_FLAGS_ADD_VDSO :=3D -O2 -mcmodel=3Dtiny -fasynchronou= s-unwind-tables =20 CFLAGS_REMOVE_vgettimeofday.o =3D $(CC_FLAGS_REMOVE_VDSO) CFLAGS_REMOVE_vgetrandom.o =3D $(CC_FLAGS_REMOVE_VDSO) +CFLAGS_REMOVE_vfutex.o =3D $(CC_FLAGS_REMOVE_VDSO) =20 CFLAGS_vgettimeofday.o =3D $(CC_FLAGS_ADD_VDSO) CFLAGS_vgetrandom.o =3D $(CC_FLAGS_ADD_VDSO) +CFLAGS_vfutex.o =3D $(CC_FLAGS_ADD_VDSO) =20 ifneq ($(c-gettimeofday-y),) CFLAGS_vgettimeofday.o +=3D -include $(c-gettimeofday-y) @@ -57,6 +63,10 @@ ifneq ($(c-getrandom-y),) CFLAGS_vgetrandom.o +=3D -include $(c-getrandom-y) endif =20 +ifneq ($(c-futex-y),) + CFLAGS_vfutex.o +=3D -include $(c-futex-y) +endif + targets +=3D vdso.lds CPPFLAGS_vdso.lds +=3D -P -C -U$(ARCH) =20 diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vds= o.lds.S index 52314be29191..225f59bb81d1 100644 --- a/arch/arm64/kernel/vdso/vdso.lds.S +++ b/arch/arm64/kernel/vdso/vdso.lds.S @@ -104,6 +104,9 @@ VERSION __kernel_clock_gettime; __kernel_clock_getres; __kernel_getrandom; +#ifdef CONFIG_FUTEX_ROBUST_UNLOCK + __vdso_futex_robust_list64_try_unlock; +#endif local: *; }; } @@ -112,3 +115,9 @@ VERSION * Make the sigreturn code visible to the kernel. */ VDSO_sigtramp =3D __kernel_rt_sigreturn; + +#ifdef CONFIG_FUTEX_ROBUST_UNLOCK +VDSO_futex_list64_try_unlock_cs_start =3D __futex_list64_try_unlock_cs_sta= rt; +VDSO_futex_list64_try_unlock_cs_success =3D __futex_list64_try_unlock_cs_s= uccess; +VDSO_futex_list64_try_unlock_cs_end =3D __futex_list64_try_unlock_cs_end; +#endif diff --git a/arch/arm64/kernel/vdso/vfutex.c b/arch/arm64/kernel/vdso/vfute= x.c new file mode 100644 index 000000000000..7622aa34bfdb --- /dev/null +++ b/arch/arm64/kernel/vdso/vfutex.c @@ -0,0 +1,41 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +#include +#include + +#define LABEL(l) \ + " .globl " #l "\n" \ + #l ":\n" + +__u32 __vdso_futex_robust_list64_try_unlock(__u32 *lock, __u32 tid, __u64 = *pop) +{ + /* + * arm64_futex_robust_unlock_get_pop() depends on this exact register + * allocation to work correctly. + */ + register __u64 pop_reg asm("x2") =3D (__u64) pop; + register __u32 result_reg asm("w3") =3D 0; + __u32 val; + + asm volatile ( + " prfm pstl1strm, %[lock] \n" + "retry: \n" + " ldxr %w[val], %[lock] \n" + " cmp %w[tid], %w[val] \n" + " b.ne __futex_list64_try_unlock_cs_end\n" + " stlxr %w[result], wzr, %[lock] \n" + LABEL(__futex_list64_try_unlock_cs_start) + " cbnz %w[result], retry \n" + LABEL(__futex_list64_try_unlock_cs_success) + " str xzr, [%x[pop_reg]] \n" + LABEL(__futex_list64_try_unlock_cs_end) + + : [val] "=3D&r" (val), + [result] "=3D&r" (result_reg), + [lock] "+Q" (*lock) + : [tid] "r" (tid), + [pop_reg] "r" (pop_reg) + : "cc", "memory" + ); + + return val; +} --=20 2.55.0