From nobody Sun Feb 8 11:07:32 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D98EE1AC44D; Sun, 13 Jul 2025 07:17:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391067; cv=none; b=iC+V8+sfnoifdB2GiYH4+F9aFLKQwT5GCMbxXMmbYyQ39u4jmOiC7cUXSBNLjSvLVlKDSeBIHZeUfwGXQDmQP0mlM2hKw53sVz+y88euHkvrqml1dcnkqT4L34zJOMzNU4PepTtxxBWDfKVRXo3ey5C5Cl0cDJGHvnmc/W8eP9M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391067; c=relaxed/simple; bh=P4fHP9UFyAr3Scf7HStnOCqWcHWXOAR5PCL9m8hSZVE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=otzuMcT1cHlh2WZJXVdHMV9cxSPLXXdo0ZG9uZEVAUTchx7AOSCxSISLT95NYeGQOnVvdc23nLaZ6UMYhuUa91OJADx8OnVTUSRuW98FCaoTXsidr/h5jxvkZMzjUfLPMH6ZsbUGzE3yubjZLlGKxxd7qAeQUG1X8pqlSmzAuXI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ffjn4N7L; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ffjn4N7L" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A8B10C4CEF5; Sun, 13 Jul 2025 07:17:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1752391066; bh=P4fHP9UFyAr3Scf7HStnOCqWcHWXOAR5PCL9m8hSZVE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ffjn4N7LSxiD9F/zVzjTT8+NJhOcazHVy9FHNuKWp5JVdcf2dF29EBCJYEoqX27Jj CVGHuLbuTrj8XmzGaDcaVMFUihPaBebeM7T8yhV9ZItqEf7Fw6JovzMVIoNKL9BxJ5 bFL7jtV0VHZgCmZS1WG/HHhQOwp1UBFYngSGfAzMIlEaMqPLJAOPFsZBw19mVFOFQM 3TzOT02U5r2/lypPzC287BLucq7AC/bJIncxrGAGbgFnQ04ROTH4bZ+zp9RYJK2OTY 2rRZFvZbwfqAnEGaelbjVbn9v95V9YrnNXMqM2u7t8MS4DW1BprMc/pjgBuXbY1vdb JhyVpYKOAy7WA== From: Mike Rapoport To: Andrew Morton Cc: Andy Lutomirski , Borislav Petkov , Christophe Leroy , Daniel Gomez , Dave Hansen , Ingo Molnar , "Liam R. Howlett" , Luis Chamberlain , Mark Rutland , Masami Hiramatsu , Mike Rapoport , "H. Peter Anvin" , Peter Zijlstra , Petr Pavlu , Sami Tolvanen , Steven Rostedt , Thomas Gleixner , Yann Ylavic , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 1/8] execmem: drop unused execmem_update_copy() Date: Sun, 13 Jul 2025 10:17:23 +0300 Message-ID: <20250713071730.4117334-2-rppt@kernel.org> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250713071730.4117334-1-rppt@kernel.org> References: <20250713071730.4117334-1-rppt@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Mike Rapoport (Microsoft)" The execmem_update_copy() that used text poking was required when memory allocated from ROX cache was always read-only. Since now its permissions can be switched to read-write there is no need in a function that updates memory with text poking. Remove it. Acked-by: Peter Zijlstra (Intel) Signed-off-by: Mike Rapoport (Microsoft) --- include/linux/execmem.h | 13 ------------- mm/execmem.c | 5 ----- 2 files changed, 18 deletions(-) diff --git a/include/linux/execmem.h b/include/linux/execmem.h index 3be35680a54f..734fbe83d98e 100644 --- a/include/linux/execmem.h +++ b/include/linux/execmem.h @@ -185,19 +185,6 @@ DEFINE_FREE(execmem, void *, if (_T) execmem_free(_T)); struct vm_struct *execmem_vmap(size_t size); #endif =20 -/** - * execmem_update_copy - copy an update to executable memory - * @dst: destination address to update - * @src: source address containing the data - * @size: how many bytes of memory shold be copied - * - * Copy @size bytes from @src to @dst using text poking if the memory at - * @dst is read-only. - * - * Return: a pointer to @dst or NULL on error - */ -void *execmem_update_copy(void *dst, const void *src, size_t size); - /** * execmem_is_rox - check if execmem is read-only * @type - the execmem type to check diff --git a/mm/execmem.c b/mm/execmem.c index 2b683e7d864d..0712ebb4eb77 100644 --- a/mm/execmem.c +++ b/mm/execmem.c @@ -399,11 +399,6 @@ void execmem_free(void *ptr) vfree(ptr); } =20 -void *execmem_update_copy(void *dst, const void *src, size_t size) -{ - return text_poke_copy(dst, src, size); -} - bool execmem_is_rox(enum execmem_type type) { return !!(execmem_info->ranges[type].flags & EXECMEM_ROX_CACHE); --=20 2.47.2 From nobody Sun Feb 8 11:07:32 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 18D8F1DA60F; Sun, 13 Jul 2025 07:17:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391073; cv=none; b=Tb3wzTkGVLiaazVM4Re9UyUhvuuF/q/P4p4GavjdpHoD68vRzD5TF8j+YwijYLTnKSQE8cLSvLv0Rodl51QexFeUqCT8vPzN0r2lPztk0W0FKgdwQu3b8k4N4Y+KxxgrdZA6yni4jw1CVCLs8AYkA8tge0Uh00hpnZw7xkYMxqs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391073; c=relaxed/simple; bh=3dUBpvEhYnlflMVnyRwOxMHiJIQysIRUBLUfy0VHZwY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NW2SC9mLa5jenSToje3FP0IQMDghomwClQu5tHd1W1hOlquRsoZABSWY7HRffS6/w+99tCdaumuwrvJzcIs919f9I2P3FcqgZYrvwprXcsgKpm5P+SDCsVUVscPAqmDs2SG3NBeqdPZw4x76z4PkZPYcblvov/pwYq3We97Yeb4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kXpza8uf; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kXpza8uf" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E6C6AC4CEF9; Sun, 13 Jul 2025 07:17:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1752391072; bh=3dUBpvEhYnlflMVnyRwOxMHiJIQysIRUBLUfy0VHZwY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kXpza8uf1fpZTHSm35hPSgJxQLhzXrdXpH/RX0ji3Z1XguEuWUQ9JT/jx8CqQAZEl 7HSZz/Iq2CY2Fe1t5oS3TPYKeZ8v93bV5hK4tDYwRl7VVq03+XCmcv89gctSoJ1Vkq QdWcpJsQ5TQZYb23jqSLscMepwHxYvO8QHQI8xQ58e7rTS1wIimyRqgYzrPUKGF54C yBsnVy330Hp7e87+oJuxNp8rPyLFe+43pcFfqiK/hYbpWZv4YjSOfMiR+Usc/aqbQB RPcsI5rnUTl5B/5/al4NEX9Fl5x7Fj3Q5zBaqUSwisYxBQRJHWtLUz8PV56DlAiahL PWlIIXbzMDwkg== From: Mike Rapoport To: Andrew Morton Cc: Andy Lutomirski , Borislav Petkov , Christophe Leroy , Daniel Gomez , Dave Hansen , Ingo Molnar , "Liam R. Howlett" , Luis Chamberlain , Mark Rutland , Masami Hiramatsu , Mike Rapoport , "H. Peter Anvin" , Peter Zijlstra , Petr Pavlu , Sami Tolvanen , Steven Rostedt , Thomas Gleixner , Yann Ylavic , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 2/8] execmem: introduce execmem_alloc_rw() Date: Sun, 13 Jul 2025 10:17:24 +0300 Message-ID: <20250713071730.4117334-3-rppt@kernel.org> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250713071730.4117334-1-rppt@kernel.org> References: <20250713071730.4117334-1-rppt@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Mike Rapoport (Microsoft)" Some callers of execmem_alloc() require the memory to be temporarily writable even when it is allocated from ROX cache. These callers use execemem_make_temp_rw() right after the call to execmem_alloc(). Wrap this sequence in execmem_alloc_rw() API. Reviewed-by: Daniel Gomez Reviewed-by: Petr Pavlu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Mike Rapoport (Microsoft) --- arch/x86/kernel/alternative.c | 3 +-- include/linux/execmem.h | 38 ++++++++++++++++++++--------------- kernel/module/main.c | 13 ++---------- mm/execmem.c | 27 ++++++++++++++++++++++++- 4 files changed, 51 insertions(+), 30 deletions(-) diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index ea1d984166cd..526a5fef93ab 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -120,7 +120,7 @@ struct its_array its_pages; =20 static void *__its_alloc(struct its_array *pages) { - void *page __free(execmem) =3D execmem_alloc(EXECMEM_MODULE_TEXT, PAGE_SI= ZE); + void *page __free(execmem) =3D execmem_alloc_rw(EXECMEM_MODULE_TEXT, PAGE= _SIZE); if (!page) return NULL; =20 @@ -237,7 +237,6 @@ static void *its_alloc(void) if (!page) return NULL; =20 - execmem_make_temp_rw(page, PAGE_SIZE); if (pages =3D=3D &its_pages) set_memory_x((unsigned long)page, 1); =20 diff --git a/include/linux/execmem.h b/include/linux/execmem.h index 734fbe83d98e..8b61b05da7d5 100644 --- a/include/linux/execmem.h +++ b/include/linux/execmem.h @@ -67,21 +67,6 @@ enum execmem_range_flags { */ void execmem_fill_trapping_insns(void *ptr, size_t size, bool writable); =20 -/** - * execmem_make_temp_rw - temporarily remap region with read-write - * permissions - * @ptr: address of the region to remap - * @size: size of the region to remap - * - * Remaps a part of the cached large page in the ROX cache in the range - * [@ptr, @ptr + @size) as writable and not executable. The caller must - * have exclusive ownership of this range and ensure nothing will try to - * execute code in this range. - * - * Return: 0 on success or negative error code on failure. - */ -int execmem_make_temp_rw(void *ptr, size_t size); - /** * execmem_restore_rox - restore read-only-execute permissions * @ptr: address of the region to remap @@ -95,7 +80,6 @@ int execmem_make_temp_rw(void *ptr, size_t size); */ int execmem_restore_rox(void *ptr, size_t size); #else -static inline int execmem_make_temp_rw(void *ptr, size_t size) { return 0;= } static inline int execmem_restore_rox(void *ptr, size_t size) { return 0; } #endif =20 @@ -165,6 +149,28 @@ struct execmem_info *execmem_arch_setup(void); */ void *execmem_alloc(enum execmem_type type, size_t size); =20 +/** + * execmem_alloc_rw - allocate writable executable memory + * @type: type of the allocation + * @size: how many bytes of memory are required + * + * Allocates memory that will contain executable code, either generated or + * loaded from kernel modules. + * + * Allocates memory that will contain data coupled with executable code, + * like data sections in kernel modules. + * + * Forces writable permissions on the allocated memory and the caller is + * responsible to manage the permissions afterwards. + * + * For architectures that use ROX cache the permissions will be set to R+W. + * For architectures that don't use ROX cache the default permissions for = @type + * will be used as they must be writable. + * + * Return: a pointer to the allocated memory or %NULL + */ +void *execmem_alloc_rw(enum execmem_type type, size_t size); + /** * execmem_free - free executable memory * @ptr: pointer to the memory that should be freed diff --git a/kernel/module/main.c b/kernel/module/main.c index 413ac6ea3702..d009326ef7bb 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -1292,20 +1292,11 @@ static int module_memory_alloc(struct module *mod, = enum mod_mem_type type) else execmem_type =3D EXECMEM_MODULE_TEXT; =20 - ptr =3D execmem_alloc(execmem_type, size); + ptr =3D execmem_alloc_rw(execmem_type, size); if (!ptr) return -ENOMEM; =20 - if (execmem_is_rox(execmem_type)) { - int err =3D execmem_make_temp_rw(ptr, size); - - if (err) { - execmem_free(ptr); - return -ENOMEM; - } - - mod->mem[type].is_rox =3D true; - } + mod->mem[type].is_rox =3D execmem_is_rox(execmem_type); =20 /* * The pointer to these blocks of memory are stored on the module diff --git a/mm/execmem.c b/mm/execmem.c index 0712ebb4eb77..6b040fbc5f4f 100644 --- a/mm/execmem.c +++ b/mm/execmem.c @@ -336,7 +336,7 @@ static bool execmem_cache_free(void *ptr) return true; } =20 -int execmem_make_temp_rw(void *ptr, size_t size) +static int execmem_force_rw(void *ptr, size_t size) { unsigned int nr =3D PAGE_ALIGN(size) >> PAGE_SHIFT; unsigned long addr =3D (unsigned long)ptr; @@ -358,6 +358,16 @@ int execmem_restore_rox(void *ptr, size_t size) } =20 #else /* CONFIG_ARCH_HAS_EXECMEM_ROX */ +/* + * when ROX cache is not used the permissions defined by architectures for + * execmem ranges that are updated before use (e.g. EXECMEM_MODULE_TEXT) m= ust + * be writable anyway + */ +static inline int execmem_force_rw(void *ptr, size_t size) +{ + return 0; +} + static void *execmem_cache_alloc(struct execmem_range *range, size_t size) { return NULL; @@ -387,6 +397,21 @@ void *execmem_alloc(enum execmem_type type, size_t siz= e) return kasan_reset_tag(p); } =20 +void *execmem_alloc_rw(enum execmem_type type, size_t size) +{ + void *p __free(execmem) =3D execmem_alloc(type, size); + int err; + + if (!p) + return NULL; + + err =3D execmem_force_rw(p, size); + if (err) + return NULL; + + return no_free_ptr(p); +} + void execmem_free(void *ptr) { /* --=20 2.47.2 From nobody Sun Feb 8 11:07:32 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 09C3A1DE3CA; Sun, 13 Jul 2025 07:17:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391079; cv=none; b=H0+VUU6XRSSInGPUZUWW50F8EM3Gszj5+ZLNJxee8c3027lMyP6RX7mm13bqz3HjNZwsw6GZLDzTmhcsoKF5e4klSw56OlaGWCZ53k7u1neznBS+cvyIUK2MVcDORKqHDW4yllQd2o05DanNYmjUC2Uidlel4Rxs0vEBKCCyEw8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391079; c=relaxed/simple; bh=W/Oo9Q6iVRegd5YSmpF86bYBZEvYG0hOXK4WxZm5rkc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mOkZScadMONXOn8ooM9OQFNMJ/wEPevrPsNeKS2ZgJlsVvmLGqt+45SrZR3HhbaruNwjXRjickF+8+b5y+qTDkjHK2+MJ9CG6tqnIore6TOhIWw/PWpLEM5Gq1vQRr42XnfcxsvFZ16rkFpEhz1xg6or0Py2urC8FQEAuHjDYkw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=egBv3Dk1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="egBv3Dk1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3066BC4CEE3; Sun, 13 Jul 2025 07:17:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1752391078; bh=W/Oo9Q6iVRegd5YSmpF86bYBZEvYG0hOXK4WxZm5rkc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=egBv3Dk1nZcxm40/Q5w8oGqCT3saJsCtXvsgFrsXYrDPJytNAGc243/SSi7sd/jIw auAVhF2lTjn5cMIC2OK6vmhMwH8KAHmdTjUcZ16gYELRY6g/say6hzttCd/MdK1j1V eVSTpa5BRDZHI5jGi0wrgaH6GfDlj9/C+UcGNTsdLcaDQoaFc0ch41tQTUAYYPFTo5 i73Ad5negD5cNhBZ2XWSAg2Kocv4u/Z9jlOY+RB11ckKGRWFF0j3s2plcBYBBLP5Yg NbagP9MNh/slcVqse1/+dYMBezw3WXTEdM65h/4x2eRK4yKveU2hXasM7WGTpY75x3 cxAUD6pd+zr7w== From: Mike Rapoport To: Andrew Morton Cc: Andy Lutomirski , Borislav Petkov , Christophe Leroy , Daniel Gomez , Dave Hansen , Ingo Molnar , "Liam R. Howlett" , Luis Chamberlain , Mark Rutland , Masami Hiramatsu , Mike Rapoport , "H. Peter Anvin" , Peter Zijlstra , Petr Pavlu , Sami Tolvanen , Steven Rostedt , Thomas Gleixner , Yann Ylavic , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 3/8] execmem: rework execmem_cache_free() Date: Sun, 13 Jul 2025 10:17:25 +0300 Message-ID: <20250713071730.4117334-4-rppt@kernel.org> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250713071730.4117334-1-rppt@kernel.org> References: <20250713071730.4117334-1-rppt@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Mike Rapoport (Microsoft)" Currently execmem_cache_free() ignores potential allocation failures that may happen in execmem_cache_add(). Besides, it uses text poking to fill the memory with trapping instructions before returning it to cache although it would be more efficient to make that memory writable, update it using memcpy and then restore ROX protection. Rework execmem_cache_free() so that in case of an error it will defer freeing of the memory to a delayed work. With this the happy fast path will now change permissions to RW, fill the memory with trapping instructions using memcpy, restore ROX permissions, add the memory back to the free cache and clear the relevant entry in busy_areas. If any step in the fast path fails, the entry in busy_areas will be marked as pending_free. These entries will be handled by a delayed work and freed asynchronously. To make the fast path faster, use __GFP_NORETRY for memory allocations and let asynchronous handler try harder with GFP_KERNEL. Acked-by: Peter Zijlstra (Intel) Signed-off-by: Mike Rapoport (Microsoft) --- mm/execmem.c | 125 +++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 102 insertions(+), 23 deletions(-) diff --git a/mm/execmem.c b/mm/execmem.c index 6b040fbc5f4f..4670e97f8e4e 100644 --- a/mm/execmem.c +++ b/mm/execmem.c @@ -93,8 +93,15 @@ struct execmem_cache { struct mutex mutex; struct maple_tree busy_areas; struct maple_tree free_areas; + unsigned int pending_free_cnt; /* protected by mutex */ }; =20 +/* delay to schedule asynchronous free if fast path free fails */ +#define FREE_DELAY (msecs_to_jiffies(10)) + +/* mark entries in busy_areas that should be freed asynchronously */ +#define PENDING_FREE_MASK (1 << (PAGE_SHIFT - 1)) + static struct execmem_cache execmem_cache =3D { .mutex =3D __MUTEX_INITIALIZER(execmem_cache.mutex), .busy_areas =3D MTREE_INIT_EXT(busy_areas, MT_FLAGS_LOCK_EXTERN, @@ -155,20 +162,17 @@ static void execmem_cache_clean(struct work_struct *w= ork) =20 static DECLARE_WORK(execmem_cache_clean_work, execmem_cache_clean); =20 -static int execmem_cache_add(void *ptr, size_t size) +static int execmem_cache_add_locked(void *ptr, size_t size, gfp_t gfp_mask) { struct maple_tree *free_areas =3D &execmem_cache.free_areas; - struct mutex *mutex =3D &execmem_cache.mutex; unsigned long addr =3D (unsigned long)ptr; MA_STATE(mas, free_areas, addr - 1, addr + 1); unsigned long lower, upper; void *area =3D NULL; - int err; =20 lower =3D addr; upper =3D addr + size - 1; =20 - mutex_lock(mutex); area =3D mas_walk(&mas); if (area && mas.last =3D=3D addr - 1) lower =3D mas.index; @@ -178,12 +182,14 @@ static int execmem_cache_add(void *ptr, size_t size) upper =3D mas.last; =20 mas_set_range(&mas, lower, upper); - err =3D mas_store_gfp(&mas, (void *)lower, GFP_KERNEL); - mutex_unlock(mutex); - if (err) - return err; + return mas_store_gfp(&mas, (void *)lower, gfp_mask); +} =20 - return 0; +static int execmem_cache_add(void *ptr, size_t size, gfp_t gfp_mask) +{ + guard(mutex)(&execmem_cache.mutex); + + return execmem_cache_add_locked(ptr, size, gfp_mask); } =20 static bool within_range(struct execmem_range *range, struct ma_state *mas, @@ -278,7 +284,7 @@ static int execmem_cache_populate(struct execmem_range = *range, size_t size) if (err) goto err_free_mem; =20 - err =3D execmem_cache_add(p, alloc_size); + err =3D execmem_cache_add(p, alloc_size, GFP_KERNEL); if (err) goto err_reset_direct_map; =20 @@ -307,29 +313,102 @@ static void *execmem_cache_alloc(struct execmem_rang= e *range, size_t size) return __execmem_cache_alloc(range, size); } =20 +static inline bool is_pending_free(void *ptr) +{ + return ((unsigned long)ptr & PENDING_FREE_MASK); +} + +static inline void *pending_free_set(void *ptr) +{ + return (void *)((unsigned long)ptr | PENDING_FREE_MASK); +} + +static inline void *pending_free_clear(void *ptr) +{ + return (void *)((unsigned long)ptr & ~PENDING_FREE_MASK); +} + +static int execmem_force_rw(void *ptr, size_t size); + +static int __execmem_cache_free(struct ma_state *mas, void *ptr, gfp_t gfp= _mask) +{ + size_t size =3D mas_range_len(mas); + int err; + + err =3D execmem_force_rw(ptr, size); + if (err) + return err; + + execmem_fill_trapping_insns(ptr, size, /* writable =3D */ true); + execmem_restore_rox(ptr, size); + + err =3D execmem_cache_add_locked(ptr, size, gfp_mask); + if (err) + return err; + + mas_store_gfp(mas, NULL, gfp_mask); + return 0; +} + +static void execmem_cache_free_slow(struct work_struct *work); +static DECLARE_DELAYED_WORK(execmem_cache_free_work, execmem_cache_free_sl= ow); + +static void execmem_cache_free_slow(struct work_struct *work) +{ + struct maple_tree *busy_areas =3D &execmem_cache.busy_areas; + MA_STATE(mas, busy_areas, 0, ULONG_MAX); + void *area; + + guard(mutex)(&execmem_cache.mutex); + + if (!execmem_cache.pending_free_cnt) + return; + + mas_for_each(&mas, area, ULONG_MAX) { + if (!is_pending_free(area)) + continue; + + area =3D pending_free_clear(area); + if (__execmem_cache_free(&mas, area, GFP_KERNEL)) + continue; + + execmem_cache.pending_free_cnt--; + } + + if (execmem_cache.pending_free_cnt) + schedule_delayed_work(&execmem_cache_free_work, FREE_DELAY); + else + schedule_work(&execmem_cache_clean_work); +} + static bool execmem_cache_free(void *ptr) { struct maple_tree *busy_areas =3D &execmem_cache.busy_areas; - struct mutex *mutex =3D &execmem_cache.mutex; unsigned long addr =3D (unsigned long)ptr; MA_STATE(mas, busy_areas, addr, addr); - size_t size; void *area; + int err; + + guard(mutex)(&execmem_cache.mutex); =20 - mutex_lock(mutex); area =3D mas_walk(&mas); - if (!area) { - mutex_unlock(mutex); + if (!area) return false; - } - size =3D mas_range_len(&mas); =20 - mas_store_gfp(&mas, NULL, GFP_KERNEL); - mutex_unlock(mutex); - - execmem_fill_trapping_insns(ptr, size, /* writable =3D */ false); - - execmem_cache_add(ptr, size); + err =3D __execmem_cache_free(&mas, area, GFP_KERNEL | __GFP_NORETRY); + if (err) { + /* + * mas points to exact slot we've got the area from, nothing + * else can modify the tree because of the mutex, so there + * won't be any allocations in mas_store_gfp() and it will just + * change the pointer. + */ + area =3D pending_free_set(area); + mas_store_gfp(&mas, area, GFP_KERNEL); + execmem_cache.pending_free_cnt++; + schedule_delayed_work(&execmem_cache_free_work, FREE_DELAY); + return true; + } =20 schedule_work(&execmem_cache_clean_work); =20 --=20 2.47.2 From nobody Sun Feb 8 11:07:32 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96D1A1D90A5; Sun, 13 Jul 2025 07:18:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391085; cv=none; b=PQS1HNqUo4Ci42/GokypusnsLewo470gbCL/eoBBev2knJnqV0sU0eH42OGo54laVfB+u5U+d6U07zZX9jFv1DYemXY3kBUuHGIL1HEEyIeiipk4wB3ip1oetBzho/aIglqJWxaPIbKAeTUNYvF50jDsJ9/j6FlQ276e8trgEbQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391085; c=relaxed/simple; bh=3tPQQyB9eXTIb3XNbybVUxCt9L6HRc0Tde1G6CrS+SQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Gn1gizr84Hi1mGkUcsc91vasLY9m48erqcUIGJs+0A/elSqb3Fh2+R1KqIYO/sBQ/mTO0KXkA4nbrxuFeh88sDZ4JCP5jlfB7OWWUmQQyAHnwrzkVtNX914sQeS69gtm7+gatetfbg3En854seJAeXsn2MOYpNT3ipWN2070XFs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fwCRotWB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fwCRotWB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6E472C4CEE3; Sun, 13 Jul 2025 07:17:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1752391085; bh=3tPQQyB9eXTIb3XNbybVUxCt9L6HRc0Tde1G6CrS+SQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fwCRotWBhQXhBmXvcSd99JiYdtJb2P07WEbN614ZfrvI6K1Q+ujga1LEyCHX31fku iQsiVQsBNJ6jq5+JEyMu+srF98RKnOpITckdeQRJCN1uOPniDG2nhQ4sybFfUmCqfl xhAtozenT7QZMyXx9IIK9ceeYztE/jSTo08w1nh7lGHoi7ylXkOoU6gEBnfhHl39zb HbPMD4QmunRUuGcD7D41Yy5PQBwwLjgxTspYzU99OEwWzqx1qGrU/LiOI6yhv4WMad oh4CInED6JTNScnbKLZF8ZyJwecbuO1/9IXQf/Fk0H6AoZ612pkaZmNe2W04dWuVAo defDw0ACOFM5Q== From: Mike Rapoport To: Andrew Morton Cc: Andy Lutomirski , Borislav Petkov , Christophe Leroy , Daniel Gomez , Dave Hansen , Ingo Molnar , "Liam R. Howlett" , Luis Chamberlain , Mark Rutland , Masami Hiramatsu , Mike Rapoport , "H. Peter Anvin" , Peter Zijlstra , Petr Pavlu , Sami Tolvanen , Steven Rostedt , Thomas Gleixner , Yann Ylavic , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 4/8] execmem: move execmem_force_rw() and execmem_restore_rox() before use Date: Sun, 13 Jul 2025 10:17:26 +0300 Message-ID: <20250713071730.4117334-5-rppt@kernel.org> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250713071730.4117334-1-rppt@kernel.org> References: <20250713071730.4117334-1-rppt@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Mike Rapoport (Microsoft)" to avoid static declarations. Acked-by: Peter Zijlstra (Intel) Signed-off-by: Mike Rapoport (Microsoft) --- mm/execmem.c | 44 +++++++++++++++++++++----------------------- 1 file changed, 21 insertions(+), 23 deletions(-) diff --git a/mm/execmem.c b/mm/execmem.c index 4670e97f8e4e..056d3caaf4a1 100644 --- a/mm/execmem.c +++ b/mm/execmem.c @@ -137,6 +137,27 @@ static int execmem_set_direct_map_valid(struct vm_stru= ct *vm, bool valid) return err; } =20 +static int execmem_force_rw(void *ptr, size_t size) +{ + unsigned int nr =3D PAGE_ALIGN(size) >> PAGE_SHIFT; + unsigned long addr =3D (unsigned long)ptr; + int ret; + + ret =3D set_memory_nx(addr, nr); + if (ret) + return ret; + + return set_memory_rw(addr, nr); +} + +int execmem_restore_rox(void *ptr, size_t size) +{ + unsigned int nr =3D PAGE_ALIGN(size) >> PAGE_SHIFT; + unsigned long addr =3D (unsigned long)ptr; + + return set_memory_rox(addr, nr); +} + static void execmem_cache_clean(struct work_struct *work) { struct maple_tree *free_areas =3D &execmem_cache.free_areas; @@ -328,8 +349,6 @@ static inline void *pending_free_clear(void *ptr) return (void *)((unsigned long)ptr & ~PENDING_FREE_MASK); } =20 -static int execmem_force_rw(void *ptr, size_t size); - static int __execmem_cache_free(struct ma_state *mas, void *ptr, gfp_t gfp= _mask) { size_t size =3D mas_range_len(mas); @@ -415,27 +434,6 @@ static bool execmem_cache_free(void *ptr) return true; } =20 -static int execmem_force_rw(void *ptr, size_t size) -{ - unsigned int nr =3D PAGE_ALIGN(size) >> PAGE_SHIFT; - unsigned long addr =3D (unsigned long)ptr; - int ret; - - ret =3D set_memory_nx(addr, nr); - if (ret) - return ret; - - return set_memory_rw(addr, nr); -} - -int execmem_restore_rox(void *ptr, size_t size) -{ - unsigned int nr =3D PAGE_ALIGN(size) >> PAGE_SHIFT; - unsigned long addr =3D (unsigned long)ptr; - - return set_memory_rox(addr, nr); -} - #else /* CONFIG_ARCH_HAS_EXECMEM_ROX */ /* * when ROX cache is not used the permissions defined by architectures for --=20 2.47.2 From nobody Sun Feb 8 11:07:32 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 909BD1EDA2B; Sun, 13 Jul 2025 07:18:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391091; cv=none; b=nCg2Cr0w6mPQymdvt4WT1Xk88DEHcTlGi2sGNHVDBKuNQ9a73Ox4nf1gPCNr35RCaSxzysNKpmQ6rR1hNsHo+fD9FdVl/7064o/ftE76RlgBvMLhZnCtjjFaC9hiEewoD20ZDUwCEEFGUUOh0vrVrQYE3SepxHRAFXE9XcFZ+nk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391091; c=relaxed/simple; bh=2q/JtK0Q0Ijdxbx3Xk3DGzA5d9fd0T9sey9wBFRhk6c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cGr9XhmvuzIHsfDFO611mqOIHTFGhHNdDmm0Wi5AYqnbOSGWCzk/9FSNZVg7UDWNjbdMQcvNsl6ZZQJHT85aLAv3WvBs7jdNM1gWT9FEwOSGby3xvdbk+VrCVj5N0RBGuzGlHUueS6igD6Oy4GoqEkarizw3wq89Iky39mRn0vA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NwSDMb+G; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NwSDMb+G" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B0F54C4CEF1; Sun, 13 Jul 2025 07:18:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1752391091; bh=2q/JtK0Q0Ijdxbx3Xk3DGzA5d9fd0T9sey9wBFRhk6c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NwSDMb+GR3jpqmG1F3gNIPFiP9ubGEaHCEyQ7aMu/Y+Xknb4nxFNImBJJYI51QT6P KgGd/2j6MCWyzEQY3jR902Xj3FabA/SKh3ArnVhr6rflgjyjnJI8uduaLAFVDsm7aR RBz9IXjMkl6u/R4v4J+/kKBKq8au9kpkwKHfkHL3p61+vsi5260EwHVIYx77QZfgQ7 IW+D6YNOOT2jpjr2mEWmBuwmXfDSfRVinE+B9zcAHNrPHtnwHt+V+ts8VlGyuklqVN xyS49/X79Ep8paxlLZOmnLZmIxGdhVWVce5JTdmgjIaYq3ncmYRDpPkSx0V+8BGWHa edXw+gArcXUjA== From: Mike Rapoport To: Andrew Morton Cc: Andy Lutomirski , Borislav Petkov , Christophe Leroy , Daniel Gomez , Dave Hansen , Ingo Molnar , "Liam R. Howlett" , Luis Chamberlain , Mark Rutland , Masami Hiramatsu , Mike Rapoport , "H. Peter Anvin" , Peter Zijlstra , Petr Pavlu , Sami Tolvanen , Steven Rostedt , Thomas Gleixner , Yann Ylavic , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 5/8] execmem: add fallback for failures in vmalloc(VM_ALLOW_HUGE_VMAP) Date: Sun, 13 Jul 2025 10:17:27 +0300 Message-ID: <20250713071730.4117334-6-rppt@kernel.org> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250713071730.4117334-1-rppt@kernel.org> References: <20250713071730.4117334-1-rppt@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Mike Rapoport (Microsoft)" When execmem populates ROX cache it uses vmalloc(VM_ALLOW_HUGE_VMAP). Although vmalloc falls back to allocating base pages if high order allocation fails, it may happen that it still cannot allocate enough memory. Right now ROX cache is only used by modules and in majority of cases the allocations happen at boot time when there's plenty of free memory, but upcoming enabling ROX cache for ftrace and kprobes would mean that execmem allocations can happen when the system is under memory pressure and a failure to allocate large page worth of memory becomes more likely. Fallback to regular vmalloc() if vmalloc(VM_ALLOW_HUGE_VMAP) fails. Acked-by: Peter Zijlstra (Intel) Signed-off-by: Mike Rapoport (Microsoft) --- mm/execmem.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/execmem.c b/mm/execmem.c index 056d3caaf4a1..04c35c3a9361 100644 --- a/mm/execmem.c +++ b/mm/execmem.c @@ -291,6 +291,11 @@ static int execmem_cache_populate(struct execmem_range= *range, size_t size) =20 alloc_size =3D round_up(size, PMD_SIZE); p =3D execmem_vmalloc(range, alloc_size, PAGE_KERNEL, vm_flags); + if (!p) { + alloc_size =3D size; + p =3D execmem_vmalloc(range, alloc_size, PAGE_KERNEL, vm_flags); + } + if (!p) return err; =20 @@ -462,7 +467,7 @@ void *execmem_alloc(enum execmem_type type, size_t size) bool use_cache =3D range->flags & EXECMEM_ROX_CACHE; unsigned long vm_flags =3D VM_FLUSH_RESET_PERMS; pgprot_t pgprot =3D range->pgprot; - void *p; + void *p =3D NULL; =20 size =3D PAGE_ALIGN(size); =20 --=20 2.47.2 From nobody Sun Feb 8 11:07:32 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14D261DF756; Sun, 13 Jul 2025 07:18:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391098; cv=none; b=ci22WhYglMnJIcZmrpvM54N5tOXuXOi//vM5SOyXSUzwWKDavoME3kOgd7EHSIMBmk4lBHjj9VYNDMic+e2SSJb+Z5VcYUyUKvRaLU/O/GTye3tzVWD8lFW9f6URZmP36HSlwQzXzxNZo6UBbPOGnFvzTd2pivap74Ned4Rr6zU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391098; c=relaxed/simple; bh=bidj/q64QU+7yTnn+7Fdxeop4Ya6ZnjaNbOgpoV8vTA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HoHSRJd1FCBXpv+v5EgJwujEoD5n43GjX7M5cvB5WZH6riSikHSZFbIQA2NO4Zmxt7PcK+H1s3MpG7W00EnXSEaVeGqhSflFkwS0henSCNk+3AXiYYfUSk+HGI/fzTA9kpx2Wctxxzn+7q42Pz0TSTMG8W5vgwPhGRELHRBWSMc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qGwTKuxZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qGwTKuxZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EFA2BC4CEE3; Sun, 13 Jul 2025 07:18:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1752391097; bh=bidj/q64QU+7yTnn+7Fdxeop4Ya6ZnjaNbOgpoV8vTA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qGwTKuxZerSIN55A/xBAJe0q0FfGdCZ4FMRonrPNNRAH+1fDtxxN+n8bx18oax0Jz W6PY2ndfl+tJi7BbZRpVb8av8GGe/6IN1v6AVqmAz7UjuG1KPwJEDWVRKmviqdCoR6 RrVO+USneuiLQmildLfpCcrdN0tn9FNkTxdW4SazIX6PvmeAuzDazSlZgmLVlATSgp ORvnWAwza4lVpg5gdfH4pLaEK7ULljXB1jD/p5CXXifh0lC+6GHKq32WyUHhuXVIr6 IxCNtCwg3Rkk5/BG+KJffbYTVDmrr8A6gSrHhyPgj7wHgq12R+jG5ECaXJiQe9kbQx 1BrE2MPjlqa9w== From: Mike Rapoport To: Andrew Morton Cc: Andy Lutomirski , Borislav Petkov , Christophe Leroy , Daniel Gomez , Dave Hansen , Ingo Molnar , "Liam R. Howlett" , Luis Chamberlain , Mark Rutland , Masami Hiramatsu , Mike Rapoport , "H. Peter Anvin" , Peter Zijlstra , Petr Pavlu , Sami Tolvanen , Steven Rostedt , Thomas Gleixner , Yann Ylavic , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 6/8] execmem: drop writable parameter from execmem_fill_trapping_insns() Date: Sun, 13 Jul 2025 10:17:28 +0300 Message-ID: <20250713071730.4117334-7-rppt@kernel.org> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250713071730.4117334-1-rppt@kernel.org> References: <20250713071730.4117334-1-rppt@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Mike Rapoport (Microsoft)" After update of execmem_cache_free() that made memory writable before updating it, there is no need to update read only memory, so the writable parameter to execmem_fill_trapping_insns() is not needed. Drop it. Acked-by: Peter Zijlstra (Intel) Signed-off-by: Mike Rapoport (Microsoft) --- arch/x86/mm/init.c | 8 ++------ include/linux/execmem.h | 3 +-- mm/execmem.c | 4 ++-- 3 files changed, 5 insertions(+), 10 deletions(-) diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index 7456df985d96..dbc63f0d538f 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -1063,13 +1063,9 @@ unsigned long arch_max_swapfile_size(void) static struct execmem_info execmem_info __ro_after_init; =20 #ifdef CONFIG_ARCH_HAS_EXECMEM_ROX -void execmem_fill_trapping_insns(void *ptr, size_t size, bool writeable) +void execmem_fill_trapping_insns(void *ptr, size_t size) { - /* fill memory with INT3 instructions */ - if (writeable) - memset(ptr, INT3_INSN_OPCODE, size); - else - text_poke_set(ptr, INT3_INSN_OPCODE, size); + memset(ptr, INT3_INSN_OPCODE, size); } #endif =20 diff --git a/include/linux/execmem.h b/include/linux/execmem.h index 8b61b05da7d5..7de229134e30 100644 --- a/include/linux/execmem.h +++ b/include/linux/execmem.h @@ -60,12 +60,11 @@ enum execmem_range_flags { * will trap * @ptr: pointer to memory to fill * @size: size of the range to fill - * @writable: is the memory poited by @ptr is writable or ROX * * A hook for architecures to fill execmem ranges with invalid instruction= s. * Architectures that use EXECMEM_ROX_CACHE must implement this. */ -void execmem_fill_trapping_insns(void *ptr, size_t size, bool writable); +void execmem_fill_trapping_insns(void *ptr, size_t size); =20 /** * execmem_restore_rox - restore read-only-execute permissions diff --git a/mm/execmem.c b/mm/execmem.c index 04c35c3a9361..0822305413ec 100644 --- a/mm/execmem.c +++ b/mm/execmem.c @@ -304,7 +304,7 @@ static int execmem_cache_populate(struct execmem_range = *range, size_t size) goto err_free_mem; =20 /* fill memory with instructions that will trap */ - execmem_fill_trapping_insns(p, alloc_size, /* writable =3D */ true); + execmem_fill_trapping_insns(p, alloc_size); =20 err =3D set_memory_rox((unsigned long)p, vm->nr_pages); if (err) @@ -363,7 +363,7 @@ static int __execmem_cache_free(struct ma_state *mas, v= oid *ptr, gfp_t gfp_mask) if (err) return err; =20 - execmem_fill_trapping_insns(ptr, size, /* writable =3D */ true); + execmem_fill_trapping_insns(ptr, size); execmem_restore_rox(ptr, size); =20 err =3D execmem_cache_add_locked(ptr, size, gfp_mask); --=20 2.47.2 From nobody Sun Feb 8 11:07:32 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 623321E22E6; Sun, 13 Jul 2025 07:18:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391104; cv=none; b=QFnx6H6uXQuto9/9B0tzkCUhR+HlnQF8cgWsLfsiMm5jWjm1MwLUIKGKW7QUtuj7WIceuGsylxuZFDfVbMxuyCtfhJ5mCONKlmgHf9YAWs42DiqBQlPZzJCjf5ZRIiSPTHCZDQ0nv86h+Y6JKDQJw0B0T/6P5tBo/45Ltj4hvMw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391104; c=relaxed/simple; bh=9YZfDECqQGQ531UjogQBUu7pWagY+AKXs1wDqpr/qhs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WrZI3DUvY8of4T23rLfKtY4/gq3nYaEIS6V4RhIwQI7pgKHRef6SO0OfkilqQvIxMDo6LAURuNc8HIoCvVjLL879Jf1OffcJ+/tPfcgoX+Ijw8H0AZkeD92fPDParmXwwmJzK0nHBoeqgupzPMvXWljoITCyLpqU6z8YNE+d2w4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KamgB060; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KamgB060" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 38D6FC4CEF1; Sun, 13 Jul 2025 07:18:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1752391103; bh=9YZfDECqQGQ531UjogQBUu7pWagY+AKXs1wDqpr/qhs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KamgB060Bvsoix98TZWks50/I13lZhrhFcsv9XHufF9YHPir24VlIQe5VkT51CH4Z AeUD3eAP+47gTJeIkA3ABhZ7bbzEsShNDKHcGpJPq55trM7HD7fWewlpUaw5bFpAzO zT/Ld6lcfyFAOc4uWlHlzn1rHlCVB6vsTk6gxwhCu3dfSJHU2LYsHLfSBraWFIFaCr xGRjaCyOt4iwAE7pLT/gIFx9WUkXhIZLigZNwfxL1nMdOC+N+2Qym6UAyA2UJcrsTf xsdpneO8e3DVCZI9ht9+RQ2V63FSsOrr2amroBsSJKgkIU3wWO4ekDAQ2kUkCfpuA9 tAcv5E/0mTyJg== From: Mike Rapoport To: Andrew Morton Cc: Andy Lutomirski , Borislav Petkov , Christophe Leroy , Daniel Gomez , Dave Hansen , Ingo Molnar , "Liam R. Howlett" , Luis Chamberlain , Mark Rutland , Masami Hiramatsu , Mike Rapoport , "H. Peter Anvin" , Peter Zijlstra , Petr Pavlu , Sami Tolvanen , Steven Rostedt , Thomas Gleixner , Yann Ylavic , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 7/8] x86/kprobes: enable EXECMEM_ROX_CACHE for kprobes allocations Date: Sun, 13 Jul 2025 10:17:29 +0300 Message-ID: <20250713071730.4117334-8-rppt@kernel.org> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250713071730.4117334-1-rppt@kernel.org> References: <20250713071730.4117334-1-rppt@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Mike Rapoport (Microsoft)" x86::alloc_insn_page() always allocates ROX memory. Instead of overriding this method, add EXECMEM_KPROBES entry in execmem_info with pgprot set to PAGE_KERNEL_ROX and use ROX cache when configuration and CPU features allow it. Acked-by: Peter Zijlstra (Intel) Signed-off-by: Mike Rapoport (Microsoft) Acked-by: Masami Hiramatsu (Google) --- arch/x86/kernel/kprobes/core.c | 18 ------------------ arch/x86/mm/init.c | 9 ++++++++- 2 files changed, 8 insertions(+), 19 deletions(-) diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index 47cb8eb138ba..6079d15dab8c 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -481,24 +481,6 @@ static int prepare_singlestep(kprobe_opcode_t *buf, st= ruct kprobe *p, return len; } =20 -/* Make page to RO mode when allocate it */ -void *alloc_insn_page(void) -{ - void *page; - - page =3D execmem_alloc(EXECMEM_KPROBES, PAGE_SIZE); - if (!page) - return NULL; - - /* - * TODO: Once additional kernel code protection mechanisms are set, ensure - * that the page was not maliciously altered and it is still zeroed. - */ - set_memory_rox((unsigned long)page, 1); - - return page; -} - /* Kprobe x86 instruction emulation - only regs->ip or IF flag modifiers */ =20 static void kprobe_emulate_ifmodifiers(struct kprobe *p, struct pt_regs *r= egs) diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index dbc63f0d538f..442fafd8ff52 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -1098,7 +1098,14 @@ struct execmem_info __init *execmem_arch_setup(void) .pgprot =3D pgprot, .alignment =3D MODULE_ALIGN, }, - [EXECMEM_KPROBES ... EXECMEM_BPF] =3D { + [EXECMEM_KPROBES] =3D { + .flags =3D flags, + .start =3D start, + .end =3D MODULES_END, + .pgprot =3D PAGE_KERNEL_ROX, + .alignment =3D MODULE_ALIGN, + }, + [EXECMEM_FTRACE ... EXECMEM_BPF] =3D { .flags =3D EXECMEM_KASAN_SHADOW, .start =3D start, .end =3D MODULES_END, --=20 2.47.2 From nobody Sun Feb 8 11:07:32 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 524B61367; Sun, 13 Jul 2025 07:18:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391110; cv=none; b=DxaT3TfhhLtjlvRu3SuWNQd3dw/5seJSXprCpWyk+a/kmo3yB27rgwog/RQDAY15t+e7aCHmA5Kuc3nGbIXOpBehyfLJPnsCqN8IqaS//TPJ206e1lQDdT2TG2ZGsKJk6uCGHJgKMWnfRf7t/Gtv3DlMEacyMFKt9Op+FJYNeMA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752391110; c=relaxed/simple; bh=W48KJjyddYpCBONIDAqxZPq1nB/qirB2tcBj7MhxtNE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IXQUwaTlGj5gJzGhR8BPG/cR7J7w0i4faz8lYbI9zwfPELYi+9EnueXNWqXrvPhjBorBFpT8vEOCXpF7Tz4Cf0/vDXC/Y+wTWdgSIF03S3kmlhmo3tTdHHKt1alO0ebVbJokTSDP3FVknfuB3og7JbnybZlvLYmTuIuSpdY9Zyw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Kko8FVru; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Kko8FVru" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 78505C4CEE3; Sun, 13 Jul 2025 07:18:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1752391110; bh=W48KJjyddYpCBONIDAqxZPq1nB/qirB2tcBj7MhxtNE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Kko8FVruo1H5ovxkMj8GTy4gtboaPCoaqQovHBDSAmOvEBYVN48O7a941dOPKJgGM o/pEbOGLWOdAkOf0JkG/h6vYJMqOyH9aP6vFVq7uWnz2fdKNq306beYBllZyYUl1tU pv9GbhCHGM4KwAjxTAb7CglQhzWcGdep1RhUMpRWd6OSaMngK2x1EtgaNVHRtiQkk7 U09qt1tgwEhsudWasaDujoJxeotNoc0Z9VMPKz959zD73V/xrMz3TLKSJt0KbjDHsL ppyeypK4EkPtO7l9Ht+aK2YdM/PT3j1lZDobitymJTLJQfWHhZrjg3Yy3XVhBsgWX3 zDU9+1czJ7zLQ== From: Mike Rapoport To: Andrew Morton Cc: Andy Lutomirski , Borislav Petkov , Christophe Leroy , Daniel Gomez , Dave Hansen , Ingo Molnar , "Liam R. Howlett" , Luis Chamberlain , Mark Rutland , Masami Hiramatsu , Mike Rapoport , "H. Peter Anvin" , Peter Zijlstra , Petr Pavlu , Sami Tolvanen , Steven Rostedt , Thomas Gleixner , Yann Ylavic , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 8/8] x86/ftrace: enable EXECMEM_ROX_CACHE for ftrace allocations Date: Sun, 13 Jul 2025 10:17:30 +0300 Message-ID: <20250713071730.4117334-9-rppt@kernel.org> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250713071730.4117334-1-rppt@kernel.org> References: <20250713071730.4117334-1-rppt@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Mike Rapoport (Microsoft)" For the most part ftrace uses text poking and can handle ROX memory. The only place that requires writable memory is create_trampoline() that updates the allocated memory and in the end makes it ROX. Use execmem_alloc_rw() in x86::ftrace::alloc_tramp() and enable ROX cache for EXECMEM_FTRACE when configuration and CPU features allow that. Acked-by: Peter Zijlstra (Intel) Signed-off-by: Mike Rapoport (Microsoft) Acked-by: Steven Rostedt (Google) Tested-by: Steven Rostedt (Google) --- arch/x86/kernel/ftrace.c | 2 +- arch/x86/mm/init.c | 9 ++++++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index 252e82bcfd2f..4450acec9390 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -263,7 +263,7 @@ void arch_ftrace_update_code(int command) =20 static inline void *alloc_tramp(unsigned long size) { - return execmem_alloc(EXECMEM_FTRACE, size); + return execmem_alloc_rw(EXECMEM_FTRACE, size); } static inline void tramp_free(void *tramp) { diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index 442fafd8ff52..bb57e93b4caf 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -1105,7 +1105,14 @@ struct execmem_info __init *execmem_arch_setup(void) .pgprot =3D PAGE_KERNEL_ROX, .alignment =3D MODULE_ALIGN, }, - [EXECMEM_FTRACE ... EXECMEM_BPF] =3D { + [EXECMEM_FTRACE] =3D { + .flags =3D flags, + .start =3D start, + .end =3D MODULES_END, + .pgprot =3D pgprot, + .alignment =3D MODULE_ALIGN, + }, + [EXECMEM_BPF] =3D { .flags =3D EXECMEM_KASAN_SHADOW, .start =3D start, .end =3D MODULES_END, --=20 2.47.2