From nobody Thu Dec 18 00:46:08 2025 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 695F279F5; Sun, 23 Feb 2025 06:21:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740291663; cv=none; b=trc9bqSKXJV2YXgXtRA+3Y+SYTN9B8qWSzJtzdauPxh5EK/NJBIkHfErMKI9jDThVRZ44oZm2+/VgcUqFl3LhKdhHCUUYE8FHIkMzG2ai1PEWwE5uwT0P/cIRZ4zlC4H9H1QeASuDjMktsU1RYNSx37AbOoea+iNzwfft6qM4N0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740291663; c=relaxed/simple; bh=z4WeWec5onOz0GyULUxJn8ppLCWKgFwNc0r+lEcc0pg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=SgKVyvOTxqjFGLxuVpus4CaXdvwZudYJfNjIUlpieKwYN/pYxEHdXN1aeWn2zIWaPpe4bjmHqzbgSjnTRkOi8SLEV2qfZkXf5cNp6AZkMfF9AVubgB0T1PNP9oDBR/v9m7qOWT6SnnkCU3g/5NdtXYsne2cScfVsk2S0yG2cr2A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Zdis9ZSC; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Zdis9ZSC" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-2210d92292eso103182985ad.1; Sat, 22 Feb 2025 22:21:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740291662; x=1740896462; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ufyJpXRplLa9hm9rmTP3gRUIzS6V2cY3a3pc+z2Os5c=; b=Zdis9ZSCZZfozzSdwHdmXrHbYav1Tog0vxMY2QcBmyCUNXIJuJEXi0n2Z9O7TVnvfn K8J4GPQ6KysL9C3G2evNyFP+VQjEda2pdJ3ivCDEp3TwK9YPVmfobiwgHyKMAbDvXgkR pYy8oyr906ebZkdspN5dhjO1pe+6BYP+bh9BRKKYNhEdLu+gKGlsYL6jMS248JCorqKT Z/5Kmd6aFPvR5w89Z7bkGgti4BEoa4O08ZFQXuvIevaRXLJ5Ot9yLXa5n5qo1wE4H9cn tO32Lal6qEftLm2uLfnbhGw28tRyR0oCV+M1u0hE4OybjxH1xphqREiFvjC4Wgi/akHz gxNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740291662; x=1740896462; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ufyJpXRplLa9hm9rmTP3gRUIzS6V2cY3a3pc+z2Os5c=; b=hHLz2Dpt++Kg/bzQEBTvjrH5+PVAqwtuGE4UgS6RftV0kHYnGhVqCqOvpZuW9E6cl5 m+oIY6Nt/LI5Q3InFVM63IM4xrJ/HHlRnNduWdKxaueJd4uxpgpl9Zaa82xmZtnhrKW6 C+i60M2mhG9iW5nFBzcsZZR6g9/RcUa71dH4IBOJbT8Y+E0Qh/bzIpVrFMeZX4Z2/Cag 6iNdYLpD6D8tYC7nPOzY4ny19ahLQKOJWHrq21JiZng+DfEak1Hd5sW5z6/pzHcHa5RB ZdCOpAC6SoeK0NXcsoRUP5cWtO77Ld78jZwwqVQX59CkXHPUkmcH3dlfH0hEwQ3sBYgi 5jGA== X-Forwarded-Encrypted: i=1; AJvYcCV9gpIoVZQwoPOCeiERyVVD4xF7VWPQwb+GgACpZtHi6j1DlGSFO3W6FHR7Q57ZW9wyxY8nwZZoCfcjzio=@vger.kernel.org X-Gm-Message-State: AOJu0YwuxjTL6Ju0frwjUbhJYLg/zb0Q5MQ9Zi2hvpVszz+axHMgjp89 TO6f5SxYnirLwD8JgvjBC3fAgGzp0EA7y6GGXIlG5bXPbwP3c2Qn X-Gm-Gg: ASbGncvMK8FJrHHnZKUjjvwddU/kcjx6L0vGTh+z0GtSY21jfcSCgOHfRj0wBw41h9u /EEpCSJ2YaOg8os3RzsoE/EzhXXVDLO9mUTEcAUOG8XE2AS6NPcz1mS+a193eRiBDM+Wi27tKmW h+tmyJgYCrDTPgjVm8kdVuLqoE6AkuvaQyy58q2YG2g/FbgRUN8iqTALiEaeawWMJI60iEtAMcH L51fvcrQnwIG6PphxLq72itQ+z3AlSnga6hObEIfLqP4GSDdCPYjTayd3aoDw9Im0osB3sQ7jft ShH+yxXXrSJSrqjGt4fIAn5Osi8Me7531cZ4jgzrCm4+ouDlwKc= X-Google-Smtp-Source: AGHT+IEPNl9mM5zcIK3tf/4Yv4mHNQs8Tdk1SzRidSAdXg+GHtAgaKb84FRZPdA4JmQKeKxO311M9Q== X-Received: by 2002:a17:903:41d0:b0:221:337:4862 with SMTP id d9443c01a7336-2219ff50e97mr142473265ad.15.1740291661607; Sat, 22 Feb 2025 22:21:01 -0800 (PST) Received: from localhost.localdomain ([39.144.244.105]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d558ed3fsm160750795ad.232.2025.02.22.22.20.57 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sat, 22 Feb 2025 22:21:01 -0800 (PST) From: Yafang Shao To: jpoimboe@kernel.org, jikos@kernel.org, mbenes@suse.cz, pmladek@suse.com, joe.lawrence@redhat.com Cc: live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, Yafang Shao Subject: [PATCH v2 1/2] livepatch: Add comment to clarify klp_add_nops() Date: Sun, 23 Feb 2025 14:20:45 +0800 Message-Id: <20250223062046.2943-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20250223062046.2943-1-laoar.shao@gmail.com> References: <20250223062046.2943-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add detailed comments to clarify the purpose of klp_add_nops() function. These comments are based on Petr's explanation[0]. Link: https://lore.kernel.org/all/Z6XUA7D0eU_YDMVp@pathway.suse.cz/ [0] Suggested-by: Petr Mladek Suggested-by: Josh Poimboeuf Signed-off-by: Yafang Shao --- kernel/livepatch/core.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c index 0cd39954d5a1..4a0fb7978d0d 100644 --- a/kernel/livepatch/core.c +++ b/kernel/livepatch/core.c @@ -601,9 +601,12 @@ static int klp_add_object_nops(struct klp_patch *patch, } =20 /* - * Add 'nop' functions which simply return to the caller to run - * the original function. The 'nop' functions are added to a - * patch to facilitate a 'replace' mode. + * Add 'nop' functions which simply return to the caller to run the + * original function. + * + * They are added only when the atomic replace mode is used and only for + * functions which are currently livepatched but are no longer included + * in the new livepatch. */ static int klp_add_nops(struct klp_patch *patch) { --=20 2.43.5 From nobody Thu Dec 18 00:46:08 2025 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 314811C84DD; Sun, 23 Feb 2025 06:21:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740291671; cv=none; b=UQ/av1p6e6ADJph82ACdz+jcCHAnfxqv8RE6frblyHEhfla/+q2rSUaOt1K2/nnEdzpNZidBk/l9s4B98L0RY66P9hqfp5t5fU58rOBvIhr4kg98DXKlUpeJ0aRl/TF1ms0RQfm9sKl3JwZsqorrGotNZRnHPJao4EaRwMhkhTs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740291671; c=relaxed/simple; bh=EAbTiBt0vbqz7Eckd3N6ONhKdF5omR8ineaSQGYw2gk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Vn60L5xSnNf3QrfJNV7nYfEMcdTGNQ70dM9gu3SQrk89xThbCCbe3h7r3it4DhUKyxI9n7btrU2CSIP0lTN041z0Bx/SPABMVEJQYuR+f0fVfgrH5tkfaJx3mXx06tfnGHoBhQEeR70RmFCwO9QPBF/bIjaSs544ZNzt+3qdPqo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=FhYHRBaC; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FhYHRBaC" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-220e6028214so73384265ad.0; Sat, 22 Feb 2025 22:21:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740291666; x=1740896466; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2Z2GJmTzeGj4EN3VSoIAcvWrzfzVxHWYzShnooMd6fI=; b=FhYHRBaCTTp2Ex4wtinD1Eb6SKd921jN06x55vPG7TEvlqKIyZ6iCITSjG1XbRF2ef QAcFe7mE89GE8Xk5vhdTwdb7IjeLKVgt3Ur3CXNUXsp1R5U3sqWTGG0olvnoSGTfqEH/ GuG4EuBKk2G+WFVy9JqdWw/MsWUBrM8rUdItcOPbpro3zURk9FmnuVUa048wAnwhpDzY xQappn0dBj8ISjlPTPBkkhSC/GRw0E4JlW6ICIv6KVOBBcUPOU0CRZnAbbrEKEqRmyYX 6tLIP0OkpOHMzsPY8+Xt5gtU1du8EkBkEOsiMkRuVnYNe3L0MajlyxJ/C1s9FJK39dXx rWcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740291666; x=1740896466; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2Z2GJmTzeGj4EN3VSoIAcvWrzfzVxHWYzShnooMd6fI=; b=six65+aky/f8DM3pl6ZVsEJy/SqbTBZ85q+y7XM+Vn4YRxW6XY900HyN/x5tcT9WhU jeH/xeCvRt5Ezhb4bT0bkbAYvqauj7ctDVxFt/p1xd8jNoNhVchHAH8yk2mblCnhj9RF B7iXYSbN0J134bB3BqAm+9FFvOGdeL8l5OIddKDTROxL7P8BaePO8FtHhB9aIKKYyyCc 2GWSOaMXiUXpNJC2Ey6F1VarpcrXQxeMc2ZKXMAm6CGo45FSHc/3iyaaMgtxaq85yeWH aU8wCEUSdM5oqVdjWKOS+MFdMxR2i+NeXpzpuEe1EKzg540RtCI9xu/jyp4mUylaPb3/ VAoA== X-Forwarded-Encrypted: i=1; AJvYcCURcI23eioAEubUpxAhgJCkOOtjOf3TUd6R2EwV1/0P5kYz2Qb5bVld3I/jFSitL/luB146vB3YCFA714Y=@vger.kernel.org X-Gm-Message-State: AOJu0YyqErnur5d20NU0N3h23kZPNgusFBBrTI03frdR+5KyGEb7g2xx Bn30y/0yqKwJx6kAHb4lkxiugLIfM/JxiMELrfL0Eri/Xx16u+lQ X-Gm-Gg: ASbGncviYCBbP7YibhlHz3V9jYVOwo5TpUZFtTz4hz8W28zSUgidywEwZ1ye1T7UqW7 CEvB2AQLjuEE79QIBu1yPuDlE9Z2MQF5v6WbqwwWIrIlqLlsmqSQFOS5le3/retxFZ9PtU5O2up I//q4wW6QOpzhqQIWliWs10cKqCsNmMD251SnpTlD8ooG8YJ1XG4mAHgiCW4IUXQU8rzonrWwFI 84StxCb+K3rFdQq/2gdbkvPA4u+UuNnfizlykxQXMUuc3D7pNJevs3L5332li5dxx7dmJoxpFRm fPOS9cVhnJhOJEVfItKl2UYpNUhkp2Sis3XfbgBtdPsPw0kTvMI= X-Google-Smtp-Source: AGHT+IE7MSQSoGKzifVL3ZzOf04hSMJvugBu/CqYY7Cb81rTd6tX1n3ErSxzrifLSnKw+/0gra+J/g== X-Received: by 2002:a17:903:22d1:b0:215:4a4e:9262 with SMTP id d9443c01a7336-2219ff8287bmr119209455ad.8.1740291666512; Sat, 22 Feb 2025 22:21:06 -0800 (PST) Received: from localhost.localdomain ([39.144.244.105]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-220d558ed3fsm160750795ad.232.2025.02.22.22.21.02 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sat, 22 Feb 2025 22:21:05 -0800 (PST) From: Yafang Shao To: jpoimboe@kernel.org, jikos@kernel.org, mbenes@suse.cz, pmladek@suse.com, joe.lawrence@redhat.com Cc: live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, Yafang Shao Subject: [PATCH v2 2/2] livepatch: Replace tasklist_lock with RCU Date: Sun, 23 Feb 2025 14:20:46 +0800 Message-Id: <20250223062046.2943-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20250223062046.2943-1-laoar.shao@gmail.com> References: <20250223062046.2943-1-laoar.shao@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The tasklist_lock in the KLP transition may cause high latency under certain workloads. To address this, we can replace it with RCU. When a new task is forked, its kernel stack is always initialized to empty[0]. As a result, we can set these new tasks to the KLP_TRANSITION_IDLE state immediately after forking. If these tasks are forked during the KLP transition but before klp_check_and_switch_task(), it is safe to switch them to the klp_target_state within klp_check_and_switch_task(). Additionally, if the klp_ftrace_handler() is triggered before the task is switched to the klp_target_state, it is also safe to perform the state transition within this ftrace handler[1]. With these changes, we can safely replace the tasklist_lock with RCU. Link: https://lore.kernel.org/all/20250213173253.ovivhuq2c5rmvkhj@jpoimboe/= [0] Link: https://lore.kernel.org/all/20250214181206.xkvxohoc4ft26uhf@jpoimboe/= [1] Signed-off-by: Yafang Shao --- include/linux/livepatch.h | 4 ++-- kernel/fork.c | 2 +- kernel/livepatch/patch.c | 7 ++++++- kernel/livepatch/transition.c | 35 ++++++++++++++--------------------- kernel/livepatch/transition.h | 1 + 5 files changed, 24 insertions(+), 25 deletions(-) diff --git a/include/linux/livepatch.h b/include/linux/livepatch.h index 51a258c24ff5..41c424120f49 100644 --- a/include/linux/livepatch.h +++ b/include/linux/livepatch.h @@ -198,7 +198,7 @@ int klp_enable_patch(struct klp_patch *); int klp_module_coming(struct module *mod); void klp_module_going(struct module *mod); =20 -void klp_copy_process(struct task_struct *child); +void klp_init_process(struct task_struct *child); void klp_update_patch_state(struct task_struct *task); =20 static inline bool klp_patch_pending(struct task_struct *task) @@ -241,7 +241,7 @@ static inline int klp_module_coming(struct module *mod)= { return 0; } static inline void klp_module_going(struct module *mod) {} static inline bool klp_patch_pending(struct task_struct *task) { return fa= lse; } static inline void klp_update_patch_state(struct task_struct *task) {} -static inline void klp_copy_process(struct task_struct *child) {} +static inline void klp_init_process(struct task_struct *child) {} =20 static inline int klp_apply_section_relocs(struct module *pmod, Elf_Shdr *sechdrs, diff --git a/kernel/fork.c b/kernel/fork.c index 735405a9c5f3..da247c4d5ec5 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -2544,7 +2544,7 @@ __latent_entropy struct task_struct *copy_process( p->exit_signal =3D args->exit_signal; } =20 - klp_copy_process(p); + klp_init_process(p); =20 sched_core_fork(p); =20 diff --git a/kernel/livepatch/patch.c b/kernel/livepatch/patch.c index 90408500e5a3..5e523a3fbb3c 100644 --- a/kernel/livepatch/patch.c +++ b/kernel/livepatch/patch.c @@ -95,7 +95,12 @@ static void notrace klp_ftrace_handler(unsigned long ip, =20 patch_state =3D current->patch_state; =20 - WARN_ON_ONCE(patch_state =3D=3D KLP_TRANSITION_IDLE); + /* If the patch_state is KLP_TRANSITION_IDLE, it indicates the + * task was forked after klp_init_transition(). For this newly + * forked task, it is safe to switch it to klp_target_state. + */ + if (patch_state =3D=3D KLP_TRANSITION_IDLE) + current->patch_state =3D klp_target_state; =20 if (patch_state =3D=3D KLP_TRANSITION_UNPATCHED) { /* diff --git a/kernel/livepatch/transition.c b/kernel/livepatch/transition.c index ba069459c101..ae4512e2acc9 100644 --- a/kernel/livepatch/transition.c +++ b/kernel/livepatch/transition.c @@ -23,7 +23,7 @@ static DEFINE_PER_CPU(unsigned long[MAX_STACK_ENTRIES], k= lp_stack_entries); =20 struct klp_patch *klp_transition_patch; =20 -static int klp_target_state =3D KLP_TRANSITION_IDLE; +int klp_target_state =3D KLP_TRANSITION_IDLE; =20 static unsigned int klp_signals_cnt; =20 @@ -294,6 +294,13 @@ static int klp_check_and_switch_task(struct task_struc= t *task, void *arg) { int ret; =20 + /* If the patch_state remains KLP_TRANSITION_IDLE at this point, it + * indicates that the task was forked after klp_init_transition(). For + * this newly forked task, it is now safe to perform the switch. + */ + if (task->patch_state =3D=3D KLP_TRANSITION_IDLE) + goto out; + if (task_curr(task) && task !=3D current) return -EBUSY; =20 @@ -301,6 +308,7 @@ static int klp_check_and_switch_task(struct task_struct= *task, void *arg) if (ret) return ret; =20 +out: clear_tsk_thread_flag(task, TIF_PATCH_PENDING); task->patch_state =3D klp_target_state; return 0; @@ -466,11 +474,11 @@ void klp_try_complete_transition(void) * Usually this will transition most (or all) of the tasks on a system * unless the patch includes changes to a very common function. */ - read_lock(&tasklist_lock); + rcu_read_lock(); for_each_process_thread(g, task) if (!klp_try_switch_task(task)) complete =3D false; - read_unlock(&tasklist_lock); + rcu_read_unlock(); =20 /* * Ditto for the idle "swapper" tasks. @@ -694,25 +702,10 @@ void klp_reverse_transition(void) } =20 /* Called from copy_process() during fork */ -void klp_copy_process(struct task_struct *child) +void klp_init_process(struct task_struct *child) { - - /* - * The parent process may have gone through a KLP transition since - * the thread flag was copied in setup_thread_stack earlier. Bring - * the task flag up to date with the parent here. - * - * The operation is serialized against all klp_*_transition() - * operations by the tasklist_lock. The only exceptions are - * klp_update_patch_state(current) and __klp_sched_try_switch(), but we - * cannot race with them because we are current. - */ - if (test_tsk_thread_flag(current, TIF_PATCH_PENDING)) - set_tsk_thread_flag(child, TIF_PATCH_PENDING); - else - clear_tsk_thread_flag(child, TIF_PATCH_PENDING); - - child->patch_state =3D current->patch_state; + clear_tsk_thread_flag(child, TIF_PATCH_PENDING); + child->patch_state =3D KLP_TRANSITION_IDLE; } =20 /* diff --git a/kernel/livepatch/transition.h b/kernel/livepatch/transition.h index 322db16233de..febcf1d50fc5 100644 --- a/kernel/livepatch/transition.h +++ b/kernel/livepatch/transition.h @@ -5,6 +5,7 @@ #include =20 extern struct klp_patch *klp_transition_patch; +extern int klp_target_state; =20 void klp_init_transition(struct klp_patch *patch, int state); void klp_cancel_transition(void); --=20 2.43.5