From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0AAEB219A93; Mon, 21 Apr 2025 21:44:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271884; cv=none; b=uPMcBThKGnf+zpFNbda71Npy/K8S+hpTHeFVXxqOEFhdcOoM+NZX6OD541wyiEezSR3ZalVkCwJilyYYDCRhllr3xhcWAGXMHayO67rpupQdQ2vB3e5I0/WZtsv9Xx8oeuFLv8clMJXkKJsVOTV+9VL7qJVra+W0nKkUMRpDVfU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271884; c=relaxed/simple; bh=ozxOQG7S4Db/4oOGw6R6e/Vcd0TPftjqn1miU1qB/8w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=P2Sos+0bpQY+k8WmN+UFf3lw8ee2/z+9NH+PrmrsuFQGjT5F31a12MXT1+ub1VEOS+oVM26k6a2WlC9Fa1dy5ivfc/2Rqy5+9NhkrxKjKu9iTfWKjdXUqQBB3DF7s52HX6+daMCxDuk2uSZLH7dJjQmmlj8RdG/pr8QHGUh9w3Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=s4tTdWbw; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="s4tTdWbw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 730B3C4CEE4; Mon, 21 Apr 2025 21:44:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745271883; bh=ozxOQG7S4Db/4oOGw6R6e/Vcd0TPftjqn1miU1qB/8w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=s4tTdWbwM5h15iwNHXqBLIFn5Bwp1Bo3lGjoG2TUdrH1qgL2l1D9reFtXhh6YQCWV 5FUg9QtFIyCI7ze6GzYY1L14djqclSMDSI99I7dIs5Hanxpa9A/QBgpQ+av48U5Ujy oCK0mEVHdlKylTdUKzb2P4FWABqnrfLzK/B8j0jgmo4cvw3EOi+lqgJRq5N2EmjeuH nWxH6oztd8Kdp7j0eqYj8e04UAlqEdr2E8DeypPAIBH9z5zsCDjo+c8EqACZL04H+C 8qobOGgH0Kjm5lP/fNoEMxuE26PTZZq3Mr/PyXJD1OMjakO+eXqVZvm7eEwSZA6E7v UT55MMHSVrWvQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 01/22] uprobes: Rename arch_uretprobe_trampoline function Date: Mon, 21 Apr 2025 23:44:01 +0200 Message-ID: <20250421214423.393661-2-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We are about to add uprobe trampoline, so cleaning up the namespace. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- arch/x86/kernel/uprobes.c | 2 +- include/linux/uprobes.h | 2 +- kernel/events/uprobes.c | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c index 6d383839e839..77050e5a4680 100644 --- a/arch/x86/kernel/uprobes.c +++ b/arch/x86/kernel/uprobes.c @@ -338,7 +338,7 @@ extern u8 uretprobe_trampoline_entry[]; extern u8 uretprobe_trampoline_end[]; extern u8 uretprobe_syscall_check[]; =20 -void *arch_uprobe_trampoline(unsigned long *psize) +void *arch_uretprobe_trampoline(unsigned long *psize) { static uprobe_opcode_t insn =3D UPROBE_SWBP_INSN; struct pt_regs *regs =3D task_pt_regs(current); diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index 516217c39094..01112f27cd21 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -224,7 +224,7 @@ extern bool arch_uprobe_ignore(struct arch_uprobe *aup,= struct pt_regs *regs); extern void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr, void *src, unsigned long len); extern void uprobe_handle_trampoline(struct pt_regs *regs); -extern void *arch_uprobe_trampoline(unsigned long *psize); +extern void *arch_uretprobe_trampoline(unsigned long *psize); extern unsigned long uprobe_get_trampoline_vaddr(void); #else /* !CONFIG_UPROBES */ struct uprobes_state { diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 4c965ba77f9f..8415c087a71f 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1727,7 +1727,7 @@ static int xol_add_vma(struct mm_struct *mm, struct x= ol_area *area) return ret; } =20 -void * __weak arch_uprobe_trampoline(unsigned long *psize) +void * __weak arch_uretprobe_trampoline(unsigned long *psize) { static uprobe_opcode_t insn =3D UPROBE_SWBP_INSN; =20 @@ -1759,7 +1759,7 @@ static struct xol_area *__create_xol_area(unsigned lo= ng vaddr) init_waitqueue_head(&area->wq); /* Reserve the 1st slot for get_trampoline_vaddr() */ set_bit(0, area->bitmap); - insns =3D arch_uprobe_trampoline(&insns_size); + insns =3D arch_uretprobe_trampoline(&insns_size); arch_uprobe_copy_ixol(area->page, 0, insns, insns_size); =20 if (!xol_add_vma(mm, area)) --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 019B8217F5C; Mon, 21 Apr 2025 21:44:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271896; cv=none; b=BzHqLwifZZEICsj1kBCA2eVNdbSDrUBg17DYqRbLiZ7x5ddKCEkWmfxLnwdisY7cwDsdXXm0NSXiZ+7Qg9LOYA18HQmy6ZjtFT4aHzUXEMbcH2exnlNDVbyLXwNoMFbi/eR6jUmp1wrGkJgw85FFvFCxbQV/8c+Y5kL5DKTRVBI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271896; c=relaxed/simple; bh=dz+mPw0D7ZbQlws9G+xf9kwzc3zEDB0hb7VnLDfl2zU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SIUIo48AFk8YobQZE3XatqttOCG2/lpMv4ZaKl3PVez9GQ5pqOyvqjMUrlq6nGGNT6vfUtL/iX36GmtqKwa+difIgRCFLiTPnggjfXy6rrsS9CuRR5uFUPYMkkn4sgCddTioSgDMwtDT5bLQFU9ZdZFyQwShEQbqt/fg60zjctU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pENTG4xi; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pENTG4xi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 017B5C4CEE4; Mon, 21 Apr 2025 21:44:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745271895; bh=dz+mPw0D7ZbQlws9G+xf9kwzc3zEDB0hb7VnLDfl2zU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pENTG4xiQaWzhixahCVTsfXrBtVy2nCOwb+dFWdIvbZ/JqSVaCjBM9TXYNE+B6+wT CQwjNzd6ybMkcEWrtrVU0/JMXgxY6KfPgXus9hN4wgIkgj6qi1f5VPh0LWKQuquR0s /PwWbVEY9LWV6YyDXCp2aYraYEkO9c3/l+wAxPXs577zPMcQCEMWnbWgpPdv9lrdBy QPBB2FgWA9tdR0DjoaaK1gc7FByeGJxVkadeeerRRfz1vG9LlWZPDbV1GDX5RU9mS/ sfXCtrsU8BQlBWKEEr4N2MGuAShcOkgM/R6bJbjPQOoCXbJabhbkua++HDw0ykcbjE nZJMKCuGpkWfw== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 02/22] uprobes: Make copy_from_page global Date: Mon, 21 Apr 2025 23:44:02 +0200 Message-ID: <20250421214423.393661-3-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Making copy_from_page global and adding uprobe prefix. Adding the uprobe prefix to copy_to_page as well for symmetry. Acked-by: Andrii Nakryiko Signed-off-by: Jiri Olsa --- include/linux/uprobes.h | 1 + kernel/events/uprobes.c | 10 +++++----- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index 01112f27cd21..7447e15559b8 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -226,6 +226,7 @@ extern void arch_uprobe_copy_ixol(struct page *page, un= signed long vaddr, extern void uprobe_handle_trampoline(struct pt_regs *regs); extern void *arch_uretprobe_trampoline(unsigned long *psize); extern unsigned long uprobe_get_trampoline_vaddr(void); +extern void uprobe_copy_from_page(struct page *page, unsigned long vaddr, = void *dst, int len); #else /* !CONFIG_UPROBES */ struct uprobes_state { }; diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 8415c087a71f..87bca004ee6a 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -177,7 +177,7 @@ bool __weak is_trap_insn(uprobe_opcode_t *insn) return is_swbp_insn(insn); } =20 -static void copy_from_page(struct page *page, unsigned long vaddr, void *d= st, int len) +void uprobe_copy_from_page(struct page *page, unsigned long vaddr, void *d= st, int len) { void *kaddr =3D kmap_atomic(page); memcpy(dst, kaddr + (vaddr & ~PAGE_MASK), len); @@ -205,7 +205,7 @@ static int verify_opcode(struct page *page, unsigned lo= ng vaddr, uprobe_opcode_t * is a trap variant; uprobes always wins over any other (gdb) * breakpoint. */ - copy_from_page(page, vaddr, &old_opcode, UPROBE_SWBP_INSN_SIZE); + uprobe_copy_from_page(page, vaddr, &old_opcode, UPROBE_SWBP_INSN_SIZE); is_swbp =3D is_swbp_insn(&old_opcode); =20 if (is_swbp_insn(new_opcode)) { @@ -1052,7 +1052,7 @@ static int __copy_insn(struct address_space *mapping,= struct file *filp, if (IS_ERR(page)) return PTR_ERR(page); =20 - copy_from_page(page, offset, insn, nbytes); + uprobe_copy_from_page(page, offset, insn, nbytes); put_page(page); =20 return 0; @@ -1398,7 +1398,7 @@ struct uprobe *uprobe_register(struct inode *inode, return ERR_PTR(-EINVAL); =20 /* - * This ensures that copy_from_page(), copy_to_page() and + * This ensures that uprobe_copy_from_page(), copy_to_page() and * __update_ref_ctr() can't cross page boundary. */ if (!IS_ALIGNED(offset, UPROBE_SWBP_INSN_SIZE)) @@ -2394,7 +2394,7 @@ static int is_trap_at_addr(struct mm_struct *mm, unsi= gned long vaddr) if (result < 0) return result; =20 - copy_from_page(page, vaddr, &opcode, UPROBE_SWBP_INSN_SIZE); + uprobe_copy_from_page(page, vaddr, &opcode, UPROBE_SWBP_INSN_SIZE); put_page(page); out: /* This needs to return true for any variant of the trap insn */ --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32C03219A93; Mon, 21 Apr 2025 21:45:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271909; cv=none; b=jvr4Ab436v3IQsq6hjI8xpQm4UbvbC9FpFV8BK3QD0tlXe2vNXBoiAeWsXL9RQXNcuNRydCgyuwm1o4N1Ogo39cUwoGXfJEwhqzME/C7rHG4k20uYe52DhpBsXwC9Zbk09VAwtf19l7wHU3Im8rFIAKC695q4VpFqTt/A/FEXr8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271909; c=relaxed/simple; bh=D85w/y2lUcAbxKEDuMXy3+L9hQ9+AnyRAYn+0nFXS8g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dFeKv3dER6bvkc5L7VFOi2Ay4id4tCWEwdOKrfVMOoBvEp1gN0xjnv8LfuDllEPrknZDadcec48i1ij9S56m5BWgVKJQZFc06ox4qPAY5nXbIj0UvZr3VzNjtZ14o3lNiwJWsn0EsQbDB9N5M5GE21tmMVcYHPr2Z7GRxvUNUiU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=S4GYO0ZF; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="S4GYO0ZF" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C48F9C4CEE4; Mon, 21 Apr 2025 21:45:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745271908; bh=D85w/y2lUcAbxKEDuMXy3+L9hQ9+AnyRAYn+0nFXS8g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S4GYO0ZFCjWwCtSf6ycECRA7aobFCWH9bFn4ePzcVZpcTiUBMwY1v5tuYZLa1lGUD wRhyfYWqiXtT4+MzdaPKcmBEq4WVaxveto3CaWTqrtH0GBw8rQed3eYhbz659YDdnV LBYJD3X1LOksgccgyUlOsAIlEMv+w8eyVoJpFkxKTjMoMBjyhH2G8vIf9+YZIimJN6 EsNiopzOSuv/qT3q2YtAb3QHb2gXUmmCMtf5OwDTjIE2hTkRZ74UJ/n4oycPxQo0CL c0aYCFFpPS5GsvMzNnal/uh30uENikRweX8XShrQ3C7MVZbR+xGtDPnmHgFHbYIdCy 2f9napgds6dnQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 03/22] uprobes: Move ref_ctr_offset update out of uprobe_write_opcode Date: Mon, 21 Apr 2025 23:44:03 +0200 Message-ID: <20250421214423.393661-4-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The uprobe_write_opcode function currently updates also refctr offset if there's one defined for uprobe. This is not handy for following changes which needs to make several updates (writes) to install or remove uprobe, but update refctr offset just once. Adding set_swbp_refctr/set_orig_refctr which makes sure refctr offset is updated. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- include/linux/uprobes.h | 2 +- kernel/events/uprobes.c | 62 ++++++++++++++++++++++++----------------- 2 files changed, 38 insertions(+), 26 deletions(-) diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index 7447e15559b8..d3496f7bc583 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -194,7 +194,7 @@ extern bool is_swbp_insn(uprobe_opcode_t *insn); extern bool is_trap_insn(uprobe_opcode_t *insn); extern unsigned long uprobe_get_swbp_addr(struct pt_regs *regs); extern unsigned long uprobe_get_trap_addr(struct pt_regs *regs); -extern int uprobe_write_opcode(struct arch_uprobe *auprobe, struct vm_area= _struct *vma, unsigned long vaddr, uprobe_opcode_t); +extern int uprobe_write_opcode(struct vm_area_struct *vma, unsigned long v= addr, uprobe_opcode_t opcode); extern struct uprobe *uprobe_register(struct inode *inode, loff_t offset, = loff_t ref_ctr_offset, struct uprobe_consumer *uc); extern int uprobe_apply(struct uprobe *uprobe, struct uprobe_consumer *uc,= bool); extern void uprobe_unregister_nosync(struct uprobe *uprobe, struct uprobe_= consumer *uc); diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 87bca004ee6a..8b31340ed1c3 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -486,13 +486,12 @@ static int __uprobe_write_opcode(struct vm_area_struc= t *vma, * Called with mm->mmap_lock held for read or write. * Return 0 (success) or a negative errno. */ -int uprobe_write_opcode(struct arch_uprobe *auprobe, struct vm_area_struct= *vma, - const unsigned long opcode_vaddr, uprobe_opcode_t opcode) +int uprobe_write_opcode(struct vm_area_struct *vma, const unsigned long op= code_vaddr, + uprobe_opcode_t opcode) { const unsigned long vaddr =3D opcode_vaddr & PAGE_MASK; struct mm_struct *mm =3D vma->vm_mm; - struct uprobe *uprobe; - int ret, is_register, ref_ctr_updated =3D 0; + int ret, is_register; unsigned int gup_flags =3D FOLL_FORCE; struct mmu_notifier_range range; struct folio_walk fw; @@ -500,7 +499,6 @@ int uprobe_write_opcode(struct arch_uprobe *auprobe, st= ruct vm_area_struct *vma, struct page *page; =20 is_register =3D is_swbp_insn(&opcode); - uprobe =3D container_of(auprobe, struct uprobe, arch); =20 if (WARN_ON_ONCE(!is_cow_mapping(vma->vm_flags))) return -EINVAL; @@ -528,17 +526,6 @@ int uprobe_write_opcode(struct arch_uprobe *auprobe, s= truct vm_area_struct *vma, goto out; } =20 - /* We are going to replace instruction, update ref_ctr. */ - if (!ref_ctr_updated && uprobe->ref_ctr_offset) { - ret =3D update_ref_ctr(uprobe, mm, is_register ? 1 : -1); - if (ret) { - folio_put(folio); - goto out; - } - - ref_ctr_updated =3D 1; - } - ret =3D 0; if (unlikely(!folio_test_anon(folio))) { VM_WARN_ON_ONCE(is_register); @@ -580,10 +567,6 @@ int uprobe_write_opcode(struct arch_uprobe *auprobe, s= truct vm_area_struct *vma, } =20 out: - /* Revert back reference counter if instruction update failed. */ - if (ret < 0 && is_register && ref_ctr_updated) - update_ref_ctr(uprobe, mm, -1); - /* try collapse pmd for compound page */ if (ret > 0) collapse_pte_mapped_thp(mm, vaddr, false); @@ -603,7 +586,27 @@ int uprobe_write_opcode(struct arch_uprobe *auprobe, s= truct vm_area_struct *vma, int __weak set_swbp(struct arch_uprobe *auprobe, struct vm_area_struct *vm= a, unsigned long vaddr) { - return uprobe_write_opcode(auprobe, vma, vaddr, UPROBE_SWBP_INSN); + return uprobe_write_opcode(vma, vaddr, UPROBE_SWBP_INSN); +} + +static int set_swbp_refctr(struct uprobe *uprobe, struct vm_area_struct *v= ma, unsigned long vaddr) +{ + struct mm_struct *mm =3D vma->vm_mm; + int err; + + /* We are going to replace instruction, update ref_ctr. */ + if (uprobe->ref_ctr_offset) { + err =3D update_ref_ctr(uprobe, mm, 1); + if (err) + return err; + } + + err =3D set_swbp(&uprobe->arch, vma, vaddr); + + /* Revert back reference counter if instruction update failed. */ + if (err && uprobe->ref_ctr_offset) + update_ref_ctr(uprobe, mm, -1); + return err; } =20 /** @@ -618,8 +621,17 @@ int __weak set_swbp(struct arch_uprobe *auprobe, struc= t vm_area_struct *vma, int __weak set_orig_insn(struct arch_uprobe *auprobe, struct vm_area_struct *vma, unsigned long vaddr) { - return uprobe_write_opcode(auprobe, vma, vaddr, - *(uprobe_opcode_t *)&auprobe->insn); + return uprobe_write_opcode(vma, vaddr, *(uprobe_opcode_t *)&auprobe->insn= ); +} + +static int set_orig_refctr(struct uprobe *uprobe, struct vm_area_struct *v= ma, unsigned long vaddr) +{ + int err =3D set_orig_insn(&uprobe->arch, vma, vaddr); + + /* Revert back reference counter even if instruction update failed. */ + if (uprobe->ref_ctr_offset) + update_ref_ctr(uprobe, vma->vm_mm, -1); + return err; } =20 /* uprobe should have guaranteed positive refcount */ @@ -1158,7 +1170,7 @@ static int install_breakpoint(struct uprobe *uprobe, = struct vm_area_struct *vma, if (first_uprobe) set_bit(MMF_HAS_UPROBES, &mm->flags); =20 - ret =3D set_swbp(&uprobe->arch, vma, vaddr); + ret =3D set_swbp_refctr(uprobe, vma, vaddr); if (!ret) clear_bit(MMF_RECALC_UPROBES, &mm->flags); else if (first_uprobe) @@ -1173,7 +1185,7 @@ static int remove_breakpoint(struct uprobe *uprobe, s= truct vm_area_struct *vma, struct mm_struct *mm =3D vma->vm_mm; =20 set_bit(MMF_RECALC_UPROBES, &mm->flags); - return set_orig_insn(&uprobe->arch, vma, vaddr); + return set_orig_refctr(uprobe, vma, vaddr); } =20 struct map_info { --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8EC1515D1; Mon, 21 Apr 2025 21:45:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271921; cv=none; b=VacFvTKL6ut+ABtaqEZGLBDB0aLm7sSaiQslrn8zaCCh17oxgMyGT583a9Kl/3ga9QObUBxQ0NXH/m92iKuaJOPcrR551svM/oyKiO9bvQuEZM9Dp40mWgdcCs6KTnOBhls1VU2FJNXpgHJ9YxrwKy9cUKliq2woxVsTnVvoXn4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271921; c=relaxed/simple; bh=1h8Lt5sfu3I36yNkn7tVNN8qftvnhgevSwlEaG90tM4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UYdop878Z6kiKJx1hXIz5uP3n3Ck0WI2IO0cDZlkU8TPsyC6CgZscYcg477bYxaut87I1jWipBjFnH62Zvrln1/yIbgaoX8uGtNAn86rsP6zJLDCbGBWIIo9h/h/HMnvhg5OkshmbA8OtpFLnang0K48Cg6LZREBD8wuNt6iWBA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OgS20hVa; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OgS20hVa" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C4F98C4CEE4; Mon, 21 Apr 2025 21:45:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745271921; bh=1h8Lt5sfu3I36yNkn7tVNN8qftvnhgevSwlEaG90tM4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OgS20hVaRmYbyUn3HeoV1Xl+DTNeZobY/dlY0dL4kZ2M9WEwn7uyO025YtCsJTiOP ytbji9NNI/JDXMtqHMmDc9lXh8NZdHDnLMSwNGz9CID5AaNV2JOHXmjOqsrS3JV//g fd57+smsSrtdohiUELx8ho2LS9wbXf3qLv/Wx6yHj5ZgBOd4QQrDBXARbWuwVEwdKr ukpu5i3VkkqSjeIpIsL1UpdtPKGJIaJIr9FhDc/Mr4br9vKM0BuDTXxNKoyUkN7BsW bBcGhbR0bMKoAz5TR6Quq60KGOcY47EPGCgo2FfPQUG4MzKb5DY8P1DwbjXQ3D1cG6 fDRQ2OidgI+SQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 04/22] uprobes: Add uprobe_write function Date: Mon, 21 Apr 2025 23:44:04 +0200 Message-ID: <20250421214423.393661-5-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding uprobe_write function that does what uprobe_write_opcode did so far, but allows to pass verify callback function that checks the memory location before writing the opcode. It will be used in following changes to implement specific checking logic for instruction update. The uprobe_write_opcode now calls uprobe_write with verify_opcode as the verify callback. Signed-off-by: Jiri Olsa --- include/linux/uprobes.h | 5 +++++ kernel/events/uprobes.c | 14 ++++++++++---- 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index d3496f7bc583..09fe93816173 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -187,6 +187,9 @@ struct uprobes_state { struct xol_area *xol_area; }; =20 +typedef int (*uprobe_write_verify_t)(struct page *page, unsigned long vadd= r, + uprobe_opcode_t *opcode); + extern void __init uprobes_init(void); extern int set_swbp(struct arch_uprobe *aup, struct vm_area_struct *vma, u= nsigned long vaddr); extern int set_orig_insn(struct arch_uprobe *aup, struct vm_area_struct *v= ma, unsigned long vaddr); @@ -195,6 +198,8 @@ extern bool is_trap_insn(uprobe_opcode_t *insn); extern unsigned long uprobe_get_swbp_addr(struct pt_regs *regs); extern unsigned long uprobe_get_trap_addr(struct pt_regs *regs); extern int uprobe_write_opcode(struct vm_area_struct *vma, unsigned long v= addr, uprobe_opcode_t opcode); +extern int uprobe_write(struct vm_area_struct *vma, const unsigned long op= code_vaddr, + uprobe_opcode_t opcode, uprobe_write_verify_t verify); extern struct uprobe *uprobe_register(struct inode *inode, loff_t offset, = loff_t ref_ctr_offset, struct uprobe_consumer *uc); extern int uprobe_apply(struct uprobe *uprobe, struct uprobe_consumer *uc,= bool); extern void uprobe_unregister_nosync(struct uprobe *uprobe, struct uprobe_= consumer *uc); diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 8b31340ed1c3..3c5dc86bfe65 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -399,7 +399,7 @@ static bool orig_page_is_identical(struct vm_area_struc= t *vma, return identical; } =20 -static int __uprobe_write_opcode(struct vm_area_struct *vma, +static int __uprobe_write(struct vm_area_struct *vma, struct folio_walk *fw, struct folio *folio, unsigned long opcode_vaddr, uprobe_opcode_t opcode) { @@ -488,6 +488,12 @@ static int __uprobe_write_opcode(struct vm_area_struct= *vma, */ int uprobe_write_opcode(struct vm_area_struct *vma, const unsigned long op= code_vaddr, uprobe_opcode_t opcode) +{ + return uprobe_write(vma, opcode_vaddr, opcode, verify_opcode); +} + +int uprobe_write(struct vm_area_struct *vma, const unsigned long opcode_va= ddr, + uprobe_opcode_t opcode, uprobe_write_verify_t verify) { const unsigned long vaddr =3D opcode_vaddr & PAGE_MASK; struct mm_struct *mm =3D vma->vm_mm; @@ -508,7 +514,7 @@ int uprobe_write_opcode(struct vm_area_struct *vma, con= st unsigned long opcode_v * page that we can safely modify. Use FOLL_WRITE to trigger a write * fault if required. When unregistering, we might be lucky and the * anon page is already gone. So defer write faults until really - * required. Use FOLL_SPLIT_PMD, because __uprobe_write_opcode() + * required. Use FOLL_SPLIT_PMD, because __uprobe_write() * cannot deal with PMDs yet. */ if (is_register) @@ -520,7 +526,7 @@ int uprobe_write_opcode(struct vm_area_struct *vma, con= st unsigned long opcode_v goto out; folio =3D page_folio(page); =20 - ret =3D verify_opcode(page, opcode_vaddr, &opcode); + ret =3D verify(page, opcode_vaddr, &opcode); if (ret <=3D 0) { folio_put(folio); goto out; @@ -548,7 +554,7 @@ int uprobe_write_opcode(struct vm_area_struct *vma, con= st unsigned long opcode_v /* Walk the page tables again, to perform the actual update. */ if (folio_walk_start(&fw, vma, vaddr, 0)) { if (fw.page =3D=3D page) - ret =3D __uprobe_write_opcode(vma, &fw, folio, opcode_vaddr, opcode); + ret =3D __uprobe_write(vma, &fw, folio, opcode_vaddr, opcode); folio_walk_end(&fw, vma); } =20 --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 85DAE1D95A3; Mon, 21 Apr 2025 21:45:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271933; cv=none; b=uDxwY6C42rELkzxvNVu0ThqyvFgx7PCidoqHXoKgwRq5J8U7vzMYW8EJTL/zlatPuxUVHrfiBzcItN8n0C0qQkl8wBnUvxXsO2Lwc4ViyfZQkX3UkKvCEq2DDlM7FSFPERClxp2X63na55pqmTX8wWq7+5rwnigiEDNT2MM7FDQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271933; c=relaxed/simple; bh=WAr4VO6L2RSNG4Tm5YcaCBQFDkvAdBO38Tj4GEwe010=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cpEHNnASJ6O3IheNQ5eQka8Hy6Mu3HOYgZ6ah3LmH73iS6J0F6N+CkT8dNb5dkl6CaAbxD48n267Df9IxeAQEttngvb8F2pwsndl9l0ByNgot8gVtkULsVKKq3PXpNbimqC1SDVBPBsHCl4h6nx22osyvuBuLY8o9WaFMfFelLs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Bxx6CmQa; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Bxx6CmQa" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CE185C4CEEA; Mon, 21 Apr 2025 21:45:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745271932; bh=WAr4VO6L2RSNG4Tm5YcaCBQFDkvAdBO38Tj4GEwe010=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Bxx6CmQamaajAJ9qAbBIp+0TcE37OqNeN3UC6LGWz5oRqGeK1J10tWhI/fvB48Nw7 wzV6ea07yGmgVk3pqpCR+bj1eQ2ySpfcs6Swfu6v6SQTXBtN4HeUvu7cLUoeqmYOLW YbFjEdCB4FrEsDSOar08XqgDLCQ2q9eK0+pBtVWgU55evSZmfiVc+a1qBM58aMaKpD b1+ISufxdZeCtwow9AyyTltfkLq4NrzFYZXKCM5dJt7AWPNutequTXMvko5LpQ1J8e qanwjD0PBRR83WbRCMilmxLyORhWvbVHbuV7DF7fa95fdZMULEZo3FeRH5MKIckM9F lqN3o5Ul8H1WA== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 05/22] uprobes: Add nbytes argument to uprobe_write Date: Mon, 21 Apr 2025 23:44:05 +0200 Message-ID: <20250421214423.393661-6-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding nbytes argument to uprobe_write and related functions as preparation for writing whole instructions in following changes. Also renaming opcode arguments to insn, which seems to fit better. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- include/linux/uprobes.h | 6 +++--- kernel/events/uprobes.c | 27 ++++++++++++++------------- 2 files changed, 17 insertions(+), 16 deletions(-) diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index 09fe93816173..b86a2f0475a4 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -188,7 +188,7 @@ struct uprobes_state { }; =20 typedef int (*uprobe_write_verify_t)(struct page *page, unsigned long vadd= r, - uprobe_opcode_t *opcode); + uprobe_opcode_t *insn, int nbytes); =20 extern void __init uprobes_init(void); extern int set_swbp(struct arch_uprobe *aup, struct vm_area_struct *vma, u= nsigned long vaddr); @@ -198,8 +198,8 @@ extern bool is_trap_insn(uprobe_opcode_t *insn); extern unsigned long uprobe_get_swbp_addr(struct pt_regs *regs); extern unsigned long uprobe_get_trap_addr(struct pt_regs *regs); extern int uprobe_write_opcode(struct vm_area_struct *vma, unsigned long v= addr, uprobe_opcode_t opcode); -extern int uprobe_write(struct vm_area_struct *vma, const unsigned long op= code_vaddr, - uprobe_opcode_t opcode, uprobe_write_verify_t verify); +extern int uprobe_write(struct vm_area_struct *vma, const unsigned long in= sn_vaddr, + uprobe_opcode_t *insn, int nbytes, uprobe_write_verify_t verify); extern struct uprobe *uprobe_register(struct inode *inode, loff_t offset, = loff_t ref_ctr_offset, struct uprobe_consumer *uc); extern int uprobe_apply(struct uprobe *uprobe, struct uprobe_consumer *uc,= bool); extern void uprobe_unregister_nosync(struct uprobe *uprobe, struct uprobe_= consumer *uc); diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 3c5dc86bfe65..6dc7f0b2756d 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -191,7 +191,8 @@ static void copy_to_page(struct page *page, unsigned lo= ng vaddr, const void *src kunmap_atomic(kaddr); } =20 -static int verify_opcode(struct page *page, unsigned long vaddr, uprobe_op= code_t *new_opcode) +static int verify_opcode(struct page *page, unsigned long vaddr, uprobe_op= code_t *insn, + int nbytes) { uprobe_opcode_t old_opcode; bool is_swbp; @@ -208,7 +209,7 @@ static int verify_opcode(struct page *page, unsigned lo= ng vaddr, uprobe_opcode_t uprobe_copy_from_page(page, vaddr, &old_opcode, UPROBE_SWBP_INSN_SIZE); is_swbp =3D is_swbp_insn(&old_opcode); =20 - if (is_swbp_insn(new_opcode)) { + if (is_swbp_insn(insn)) { if (is_swbp) /* register: already installed? */ return 0; } else { @@ -401,10 +402,10 @@ static bool orig_page_is_identical(struct vm_area_str= uct *vma, =20 static int __uprobe_write(struct vm_area_struct *vma, struct folio_walk *fw, struct folio *folio, - unsigned long opcode_vaddr, uprobe_opcode_t opcode) + unsigned long insn_vaddr, uprobe_opcode_t *insn, int nbytes) { - const unsigned long vaddr =3D opcode_vaddr & PAGE_MASK; - const bool is_register =3D !!is_swbp_insn(&opcode); + const unsigned long vaddr =3D insn_vaddr & PAGE_MASK; + const bool is_register =3D !!is_swbp_insn(insn); bool pmd_mappable; =20 /* For now, we'll only handle PTE-mapped folios. */ @@ -429,7 +430,7 @@ static int __uprobe_write(struct vm_area_struct *vma, */ flush_cache_page(vma, vaddr, pte_pfn(fw->pte)); fw->pte =3D ptep_clear_flush(vma, vaddr, fw->ptep); - copy_to_page(fw->page, opcode_vaddr, &opcode, UPROBE_SWBP_INSN_SIZE); + copy_to_page(fw->page, insn_vaddr, insn, nbytes); =20 /* * When unregistering, we may only zap a PTE if uffd is disabled and @@ -489,13 +490,13 @@ static int __uprobe_write(struct vm_area_struct *vma, int uprobe_write_opcode(struct vm_area_struct *vma, const unsigned long op= code_vaddr, uprobe_opcode_t opcode) { - return uprobe_write(vma, opcode_vaddr, opcode, verify_opcode); + return uprobe_write(vma, opcode_vaddr, &opcode, UPROBE_SWBP_INSN_SIZE, ve= rify_opcode); } =20 -int uprobe_write(struct vm_area_struct *vma, const unsigned long opcode_va= ddr, - uprobe_opcode_t opcode, uprobe_write_verify_t verify) +int uprobe_write(struct vm_area_struct *vma, const unsigned long insn_vadd= r, + uprobe_opcode_t *insn, int nbytes, uprobe_write_verify_t verify) { - const unsigned long vaddr =3D opcode_vaddr & PAGE_MASK; + const unsigned long vaddr =3D insn_vaddr & PAGE_MASK; struct mm_struct *mm =3D vma->vm_mm; int ret, is_register; unsigned int gup_flags =3D FOLL_FORCE; @@ -504,7 +505,7 @@ int uprobe_write(struct vm_area_struct *vma, const unsi= gned long opcode_vaddr, struct folio *folio; struct page *page; =20 - is_register =3D is_swbp_insn(&opcode); + is_register =3D is_swbp_insn(insn); =20 if (WARN_ON_ONCE(!is_cow_mapping(vma->vm_flags))) return -EINVAL; @@ -526,7 +527,7 @@ int uprobe_write(struct vm_area_struct *vma, const unsi= gned long opcode_vaddr, goto out; folio =3D page_folio(page); =20 - ret =3D verify(page, opcode_vaddr, &opcode); + ret =3D verify(page, insn_vaddr, insn, nbytes); if (ret <=3D 0) { folio_put(folio); goto out; @@ -554,7 +555,7 @@ int uprobe_write(struct vm_area_struct *vma, const unsi= gned long opcode_vaddr, /* Walk the page tables again, to perform the actual update. */ if (folio_walk_start(&fw, vma, vaddr, 0)) { if (fw.page =3D=3D page) - ret =3D __uprobe_write(vma, &fw, folio, opcode_vaddr, opcode); + ret =3D __uprobe_write(vma, &fw, folio, insn_vaddr, insn, nbytes); folio_walk_end(&fw, vma); } =20 --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0A5021ABAD; Mon, 21 Apr 2025 21:45:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271945; cv=none; b=YpiJveEYsiUxoODo/YX/38+nPn/2q669du3FJww95B3mywTaVI0DjldYGCIjDBvyuHdJzojYOJVuHhmWRSFF8QcbgOoUZeXQXLXAXNT4M5wMD3tRAAuiXH52TDvmNJwwxZTtMaVUhaLWMeP42skNE/z7QFWMY6pL/10s9msoXF0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271945; c=relaxed/simple; bh=q0HSJq8brWtnkLHhuXhWrk+19fIQLZuRxYst30BTAT0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=l3uw7x4RYKx9mo2Fvv86AD7AUW54juQ6oiBhKatIdpZs+Sup9jqeZbbAMlUENvXHvr+1f59UII/0HRul5O5scLhQcIooI/OeN7tXYe642nNkHwB4oXLpcRSyToEs0Y6HIvfLfZ2hID7RqBO7+W56GjUfDKzvgPRwwLcV2JOjJio= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=tfnF32vF; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="tfnF32vF" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A1D73C4CEEA; Mon, 21 Apr 2025 21:45:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745271945; bh=q0HSJq8brWtnkLHhuXhWrk+19fIQLZuRxYst30BTAT0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tfnF32vFpurWGYwhwhhehrWxQdnZjOS4Y60E6yQYYPKswy5yzjbgg0cwBNSDHxPfw x9d4AjZLaTHuClc9WgcV9O960O/+26Jvf1vBcK7+4ZoiLZQQr8UOuMcJFBl8yPYzcq eAwryTQSRV/V2OgXOEgLqtEIknbZPji0IrbgsRRNOyJO9q8C7PzXj7vjhbZ4JBgeEi CPTR9XgIrn4KTm8JZxHuhgxeiwDGuc8Vt9LONQMLYC45wcI+dqWMlXs9n73+pjlSB9 ragCvSDRaXtGx4fwQcZ4tKZrY/4sZ/eliVDgurUZg3saAEGXFC3t+hmYWg+j188jWf mhNJmtJsvvH1Q== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 06/22] uprobes: Add is_register argument to uprobe_write and uprobe_write_opcode Date: Mon, 21 Apr 2025 23:44:06 +0200 Message-ID: <20250421214423.393661-7-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The uprobe_write has special path to restore the original page when we write original instruction back. This happens when uprobe_write detects that we want to write anything else but breakpoint instruction. Moving the detection away and passing it to uprobe_write as argument, so it's possible to write different instructions (other than just breakpoint and rest). Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- arch/arm/probes/uprobes/core.c | 2 +- include/linux/uprobes.h | 5 +++-- kernel/events/uprobes.c | 22 +++++++++++----------- 3 files changed, 15 insertions(+), 14 deletions(-) diff --git a/arch/arm/probes/uprobes/core.c b/arch/arm/probes/uprobes/core.c index 885e0c5e8c20..3d96fb41d624 100644 --- a/arch/arm/probes/uprobes/core.c +++ b/arch/arm/probes/uprobes/core.c @@ -30,7 +30,7 @@ int set_swbp(struct arch_uprobe *auprobe, struct vm_area_= struct *vma, unsigned long vaddr) { return uprobe_write_opcode(auprobe, vma, vaddr, - __opcode_to_mem_arm(auprobe->bpinsn)); + __opcode_to_mem_arm(auprobe->bpinsn), true); } =20 bool arch_uprobe_ignore(struct arch_uprobe *auprobe, struct pt_regs *regs) diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index b86a2f0475a4..6af61e977bfb 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -197,9 +197,10 @@ extern bool is_swbp_insn(uprobe_opcode_t *insn); extern bool is_trap_insn(uprobe_opcode_t *insn); extern unsigned long uprobe_get_swbp_addr(struct pt_regs *regs); extern unsigned long uprobe_get_trap_addr(struct pt_regs *regs); -extern int uprobe_write_opcode(struct vm_area_struct *vma, unsigned long v= addr, uprobe_opcode_t opcode); +extern int uprobe_write_opcode(struct vm_area_struct *vma, unsigned long v= addr, + uprobe_opcode_t opcode, bool is_register); extern int uprobe_write(struct vm_area_struct *vma, const unsigned long in= sn_vaddr, - uprobe_opcode_t *insn, int nbytes, uprobe_write_verify_t verify); + uprobe_opcode_t *insn, int nbytes, uprobe_write_verify_t verify, bool i= s_register); extern struct uprobe *uprobe_register(struct inode *inode, loff_t offset, = loff_t ref_ctr_offset, struct uprobe_consumer *uc); extern int uprobe_apply(struct uprobe *uprobe, struct uprobe_consumer *uc,= bool); extern void uprobe_unregister_nosync(struct uprobe *uprobe, struct uprobe_= consumer *uc); diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 6dc7f0b2756d..c8d88060dfbf 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -402,10 +402,10 @@ static bool orig_page_is_identical(struct vm_area_str= uct *vma, =20 static int __uprobe_write(struct vm_area_struct *vma, struct folio_walk *fw, struct folio *folio, - unsigned long insn_vaddr, uprobe_opcode_t *insn, int nbytes) + unsigned long insn_vaddr, uprobe_opcode_t *insn, int nbytes, + bool is_register) { const unsigned long vaddr =3D insn_vaddr & PAGE_MASK; - const bool is_register =3D !!is_swbp_insn(insn); bool pmd_mappable; =20 /* For now, we'll only handle PTE-mapped folios. */ @@ -488,25 +488,25 @@ static int __uprobe_write(struct vm_area_struct *vma, * Return 0 (success) or a negative errno. */ int uprobe_write_opcode(struct vm_area_struct *vma, const unsigned long op= code_vaddr, - uprobe_opcode_t opcode) + uprobe_opcode_t opcode, bool is_register) { - return uprobe_write(vma, opcode_vaddr, &opcode, UPROBE_SWBP_INSN_SIZE, ve= rify_opcode); + return uprobe_write(vma, opcode_vaddr, &opcode, UPROBE_SWBP_INSN_SIZE, + verify_opcode, is_register); } =20 int uprobe_write(struct vm_area_struct *vma, const unsigned long insn_vadd= r, - uprobe_opcode_t *insn, int nbytes, uprobe_write_verify_t verify) + uprobe_opcode_t *insn, int nbytes, uprobe_write_verify_t verify, + bool is_register) { const unsigned long vaddr =3D insn_vaddr & PAGE_MASK; struct mm_struct *mm =3D vma->vm_mm; - int ret, is_register; + int ret; unsigned int gup_flags =3D FOLL_FORCE; struct mmu_notifier_range range; struct folio_walk fw; struct folio *folio; struct page *page; =20 - is_register =3D is_swbp_insn(insn); - if (WARN_ON_ONCE(!is_cow_mapping(vma->vm_flags))) return -EINVAL; =20 @@ -555,7 +555,7 @@ int uprobe_write(struct vm_area_struct *vma, const unsi= gned long insn_vaddr, /* Walk the page tables again, to perform the actual update. */ if (folio_walk_start(&fw, vma, vaddr, 0)) { if (fw.page =3D=3D page) - ret =3D __uprobe_write(vma, &fw, folio, insn_vaddr, insn, nbytes); + ret =3D __uprobe_write(vma, &fw, folio, insn_vaddr, insn, nbytes, is_re= gister); folio_walk_end(&fw, vma); } =20 @@ -593,7 +593,7 @@ int uprobe_write(struct vm_area_struct *vma, const unsi= gned long insn_vaddr, int __weak set_swbp(struct arch_uprobe *auprobe, struct vm_area_struct *vm= a, unsigned long vaddr) { - return uprobe_write_opcode(vma, vaddr, UPROBE_SWBP_INSN); + return uprobe_write_opcode(vma, vaddr, UPROBE_SWBP_INSN, true); } =20 static int set_swbp_refctr(struct uprobe *uprobe, struct vm_area_struct *v= ma, unsigned long vaddr) @@ -628,7 +628,7 @@ static int set_swbp_refctr(struct uprobe *uprobe, struc= t vm_area_struct *vma, un int __weak set_orig_insn(struct arch_uprobe *auprobe, struct vm_area_struct *vma, unsigned long vaddr) { - return uprobe_write_opcode(vma, vaddr, *(uprobe_opcode_t *)&auprobe->insn= ); + return uprobe_write_opcode(vma, vaddr, *(uprobe_opcode_t *)&auprobe->insn= , false); } =20 static int set_orig_refctr(struct uprobe *uprobe, struct vm_area_struct *v= ma, unsigned long vaddr) --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 699CC19B5B8; Mon, 21 Apr 2025 21:45:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271957; cv=none; b=egxn7igcml5XrGNZKotX+OvYs+hhd94POQM/OWIgSkh86iiVYmZfqaJJ+zdRmGBMYsnmf0vuC2lYr0wzM9sOf4ulPoC6jU4O0m2zMWAgUQut7nqhwHns43H9OdsU33a3tsZPD8q6b99Xmgp7F4JxpRcI51wGMj8w3qFnrR0Mln4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271957; c=relaxed/simple; bh=5HVHWaig0HEBawYxtqQdeilqzoxF5+c66y72SyzkBlw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qwm6nQH9FOV9+5Bh5VGt4FGhUPYq91Fe5Gpec0XsCcxevP9gKMwq+duLV6IXcNaaSr9cPlJywRleXAUpHykchNAI53HM78cZG0N3vHFdMWbXDz1rJlCxMNkUeWV0g1cL4x4/YtY5vcLhDCTj5w9KFjKMGvccKZZl1lwbSQYvpjk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UEQkmuWN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UEQkmuWN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9DC92C4CEE4; Mon, 21 Apr 2025 21:45:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745271956; bh=5HVHWaig0HEBawYxtqQdeilqzoxF5+c66y72SyzkBlw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UEQkmuWNapw0a58kRsvpiDRxddPLJVjMIvlClYAdMNORc1Rz9xLDu5dNPvgC4pn6M hSse5BTgRuCocPCoe1a0/a9RC02t0Rj6tJaXUolVGfvB01XtrhjLUy8iDbC5uOOwew COlDoe6sqtetk9QI9/vFh6uAxFCHSWR2ocOIix+zmyLOd+n06sLm1MtzjUly4H00C2 Yt8quWeij+vrEP1adkBnUjMUBa4ib81Tys20wUTp1DJw15LTmR7aZjZ5c6EBcGdyH+ uxunl9svtx5HNGxd7ZFwlg8Ts6ZgFUMyMzkFij3nzWDnlQr4xpsZw5zyg9/QvIKPrU t1uUpf2LQJZMg== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 07/22] uprobes: Remove breakpoint in unapply_uprobe under mmap_write_lock Date: Mon, 21 Apr 2025 23:44:07 +0200 Message-ID: <20250421214423.393661-8-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently unapply_uprobe takes mmap_read_lock, but it might call remove_breakpoint which eventually changes user pages. Current code writes either breakpoint or original instruction, so it can probably go away with that, but with the upcoming change that writes multiple instructions on the probed address we need to ensure that any update to mm's pages is exclusive. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- kernel/events/uprobes.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index c8d88060dfbf..d256c695d7ff 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1483,7 +1483,7 @@ static int unapply_uprobe(struct uprobe *uprobe, stru= ct mm_struct *mm) struct vm_area_struct *vma; int err =3D 0; =20 - mmap_read_lock(mm); + mmap_write_lock(mm); for_each_vma(vmi, vma) { unsigned long vaddr; loff_t offset; @@ -1500,7 +1500,7 @@ static int unapply_uprobe(struct uprobe *uprobe, stru= ct mm_struct *mm) vaddr =3D offset_to_vaddr(vma, uprobe->offset); err |=3D remove_breakpoint(uprobe, vma, vaddr); } - mmap_read_unlock(mm); + mmap_write_unlock(mm); =20 return err; } --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 39BEF21ADD3; Mon, 21 Apr 2025 21:46:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271969; cv=none; b=Qi87FDqJkQUb8dWqFYRnMXNaF9cks6yKIAMZZH0g1MRNq3RkjtM9RezmJOlUibeSoslDPdUpZigt9sxWEFPaMu86GRPdxuU4wPoC6NGYbBAepuXDThTwaw5A4exGltw8LzDxRD72WkkVUzdTZPlozzNAsX7huhlnqUIlhjhxU5g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271969; c=relaxed/simple; bh=BkvZFQCxGsH5FDBVK4gs3h4kzREZE2mXKj7RzTAZsJQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=h8ds1vfR7Id5iWVQazhgy7uXxFkMklOqTCa38bCkN4RDpfmQkfoPo+6sWlFsGymHmjfxeVM54zartJ4W3Lt0lpoQOYwymj27mUX2QH3UTSlZIO3lM6968sJfFZxyjonSOj89aGLs+VHUIF77Fp1OQazt+RlcD23Sibad03F3p/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HPdcJd9q; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HPdcJd9q" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B986AC4CEE4; Mon, 21 Apr 2025 21:46:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745271969; bh=BkvZFQCxGsH5FDBVK4gs3h4kzREZE2mXKj7RzTAZsJQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HPdcJd9qbOQ37y6WEn5j7hewbxBwsFRGOIcXbRMC5fdGCsxAx9N59VdY091chhVv4 KyI/CkthyN9bJ+bw7m4Jbr/n+MpHkCzF4b8luu/w7PYcmbPFqD5PWsh6YSyovXloMR kwN0h3dddUeAfN5g6st2EJAh+8Ygs7gSroOM+tkoaok0sJGaAgcBY95xvXgI1Z004+ N1HwDL+lLs30VJRKatMXetfi1GWVOMG6ecvdbbrMzlPcuunoawE4536KQkym+4hwYi qHqP++nuP82I1kpSD/bC70q6BFrfFFMZ/SycxvAB39UClQAFSzL/IJyKEf7bRvR7o8 7QCBSrQKah0uQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 08/22] uprobes/x86: Add mapping for optimized uprobe trampolines Date: Mon, 21 Apr 2025 23:44:08 +0200 Message-ID: <20250421214423.393661-9-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding support to add special mapping for for user space trampoline with following functions: uprobe_trampoline_get - find or add uprobe_trampoline uprobe_trampoline_put - remove or destroy uprobe_trampoline The user space trampoline is exported as arch specific user space special mapping through tramp_mapping, which is initialized in following changes with new uprobe syscall. The uprobe trampoline needs to be callable/reachable from the probed addres= s, so while searching for available address we use is_reachable_by_call functi= on to decide if the uprobe trampoline is callable from the probe address. All uprobe_trampoline objects are stored in uprobes_state object and are cleaned up when the process mm_struct goes down. Adding new arch hooks for that, because this change is x86_64 specific. Locking is provided by callers in following changes. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- arch/x86/kernel/uprobes.c | 131 ++++++++++++++++++++++++++++++++++++++ include/linux/uprobes.h | 6 ++ kernel/events/uprobes.c | 10 +++ kernel/fork.c | 1 + 4 files changed, 148 insertions(+) diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c index 77050e5a4680..023c55d52138 100644 --- a/arch/x86/kernel/uprobes.c +++ b/arch/x86/kernel/uprobes.c @@ -608,6 +608,137 @@ static void riprel_post_xol(struct arch_uprobe *aupro= be, struct pt_regs *regs) *sr =3D utask->autask.saved_scratch_register; } } + +static int tramp_mremap(const struct vm_special_mapping *sm, struct vm_are= a_struct *new_vma) +{ + return -EPERM; +} + +static struct page *tramp_mapping_pages[2] __ro_after_init; + +static struct vm_special_mapping tramp_mapping =3D { + .name =3D "[uprobes-trampoline]", + .mremap =3D tramp_mremap, + .pages =3D tramp_mapping_pages, +}; + +struct uprobe_trampoline { + struct hlist_node node; + unsigned long vaddr; + atomic64_t ref; +}; + +static bool is_reachable_by_call(unsigned long vtramp, unsigned long vaddr) +{ + long delta =3D (long)(vaddr + 5 - vtramp); + + return delta >=3D INT_MIN && delta <=3D INT_MAX; +} + +static unsigned long find_nearest_page(unsigned long vaddr) +{ + struct vm_area_struct *vma, *prev =3D NULL; + unsigned long prev_vm_end =3D PAGE_SIZE; + VMA_ITERATOR(vmi, current->mm, 0); + + vma =3D vma_next(&vmi); + while (vma) { + if (prev) + prev_vm_end =3D prev->vm_end; + if (vma->vm_start - prev_vm_end >=3D PAGE_SIZE) { + if (is_reachable_by_call(prev_vm_end, vaddr)) + return prev_vm_end; + if (is_reachable_by_call(vma->vm_start - PAGE_SIZE, vaddr)) + return vma->vm_start - PAGE_SIZE; + } + prev =3D vma; + vma =3D vma_next(&vmi); + } + + return 0; +} + +static struct uprobe_trampoline *create_uprobe_trampoline(unsigned long va= ddr) +{ + struct pt_regs *regs =3D task_pt_regs(current); + struct mm_struct *mm =3D current->mm; + struct uprobe_trampoline *tramp; + struct vm_area_struct *vma; + + if (!user_64bit_mode(regs)) + return NULL; + + vaddr =3D find_nearest_page(vaddr); + if (!vaddr) + return NULL; + + tramp =3D kzalloc(sizeof(*tramp), GFP_KERNEL); + if (unlikely(!tramp)) + return NULL; + + atomic64_set(&tramp->ref, 1); + tramp->vaddr =3D vaddr; + + vma =3D _install_special_mapping(mm, tramp->vaddr, PAGE_SIZE, + VM_READ|VM_EXEC|VM_MAYEXEC|VM_MAYREAD|VM_DONTCOPY|VM_IO, + &tramp_mapping); + if (IS_ERR(vma)) + goto free_area; + return tramp; + +free_area: + kfree(tramp); + return NULL; +} + +__maybe_unused +static struct uprobe_trampoline *uprobe_trampoline_get(unsigned long vaddr) +{ + struct uprobes_state *state =3D ¤t->mm->uprobes_state; + struct uprobe_trampoline *tramp =3D NULL; + + hlist_for_each_entry(tramp, &state->head_tramps, node) { + if (is_reachable_by_call(tramp->vaddr, vaddr)) { + atomic64_inc(&tramp->ref); + return tramp; + } + } + + tramp =3D create_uprobe_trampoline(vaddr); + if (!tramp) + return NULL; + + hlist_add_head(&tramp->node, &state->head_tramps); + return tramp; +} + +static void destroy_uprobe_trampoline(struct uprobe_trampoline *tramp) +{ + hlist_del(&tramp->node); + kfree(tramp); +} + +__maybe_unused +static void uprobe_trampoline_put(struct uprobe_trampoline *tramp) +{ + if (tramp && atomic64_dec_and_test(&tramp->ref)) + destroy_uprobe_trampoline(tramp); +} + +void arch_uprobe_init_state(struct mm_struct *mm) +{ + INIT_HLIST_HEAD(&mm->uprobes_state.head_tramps); +} + +void arch_uprobe_clear_state(struct mm_struct *mm) +{ + struct uprobes_state *state =3D &mm->uprobes_state; + struct uprobe_trampoline *tramp; + struct hlist_node *n; + + hlist_for_each_entry_safe(tramp, n, &state->head_tramps, node) + destroy_uprobe_trampoline(tramp); +} #else /* 32-bit: */ /* * No RIP-relative addressing on 32-bit diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index 6af61e977bfb..bc532d086813 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -17,6 +17,7 @@ #include #include #include +#include =20 struct uprobe; struct vm_area_struct; @@ -185,6 +186,9 @@ struct xol_area; =20 struct uprobes_state { struct xol_area *xol_area; +#ifdef CONFIG_X86_64 + struct hlist_head head_tramps; +#endif }; =20 typedef int (*uprobe_write_verify_t)(struct page *page, unsigned long vadd= r, @@ -233,6 +237,8 @@ extern void uprobe_handle_trampoline(struct pt_regs *re= gs); extern void *arch_uretprobe_trampoline(unsigned long *psize); extern unsigned long uprobe_get_trampoline_vaddr(void); extern void uprobe_copy_from_page(struct page *page, unsigned long vaddr, = void *dst, int len); +extern void arch_uprobe_clear_state(struct mm_struct *mm); +extern void arch_uprobe_init_state(struct mm_struct *mm); #else /* !CONFIG_UPROBES */ struct uprobes_state { }; diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index d256c695d7ff..a3107f63f295 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1812,6 +1812,14 @@ static struct xol_area *get_xol_area(void) return area; } =20 +void __weak arch_uprobe_clear_state(struct mm_struct *mm) +{ +} + +void __weak arch_uprobe_init_state(struct mm_struct *mm) +{ +} + /* * uprobe_clear_state - Free the area allocated for slots. */ @@ -1823,6 +1831,8 @@ void uprobe_clear_state(struct mm_struct *mm) delayed_uprobe_remove(NULL, mm); mutex_unlock(&delayed_uprobe_lock); =20 + arch_uprobe_clear_state(mm); + if (!area) return; =20 diff --git a/kernel/fork.c b/kernel/fork.c index c4b26cd8998b..4c2df3816728 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1269,6 +1269,7 @@ static void mm_init_uprobes_state(struct mm_struct *m= m) { #ifdef CONFIG_UPROBES mm->uprobes_state.xol_area =3D NULL; + arch_uprobe_init_state(mm); #endif } =20 --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C8F31E5B71; Mon, 21 Apr 2025 21:46:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271981; cv=none; b=Y8l/wDFDpOpXkES636fVpmx2nNHDcnHUZVg+FVySv6DLWwHtzzuZua76RkYQbnr/dkSkYAUoYFHzF4Kdj2DRYWOqrQ6IRXkT7qYTwtF066X9loWZMk5lZYuRBzYghp2N7HP6rgw9xHyauQCxwazYjuwcoAnAZrqtTB/POgd/i/s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271981; c=relaxed/simple; bh=gos/eCQQ6g+MQoN29cxGRcV0jPzVNlfLhiU25yTdcvE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XEsPE6fy8JJ+y+5ywmd3gfgdVTxyXZrLQrP5K0yToBkacAlWzC1fF/dJz7yxGyokl+dhOldn//xHv6RKh/kwfWdT7/HwhnyCHAuZC7tv3j6XRF9mEmWyjSzjRQPDQv9twC/WQuPCYVC4+UxC2nUZVe5KIqA8f2nyVv+BmkdbCF8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Tt9Duniv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Tt9Duniv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7C381C4CEE4; Mon, 21 Apr 2025 21:46:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745271981; bh=gos/eCQQ6g+MQoN29cxGRcV0jPzVNlfLhiU25yTdcvE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Tt9Duniv6YSgQskEH3ao0IoIkU8a0esqFXQjyyc42pGfHIgGNbp9FQtPi6KOvQIj7 3nqBOPewumLBEgGyxF8q3mjNnKeFIpLqlkWkds/yk620UVTI1d+M+YqOxH5Rc2i4WQ cbQeQ1CbijRfmFBFrIeBhxDM3v1LYI3+t25A5UlRUtt5TLHOZOxIwiI+Bb8cfyDZ2p AfcnTx18FEMfp/I7MHkxvlOFqRydYuoFetXe//9SvTLSq5ICKqtq7luyKDDT8BgUf0 0i4c/Q6AXnfQKTP2FLpw7zHH7P5m4Jzpepc6RROoT4NeW1n8xtxOnlL9INYhmEFlcx lmpPrYkWLWOJQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 09/22] uprobes/x86: Add uprobe syscall to speed up uprobe Date: Mon, 21 Apr 2025 23:44:09 +0200 Message-ID: <20250421214423.393661-10-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding new uprobe syscall that calls uprobe handlers for given 'breakpoint' address. The idea is that the 'breakpoint' address calls the user space trampoline which executes the uprobe syscall. The syscall handler reads the return address of the initial call to retrieve the original 'breakpoint' address. With this address we find the related uprobe object and call its consumers. Adding the arch_uprobe_trampoline_mapping function that provides uprobe trampoline mapping. This mapping is backed with one global page initialized at __init time and shared by the all the mapping instances. We do not allow to execute uprobe syscall if the caller is not from uprobe trampoline mapping. The uprobe syscall ensures the consumer (bpf program) sees registers values in the state before the trampoline was called. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- arch/x86/entry/syscalls/syscall_64.tbl | 1 + arch/x86/kernel/uprobes.c | 122 +++++++++++++++++++++++++ include/linux/syscalls.h | 2 + include/linux/uprobes.h | 1 + kernel/events/uprobes.c | 17 ++++ kernel/sys_ni.c | 1 + 6 files changed, 144 insertions(+) diff --git a/arch/x86/entry/syscalls/syscall_64.tbl b/arch/x86/entry/syscal= ls/syscall_64.tbl index cfb5ca41e30d..9fd1291e7bdf 100644 --- a/arch/x86/entry/syscalls/syscall_64.tbl +++ b/arch/x86/entry/syscalls/syscall_64.tbl @@ -345,6 +345,7 @@ 333 common io_pgetevents sys_io_pgetevents 334 common rseq sys_rseq 335 common uretprobe sys_uretprobe +336 common uprobe sys_uprobe # don't use numbers 387 through 423, add new calls after the last # 'common' entry 424 common pidfd_send_signal sys_pidfd_send_signal diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c index 023c55d52138..01b3035e01ea 100644 --- a/arch/x86/kernel/uprobes.c +++ b/arch/x86/kernel/uprobes.c @@ -739,6 +739,128 @@ void arch_uprobe_clear_state(struct mm_struct *mm) hlist_for_each_entry_safe(tramp, n, &state->head_tramps, node) destroy_uprobe_trampoline(tramp); } + +static bool __in_uprobe_trampoline(unsigned long ip) +{ + struct vm_area_struct *vma =3D vma_lookup(current->mm, ip); + + return vma && vma_is_special_mapping(vma, &tramp_mapping); +} + +static bool in_uprobe_trampoline(unsigned long ip) +{ + struct mm_struct *mm =3D current->mm; + bool found, retry =3D true; + unsigned int seq; + + rcu_read_lock(); + if (mmap_lock_speculate_try_begin(mm, &seq)) { + found =3D __in_uprobe_trampoline(ip); + retry =3D mmap_lock_speculate_retry(mm, seq); + } + rcu_read_unlock(); + + if (retry) { + mmap_read_lock(mm); + found =3D __in_uprobe_trampoline(ip); + mmap_read_unlock(mm); + } + return found; +} + +SYSCALL_DEFINE0(uprobe) +{ + struct pt_regs *regs =3D task_pt_regs(current); + unsigned long ip, sp, ax_r11_cx_ip[4]; + int err; + + /* Allow execution only from uprobe trampolines. */ + if (!in_uprobe_trampoline(regs->ip)) + goto sigill; + + err =3D copy_from_user(ax_r11_cx_ip, (void __user *)regs->sp, sizeof(ax_r= 11_cx_ip)); + if (err) + goto sigill; + + ip =3D regs->ip; + + /* + * expose the "right" values of ax/r11/cx/ip/sp to uprobe_consumer/s, plu= s: + * - adjust ip to the probe address, call saved next instruction address + * - adjust sp to the probe's stack frame (check trampoline code) + */ + regs->ax =3D ax_r11_cx_ip[0]; + regs->r11 =3D ax_r11_cx_ip[1]; + regs->cx =3D ax_r11_cx_ip[2]; + regs->ip =3D ax_r11_cx_ip[3] - 5; + regs->sp +=3D sizeof(ax_r11_cx_ip); + regs->orig_ax =3D -1; + + sp =3D regs->sp; + + handle_syscall_uprobe(regs, regs->ip); + + /* + * Some of the uprobe consumers has changed sp, we can do nothing, + * just return via iret. + */ + if (regs->sp !=3D sp) + return regs->ax; + + regs->sp -=3D sizeof(ax_r11_cx_ip); + + /* for the case uprobe_consumer has changed ax/r11/cx */ + ax_r11_cx_ip[0] =3D regs->ax; + ax_r11_cx_ip[1] =3D regs->r11; + ax_r11_cx_ip[2] =3D regs->cx; + + /* keep return address unless we are instructed otherwise */ + if (ax_r11_cx_ip[3] - 5 !=3D regs->ip) + ax_r11_cx_ip[3] =3D regs->ip; + + regs->ip =3D ip; + + err =3D copy_to_user((void __user *)regs->sp, ax_r11_cx_ip, sizeof(ax_r11= _cx_ip)); + if (err) + goto sigill; + + /* ensure sysret, see do_syscall_64() */ + regs->r11 =3D regs->flags; + regs->cx =3D regs->ip; + return 0; + +sigill: + force_sig(SIGILL); + return -1; +} + +asm ( + ".pushsection .rodata\n" + ".balign " __stringify(PAGE_SIZE) "\n" + "uprobe_trampoline_entry:\n" + "push %rcx\n" + "push %r11\n" + "push %rax\n" + "movq $" __stringify(__NR_uprobe) ", %rax\n" + "syscall\n" + "pop %rax\n" + "pop %r11\n" + "pop %rcx\n" + "ret\n" + ".balign " __stringify(PAGE_SIZE) "\n" + ".popsection\n" +); + +extern u8 uprobe_trampoline_entry[]; + +static int __init arch_uprobes_init(void) +{ + tramp_mapping_pages[0] =3D virt_to_page(uprobe_trampoline_entry); + return 0; +} + +late_initcall(arch_uprobes_init); + #else /* 32-bit: */ /* * No RIP-relative addressing on 32-bit diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h index e5603cc91963..b0cc60f1c458 100644 --- a/include/linux/syscalls.h +++ b/include/linux/syscalls.h @@ -998,6 +998,8 @@ asmlinkage long sys_ioperm(unsigned long from, unsigned= long num, int on); =20 asmlinkage long sys_uretprobe(void); =20 +asmlinkage long sys_uprobe(void); + /* pciconfig: alpha, arm, arm64, ia64, sparc */ asmlinkage long sys_pciconfig_read(unsigned long bus, unsigned long dfn, unsigned long off, unsigned long len, diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index bc532d086813..bbe218ff16cc 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -239,6 +239,7 @@ extern unsigned long uprobe_get_trampoline_vaddr(void); extern void uprobe_copy_from_page(struct page *page, unsigned long vaddr, = void *dst, int len); extern void arch_uprobe_clear_state(struct mm_struct *mm); extern void arch_uprobe_init_state(struct mm_struct *mm); +extern void handle_syscall_uprobe(struct pt_regs *regs, unsigned long bp_v= addr); #else /* !CONFIG_UPROBES */ struct uprobes_state { }; diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index a3107f63f295..97a7b9f0c7ca 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -2782,6 +2782,23 @@ static void handle_swbp(struct pt_regs *regs) rcu_read_unlock_trace(); } =20 +void handle_syscall_uprobe(struct pt_regs *regs, unsigned long bp_vaddr) +{ + struct uprobe *uprobe; + int is_swbp; + + guard(rcu_tasks_trace)(); + + uprobe =3D find_active_uprobe_rcu(bp_vaddr, &is_swbp); + if (!uprobe) + return; + if (!get_utask()) + return; + if (arch_uprobe_ignore(&uprobe->arch, regs)) + return; + handler_chain(uprobe, regs); +} + /* * Perform required fix-ups and disable singlestep. * Allow pending signals to take effect. diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c index c00a86931f8c..bf5d05c635ff 100644 --- a/kernel/sys_ni.c +++ b/kernel/sys_ni.c @@ -392,3 +392,4 @@ COND_SYSCALL(setuid16); COND_SYSCALL(rseq); =20 COND_SYSCALL(uretprobe); +COND_SYSCALL(uprobe); --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CB0C21A433; Mon, 21 Apr 2025 21:46:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271993; cv=none; b=SwfW+K828JRytkdVqzx+hpUPX/2H4+9aJGWdEvmRV/oqWCkevAhTvLjcejNCLv0uoOr7bCnxgqHUjhBIDUgg/8HhzZs8zY58uXEVGSP+zmk1STYwGk32vVhuRTVT94hKQ6FzHVqNAr783Ii9dzkBBfCTVbdoFnZPAZ0xJFXl+yA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745271993; c=relaxed/simple; bh=HE1Bda5NpWZHr9k9j8IGFS/Nnrltuhq/KagO36pv+U4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GlJhPOwM2c58gr/XzIqQjGFW3CDtxRZfZ0tdv9XSyOGfzrGLhcgEVmvFgsKY28Ij+Ow5j/CII8YdeGJdoseK/mHX8B/c6xCmvmTXhtZVUqceOwBI5gCrbaCIFdCDU1kDdl35k7Jw6wlSXSB4GIFXF5v4Xw84Rk92HJS+Lw6wKZs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Z0RZe5LI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Z0RZe5LI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5EAAEC4CEE4; Mon, 21 Apr 2025 21:46:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745271992; bh=HE1Bda5NpWZHr9k9j8IGFS/Nnrltuhq/KagO36pv+U4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Z0RZe5LI3fm7ovDloTzx8tHBUlC3aEaoVo6/evdsLbcZrFbiagu4noeE8m31VRpup g+UsHXWLN/ZhPK6Q0qdJ96ZvqazrNVY45D3nqJlcUy5HWInkLnwdOpZ7XCVQ+UFs5x LtUOrYSeDCe8UdBjL6ohyIqA+J420I51CVaro/m6U3cyEg8Rl1c7AWVeTvWFKJiOJk LjZjwfpOX9cN11AR1QnGhFMBWklN2mSFdZ2LiRD1xtk86rJaODyLwISE+GWU/rSP9Q 8vSmDxt5+NLUh4ZB+KEMycKGWfN+zZx+GBP2jKYDMJCE5XWChfaLth9n6odVtZvKBp 1szimup3//1gQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 10/22] uprobes/x86: Add support to optimize uprobes Date: Mon, 21 Apr 2025 23:44:10 +0200 Message-ID: <20250421214423.393661-11-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Putting together all the previously added pieces to support optimized uprobes on top of 5-byte nop instruction. The current uprobe execution goes through following: - installs breakpoint instruction over original instruction - exception handler hit and calls related uprobe consumers - and either simulates original instruction or does out of line single st= ep execution of it - returns to user space The optimized uprobe path does following: - checks the original instruction is 5-byte nop (plus other checks) - adds (or uses existing) user space trampoline with uprobe syscall - overwrites original instruction (5-byte nop) with call to user space trampoline - the user space trampoline executes uprobe syscall that calls related up= robe consumers - trampoline returns back to next instruction This approach won't speed up all uprobes as it's limited to using nop5 as original instruction, but we plan to use nop5 as USDT probe instruction (which currently uses single byte nop) and speed up the USDT probes. The arch_uprobe_optimize triggers the uprobe optimization and is called aft= er first uprobe hit. I originally had it called on uprobe installation but then it clashed with elf loader, because the user space trampoline was added in a place where loader might need to put elf segments, so I decided to do it af= ter first uprobe hit when loading is done. The uprobe is un-optimized in arch specific set_orig_insn call. The instruction overwrite is x86 arch specific and needs to go through 3 up= dates: (on top of nop5 instruction) - write int3 into 1st byte - write last 4 bytes of the call instruction - update the call instruction opcode And cleanup goes though similar reverse stages: - overwrite call opcode with breakpoint (int3) - write last 4 bytes of the nop5 instruction - write the nop5 first instruction byte We do not unmap and release uprobe trampoline when it's no longer needed, because there's no easy way to make sure none of the threads is still inside the trampoline. But we do not waste memory, because there's just single page for all the uprobe trampoline mappings. We do waste frame on page mapping for every 4GB by keeping the uprobe trampoline page mapped, but that seems ok. We take the benefit from the fact that set_swbp and set_orig_insn are called under mmap_write_lock(mm), so we can use the current instruction as the state the uprobe is in - nop5/breakpoint/call trampoline - and decide the needed action (optimize/un-optimize) based on that. Attaching the speed up from benchs/run_bench_uprobes.sh script: current: usermode-count : 152.604 =C2=B1 0.044M/s syscall-count : 13.359 =C2=B1 0.042M/s --> uprobe-nop : 3.229 =C2=B1 0.002M/s uprobe-push : 3.086 =C2=B1 0.004M/s uprobe-ret : 1.114 =C2=B1 0.004M/s uprobe-nop5 : 1.121 =C2=B1 0.005M/s uretprobe-nop : 2.145 =C2=B1 0.002M/s uretprobe-push : 2.070 =C2=B1 0.001M/s uretprobe-ret : 0.931 =C2=B1 0.001M/s uretprobe-nop5 : 0.957 =C2=B1 0.001M/s after the change: usermode-count : 152.448 =C2=B1 0.244M/s syscall-count : 14.321 =C2=B1 0.059M/s uprobe-nop : 3.148 =C2=B1 0.007M/s uprobe-push : 2.976 =C2=B1 0.004M/s uprobe-ret : 1.068 =C2=B1 0.003M/s --> uprobe-nop5 : 7.038 =C2=B1 0.007M/s uretprobe-nop : 2.109 =C2=B1 0.004M/s uretprobe-push : 2.035 =C2=B1 0.001M/s uretprobe-ret : 0.908 =C2=B1 0.001M/s uretprobe-nop5 : 3.377 =C2=B1 0.009M/s I see bit more speed up on Intel (above) compared to AMD. The big nop5 speed up is partly due to emulating nop5 and partly due to optimization. The key speed up we do this for is the USDT switch from nop to nop5: uprobe-nop : 3.148 =C2=B1 0.007M/s uprobe-nop5 : 7.038 =C2=B1 0.007M/s Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- arch/x86/include/asm/uprobes.h | 7 + arch/x86/kernel/uprobes.c | 281 ++++++++++++++++++++++++++++++++- include/linux/uprobes.h | 6 +- kernel/events/uprobes.c | 15 +- 4 files changed, 301 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/uprobes.h b/arch/x86/include/asm/uprobes.h index 678fb546f0a7..1ee2e5115955 100644 --- a/arch/x86/include/asm/uprobes.h +++ b/arch/x86/include/asm/uprobes.h @@ -20,6 +20,11 @@ typedef u8 uprobe_opcode_t; #define UPROBE_SWBP_INSN 0xcc #define UPROBE_SWBP_INSN_SIZE 1 =20 +enum { + ARCH_UPROBE_FLAG_CAN_OPTIMIZE =3D 0, + ARCH_UPROBE_FLAG_OPTIMIZE_FAIL =3D 1, +}; + struct uprobe_xol_ops; =20 struct arch_uprobe { @@ -45,6 +50,8 @@ struct arch_uprobe { u8 ilen; } push; }; + + unsigned long flags; }; =20 struct arch_uprobe_task { diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c index 01b3035e01ea..d5ef04a1626d 100644 --- a/arch/x86/kernel/uprobes.c +++ b/arch/x86/kernel/uprobes.c @@ -18,6 +18,7 @@ #include #include #include +#include =20 /* Post-execution fixups. */ =20 @@ -691,7 +692,6 @@ static struct uprobe_trampoline *create_uprobe_trampoli= ne(unsigned long vaddr) return NULL; } =20 -__maybe_unused static struct uprobe_trampoline *uprobe_trampoline_get(unsigned long vaddr) { struct uprobes_state *state =3D ¤t->mm->uprobes_state; @@ -718,7 +718,6 @@ static void destroy_uprobe_trampoline(struct uprobe_tra= mpoline *tramp) kfree(tramp); } =20 -__maybe_unused static void uprobe_trampoline_put(struct uprobe_trampoline *tramp) { if (tramp && atomic64_dec_and_test(&tramp->ref)) @@ -861,6 +860,277 @@ static int __init arch_uprobes_init(void) =20 late_initcall(arch_uprobes_init); =20 +enum { + OPT_PART, + OPT_INSN, + UNOPT_INT3, + UNOPT_PART, +}; + +struct write_opcode_ctx { + unsigned long base; + int update; +}; + +static int is_call_insn(uprobe_opcode_t *insn) +{ + return *insn =3D=3D CALL_INSN_OPCODE; +} + +static int verify_insn(struct page *page, unsigned long vaddr, uprobe_opco= de_t *new_opcode, + int nbytes, void *data) +{ + struct write_opcode_ctx *ctx =3D data; + uprobe_opcode_t old_opcode[5]; + + uprobe_copy_from_page(page, ctx->base, (uprobe_opcode_t *) &old_opcode, 5= ); + + switch (ctx->update) { + case OPT_PART: + case OPT_INSN: + if (is_swbp_insn(&old_opcode[0])) + return 1; + break; + case UNOPT_INT3: + if (is_call_insn(&old_opcode[0])) + return 1; + break; + case UNOPT_PART: + if (is_swbp_insn(&old_opcode[0])) + return 1; + break; + } + + return -1; +} + +static int write_insn(struct vm_area_struct *vma, unsigned long vaddr, + uprobe_opcode_t *insn, int nbytes, void *ctx) +{ + return uprobe_write(vma, vaddr, insn, nbytes, verify_insn, true, ctx); +} + +static void relative_call(void *dest, long from, long to) +{ + struct __packed __arch_relative_insn { + u8 op; + s32 raddr; + } *insn; + + insn =3D (struct __arch_relative_insn *)dest; + insn->raddr =3D (s32)(to - (from + 5)); + insn->op =3D CALL_INSN_OPCODE; +} + +static int swbp_optimize(struct vm_area_struct *vma, unsigned long vaddr, = unsigned long tramp) +{ + struct write_opcode_ctx ctx =3D { + .base =3D vaddr, + }; + char call[5]; + int err; + + relative_call(call, vaddr, tramp); + + /* + * We are in state where breakpoint (int3) is installed on top of first + * byte of the nop5 instruction. We will do following steps to overwrite + * this to call instruction: + * + * - sync cores + * - write last 4 bytes of the call instruction + * - sync cores + * - update the call instruction opcode + */ + + text_poke_sync(); + + ctx.update =3D OPT_PART; + err =3D write_insn(vma, vaddr + 1, call + 1, 4, &ctx); + if (err) + return err; + + text_poke_sync(); + + ctx.update =3D OPT_INSN; + return write_insn(vma, vaddr, call, 1, &ctx); +} + +static int swbp_unoptimize(struct arch_uprobe *auprobe, struct vm_area_str= uct *vma, + unsigned long vaddr) +{ + uprobe_opcode_t int3 =3D UPROBE_SWBP_INSN; + struct write_opcode_ctx ctx =3D { + .base =3D vaddr, + }; + int err; + + /* + * We need to overwrite call instruction into nop5 instruction with + * breakpoint (int3) installed on top of its first byte. We will: + * + * - overwrite call opcode with breakpoint (int3) + * - sync cores + * - write last 4 bytes of the nop5 instruction + * - sync cores + */ + + ctx.update =3D UNOPT_INT3; + err =3D write_insn(vma, vaddr, &int3, 1, &ctx); + if (err) + return err; + + text_poke_sync(); + + ctx.update =3D UNOPT_PART; + err =3D write_insn(vma, vaddr + 1, (uprobe_opcode_t *) auprobe->insn + 1,= 4, &ctx); + + text_poke_sync(); + return err; +} + +static int copy_from_vaddr(struct mm_struct *mm, unsigned long vaddr, void= *dst, int len) +{ + unsigned int gup_flags =3D FOLL_FORCE|FOLL_SPLIT_PMD; + struct vm_area_struct *vma; + struct page *page; + + page =3D get_user_page_vma_remote(mm, vaddr, gup_flags, &vma); + if (IS_ERR(page)) + return PTR_ERR(page); + uprobe_copy_from_page(page, vaddr, dst, len); + put_page(page); + return 0; +} + +static bool __is_optimized(uprobe_opcode_t *insn, unsigned long vaddr) +{ + struct __packed __arch_relative_insn { + u8 op; + s32 raddr; + } *call =3D (struct __arch_relative_insn *) insn; + + if (!is_call_insn(insn)) + return false; + return __in_uprobe_trampoline(vaddr + 5 + call->raddr); +} + +static int is_optimized(struct mm_struct *mm, unsigned long vaddr, bool *o= ptimized) +{ + uprobe_opcode_t insn[5]; + int err; + + err =3D copy_from_vaddr(mm, vaddr, &insn, 5); + if (err) + return err; + *optimized =3D __is_optimized((uprobe_opcode_t *)&insn, vaddr); + return 0; +} + +static bool should_optimize(struct arch_uprobe *auprobe) +{ + return !test_bit(ARCH_UPROBE_FLAG_OPTIMIZE_FAIL, &auprobe->flags) && + test_bit(ARCH_UPROBE_FLAG_CAN_OPTIMIZE, &auprobe->flags); +} + +int set_swbp(struct arch_uprobe *auprobe, struct vm_area_struct *vma, + unsigned long vaddr) +{ + if (should_optimize(auprobe)) { + bool optimized =3D false; + int err; + + /* + * We could race with another thread that already optimized the probe, + * so let's not overwrite it with int3 again in this case. + */ + err =3D is_optimized(vma->vm_mm, vaddr, &optimized); + if (err || optimized) + return err; + } + return uprobe_write_opcode(vma, vaddr, UPROBE_SWBP_INSN, true); +} + +int set_orig_insn(struct arch_uprobe *auprobe, struct vm_area_struct *vma, + unsigned long vaddr) +{ + if (test_bit(ARCH_UPROBE_FLAG_CAN_OPTIMIZE, &auprobe->flags)) { + struct mm_struct *mm =3D vma->vm_mm; + bool optimized =3D false; + int err; + + err =3D is_optimized(mm, vaddr, &optimized); + if (err) + return err; + if (optimized) + WARN_ON_ONCE(swbp_unoptimize(auprobe, vma, vaddr)); + } + return uprobe_write_opcode(vma, vaddr, *(uprobe_opcode_t *)&auprobe->insn= , false); +} + +static int __arch_uprobe_optimize(struct mm_struct *mm, unsigned long vadd= r) +{ + struct uprobe_trampoline *tramp; + struct vm_area_struct *vma; + int err =3D 0; + + vma =3D find_vma(mm, vaddr); + if (!vma) + return -1; + tramp =3D uprobe_trampoline_get(vaddr); + if (!tramp) + return -1; + err =3D swbp_optimize(vma, vaddr, tramp->vaddr); + if (WARN_ON_ONCE(err)) + uprobe_trampoline_put(tramp); + return err; +} + +void arch_uprobe_optimize(struct arch_uprobe *auprobe, unsigned long vaddr) +{ + struct mm_struct *mm =3D current->mm; + uprobe_opcode_t insn[5]; + + /* + * Do not optimize if shadow stack is enabled, the return address hijack + * code in arch_uretprobe_hijack_return_addr updates wrong frame when + * the entry uprobe is optimized and the shadow stack crashes the app. + */ + if (shstk_is_enabled()) + return; + + if (!should_optimize(auprobe)) + return; + + mmap_write_lock(mm); + + /* + * Check if some other thread already optimized the uprobe for us, + * if it's the case just go away silently. + */ + if (copy_from_vaddr(mm, vaddr, &insn, 5)) + goto unlock; + if (!is_swbp_insn((uprobe_opcode_t*) &insn)) + goto unlock; + + /* + * If we fail to optimize the uprobe we set the fail bit so the + * above should_optimize will fail from now on. + */ + if (__arch_uprobe_optimize(mm, vaddr)) + set_bit(ARCH_UPROBE_FLAG_OPTIMIZE_FAIL, &auprobe->flags); + +unlock: + mmap_write_unlock(mm); +} + +static bool can_optimize(struct arch_uprobe *auprobe, unsigned long vaddr) +{ + if (memcmp(&auprobe->insn, x86_nops[5], 5)) + return false; + /* We can't do cross page atomic writes yet. */ + return PAGE_SIZE - (vaddr & ~PAGE_MASK) >=3D 5; +} #else /* 32-bit: */ /* * No RIP-relative addressing on 32-bit @@ -874,6 +1144,10 @@ static void riprel_pre_xol(struct arch_uprobe *auprob= e, struct pt_regs *regs) static void riprel_post_xol(struct arch_uprobe *auprobe, struct pt_regs *r= egs) { } +static bool can_optimize(struct arch_uprobe *auprobe, unsigned long vaddr) +{ + return false; +} #endif /* CONFIG_X86_64 */ =20 struct uprobe_xol_ops { @@ -1240,6 +1514,9 @@ int arch_uprobe_analyze_insn(struct arch_uprobe *aupr= obe, struct mm_struct *mm, if (ret) return ret; =20 + if (can_optimize(auprobe, addr)) + set_bit(ARCH_UPROBE_FLAG_CAN_OPTIMIZE, &auprobe->flags); + ret =3D branch_setup_xol_ops(auprobe, &insn); if (ret !=3D -ENOSYS) return ret; diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index bbe218ff16cc..d4c1fed9a9e4 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -192,7 +192,7 @@ struct uprobes_state { }; =20 typedef int (*uprobe_write_verify_t)(struct page *page, unsigned long vadd= r, - uprobe_opcode_t *insn, int nbytes); + uprobe_opcode_t *insn, int nbytes, void *data); =20 extern void __init uprobes_init(void); extern int set_swbp(struct arch_uprobe *aup, struct vm_area_struct *vma, u= nsigned long vaddr); @@ -204,7 +204,8 @@ extern unsigned long uprobe_get_trap_addr(struct pt_reg= s *regs); extern int uprobe_write_opcode(struct vm_area_struct *vma, unsigned long v= addr, uprobe_opcode_t opcode, bool is_register); extern int uprobe_write(struct vm_area_struct *vma, const unsigned long in= sn_vaddr, - uprobe_opcode_t *insn, int nbytes, uprobe_write_verify_t verify, bool i= s_register); + uprobe_opcode_t *insn, int nbytes, uprobe_write_verify_t verify, bool i= s_register, + void *data); extern struct uprobe *uprobe_register(struct inode *inode, loff_t offset, = loff_t ref_ctr_offset, struct uprobe_consumer *uc); extern int uprobe_apply(struct uprobe *uprobe, struct uprobe_consumer *uc,= bool); extern void uprobe_unregister_nosync(struct uprobe *uprobe, struct uprobe_= consumer *uc); @@ -240,6 +241,7 @@ extern void uprobe_copy_from_page(struct page *page, un= signed long vaddr, void * extern void arch_uprobe_clear_state(struct mm_struct *mm); extern void arch_uprobe_init_state(struct mm_struct *mm); extern void handle_syscall_uprobe(struct pt_regs *regs, unsigned long bp_v= addr); +extern void arch_uprobe_optimize(struct arch_uprobe *auprobe, unsigned lon= g vaddr); #else /* !CONFIG_UPROBES */ struct uprobes_state { }; diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 97a7b9f0c7ca..408a134c1a7b 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -192,7 +192,7 @@ static void copy_to_page(struct page *page, unsigned lo= ng vaddr, const void *src } =20 static int verify_opcode(struct page *page, unsigned long vaddr, uprobe_op= code_t *insn, - int nbytes) + int nbytes, void *data) { uprobe_opcode_t old_opcode; bool is_swbp; @@ -491,12 +491,12 @@ int uprobe_write_opcode(struct vm_area_struct *vma, c= onst unsigned long opcode_v uprobe_opcode_t opcode, bool is_register) { return uprobe_write(vma, opcode_vaddr, &opcode, UPROBE_SWBP_INSN_SIZE, - verify_opcode, is_register); + verify_opcode, is_register, NULL); } =20 int uprobe_write(struct vm_area_struct *vma, const unsigned long insn_vadd= r, uprobe_opcode_t *insn, int nbytes, uprobe_write_verify_t verify, - bool is_register) + bool is_register, void *data) { const unsigned long vaddr =3D insn_vaddr & PAGE_MASK; struct mm_struct *mm =3D vma->vm_mm; @@ -527,7 +527,7 @@ int uprobe_write(struct vm_area_struct *vma, const unsi= gned long insn_vaddr, goto out; folio =3D page_folio(page); =20 - ret =3D verify(page, insn_vaddr, insn, nbytes); + ret =3D verify(page, insn_vaddr, insn, nbytes, data); if (ret <=3D 0) { folio_put(folio); goto out; @@ -2707,6 +2707,10 @@ bool __weak arch_uretprobe_is_alive(struct return_in= stance *ret, enum rp_check c return true; } =20 +void __weak arch_uprobe_optimize(struct arch_uprobe *auprobe, unsigned lon= g vaddr) +{ +} + /* * Run handler and ask thread to singlestep. * Ensure all non-fatal signals cannot interrupt thread while it singleste= ps. @@ -2771,6 +2775,9 @@ static void handle_swbp(struct pt_regs *regs) =20 handler_chain(uprobe, regs); =20 + /* Try to optimize after first hit. */ + arch_uprobe_optimize(&uprobe->arch, bp_vaddr); + if (arch_uprobe_skip_sstep(&uprobe->arch, regs)) goto out; =20 --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0EA921ABDC; Mon, 21 Apr 2025 21:46:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272005; cv=none; b=jdvO8/guHXtR7Jwv9D1wIQT3bZJc2v1PJT4qDOeHahpBDCihOagHgdngY2gkjJvbXz4k1GSChCDegCWE4oVMihZXMN4huPIuPEbTlljQgRbsQBfASBCG11UWpB237ha+ULGQdEgos4jDkwEo1p93/4N8wooseVYmUPCvUCTIfIk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272005; c=relaxed/simple; bh=ArFi6LYnws/zK5pbLN4Y6WDU1aVkoTA4UhqE31uFzpc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iFyATc3O9uIv+GSCOZ0c+fvpUi7a4WoGb/Zf9mZ7UjcFMiiNpD+D//DrDA7Mv81hN4XILDQy/Ac8NwlhwqYtrENGO28i0jiRl256Ipmwg9PvYQ1YfuT9yC155nitp75ya5GbWDZ/ebFBfryQK1o9e7BVLs8s4PciX+y22ATKnzI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=XFbitown; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="XFbitown" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 73373C4CEEA; Mon, 21 Apr 2025 21:46:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745272004; bh=ArFi6LYnws/zK5pbLN4Y6WDU1aVkoTA4UhqE31uFzpc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XFbitownCgJkrcx4SgKZj55+9VkZsd8302JYCpb6h4fdy+5cl8LqwYAozlGxIB8kn i8TyXOvEL4Zk5eqWbbEf4HPXqiv1l/uw+Xrxwve1UKDCyo3DVvhqRrzc0XXGzMtB6l M3zIOhNOu8XAtaH5K9h1CTFIbhYst/OZ3UOn25PAM4asosE5/T2/jycBfnQkVwfwMl h3ImtZuj31sGZqtxKUXHKvQfMlp6tlI2y7JMv+o2ZvfhzwaYxX2fqzDw1Wn6Kq8kfg 177Gp8T3X0XqfGnZWlVGtJ6jDzU2d25Ubt571iq5tzzIJmwVdZwfZFGhJYOXQFyG0I FC+c4PrvOhnUQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 11/22] selftests/bpf: Use 5-byte nop for x86 usdt probes Date: Mon, 21 Apr 2025 23:44:11 +0200 Message-ID: <20250421214423.393661-12-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Using 5-byte nop for x86 usdt probes so we can switch to optimized uprobe them. Signed-off-by: Jiri Olsa --- tools/testing/selftests/bpf/sdt.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/sdt.h b/tools/testing/selftests/bp= f/sdt.h index 1fcfa5160231..1d62c06f5ddc 100644 --- a/tools/testing/selftests/bpf/sdt.h +++ b/tools/testing/selftests/bpf/sdt.h @@ -236,6 +236,13 @@ __extension__ extern unsigned long long __sdt_unsp; #define _SDT_NOP nop #endif =20 +/* Use 5 byte nop for x86_64 to allow optimizing uprobes. */ +#if defined(__x86_64__) +# define _SDT_DEF_NOP _SDT_ASM_5(990: .byte 0x0f, 0x1f, 0x44, 0x00, 0x00) +#else +# define _SDT_DEF_NOP _SDT_ASM_1(990: _SDT_NOP) +#endif + #define _SDT_NOTE_NAME "stapsdt" #define _SDT_NOTE_TYPE 3 =20 @@ -288,7 +295,7 @@ __extension__ extern unsigned long long __sdt_unsp; =20 #define _SDT_ASM_BODY(provider, name, pack_args, args, ...) \ _SDT_DEF_MACROS \ - _SDT_ASM_1(990: _SDT_NOP) \ + _SDT_DEF_NOP \ _SDT_ASM_3( .pushsection .note.stapsdt,_SDT_ASM_AUTOGROUP,"note") \ _SDT_ASM_1( .balign 4) \ _SDT_ASM_3( .4byte 992f-991f, 994f-993f, _SDT_NOTE_TYPE) \ --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9C9021A421; Mon, 21 Apr 2025 21:46:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272017; cv=none; b=mb6Kta4QWXeZEi6/ikl13sHz3V/rQP8CV8hE/CHTKdyrROwQgMBQV06bD0O3v/yhT9Ez9eJp/OA9OpCHP22jkOggNIIbjlc5vUXV9GCZWsPIzE7bfurx0+TAfcWLT5aErcliyB/fHc61TuUlfTWHr4TfdKv/ZKDHwnqeS7D6AT0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272017; c=relaxed/simple; bh=QHeRP+3vFIU4kzTD64Z50sxitOHVSEPLwnAxE9SvpDo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bDHnH61Zy1tCVtbvCqOZAA6sV+t8lbIWUH+UnqFfy2tmD4+R+ckdqD4qSsDSIgrROEQAR65ExhASfdmcHOWKLA+VVJiIHyJmFYuhRjNm7Qg3f2oxlI0PygqHLe4nslJ5+1xun2Qj13xSF8EXO/NUpVAURB58LVjyXTAOAL47aZ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qWTKbciV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qWTKbciV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B944EC4CEE4; Mon, 21 Apr 2025 21:46:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745272017; bh=QHeRP+3vFIU4kzTD64Z50sxitOHVSEPLwnAxE9SvpDo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qWTKbciVgJUk4YqDNH6wXkskafiaPg5xNbUEjP8L8y89AXwTJpUfzZmt7Qi5blWzb yLM5TN3u4Hsqy+HAzpBS1AscYTO4IrX4QObhNLHKGaGK7TucNjVCtGgOrGlM7FfqJJ ysAIF+dRH0zJ7RlaYu9QEHDAHLwuH8qCVbrUNeeaU0NBX9FWyLS8VhurN7qJCxItpS NR/ARwZ5SDlR4rkCmQDtPfFC3zdXcWxw47g7stC3CexXjrzOYm7liRwNjaa7HmAhWN dRmP1dlPhRQz2JzF0u3tipdb/u9uZCT9b80jc8HLQtYlgNP3FfbEJ2eooeBYfX/lxt FIjEa+qP9rnYQ== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 12/22] selftests/bpf: Reorg the uprobe_syscall test function Date: Mon, 21 Apr 2025 23:44:12 +0200 Message-ID: <20250421214423.393661-13-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding __test_uprobe_syscall with non x86_64 stub to execute all the tests, so we don't need to keep adding non x86_64 stub functions for new tests. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- .../selftests/bpf/prog_tests/uprobe_syscall.c | 34 +++++++------------ 1 file changed, 12 insertions(+), 22 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c b/tool= s/testing/selftests/bpf/prog_tests/uprobe_syscall.c index c397336fe1ed..2b00f16406c8 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c @@ -350,29 +350,8 @@ static void test_uretprobe_shadow_stack(void) =20 ARCH_PRCTL(ARCH_SHSTK_DISABLE, ARCH_SHSTK_SHSTK); } -#else -static void test_uretprobe_regs_equal(void) -{ - test__skip(); -} - -static void test_uretprobe_regs_change(void) -{ - test__skip(); -} - -static void test_uretprobe_syscall_call(void) -{ - test__skip(); -} =20 -static void test_uretprobe_shadow_stack(void) -{ - test__skip(); -} -#endif - -void test_uprobe_syscall(void) +static void __test_uprobe_syscall(void) { if (test__start_subtest("uretprobe_regs_equal")) test_uretprobe_regs_equal(); @@ -383,3 +362,14 @@ void test_uprobe_syscall(void) if (test__start_subtest("uretprobe_shadow_stack")) test_uretprobe_shadow_stack(); } +#else +static void __test_uprobe_syscall(void) +{ + test__skip(); +} +#endif + +void test_uprobe_syscall(void) +{ + __test_uprobe_syscall(); +} --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F33821B909; Mon, 21 Apr 2025 21:47:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272029; cv=none; b=Jfb3hfoTt6rO66DjcevkbyJyQT5k4B+sf7NhNcNsMGneftinTREo5b4JiodUKnp6cGzcyCIvLoCAfS7jpFPwIV/++DWwFdNtdCnjJz52cKUqplrfinVW17Himdhc9pPb1o4Ln7MZ7tZ4BI/kcYd9cTQ0O8k5uwWhOpWRAj/1w5Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272029; c=relaxed/simple; bh=QVQ6qfmvUrSx0DydWdPijoOQliv6RaIe4HgM86uPyfU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fpNZBMk845/qX6BExU5SiG01ECrryT5JxiXc4bUyvQ3jM8vPDDnlAaie1NA2IaYvOybUb9hz/VMYxyVugrzgb5Thh6U0Di7whDPJNFXaOl5aD86QwFEJWWTB0VGePUrKtB99bBNd/lWNqiEzn2YqLoW8b0HRXmeQGmm2e3t8cCk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=S3ref8Qx; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="S3ref8Qx" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B480DC4CEE4; Mon, 21 Apr 2025 21:47:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745272029; bh=QVQ6qfmvUrSx0DydWdPijoOQliv6RaIe4HgM86uPyfU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S3ref8QxeNdSlGO49mcYKZDWNLyPqc7ezuxj8vPUPuWH4HA95NObE8dnpxIC0Fla3 WXGQk6YwBTgFEdGr8OEvZqVS7d1Jro7DbM59Ys3xgx0l9lJPO066Eg8zTMDGgY/3UY 0MsR0dmNjl5/+uWTzQhJECP8vQC7RSEGvm00jWSe3WKYGxL/BoyEpUiSyZnZ0Poobm LrHwA3f7yfVCvmu+gFkYFRHE0rMUNaULmepbscTEfqLpxh0GkulslN1/kNF82YtRjO mU6iqn0E8FrZFnydT7FfDTT75Lb1UR3YugYvLicSxJ+U+qzbgjgsHUkZ8quC5ntVIJ aWh+xP9aLDLNw== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 13/22] selftests/bpf: Rename uprobe_syscall_executed prog to test_uretprobe_multi Date: Mon, 21 Apr 2025 23:44:13 +0200 Message-ID: <20250421214423.393661-14-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Renaming uprobe_syscall_executed prog to test_uretprobe_multi to fit properly in the following changes that add more programs. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c | 8 ++++---- .../testing/selftests/bpf/progs/uprobe_syscall_executed.c | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c b/tool= s/testing/selftests/bpf/prog_tests/uprobe_syscall.c index 2b00f16406c8..3c74a079e6d9 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c @@ -277,10 +277,10 @@ static void test_uretprobe_syscall_call(void) _exit(0); } =20 - skel->links.test =3D bpf_program__attach_uprobe_multi(skel->progs.test, p= id, - "/proc/self/exe", - "uretprobe_syscall_call", &opts); - if (!ASSERT_OK_PTR(skel->links.test, "bpf_program__attach_uprobe_multi")) + skel->links.test_uretprobe_multi =3D bpf_program__attach_uprobe_multi(ske= l->progs.test_uretprobe_multi, + pid, "/proc/self/exe", + "uretprobe_syscall_call", &opts); + if (!ASSERT_OK_PTR(skel->links.test_uretprobe_multi, "bpf_program__attach= _uprobe_multi")) goto cleanup; =20 /* kick the child */ diff --git a/tools/testing/selftests/bpf/progs/uprobe_syscall_executed.c b/= tools/testing/selftests/bpf/progs/uprobe_syscall_executed.c index 0d7f1a7db2e2..2e1b689ed4fb 100644 --- a/tools/testing/selftests/bpf/progs/uprobe_syscall_executed.c +++ b/tools/testing/selftests/bpf/progs/uprobe_syscall_executed.c @@ -10,8 +10,8 @@ char _license[] SEC("license") =3D "GPL"; int executed =3D 0; =20 SEC("uretprobe.multi") -int test(struct pt_regs *regs) +int test_uretprobe_multi(struct pt_regs *ctx) { - executed =3D 1; + executed++; return 0; } --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D4C3D21CC44; Mon, 21 Apr 2025 21:47:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272041; cv=none; b=WWmSUg4jK5vIG/dBTIsVQAf44Xk706MF8aN0pkQEZkvcy12SeL9pVaSiM8un0oRdSLRSCs5L4pURfkqDgPf5VfMOF/xkcgo4v6tOkg+Ze5XLjyy2claEHs6cSNUt3QmHCNcpEZjQ+9+g/gjlip29Fuu7CW0C0FrU5QAXCM644v0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272041; c=relaxed/simple; bh=+nZREEM3gS258RP/nFqPYiILlPVFY0mElLL3BfF6IUU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=a8UOVoMn8JT70ikyryq/4wyoGWc3Z0ZhE4qcipovEL3CSn0p/gNJa5vKuHfSBC36u+untWZTkk/x0oRCctccHF8i3UKeGkPZwlq7VsDxzWt+d+/GQHYCom6V7ib4BzHjMcuO1s6oUN1TJHbf6Srvgc0H/vVKux2eyaErH3yZr8w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=b5eTMU59; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="b5eTMU59" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 65BA2C4CEE4; Mon, 21 Apr 2025 21:47:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745272041; bh=+nZREEM3gS258RP/nFqPYiILlPVFY0mElLL3BfF6IUU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b5eTMU59tRibTLxHH03Y2h6I7Lcd47vV9/QU0Pge3QkZtO6qfEOdBLu3IPvOH6dJD guVKSYvBH1LMvaqCCFIy3BqQj58avYgc01i4nw+BtJ/mT/UUjip3Q7NRsAtsZTK1Tt zm7cKYWr4GW6/z3B4eBxIE+uz5ej0LPvSevMU2z+yM8Ys5yCxdUfviejxF3mWIPd1P JGjhF7IH313fHV1di8gQr996QVT9MOVzEfDwdzNQ0Z6NEywBBoAw/icSnd8oCl5KgJ 7noY5ZmmJfDXmjtBc7E55kkq5iSNT5oZrI8b/2OhI5OMZV3+9cV0mSMtCHUHW5Rvxo 5SBK+ZipFMAvg== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 14/22] selftests/bpf: Add uprobe/usdt syscall tests Date: Mon, 21 Apr 2025 23:44:14 +0200 Message-ID: <20250421214423.393661-15-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding tests for optimized uprobe/usdt probes. Checking that we get expected trampoline and attached bpf programs get executed properly. Signed-off-by: Jiri Olsa --- .../selftests/bpf/prog_tests/uprobe_syscall.c | 278 +++++++++++++++++- .../bpf/progs/uprobe_syscall_executed.c | 37 +++ 2 files changed, 314 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c b/tool= s/testing/selftests/bpf/prog_tests/uprobe_syscall.c index 3c74a079e6d9..16effe0bca1d 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c @@ -14,6 +14,9 @@ #include #include "uprobe_syscall.skel.h" #include "uprobe_syscall_executed.skel.h" +#include "sdt.h" + +#pragma GCC diagnostic ignored "-Wattributes" =20 __naked unsigned long uretprobe_regs_trigger(void) { @@ -301,6 +304,262 @@ static void test_uretprobe_syscall_call(void) close(go[0]); } =20 +#define TRAMP "[uprobes-trampoline]" + +__attribute__((aligned(16))) +__nocf_check __weak __naked void uprobe_test(void) +{ + asm volatile (" \n" + ".byte 0x0f, 0x1f, 0x44, 0x00, 0x00 \n" + "ret \n" + ); +} + +__attribute__((aligned(16))) +__nocf_check __weak void usdt_test(void) +{ + STAP_PROBE(optimized_uprobe, usdt); +} + +static int find_uprobes_trampoline(void **start, void **end) +{ + char line[128]; + int ret =3D -1; + FILE *maps; + + maps =3D fopen("/proc/self/maps", "r"); + if (!maps) { + fprintf(stderr, "cannot open maps\n"); + return -1; + } + + while (fgets(line, sizeof(line), maps)) { + int m =3D -1; + + /* We care only about private r-x mappings. */ + if (sscanf(line, "%p-%p r-xp %*x %*x:%*x %*u %n", start, end, &m) !=3D 2) + continue; + if (m < 0) + continue; + if (!strncmp(&line[m], TRAMP, sizeof(TRAMP)-1)) { + ret =3D 0; + break; + } + } + + fclose(maps); + return ret; +} + +static unsigned char nop5[5] =3D { 0x0f, 0x1f, 0x44, 0x00, 0x00 }; + +static void *find_nop5(void *fn) +{ + int i; + + for (i =3D 0; i < 10; i++) { + if (!memcmp(nop5, fn + i, 5)) + return fn + i; + } + return NULL; +} + +typedef void (__attribute__((nocf_check)) *trigger_t)(void); + +static bool shstk_is_enabled; + +static void check_attach(struct uprobe_syscall_executed *skel, trigger_t t= rigger, + void *addr, int executed) +{ + void *tramp_start, *tramp_end; + struct __arch_relative_insn { + u8 op; + s32 raddr; + } __packed *call; + s32 delta; + u8 *bp; + + /* Uprobe gets optimized after first trigger, so let's press twice. */ + trigger(); + trigger(); + + if (!shstk_is_enabled && + !ASSERT_OK(find_uprobes_trampoline(&tramp_start, &tramp_end), "uprobe= s_trampoline")) + return; + + /* Make sure bpf program got executed.. */ + ASSERT_EQ(skel->bss->executed, executed, "executed"); + + if (shstk_is_enabled) { + /* .. and check optimization is disabled under shadow stack. */ + bp =3D (u8 *) addr; + ASSERT_EQ(*bp, 0xcc, "int3"); + } else { + /* .. and check the trampoline is as expected. */ + call =3D (struct __arch_relative_insn *) addr; + delta =3D (unsigned long) tramp_start - ((unsigned long) addr + 5); + + ASSERT_EQ(call->op, 0xe8, "call"); + ASSERT_EQ(call->raddr, delta, "delta"); + ASSERT_EQ(tramp_end - tramp_start, 4096, "size"); + } +} + +static void check_detach(struct uprobe_syscall_executed *skel, trigger_t t= rigger, void *addr) +{ + void *tramp_start, *tramp_end; + + /* [uprobes_trampoline] stays after detach */ + ASSERT_OK(!shstk_is_enabled && + find_uprobes_trampoline(&tramp_start, &tramp_end), "uprobes_trampoline= "); + ASSERT_OK(memcmp(addr, nop5, 5), "nop5"); +} + +static void check(struct uprobe_syscall_executed *skel, struct bpf_link *l= ink, + trigger_t trigger, void *addr, int executed) +{ + check_attach(skel, trigger, addr, executed); + bpf_link__destroy(link); + check_detach(skel, trigger, addr); +} + +static void test_uprobe_legacy(void) +{ + struct uprobe_syscall_executed *skel =3D NULL; + LIBBPF_OPTS(bpf_uprobe_opts, opts, + .retprobe =3D true, + ); + struct bpf_link *link; + unsigned long offset; + + offset =3D get_uprobe_offset(&uprobe_test); + if (!ASSERT_GE(offset, 0, "get_uprobe_offset")) + goto cleanup; + + /* uprobe */ + skel =3D uprobe_syscall_executed__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_syscall_executed__open_and_load")) + return; + + link =3D bpf_program__attach_uprobe_opts(skel->progs.test_uprobe, + 0, "/proc/self/exe", offset, NULL); + if (!ASSERT_OK_PTR(link, "bpf_program__attach_uprobe_opts")) + goto cleanup; + + check(skel, link, uprobe_test, uprobe_test, 2); + + /* uretprobe */ + skel->bss->executed =3D 0; + + link =3D bpf_program__attach_uprobe_opts(skel->progs.test_uretprobe, + 0, "/proc/self/exe", offset, &opts); + if (!ASSERT_OK_PTR(link, "bpf_program__attach_uprobe_opts")) + goto cleanup; + + check(skel, link, uprobe_test, uprobe_test, 2); + +cleanup: + uprobe_syscall_executed__destroy(skel); +} + +static void test_uprobe_multi(void) +{ + struct uprobe_syscall_executed *skel =3D NULL; + LIBBPF_OPTS(bpf_uprobe_multi_opts, opts); + struct bpf_link *link; + unsigned long offset; + + offset =3D get_uprobe_offset(&uprobe_test); + if (!ASSERT_GE(offset, 0, "get_uprobe_offset")) + goto cleanup; + + opts.offsets =3D &offset; + opts.cnt =3D 1; + + skel =3D uprobe_syscall_executed__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_syscall_executed__open_and_load")) + return; + + /* uprobe.multi */ + link =3D bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_multi, + 0, "/proc/self/exe", NULL, &opts); + if (!ASSERT_OK_PTR(link, "bpf_program__attach_uprobe_multi")) + goto cleanup; + + check(skel, link, uprobe_test, uprobe_test, 2); + + /* uretprobe.multi */ + skel->bss->executed =3D 0; + opts.retprobe =3D true; + link =3D bpf_program__attach_uprobe_multi(skel->progs.test_uretprobe_mult= i, + 0, "/proc/self/exe", NULL, &opts); + if (!ASSERT_OK_PTR(link, "bpf_program__attach_uprobe_multi")) + goto cleanup; + + check(skel, link, uprobe_test, uprobe_test, 2); + +cleanup: + uprobe_syscall_executed__destroy(skel); +} + +static void test_uprobe_session(void) +{ + struct uprobe_syscall_executed *skel =3D NULL; + LIBBPF_OPTS(bpf_uprobe_multi_opts, opts, + .session =3D true, + ); + struct bpf_link *link; + unsigned long offset; + + offset =3D get_uprobe_offset(&uprobe_test); + if (!ASSERT_GE(offset, 0, "get_uprobe_offset")) + goto cleanup; + + opts.offsets =3D &offset; + opts.cnt =3D 1; + + skel =3D uprobe_syscall_executed__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_syscall_executed__open_and_load")) + return; + + link =3D bpf_program__attach_uprobe_multi(skel->progs.test_uprobe_session, + 0, "/proc/self/exe", NULL, &opts); + if (!ASSERT_OK_PTR(link, "bpf_program__attach_uprobe_multi")) + goto cleanup; + + check(skel, link, uprobe_test, uprobe_test, 4); + +cleanup: + uprobe_syscall_executed__destroy(skel); +} + +static void test_uprobe_usdt(void) +{ + struct uprobe_syscall_executed *skel; + struct bpf_link *link; + void *addr; + + errno =3D 0; + addr =3D find_nop5(usdt_test); + if (!ASSERT_OK_PTR(addr, "find_nop5")) + return; + + skel =3D uprobe_syscall_executed__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_syscall_executed__open_and_load")) + return; + + link =3D bpf_program__attach_usdt(skel->progs.test_usdt, + -1 /* all PIDs */, "/proc/self/exe", + "optimized_uprobe", "usdt", NULL); + if (!ASSERT_OK_PTR(link, "bpf_program__attach_usdt")) + goto cleanup; + + check(skel, link, usdt_test, addr, 2); + +cleanup: + uprobe_syscall_executed__destroy(skel); +} + /* * Borrowed from tools/testing/selftests/x86/test_shadow_stack.c. * @@ -343,11 +602,20 @@ static void test_uretprobe_shadow_stack(void) return; } =20 - /* Run all of the uretprobe tests. */ + /* Run all the tests with shadow stack in place. */ + shstk_is_enabled =3D true; + test_uretprobe_regs_equal(); test_uretprobe_regs_change(); test_uretprobe_syscall_call(); =20 + test_uprobe_legacy(); + test_uprobe_multi(); + test_uprobe_session(); + test_uprobe_usdt(); + + shstk_is_enabled =3D false; + ARCH_PRCTL(ARCH_SHSTK_DISABLE, ARCH_SHSTK_SHSTK); } =20 @@ -361,6 +629,14 @@ static void __test_uprobe_syscall(void) test_uretprobe_syscall_call(); if (test__start_subtest("uretprobe_shadow_stack")) test_uretprobe_shadow_stack(); + if (test__start_subtest("uprobe_legacy")) + test_uprobe_legacy(); + if (test__start_subtest("uprobe_multi")) + test_uprobe_multi(); + if (test__start_subtest("uprobe_session")) + test_uprobe_session(); + if (test__start_subtest("uprobe_usdt")) + test_uprobe_usdt(); } #else static void __test_uprobe_syscall(void) diff --git a/tools/testing/selftests/bpf/progs/uprobe_syscall_executed.c b/= tools/testing/selftests/bpf/progs/uprobe_syscall_executed.c index 2e1b689ed4fb..7bb4338c3ee2 100644 --- a/tools/testing/selftests/bpf/progs/uprobe_syscall_executed.c +++ b/tools/testing/selftests/bpf/progs/uprobe_syscall_executed.c @@ -1,6 +1,8 @@ // SPDX-License-Identifier: GPL-2.0 #include "vmlinux.h" #include +#include +#include #include =20 struct pt_regs regs; @@ -9,9 +11,44 @@ char _license[] SEC("license") =3D "GPL"; =20 int executed =3D 0; =20 +SEC("uprobe") +int BPF_UPROBE(test_uprobe) +{ + executed++; + return 0; +} + +SEC("uretprobe") +int BPF_URETPROBE(test_uretprobe) +{ + executed++; + return 0; +} + +SEC("uprobe.multi") +int test_uprobe_multi(struct pt_regs *ctx) +{ + executed++; + return 0; +} + SEC("uretprobe.multi") int test_uretprobe_multi(struct pt_regs *ctx) { executed++; return 0; } + +SEC("uprobe.session") +int test_uprobe_session(struct pt_regs *ctx) +{ + executed++; + return 0; +} + +SEC("usdt") +int test_usdt(struct pt_regs *ctx) +{ + executed++; + return 0; +} --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F2A31D63D3; Mon, 21 Apr 2025 21:47:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272053; cv=none; b=Z9tAMJPufDlcPbLUueqJOWVZS3j4NqIt5l3Oduarnmrzq+BDOYtkHSAFmtzHZWRiEDaubtPSUnOXj3a0woQD7E0iKOg2eqHUcb4GAiaNxprs53XtlIzAmxSVAtqSnhnS43F/PoQDQZlT6CU1O4zWuKLzMI11jwRIISgCuIRtDRg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272053; c=relaxed/simple; bh=aI9whqPW5DtGUUIX0VMcRWnf+t2wx4QcdG0DrI/uDXs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qqbVI5EzCZPFMz3INxxY411mWCGarhg70enDrIwgSZ61Jzje0Qg24NyV/98DWgwuP1u3nl7L3UAy4v/Bi8vdHoAZYamRwi/dD3D/uAsPLZ36QHkexLnplgRRnFNML/rzpJSqk8lyafNiKkq0x4aQlmtxOwYgg+h9B4TYpiPrkMY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=u979u3Wd; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="u979u3Wd" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B1C61C4CEE4; Mon, 21 Apr 2025 21:47:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745272053; bh=aI9whqPW5DtGUUIX0VMcRWnf+t2wx4QcdG0DrI/uDXs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=u979u3WdPag7zDwXfjeoNiYIqVhIYNhB+6laUxSASVVcK4pap7B6BAA8y5w/WVxWT LsTHMTbC/Iez77n+fDo2oceSNi93VP6vNArd3La4UMWo9A3fhjsfO5mBddXLHTZcv+ BS9iJ6zDoRCkVdpGSrw8Xu0eoI8LzNG8ZkzuV8Q15cbdvIt2TJaa9U5aFZ69NDyIVU I+DpRcKU+4p9tGpeuNpqTFVIppl8IuTSbI8ZcGqeiW1KkSCvnctbWGshZtAr4vQKIT gXwkve0/MfGaIj0B9rJs59FucfuzK9e7uXXdR2Nx5409+/rczHLsI6KPHW6Benuv+3 +fg+rdrPVSibA== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 15/22] selftests/bpf: Add hit/attach/detach race optimized uprobe test Date: Mon, 21 Apr 2025 23:44:15 +0200 Message-ID: <20250421214423.393661-16-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding test that makes sure parallel execution of the uprobe and attach/detach of optimized uprobe on it works properly. Signed-off-by: Jiri Olsa --- .../selftests/bpf/prog_tests/uprobe_syscall.c | 74 +++++++++++++++++++ 1 file changed, 74 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c b/tool= s/testing/selftests/bpf/prog_tests/uprobe_syscall.c index 16effe0bca1d..57ef1207c3f5 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c @@ -619,6 +619,78 @@ static void test_uretprobe_shadow_stack(void) ARCH_PRCTL(ARCH_SHSTK_DISABLE, ARCH_SHSTK_SHSTK); } =20 +static volatile bool race_stop; + +static void *worker_trigger(void *arg) +{ + unsigned long rounds =3D 0; + + while (!race_stop) { + uprobe_test(); + rounds++; + } + + printf("tid %d trigger rounds: %lu\n", gettid(), rounds); + return NULL; +} + +static void *worker_attach(void *arg) +{ + struct uprobe_syscall_executed *skel; + unsigned long rounds =3D 0, offset; + + offset =3D get_uprobe_offset(&uprobe_test); + if (!ASSERT_GE(offset, 0, "get_uprobe_offset")) + return NULL; + + skel =3D uprobe_syscall_executed__open_and_load(); + if (!ASSERT_OK_PTR(skel, "uprobe_syscall_executed__open_and_load")) + return NULL; + + while (!race_stop) { + skel->links.test_uprobe =3D bpf_program__attach_uprobe_opts(skel->progs.= test_uprobe, + 0, "/proc/self/exe", offset, NULL); + if (!ASSERT_OK_PTR(skel->links.test_uprobe, "bpf_program__attach_uprobe_= opts")) + break; + + bpf_link__destroy(skel->links.test_uprobe); + skel->links.test_uprobe =3D NULL; + rounds++; + } + + printf("tid %d attach rounds: %lu hits: %d\n", gettid(), rounds, skel->bs= s->executed); + uprobe_syscall_executed__destroy(skel); + return NULL; +} + +static void test_uprobe_race(void) +{ + int err, i, nr_threads; + pthread_t *threads; + + nr_threads =3D libbpf_num_possible_cpus(); + if (!ASSERT_GE(nr_threads, 0, "libbpf_num_possible_cpus")) + return; + + threads =3D malloc(sizeof(*threads) * nr_threads); + if (!ASSERT_OK_PTR(threads, "malloc")) + return; + + for (i =3D 0; i < nr_threads; i++) { + err =3D pthread_create(&threads[i], NULL, i % 2 ? worker_trigger : worke= r_attach, + NULL); + if (!ASSERT_OK(err, "pthread_create")) + goto cleanup; + } + + sleep(4); + +cleanup: + race_stop =3D true; + for (nr_threads =3D i, i =3D 0; i < nr_threads; i++) + pthread_join(threads[i], NULL); +} + static void __test_uprobe_syscall(void) { if (test__start_subtest("uretprobe_regs_equal")) @@ -637,6 +709,8 @@ static void __test_uprobe_syscall(void) test_uprobe_session(); if (test__start_subtest("uprobe_usdt")) test_uprobe_usdt(); + if (test__start_subtest("uprobe_race")) + test_uprobe_race(); } #else static void __test_uprobe_syscall(void) --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9DE921B9FC; Mon, 21 Apr 2025 21:47:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272066; cv=none; b=tDzyumZsDF1802edCzvyGfZnrZ1bbb7EpTvodizXhuwSDppeSugsFUOOpgjAvkSrCBJBNotzR3ooxiSpeF7CPf4406NkDXkwZ10F+opqpP4CuI6YfJiaaCGVE59IvoC0iGKRVpVh1lD48oILlPuTQDHiWQ8ziNTYm4n2RE5Ykjk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272066; c=relaxed/simple; bh=SREC6BOLxkMw16dJMZEBgmN7wOFw0w7YqTHRg+lx7B8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HuSpwZ+ABNVwBuoI+V6+KMxUmsopRQnWeNW2MDrVmYT0O9up46C4yb4mLhdsgO7UZW+cciIbBnAfYAPWMKbSETHkouCfdXl80dfDIvYo2Yfu5wVIdWXWpnh8pkEa+BLIwwjwU/OXiLjc27Df5lVwXvxSuB/R6fYXwmyHQ0JQPjE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RqFFJ7Zl; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RqFFJ7Zl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8E7D0C4CEE4; Mon, 21 Apr 2025 21:47:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745272065; bh=SREC6BOLxkMw16dJMZEBgmN7wOFw0w7YqTHRg+lx7B8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RqFFJ7ZlSWWVKpVAwBRwz3KNuAh4xFdJiRB9GO9GdvF5u8Bap50B7QhaobvTouMQM +6X/nz7OhNTxmsgVcchSItkIdl3J8JJzYiLzT22RSRAPuCtu67STa66d0sRHeO2/rU Nsspe1zW5vOq/OLmZRtc+dQ76bgh09HMswKzQrFF/YpVDYBg/Le7+BEVOM0IzrZ+Ty Kv012+dJp9pKizTOePoCsy3KCy/7gDnRhaqrem3l2/kA901sbH1EkI7abmNRli0vSD ihDawBFnxl8dSm8royf/2Mtb53jVHYzzbvWu6g1ZkCQrzEfoV8gj3onhlRxSRcDUPb Wh4ccspI4m2fg== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 16/22] selftests/bpf: Add uprobe syscall sigill signal test Date: Mon, 21 Apr 2025 23:44:16 +0200 Message-ID: <20250421214423.393661-17-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make sure that calling uprobe syscall from outside uprobe trampoline results in sigill signal. Signed-off-by: Jiri Olsa --- .../selftests/bpf/prog_tests/uprobe_syscall.c | 36 +++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c b/tool= s/testing/selftests/bpf/prog_tests/uprobe_syscall.c index 57ef1207c3f5..f001986981ab 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c @@ -691,6 +691,40 @@ static void test_uprobe_race(void) pthread_join(threads[i], NULL); } =20 +#ifndef __NR_uprobe +#define __NR_uprobe 336 +#endif + +static void test_uprobe_sigill(void) +{ + int status, err, pid; + + pid =3D fork(); + if (!ASSERT_GE(pid, 0, "fork")) + return; + /* child */ + if (pid =3D=3D 0) { + asm volatile ( + "pushq %rax\n" + "pushq %rcx\n" + "pushq %r11\n" + "movq $" __stringify(__NR_uprobe) ", %rax\n" + "syscall\n" + "popq %r11\n" + "popq %rcx\n" + "retq\n" + ); + exit(0); + } + + err =3D waitpid(pid, &status, 0); + ASSERT_EQ(err, pid, "waitpid"); + + /* verify the child got killed with SIGILL */ + ASSERT_EQ(WIFSIGNALED(status), 1, "WIFSIGNALED"); + ASSERT_EQ(WTERMSIG(status), SIGILL, "WTERMSIG"); +} + static void __test_uprobe_syscall(void) { if (test__start_subtest("uretprobe_regs_equal")) @@ -711,6 +745,8 @@ static void __test_uprobe_syscall(void) test_uprobe_usdt(); if (test__start_subtest("uprobe_race")) test_uprobe_race(); + if (test__start_subtest("uprobe_sigill")) + test_uprobe_sigill(); } #else static void __test_uprobe_syscall(void) --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75D0521E0BE; Mon, 21 Apr 2025 21:47:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272077; cv=none; b=dbxiEhNFPD0ezpFDQV6Lr9FBAbC76X9Lqfwax4t/cvsBikRiMjSMbeegnLmpXTNGQ3I3O0Ftln7bqbXLTPeF4119Kb+ds6C3SHpzNzbRGYY+qMOYYO2gQ08VepY6d1WvIe4ObWGBZd1bOUfe99jT2x3D3+z4iPwyxwtwTb9Ugys= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272077; c=relaxed/simple; bh=HTzBLdxwEvxQBtSDIW0CoRoNOhQjNbfqNb3hpAJa9fU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GOHKholDEU98iZCyebkPxjBBprgK6AdzoindhmKzgPOFAX3QzjWaVb4g00yQcwGmUHrjLLi9MuTR4MCsLZ+a+MV9Zpt0OUmFqviV/8LwJR+DN7pLQTq13mV5LEubAympaLyeqlmDoWhS0VZ0MRorA2Pn19BIb3S+oXJHXw89vZQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jc4v+KUa; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jc4v+KUa" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D347AC4CEE4; Mon, 21 Apr 2025 21:47:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745272076; bh=HTzBLdxwEvxQBtSDIW0CoRoNOhQjNbfqNb3hpAJa9fU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jc4v+KUa0x/8wLpfA1d5p4RhU4Xq+IkA9ryJGtXeHS9wru2elBUuYiMcBNf3jYijU pK2gAsMJ5qLGOXz/m44pR74/sbcoPc2ORfT04J4vX8FPfueu0vOhndAlvB+tGdk8/q i0inec7k/yEn+GhYjwrao1Ca4k7Gekqewl5zik9PX1kftgu4HQB1lUQj86TSOo/PMq Gg/SCQyTGJWEb7K5pGrVFlwgrZNcjVJYOm118oVtfmyFj/cA3ibs4OBhMwpL998sBb Xa0o4khYTG1kBlxrMBg0v+JHOT6kOg66KiutTFzO0tHSid6B+n2OeD/9dki/dkQKN7 TXLWwsq8zvFiA== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 17/22] selftests/bpf: Add optimized usdt variant for basic usdt test Date: Mon, 21 Apr 2025 23:44:17 +0200 Message-ID: <20250421214423.393661-18-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding optimized usdt variant for basic usdt test to check that usdt arguments are properly passed in optimized code path. Signed-off-by: Jiri Olsa Acked-by: Andrii Nakryiko --- tools/testing/selftests/bpf/prog_tests/usdt.c | 38 ++++++++++++------- 1 file changed, 25 insertions(+), 13 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/usdt.c b/tools/testing/= selftests/bpf/prog_tests/usdt.c index 495d66414b57..3a5b5230bfa0 100644 --- a/tools/testing/selftests/bpf/prog_tests/usdt.c +++ b/tools/testing/selftests/bpf/prog_tests/usdt.c @@ -40,12 +40,19 @@ static void __always_inline trigger_func(int x) { } } =20 -static void subtest_basic_usdt(void) +static void subtest_basic_usdt(bool optimized) { LIBBPF_OPTS(bpf_usdt_opts, opts); struct test_usdt *skel; struct test_usdt__bss *bss; - int err, i; + int err, i, called; + +#define TRIGGER(x) ({ \ + trigger_func(x); \ + if (optimized) \ + trigger_func(x); \ + optimized ? 2 : 1; \ + }) =20 skel =3D test_usdt__open_and_load(); if (!ASSERT_OK_PTR(skel, "skel_open")) @@ -66,11 +73,11 @@ static void subtest_basic_usdt(void) if (!ASSERT_OK_PTR(skel->links.usdt0, "usdt0_link")) goto cleanup; =20 - trigger_func(1); + called =3D TRIGGER(1); =20 - ASSERT_EQ(bss->usdt0_called, 1, "usdt0_called"); - ASSERT_EQ(bss->usdt3_called, 1, "usdt3_called"); - ASSERT_EQ(bss->usdt12_called, 1, "usdt12_called"); + ASSERT_EQ(bss->usdt0_called, called, "usdt0_called"); + ASSERT_EQ(bss->usdt3_called, called, "usdt3_called"); + ASSERT_EQ(bss->usdt12_called, called, "usdt12_called"); =20 ASSERT_EQ(bss->usdt0_cookie, 0xcafedeadbeeffeed, "usdt0_cookie"); ASSERT_EQ(bss->usdt0_arg_cnt, 0, "usdt0_arg_cnt"); @@ -119,11 +126,11 @@ static void subtest_basic_usdt(void) * bpf_program__attach_usdt() handles this properly and attaches to * all possible places of USDT invocation. */ - trigger_func(2); + called +=3D TRIGGER(2); =20 - ASSERT_EQ(bss->usdt0_called, 2, "usdt0_called"); - ASSERT_EQ(bss->usdt3_called, 2, "usdt3_called"); - ASSERT_EQ(bss->usdt12_called, 2, "usdt12_called"); + ASSERT_EQ(bss->usdt0_called, called, "usdt0_called"); + ASSERT_EQ(bss->usdt3_called, called, "usdt3_called"); + ASSERT_EQ(bss->usdt12_called, called, "usdt12_called"); =20 /* only check values that depend on trigger_func()'s input value */ ASSERT_EQ(bss->usdt3_args[0], 2, "usdt3_arg1"); @@ -142,9 +149,9 @@ static void subtest_basic_usdt(void) if (!ASSERT_OK_PTR(skel->links.usdt3, "usdt3_reattach")) goto cleanup; =20 - trigger_func(3); + called +=3D TRIGGER(3); =20 - ASSERT_EQ(bss->usdt3_called, 3, "usdt3_called"); + ASSERT_EQ(bss->usdt3_called, called, "usdt3_called"); /* this time usdt3 has custom cookie */ ASSERT_EQ(bss->usdt3_cookie, 0xBADC00C51E, "usdt3_cookie"); ASSERT_EQ(bss->usdt3_arg_cnt, 3, "usdt3_arg_cnt"); @@ -158,6 +165,7 @@ static void subtest_basic_usdt(void) =20 cleanup: test_usdt__destroy(skel); +#undef TRIGGER } =20 unsigned short test_usdt_100_semaphore SEC(".probes"); @@ -419,7 +427,11 @@ static void subtest_urandom_usdt(bool auto_attach) void test_usdt(void) { if (test__start_subtest("basic")) - subtest_basic_usdt(); + subtest_basic_usdt(false); +#ifdef __x86_64__ + if (test__start_subtest("basic_optimized")) + subtest_basic_usdt(true); +#endif if (test__start_subtest("multispec")) subtest_multispec_usdt(); if (test__start_subtest("urand_auto_attach")) --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91B0621E0BE; Mon, 21 Apr 2025 21:48:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272089; cv=none; b=ofEWriG0EEx2pTUEm5Emu6/oVMyxGGXeTey6npNXlnZM/sLeYZhnbklwcpFQcV+DM3oM/QqtcD3rn0ENQKuWwazZ/UhloZujwygWftTwVnxkbLaBxktv3OXQIDjhWzB5/2hr6/cK6yXAnoTJ/jQgyDr9qWFVWiFkia2Vh8jkW3c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272089; c=relaxed/simple; bh=w7qqIh17fSwCFSWXvO2PicQUXbzrbsGmg5JeXxLIKAY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rmkfC0cG5+XoGbVnOA9KycRPvmSsCoeudWo/wESXucyI1qUts9I+ufmmIXFZwu1uGlm8ZZO4RkrjHAtj7CKehY1arrDYKzZkRl67FxmKVs0x9Fm9rlLNYU0dQdGDgkWwxiija+VV8FyQNHXvS9aVZhZKiYmMRZq9HsHl2VHLY3Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=s7VECdYj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="s7VECdYj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6B528C4CEE4; Mon, 21 Apr 2025 21:48:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745272089; bh=w7qqIh17fSwCFSWXvO2PicQUXbzrbsGmg5JeXxLIKAY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=s7VECdYj7kWyp3cAmNmvoXHdL+inrEdfGlNpyvwkzirgrLje6ufeILvXRok4+b24p hi7EmaEtFfQMuZBjEJvb9VwxvmLliYjo0epa0HbIUmvXtxJFSNIGEnqPRDWyxtvvX6 iZ15hlQX7ZkBvYzr1WcofbLuKKiL2vAI8uRhqC0kMIq4m9GYqp8nhH+Bys5F54WAx4 0AdEobcpPaUpTbLdC94+UmMUQIhLt8Xmc8FdHoaVTcOP72ngcl5rBos3mFM8Rys8nw 3WoWCOAT0NViSCSoeC2VLt6cS/JNb4dYHN18CqGW/lzJTEeNeYPoIq1qLCNLQHWNl4 OeLF7IBa07W5A== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 18/22] selftests/bpf: Add uprobe_regs_equal test Date: Mon, 21 Apr 2025 23:44:18 +0200 Message-ID: <20250421214423.393661-19-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Changing uretprobe_regs_trigger to allow the test for both uprobe and uretprobe and renaming it to uprobe_regs_equal. We check that both uprobe and uretprobe probes (bpf programs) see expected registers with few exceptions. Signed-off-by: Jiri Olsa --- .../selftests/bpf/prog_tests/uprobe_syscall.c | 58 ++++++++++++++----- .../selftests/bpf/progs/uprobe_syscall.c | 4 +- 2 files changed, 45 insertions(+), 17 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c b/tool= s/testing/selftests/bpf/prog_tests/uprobe_syscall.c index f001986981ab..6d88c5b0f6aa 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c @@ -18,15 +18,17 @@ =20 #pragma GCC diagnostic ignored "-Wattributes" =20 -__naked unsigned long uretprobe_regs_trigger(void) +__attribute__((aligned(16))) +__nocf_check __weak __naked unsigned long uprobe_regs_trigger(void) { asm volatile ( - "movq $0xdeadbeef, %rax\n" + ".byte 0x0f, 0x1f, 0x44, 0x00, 0x00 \n" + "movq $0xdeadbeef, %rax \n" "ret\n" ); } =20 -__naked void uretprobe_regs(struct pt_regs *before, struct pt_regs *after) +__naked void uprobe_regs(struct pt_regs *before, struct pt_regs *after) { asm volatile ( "movq %r15, 0(%rdi)\n" @@ -47,15 +49,17 @@ __naked void uretprobe_regs(struct pt_regs *before, str= uct pt_regs *after) "movq $0, 120(%rdi)\n" /* orig_rax */ "movq $0, 128(%rdi)\n" /* rip */ "movq $0, 136(%rdi)\n" /* cs */ + "pushq %rax\n" "pushf\n" "pop %rax\n" "movq %rax, 144(%rdi)\n" /* eflags */ + "pop %rax\n" "movq %rsp, 152(%rdi)\n" /* rsp */ "movq $0, 160(%rdi)\n" /* ss */ =20 /* save 2nd argument */ "pushq %rsi\n" - "call uretprobe_regs_trigger\n" + "call uprobe_regs_trigger\n" =20 /* save return value and load 2nd argument pointer to rax */ "pushq %rax\n" @@ -95,25 +99,37 @@ __naked void uretprobe_regs(struct pt_regs *before, str= uct pt_regs *after) ); } =20 -static void test_uretprobe_regs_equal(void) +static void test_uprobe_regs_equal(bool retprobe) { + LIBBPF_OPTS(bpf_uprobe_opts, opts, + .retprobe =3D retprobe, + ); struct uprobe_syscall *skel =3D NULL; struct pt_regs before =3D {}, after =3D {}; unsigned long *pb =3D (unsigned long *) &before; unsigned long *pa =3D (unsigned long *) &after; unsigned long *pp; + unsigned long offset; unsigned int i, cnt; - int err; + + offset =3D get_uprobe_offset(&uprobe_regs_trigger); + if (!ASSERT_GE(offset, 0, "get_uprobe_offset")) + return; =20 skel =3D uprobe_syscall__open_and_load(); if (!ASSERT_OK_PTR(skel, "uprobe_syscall__open_and_load")) goto cleanup; =20 - err =3D uprobe_syscall__attach(skel); - if (!ASSERT_OK(err, "uprobe_syscall__attach")) + skel->links.probe =3D bpf_program__attach_uprobe_opts(skel->progs.probe, + 0, "/proc/self/exe", offset, &opts); + if (!ASSERT_OK_PTR(skel->links.probe, "bpf_program__attach_uprobe_opts")) goto cleanup; =20 - uretprobe_regs(&before, &after); + /* make sure uprobe gets optimized */ + if (!retprobe) + uprobe_regs_trigger(); + + uprobe_regs(&before, &after); =20 pp =3D (unsigned long *) &skel->bss->regs; cnt =3D sizeof(before)/sizeof(*pb); @@ -122,7 +138,7 @@ static void test_uretprobe_regs_equal(void) unsigned int offset =3D i * sizeof(unsigned long); =20 /* - * Check register before and after uretprobe_regs_trigger call + * Check register before and after uprobe_regs_trigger call * that triggers the uretprobe. */ switch (offset) { @@ -136,7 +152,7 @@ static void test_uretprobe_regs_equal(void) =20 /* * Check register seen from bpf program and register after - * uretprobe_regs_trigger call + * uprobe_regs_trigger call (with rax exception, check below). */ switch (offset) { /* @@ -149,6 +165,15 @@ static void test_uretprobe_regs_equal(void) case offsetof(struct pt_regs, rsp): case offsetof(struct pt_regs, ss): break; + /* + * uprobe does not see return value in rax, it needs to see the + * original (before) rax value + */ + case offsetof(struct pt_regs, rax): + if (!retprobe) { + ASSERT_EQ(pp[i], pb[i], "uprobe rax prog-before value check"); + break; + } default: if (!ASSERT_EQ(pp[i], pa[i], "register prog-after value check")) fprintf(stdout, "failed register offset %u\n", offset); @@ -186,13 +211,13 @@ static void test_uretprobe_regs_change(void) unsigned long cnt =3D sizeof(before)/sizeof(*pb); unsigned int i, err, offset; =20 - offset =3D get_uprobe_offset(uretprobe_regs_trigger); + offset =3D get_uprobe_offset(uprobe_regs_trigger); =20 err =3D write_bpf_testmod_uprobe(offset); if (!ASSERT_OK(err, "register_uprobe")) return; =20 - uretprobe_regs(&before, &after); + uprobe_regs(&before, &after); =20 err =3D write_bpf_testmod_uprobe(0); if (!ASSERT_OK(err, "unregister_uprobe")) @@ -605,7 +630,8 @@ static void test_uretprobe_shadow_stack(void) /* Run all the tests with shadow stack in place. */ shstk_is_enabled =3D true; =20 - test_uretprobe_regs_equal(); + test_uprobe_regs_equal(false); + test_uprobe_regs_equal(true); test_uretprobe_regs_change(); test_uretprobe_syscall_call(); =20 @@ -728,7 +754,7 @@ static void test_uprobe_sigill(void) static void __test_uprobe_syscall(void) { if (test__start_subtest("uretprobe_regs_equal")) - test_uretprobe_regs_equal(); + test_uprobe_regs_equal(true); if (test__start_subtest("uretprobe_regs_change")) test_uretprobe_regs_change(); if (test__start_subtest("uretprobe_syscall_call")) @@ -747,6 +773,8 @@ static void __test_uprobe_syscall(void) test_uprobe_race(); if (test__start_subtest("uprobe_sigill")) test_uprobe_sigill(); + if (test__start_subtest("uprobe_regs_equal")) + test_uprobe_regs_equal(false); } #else static void __test_uprobe_syscall(void) diff --git a/tools/testing/selftests/bpf/progs/uprobe_syscall.c b/tools/tes= ting/selftests/bpf/progs/uprobe_syscall.c index 8a4fa6c7ef59..e08c31669e5a 100644 --- a/tools/testing/selftests/bpf/progs/uprobe_syscall.c +++ b/tools/testing/selftests/bpf/progs/uprobe_syscall.c @@ -7,8 +7,8 @@ struct pt_regs regs; =20 char _license[] SEC("license") =3D "GPL"; =20 -SEC("uretprobe//proc/self/exe:uretprobe_regs_trigger") -int uretprobe(struct pt_regs *ctx) +SEC("uprobe") +int probe(struct pt_regs *ctx) { __builtin_memcpy(®s, ctx, sizeof(regs)); return 0; --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1432921A447; Mon, 21 Apr 2025 21:48:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272101; cv=none; b=kH7B6PitoM8l2ROY92aO9CtvNPJx7pwW+Qe3podU8EKUXtmMnikNrO8jaef0/3cNZ8QLuJncxWW823rWfz1s/BiEiyWegXsFj+eFCjPxm6GKTNsKpwYPryb+4EmPhkfn4JNyqBF05vZsksw0NPc01MBhk8m0bCWvW+nwrasHfWs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272101; c=relaxed/simple; bh=7eDFb2bAXwqgRpr5YRBO8GLWorMSw54LpDK70BKgOtE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PSPod62VfppkooUOv+WonGT5Gp2TZrPyukV+s7/pbXnLuQLD2T6o/3IRbKhyVfHpus6jbJQK/2Amrwv/2+GvPSkXvO2ySdIvQoHBCgl/sYfsTYvDrlQcR2w8dYXMSgYgVhxAjGySGKWMLLNebEK8nx0277i1K0rb3LaGSaRkOFY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lGcqbKEG; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lGcqbKEG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 670D0C4CEE4; Mon, 21 Apr 2025 21:48:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745272100; bh=7eDFb2bAXwqgRpr5YRBO8GLWorMSw54LpDK70BKgOtE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lGcqbKEGKHShum4cV4RIDj+V8SixsqAxJuVMFNKxGquiyXwTYwdT+r2uTKl8/PC7t gVfBF8BdIU5KbCyXO391IqhtyPeNg5ANlZ0wSu/aBGCB59/wn9d2y6YlyccSI7BIg4 uPwqbaEhoZEJl0wTPqfXMFZ6WHs1kpGJG9hjstiVcm2jhQMOBVpZjxD9VngL1Svvcy AxfWHjH+i/yvRmKi6G9txmqSX6N/94kOZHpJpIrZ+xH30T9oVGm0bUyCzNc7W6zHtr 4dJQSu6pefcaDBRB6cU8xk/P76Jd7p+o65up+iSur5oO6GkmqZRAdOiqtCie8tDGKJ bnb6wQ/WfTrTw== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 19/22] selftests/bpf: Change test_uretprobe_regs_change for uprobe and uretprobe Date: Mon, 21 Apr 2025 23:44:19 +0200 Message-ID: <20250421214423.393661-20-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Changing the test_uretprobe_regs_change test to test both uprobe and uretprobe by adding entry consumer handler to the testmod and making it to change one of the registers. Making sure that changed values both uprobe and uretprobe handlers propagate to the user space. Signed-off-by: Jiri Olsa --- .../selftests/bpf/prog_tests/uprobe_syscall.c | 12 ++++++++---- tools/testing/selftests/bpf/test_kmods/bpf_testmod.c | 11 +++++++++-- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c b/tool= s/testing/selftests/bpf/prog_tests/uprobe_syscall.c index 6d88c5b0f6aa..684f8ab2e7f8 100644 --- a/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c +++ b/tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c @@ -203,7 +203,7 @@ static int write_bpf_testmod_uprobe(unsigned long offse= t) return ret !=3D n ? (int) ret : 0; } =20 -static void test_uretprobe_regs_change(void) +static void test_regs_change(void) { struct pt_regs before =3D {}, after =3D {}; unsigned long *pb =3D (unsigned long *) &before; @@ -217,6 +217,9 @@ static void test_uretprobe_regs_change(void) if (!ASSERT_OK(err, "register_uprobe")) return; =20 + /* make sure uprobe gets optimized */ + uprobe_regs_trigger(); + uprobe_regs(&before, &after); =20 err =3D write_bpf_testmod_uprobe(0); @@ -632,7 +635,6 @@ static void test_uretprobe_shadow_stack(void) =20 test_uprobe_regs_equal(false); test_uprobe_regs_equal(true); - test_uretprobe_regs_change(); test_uretprobe_syscall_call(); =20 test_uprobe_legacy(); @@ -640,6 +642,8 @@ static void test_uretprobe_shadow_stack(void) test_uprobe_session(); test_uprobe_usdt(); =20 + test_regs_change(); + shstk_is_enabled =3D false; =20 ARCH_PRCTL(ARCH_SHSTK_DISABLE, ARCH_SHSTK_SHSTK); @@ -755,8 +759,6 @@ static void __test_uprobe_syscall(void) { if (test__start_subtest("uretprobe_regs_equal")) test_uprobe_regs_equal(true); - if (test__start_subtest("uretprobe_regs_change")) - test_uretprobe_regs_change(); if (test__start_subtest("uretprobe_syscall_call")) test_uretprobe_syscall_call(); if (test__start_subtest("uretprobe_shadow_stack")) @@ -775,6 +777,8 @@ static void __test_uprobe_syscall(void) test_uprobe_sigill(); if (test__start_subtest("uprobe_regs_equal")) test_uprobe_regs_equal(false); + if (test__start_subtest("regs_change")) + test_regs_change(); } #else static void __test_uprobe_syscall(void) diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/t= esting/selftests/bpf/test_kmods/bpf_testmod.c index f38eaf0d35ef..5a3dc463ace5 100644 --- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c @@ -496,15 +496,21 @@ static struct bin_attribute bin_attr_bpf_testmod_file= __ro_after_init =3D { */ #ifdef __x86_64__ =20 +static int +uprobe_handler(struct uprobe_consumer *self, struct pt_regs *regs, __u64 *= data) +{ + regs->cx =3D 0x87654321feebdaed; + return 0; +} + static int uprobe_ret_handler(struct uprobe_consumer *self, unsigned long func, struct pt_regs *regs, __u64 *data) =20 { regs->ax =3D 0x12345678deadbeef; - regs->cx =3D 0x87654321feebdaed; regs->r11 =3D (u64) -1; - return true; + return 0; } =20 struct testmod_uprobe { @@ -516,6 +522,7 @@ struct testmod_uprobe { static DEFINE_MUTEX(testmod_uprobe_mutex); =20 static struct testmod_uprobe uprobe =3D { + .consumer.handler =3D uprobe_handler, .consumer.ret_handler =3D uprobe_ret_handler, }; =20 --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B429521A447; Mon, 21 Apr 2025 21:48:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272113; cv=none; b=Yq8H6Fd3rnnGlqILP3aJ3s/GHMtcshH6b5x5bUs42AFE2NDy0LhhPqNTMkSKQpRCVIBEjd4ECPEdbQHw9+moaiI4m8tz4y/i3X8w+FpuAHBmVARuUcNB9fPSjldQwCSOjA27mD1N6BmwHpdjOfhAQeeF4vWfeeD+/A1rizxDmuo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272113; c=relaxed/simple; bh=WTDVIR1+sFTx9ueUq/qHOJEPSPM0tSDVTCdHHw99xKw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=B3DbrW+kEhDKWOc7VCCCy4/YYCSCcUHHU3PIDAyfFo7snw71Qn1Va2smeFXzEyVWkNYK9HuqSPfIr0kPqUtna7eags0Xa8WyMce11gFY5k7TFs1OM08DSWLewZQLjfK0ink+PDJ6ChJP/bEOjJrzJM0SpB3S3RApaFASVUkeMQI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lizAADCN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lizAADCN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2FF86C4CEE4; Mon, 21 Apr 2025 21:48:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745272113; bh=WTDVIR1+sFTx9ueUq/qHOJEPSPM0tSDVTCdHHw99xKw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lizAADCNR30JhZPRnLaVJhGe31MiWbCb8B72Slt2A5PWzV8PY5tZmq2jwWirgWoIq aF1Vuv6KhOzU8OdCvjcduBwnhHFnXwLTH0AuPknbjrOsDxZ438tnXXXhEdPwiv36ZS Dx8lpTTE4OlricBbbUp0D66ftimMLGwf5yaXZoGgSGMODxLRBPFQo3nhkk0UYTBMHn UQfzWD4WllKMZNao4eNT09pJllx9sXLxCAk4iCU1xgLQoxkWG/4NHiRbW7/r/mg49t hI3a3uGPDt81kI/dWXLDLIYlSZMp84zeKvMUqNRB0Fkp70msPJ3e83mmu0RW3gpVHW v/iMvMAHhT4Wg== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: Kees Cook , Eyal Birger , bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 20/22] seccomp: passthrough uprobe systemcall without filtering Date: Mon, 21 Apr 2025 23:44:20 +0200 Message-ID: <20250421214423.393661-21-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding uprobe as another exception to the seccomp filter alongside with the uretprobe syscall. Same as the uretprobe the uprobe syscall is installed by kernel as replacement for the breakpoint exception and is limited to x86_64 arch and isn't expected to ever be supported in i386. Cc: Kees Cook Cc: Eyal Birger Signed-off-by: Jiri Olsa Reviewed-by: Kees Cook --- kernel/seccomp.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/kernel/seccomp.c b/kernel/seccomp.c index 41aa761c7738..7daf2da09e8e 100644 --- a/kernel/seccomp.c +++ b/kernel/seccomp.c @@ -741,6 +741,26 @@ seccomp_prepare_user_filter(const char __user *user_fi= lter) } =20 #ifdef SECCOMP_ARCH_NATIVE +static bool seccomp_uprobe_exception(struct seccomp_data *sd) +{ +#if defined __NR_uretprobe || defined __NR_uprobe +#ifdef SECCOMP_ARCH_COMPAT + if (sd->arch =3D=3D SECCOMP_ARCH_NATIVE) +#endif + { +#ifdef __NR_uretprobe + if (sd->nr =3D=3D __NR_uretprobe) + return true; +#endif +#ifdef __NR_uprobe + if (sd->nr =3D=3D __NR_uprobe) + return true; +#endif + } +#endif + return false; +} + /** * seccomp_is_const_allow - check if filter is constant allow with given d= ata * @fprog: The BPF programs @@ -758,13 +778,8 @@ static bool seccomp_is_const_allow(struct sock_fprog_k= ern *fprog, return false; =20 /* Our single exception to filtering. */ -#ifdef __NR_uretprobe -#ifdef SECCOMP_ARCH_COMPAT - if (sd->arch =3D=3D SECCOMP_ARCH_NATIVE) -#endif - if (sd->nr =3D=3D __NR_uretprobe) - return true; -#endif + if (seccomp_uprobe_exception(sd)) + return true; =20 for (pc =3D 0; pc < fprog->len; pc++) { struct sock_filter *insn =3D &fprog->filter[pc]; @@ -1042,6 +1057,9 @@ static const int mode1_syscalls[] =3D { __NR_seccomp_read, __NR_seccomp_write, __NR_seccomp_exit, __NR_seccomp_si= greturn, #ifdef __NR_uretprobe __NR_uretprobe, +#endif +#ifdef __NR_uprobe + __NR_uprobe, #endif -1, /* negative terminated */ }; --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2A5819B5B8; Mon, 21 Apr 2025 21:48:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272126; cv=none; b=N04H3bKTa6eJ9RLMn1EyVkkcYVxNiD8ZhID/ItF07V0ZSWXBGz1OVIUFT1e9aS0mXZTGWD+EchSn1sqLyNRDVwpD/oPj5UIv9F2l6J6YiJUAv0wSP9tnQVEDfgPMkc4Tqbh2eyP9OhV5pa3aJsYKVk2wrXSvl7pu3OKAJTz9gKo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272126; c=relaxed/simple; bh=6wEmQ+HcOSae40FunG4oi3QmQm5OmiTydiftVUH95is=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WGWQEvNnd5mPECc4QsJv3ymyrN0lEPj+3ZaXv3pNvzBsFwvIRABaFX0h0ixTUmc+2mnE/0yyvClU9At3CUNkwkjljou+9D6NZj1SmRVXHOSxgm9HP16Hx8eU/4hbxs0oCVDh3os/hS03h3QStK4s0uzTJGe8bp6DFdTyASP1QQw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=J+8YgL1w; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="J+8YgL1w" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A7B84C4CEE4; Mon, 21 Apr 2025 21:48:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745272125; bh=6wEmQ+HcOSae40FunG4oi3QmQm5OmiTydiftVUH95is=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J+8YgL1wbKcrGjQyBPE79198KuPFLTRLHvyaA6czFdMyP8hrpCiUc8TMTJEp+9IiJ VI3X/IKEq8/uTA4xz23xrFtehdnNZDU98C13oUOlJpFfLbdSe9gT/OKCAQgjJX9P3E zRuljop6ydX8I0YPCZoVk0DlHxO6NgzTHrA3iZFyd23UVfZtq8+O1j8jm52seJjDNo aelS1vKbMYbWsi+gWao9ERCamGxBW09EUK3GlMPCySlYfW7KCuIRJhigAPt7U847g3 G06/ZYPP2v6tp2nuH6KWoDqoc6ccs9X43uxDDzid+v5mScz5oH4iQB1f/5Thcl/K5/ Xa/TCCowZ6m7Q== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: Kees Cook , Eyal Birger , bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH perf/core 21/22] selftests/seccomp: validate uprobe syscall passes through seccomp Date: Mon, 21 Apr 2025 23:44:21 +0200 Message-ID: <20250421214423.393661-22-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding uprobe checks into the current uretprobe tests. All the related tests are now executed with attached uprobe or uretprobe or without any probe. Renaming the test fixture to uprobe, because it seems better. Cc: Kees Cook Cc: Eyal Birger Signed-off-by: Jiri Olsa Reviewed-by: Kees Cook --- tools/testing/selftests/seccomp/seccomp_bpf.c | 107 ++++++++++++++---- 1 file changed, 86 insertions(+), 21 deletions(-) diff --git a/tools/testing/selftests/seccomp/seccomp_bpf.c b/tools/testing/= selftests/seccomp/seccomp_bpf.c index b2f76a52215a..d566e40a6028 100644 --- a/tools/testing/selftests/seccomp/seccomp_bpf.c +++ b/tools/testing/selftests/seccomp/seccomp_bpf.c @@ -73,6 +73,14 @@ #define noinline __attribute__((noinline)) #endif =20 +#ifndef __nocf_check +#define __nocf_check __attribute__((nocf_check)) +#endif + +#ifndef __naked +#define __naked __attribute__((__naked__)) +#endif + #ifndef PR_SET_NO_NEW_PRIVS #define PR_SET_NO_NEW_PRIVS 38 #define PR_GET_NO_NEW_PRIVS 39 @@ -4899,7 +4907,36 @@ TEST(tsync_vs_dead_thread_leader) EXPECT_EQ(0, status); } =20 -noinline int probed(void) +#ifdef __x86_64__ + +/* + * We need naked probed_uprobe function. Using __nocf_check + * check to skip possible endbr64 instruction and ignoring + * -Wattributes, otherwise the compilation might fail. + */ +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wattributes" + +__naked __nocf_check noinline int probed_uprobe(void) +{ + /* + * Optimized uprobe is possible only on top of nop5 instruction. + */ + asm volatile (" \n" + ".byte 0x0f, 0x1f, 0x44, 0x00, 0x00 \n" + "ret \n" + ); +} +#pragma GCC diagnostic pop + +#else +noinline int probed_uprobe(void) +{ + return 1; +} +#endif + +noinline int probed_uretprobe(void) { return 1; } @@ -4952,35 +4989,46 @@ static ssize_t get_uprobe_offset(const void *addr) return found ? (uintptr_t)addr - start + base : -1; } =20 -FIXTURE(URETPROBE) { +FIXTURE(UPROBE) { int fd; }; =20 -FIXTURE_VARIANT(URETPROBE) { +FIXTURE_VARIANT(UPROBE) { /* - * All of the URETPROBE behaviors can be tested with either - * uretprobe attached or not + * All of the U(RET)PROBE behaviors can be tested with either + * u(ret)probe attached or not */ bool attach; + /* + * Test both uprobe and uretprobe. + */ + bool uretprobe; }; =20 -FIXTURE_VARIANT_ADD(URETPROBE, attached) { +FIXTURE_VARIANT_ADD(UPROBE, not_attached) { + .attach =3D false, + .uretprobe =3D false, +}; + +FIXTURE_VARIANT_ADD(UPROBE, uprobe_attached) { .attach =3D true, + .uretprobe =3D false, }; =20 -FIXTURE_VARIANT_ADD(URETPROBE, not_attached) { - .attach =3D false, +FIXTURE_VARIANT_ADD(UPROBE, uretprobe_attached) { + .attach =3D true, + .uretprobe =3D true, }; =20 -FIXTURE_SETUP(URETPROBE) +FIXTURE_SETUP(UPROBE) { const size_t attr_sz =3D sizeof(struct perf_event_attr); struct perf_event_attr attr; ssize_t offset; int type, bit; =20 -#ifndef __NR_uretprobe - SKIP(return, "__NR_uretprobe syscall not defined"); +#if !defined(__NR_uprobe) || !defined(__NR_uretprobe) + SKIP(return, "__NR_uprobe ot __NR_uretprobe syscalls not defined"); #endif =20 if (!variant->attach) @@ -4990,12 +5038,17 @@ FIXTURE_SETUP(URETPROBE) =20 type =3D determine_uprobe_perf_type(); ASSERT_GE(type, 0); - bit =3D determine_uprobe_retprobe_bit(); - ASSERT_GE(bit, 0); - offset =3D get_uprobe_offset(probed); + + if (variant->uretprobe) { + bit =3D determine_uprobe_retprobe_bit(); + ASSERT_GE(bit, 0); + } + + offset =3D get_uprobe_offset(variant->uretprobe ? probed_uretprobe : prob= ed_uprobe); ASSERT_GE(offset, 0); =20 - attr.config |=3D 1 << bit; + if (variant->uretprobe) + attr.config |=3D 1 << bit; attr.size =3D attr_sz; attr.type =3D type; attr.config1 =3D ptr_to_u64("/proc/self/exe"); @@ -5006,7 +5059,7 @@ FIXTURE_SETUP(URETPROBE) PERF_FLAG_FD_CLOEXEC); } =20 -FIXTURE_TEARDOWN(URETPROBE) +FIXTURE_TEARDOWN(UPROBE) { /* we could call close(self->fd), but we'd need extra filter for * that and since we are calling _exit right away.. @@ -5020,11 +5073,17 @@ static int run_probed_with_filter(struct sock_fprog= *prog) return -1; } =20 - probed(); + /* + * Uprobe is optimized after first hit, so let's hit twice. + */ + probed_uprobe(); + probed_uprobe(); + + probed_uretprobe(); return 0; } =20 -TEST_F(URETPROBE, uretprobe_default_allow) +TEST_F(UPROBE, uprobe_default_allow) { struct sock_filter filter[] =3D { BPF_STMT(BPF_RET|BPF_K, SECCOMP_RET_ALLOW), @@ -5037,7 +5096,7 @@ TEST_F(URETPROBE, uretprobe_default_allow) ASSERT_EQ(0, run_probed_with_filter(&prog)); } =20 -TEST_F(URETPROBE, uretprobe_default_block) +TEST_F(UPROBE, uprobe_default_block) { struct sock_filter filter[] =3D { BPF_STMT(BPF_LD|BPF_W|BPF_ABS, @@ -5054,11 +5113,14 @@ TEST_F(URETPROBE, uretprobe_default_block) ASSERT_EQ(0, run_probed_with_filter(&prog)); } =20 -TEST_F(URETPROBE, uretprobe_block_uretprobe_syscall) +TEST_F(UPROBE, uprobe_block_syscall) { struct sock_filter filter[] =3D { BPF_STMT(BPF_LD|BPF_W|BPF_ABS, offsetof(struct seccomp_data, nr)), +#ifdef __NR_uprobe + BPF_JUMP(BPF_JMP|BPF_JEQ|BPF_K, __NR_uprobe, 1, 2), +#endif #ifdef __NR_uretprobe BPF_JUMP(BPF_JMP|BPF_JEQ|BPF_K, __NR_uretprobe, 0, 1), #endif @@ -5073,11 +5135,14 @@ TEST_F(URETPROBE, uretprobe_block_uretprobe_syscall) ASSERT_EQ(0, run_probed_with_filter(&prog)); } =20 -TEST_F(URETPROBE, uretprobe_default_block_with_uretprobe_syscall) +TEST_F(UPROBE, uprobe_default_block_with_syscall) { struct sock_filter filter[] =3D { BPF_STMT(BPF_LD|BPF_W|BPF_ABS, offsetof(struct seccomp_data, nr)), +#ifdef __NR_uprobe + BPF_JUMP(BPF_JMP|BPF_JEQ|BPF_K, __NR_uprobe, 3, 0), +#endif #ifdef __NR_uretprobe BPF_JUMP(BPF_JMP|BPF_JEQ|BPF_K, __NR_uretprobe, 2, 0), #endif --=20 2.49.0 From nobody Tue Dec 16 00:22:13 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47F3421ADD3; Mon, 21 Apr 2025 21:48:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272137; cv=none; b=KANixgSeK8sozZAaZBZKZwO17CEUxXlJ5WenSZENzIcVz0qQu7jz0tRt+carbRe5NUV42KvbVTM0VCHkM3JYAudvPNa0uUUNTBGIFvHyVtH8lCEcm7ZeKdKw2U7jRzCdwhM2b116JS0C0lSQNJYVWKvPLyfRzE1FxWO7XsbwvbQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745272137; c=relaxed/simple; bh=Hm2/jQHfF1PjQvlOfp3fNR5vNrQhuRjESWnF+NZsiho=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OPcwfpY8SjOj0ITLJm569mPYY9/DrSZ+K6RemJTvp1rzTnApKZvmiHS7oj89zIpfNXLNZImXZbRHl5u3UTDCqk9KeP18b6rHb2hNRcjOnJSwgJ2ruzrXvbQ1sgNcsItEGv8zlvjO+Oi67+fFqs75t1o+UtwtEw1nNuY4q+EDpF8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=KlscQzeQ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="KlscQzeQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A283BC4CEE4; Mon, 21 Apr 2025 21:48:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1745272137; bh=Hm2/jQHfF1PjQvlOfp3fNR5vNrQhuRjESWnF+NZsiho=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KlscQzeQu3ySIJ/BL6x7+31hh4A6und8d8ZzIFN8cei/Nzqq60CsfOwoWnTwRsAWs nJbywcFIrNaoyAFAUJREk0gfMG4lblS2nO2LOezRkfXXkS9GXWkf/BSDs0RZ2CdwNK Xcnrv7ohxZ3CmPMIydNJLarzs76W3Z8CHO2vH/8x+GbyeICfe/YO43RrrhGcYBHTa8 5DVOEiE2JgRPQGBU9TY3eLQ14pz3eNEeMpJaA2MtX5PtChMowDUeK2LL5FGuZuvPUH /EGC6jOhizi57k2BMCOXHVL5rJN1EBT/Cqs+VvCgBs31nevTfjR+lKaER61mMrmXLd 1vawBDiKEWf9A== From: Jiri Olsa To: Oleg Nesterov , Peter Zijlstra , Andrii Nakryiko Cc: Alejandro Colomar , bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, x86@kernel.org, Song Liu , Yonghong Song , John Fastabend , Hao Luo , Steven Rostedt , Masami Hiramatsu , Alan Maguire , David Laight , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , Ingo Molnar Subject: [PATCH 22/22] man2: Add uprobe syscall page Date: Mon, 21 Apr 2025 23:44:22 +0200 Message-ID: <20250421214423.393661-23-jolsa@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250421214423.393661-1-jolsa@kernel.org> References: <20250421214423.393661-1-jolsa@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding man page for new uprobe syscall. Cc: Alejandro Colomar Signed-off-by: Jiri Olsa --- man/man2/uprobe.2 | 49 ++++++++++++++++++++++++++++++++++++++++++++ man/man2/uretprobe.2 | 2 ++ 2 files changed, 51 insertions(+) create mode 100644 man/man2/uprobe.2 diff --git a/man/man2/uprobe.2 b/man/man2/uprobe.2 new file mode 100644 index 000000000000..2b01a5ab5f3e --- /dev/null +++ b/man/man2/uprobe.2 @@ -0,0 +1,49 @@ +.\" Copyright (C) 2024, Jiri Olsa +.\" +.\" SPDX-License-Identifier: Linux-man-pages-copyleft +.\" +.TH uprobe 2 (date) "Linux man-pages (unreleased)" +.SH NAME +uprobe +\- +execute pending entry uprobes +.SH SYNOPSIS +.nf +.B int uprobe(void); +.fi +.SH DESCRIPTION +.BR uprobe () +is an alternative to breakpoint instructions +for triggering entry uprobe consumers. +.P +Calls to +.BR uprobe () +are only made from the user-space trampoline provided by the kernel. +Calls from any other place result in a +.BR SIGILL . +.SH RETURN VALUE +The return value is architecture-specific. +.SH ERRORS +.TP +.B SIGILL +.BR uprobe () +was called by a user-space program. +.SH VERSIONS +The behavior varies across systems. +.SH STANDARDS +None. +.SH HISTORY +TBD +.P +.BR uprobe () +was initially introduced for the x86_64 architecture +where it was shown to be faster than breakpoint traps. +It might be extended to other architectures. +.SH CAVEATS +.BR uprobe () +exists only to allow the invocation of entry uprobe consumers. +It should +.B never +be called directly. +.SH SEE ALSO +.BR uretprobe (2) diff --git a/man/man2/uretprobe.2 b/man/man2/uretprobe.2 index bbbfb0c59335..bb8bf4e32e5d 100644 --- a/man/man2/uretprobe.2 +++ b/man/man2/uretprobe.2 @@ -45,3 +45,5 @@ exists only to allow the invocation of return uprobe cons= umers. It should .B never be called directly. +.SH SEE ALSO +.BR uprobe (2) --=20 2.49.0