From nobody Thu Nov 28 17:34:22 2024 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1EF117ADE1 for ; Sun, 29 Sep 2024 14:42:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620973; cv=none; b=HGHC8yTSTuDWAnGPcmt9OQussKmqS19EU1hKkH8kGEefRmauS5X5Qk0oSm6ttgq/WXhq13WnnQO2brq3yEykjuAHrioKLwm8SSq/IIyfmxxORppGQ3K42/V57I3t0vSZ68s528ZuhAd2knYvxxmlUjZ3ouNc1tochMmfIYoFPag= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620973; c=relaxed/simple; bh=6WTZso5WGhvtb2/OxjllcDfJ4f6hMAM6NKTHQqMFnIQ=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=Ky97EZV5PMWtiyJjczjJmRBpZWTBjfvyQ1y1q6t+H8QoGvNpelwss41adq+mHIR4l/5VE8cEWz8vHDmb7hyOs7o9RHX26Rts9A6GkKz+Slqr8HAcPusz6YH9I3FYqae73XC3New2exo+xUqeAAVbWxy8+/JIDEsotlt2Wbp5VfI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=IH8kdwcO; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IH8kdwcO" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727620970; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=UjMkCez+I6XaIxZPG+o4NZ0Bsl2LfAkraC0d9Mrst+o=; b=IH8kdwcOpkZWw84ebQ19dbXehgBPuh/BUGMgFSC2Ijuj+eZIy877bMg+Cr6QEuhj8OulWh VM0fvV7v+oePjXsqzhpgRi2hVukSIqZbQXK29dt7xk+KjHdKU+gJ0amWsfiuzsp3ECbGWC Scm1qHzw4m1+HDCskVikMPKyfzSvdAo= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-158-doMx__8qNg-wZ2uUUCItBQ-1; Sun, 29 Sep 2024 10:42:48 -0400 X-MC-Unique: doMx__8qNg-wZ2uUUCItBQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 82CE018E6A6C; Sun, 29 Sep 2024 14:42:47 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id B8DC01944B22; Sun, 29 Sep 2024 14:42:44 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:42:34 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:30 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 1/7] uprobes: don't abuse get_utask() in pre_ssout() and prepare_uretprobe() Message-ID: <20240929144230.GA9468@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" handle_swbp() calls get_utask() before prepare_uretprobe() or pre_ssout() can be called, they can simply use current->utask which can't be NULL. Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 4b52cb2ae6d6..2a9cdd5c82d7 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1908,18 +1908,14 @@ static void cleanup_return_instances(struct uprobe_= task *utask, bool chained, =20 static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) { - struct return_instance *ri; - struct uprobe_task *utask; + struct uprobe_task *utask =3D current->utask; unsigned long orig_ret_vaddr, trampoline_vaddr; + struct return_instance *ri; bool chained; =20 if (!get_xol_area()) return; =20 - utask =3D get_utask(); - if (!utask) - return; - if (utask->depth >=3D MAX_URETPROBE_DEPTH) { printk_ratelimited(KERN_INFO "uprobe: omit uretprobe due to" " nestedness limit pid/tgid=3D%d/%d\n", @@ -1980,14 +1976,10 @@ static void prepare_uretprobe(struct uprobe *uprobe= , struct pt_regs *regs) static int pre_ssout(struct uprobe *uprobe, struct pt_regs *regs, unsigned long bp_va= ddr) { - struct uprobe_task *utask; + struct uprobe_task *utask =3D current->utask; unsigned long xol_vaddr; int err; =20 - utask =3D get_utask(); - if (!utask) - return -ENOMEM; - if (!try_get_uprobe(uprobe)) return -EINVAL; =20 --=20 2.25.1.362.g51ebf55 From nobody Thu Nov 28 17:34:22 2024 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D99B6174EFA for ; Sun, 29 Sep 2024 14:42:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620980; cv=none; b=bquOPbiXpjXiJUDK9cbijY5KqexRbhBx7om856/WBiS91V/crc7hJBNZJKAcLc3TkNwzhlFTpOoCh7J/ta8Muws/JBKXu0VJwFXbWMG+mdXjgdDHpQvrB8aPCTnYplV1ludsdzciwXQkBq8izbds268kwLv2TGQAYp9njoWXe50= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620980; c=relaxed/simple; bh=SW0a/2y9x5XuxLHYhEeUJoJubePyKUh+UH4ExDXUbI8=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=sNHiRhNPVKcAeoFwuTMP30VofADessurhTzXeS01isgJegfB8BJzRXbO35S17dBqYnybYvCAC53icqNKOEUrdb7oF2EVa3ZBOSZuHy3G4hhsUk16qesYIejCXdZy0oo89MxJpBq9FPHQ/+MywPCm/0CrIxkcxvOIGM1I0/IfoPI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=MCc6qGFq; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MCc6qGFq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727620978; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=+dC6z3bA6xQ8q0W/xhGSjQmv+2uYs7hECJrc8EdB/mM=; b=MCc6qGFqqUpHMx9uyqL7mD6GsjBtpaO1ABs6O2AKjeReIaxnkdZ3At5u7elUuKB1uP7Zsy XRS3wYSU8t7LVH09ZEq/JeI1FDhXd8CN/jFRn0aqWVRbuYqSIiwxIkBBWiNBDZFe162YdW GsapHiyjkqKQaw4bJGhUvbNYH7u4h+w= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-219-mQEtUGoCOaCeMjArfJysMQ-1; Sun, 29 Sep 2024 10:42:54 -0400 X-MC-Unique: mQEtUGoCOaCeMjArfJysMQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 206071919156; Sun, 29 Sep 2024 14:42:52 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id 6E3E41979060; Sun, 29 Sep 2024 14:42:49 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:42:38 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:35 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 2/7] uprobes: sanitiize xol_free_insn_slot() Message-ID: <20240929144235.GA9471@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" 1. Clear utask->xol_vaddr unconditionally, even if this addr is not valid, xol_free_insn_slot() should never return with utask->xol_vaddr !=3D NULL. 2. Add a comment to explain why do we need to validate slot_addr. 3. Simplify the validation above. We can simply check offset < PAGE_SIZE, unsigned underflows are fine, it should work if slot_addr < area->vaddr. 4. Kill the unnecessary "slot_nr >=3D UINSNS_PER_PAGE" check, slot_nr must be valid if offset < PAGE_SIZE. The next patches will cleanup this function even more. Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 2a9cdd5c82d7..3023714b83f2 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1683,8 +1683,8 @@ static unsigned long xol_get_insn_slot(struct uprobe = *uprobe) static void xol_free_insn_slot(struct task_struct *tsk) { struct xol_area *area; - unsigned long vma_end; unsigned long slot_addr; + unsigned long offset; =20 if (!tsk->mm || !tsk->mm->uprobes_state.xol_area || !tsk->utask) return; @@ -1693,24 +1693,21 @@ static void xol_free_insn_slot(struct task_struct *= tsk) if (unlikely(!slot_addr)) return; =20 + tsk->utask->xol_vaddr =3D 0; area =3D tsk->mm->uprobes_state.xol_area; - vma_end =3D area->vaddr + PAGE_SIZE; - if (area->vaddr <=3D slot_addr && slot_addr < vma_end) { - unsigned long offset; - int slot_nr; - - offset =3D slot_addr - area->vaddr; - slot_nr =3D offset / UPROBE_XOL_SLOT_BYTES; - if (slot_nr >=3D UINSNS_PER_PAGE) - return; + offset =3D slot_addr - area->vaddr; + /* + * slot_addr must fit into [area->vaddr, area->vaddr + PAGE_SIZE). + * This check can only fail if the "[uprobes]" vma was mremap'ed. + */ + if (offset < PAGE_SIZE) { + int slot_nr =3D offset / UPROBE_XOL_SLOT_BYTES; =20 clear_bit(slot_nr, area->bitmap); atomic_dec(&area->slot_count); smp_mb__after_atomic(); /* pairs with prepare_to_wait() */ if (waitqueue_active(&area->wq)) wake_up(&area->wq); - - tsk->utask->xol_vaddr =3D 0; } } =20 --=20 2.25.1.362.g51ebf55 From nobody Thu Nov 28 17:34:22 2024 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F93E17D354 for ; Sun, 29 Sep 2024 14:43:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620985; cv=none; b=HN+gATYtfOw47zOCfEoF94N5l3a8SwyTju1Fi8JQPMMvVOGZbGucbinFnX+V76r28DaPi7Lxke2nxhtOeztxohOcD2IgsQvezsAzfjBuRWmCV18Y5lSfrFPvwb9JC04tbMHXAAkpa68EEllWyNz6Db0j7s8eIZwOnr3oKGMI0aY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620985; c=relaxed/simple; bh=r9g2mDKo1VyZh/uQk94rwCQODx7VEt13cnsjwkriLJY=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=XzXIKbrcGHIhhSE2RCFATZI1+O+n1XTynmb2dxTl7VXGaevqGPg1nIJYx13mAM8Uv6u6Pvx8YNzU0f0fR2tanKH69HoJPz8gZFDVytWmb7eZ2DFHg+YDTLH3FmSBWLaTBm4JEm1oLFfA/bdcTTMHiJ5pkKbWIh9np+l0CEhfKsY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AMgAssKR; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AMgAssKR" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727620981; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=mEl9ds8cbxbLGs6JnmKeAQsfkw4QJ2bnOpSp9pzGta8=; b=AMgAssKRdhMznu4P+I1AFafYG15ofOaz41szQaD6HV4QqUgAqoLOWNRk841qvicIM1baBf wLBT4DDUhgY+gje97BY41zJYt1R2WFKQ+Yba8LdHscyk04jDe7iDxn3p6wakzYpjDOz3+A FeygGk7KLkvug/4DdKuKd6CpUYwWGSI= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-647-Yhzs64IxNfqpMlvjWoTwfw-1; Sun, 29 Sep 2024 10:42:58 -0400 X-MC-Unique: Yhzs64IxNfqpMlvjWoTwfw-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B460119B9ABA; Sun, 29 Sep 2024 14:42:56 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id 0ED1B1944B22; Sun, 29 Sep 2024 14:42:53 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:42:43 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:39 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 3/7] uprobes: kill the unnecessary put_uprobe/xol_free_insn_slot in uprobe_free_utask() Message-ID: <20240929144239.GA9475@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" If pre_ssout() succeeds and sets utask->active_uprobe and utask->xol_vaddr the task must not exit until it calls handle_singlestep() which does the necessary put_uprobe() and xol_free_insn_slot(). Remove put_uprobe() and xol_free_insn_slot() from uprobe_free_utask(). With this change xol_free_insn_slot() can't hit xol_area/utask/xol_vaddr =3D=3D = NULL, we can kill the unnecessary checks checks and simplify this function more. Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 24 +++++------------------- 1 file changed, 5 insertions(+), 19 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 3023714b83f2..4619de10772e 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1676,28 +1676,16 @@ static unsigned long xol_get_insn_slot(struct uprob= e *uprobe) } =20 /* - * xol_free_insn_slot - If slot was earlier allocated by - * @xol_get_insn_slot(), make the slot available for - * subsequent requests. + * xol_free_insn_slot - free the slot allocated by xol_get_insn_slot() */ static void xol_free_insn_slot(struct task_struct *tsk) { - struct xol_area *area; - unsigned long slot_addr; - unsigned long offset; - - if (!tsk->mm || !tsk->mm->uprobes_state.xol_area || !tsk->utask) - return; - - slot_addr =3D tsk->utask->xol_vaddr; - if (unlikely(!slot_addr)) - return; + struct xol_area *area =3D tsk->mm->uprobes_state.xol_area; + unsigned long offset =3D tsk->utask->xol_vaddr - area->vaddr; =20 tsk->utask->xol_vaddr =3D 0; - area =3D tsk->mm->uprobes_state.xol_area; - offset =3D slot_addr - area->vaddr; /* - * slot_addr must fit into [area->vaddr, area->vaddr + PAGE_SIZE). + * xol_vaddr must fit into [area->vaddr, area->vaddr + PAGE_SIZE). * This check can only fail if the "[uprobes]" vma was mremap'ed. */ if (offset < PAGE_SIZE) { @@ -1767,14 +1755,12 @@ void uprobe_free_utask(struct task_struct *t) if (!utask) return; =20 - if (utask->active_uprobe) - put_uprobe(utask->active_uprobe); + WARN_ON_ONCE(utask->active_uprobe || utask->xol_vaddr); =20 ri =3D utask->return_instances; while (ri) ri =3D free_ret_instance(ri); =20 - xol_free_insn_slot(t); kfree(utask); t->utask =3D NULL; } --=20 2.25.1.362.g51ebf55 From nobody Thu Nov 28 17:34:22 2024 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7567D17C992 for ; Sun, 29 Sep 2024 14:43:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620991; cv=none; b=QqSd3FIrgJBso/TD5VQ0DdWr6/1/s+LfhGVWDOsYCnM6oVg/ALb2DLk1lBtxVi3YmoRAT2edmPMkyZXDL2lR59/J4l2RoHsTDmM/nCbNF/fQmV9DTwtAHanUt4OWAmmVDKWdZzZB52K6HMsR6Lx83++r8lwSBO9gPRs3/EdgPQ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620991; c=relaxed/simple; bh=Qx3crvMAj4oJepAs7rbIw8d48jGe7BvkfQ64tpGkwjk=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=L0duagqx+9zFue/fEkF9yynnrSp0BQn5+3nnxl4JjhtqaKh0isSMkmkQ/Orbqa0XW2eitqtCQOchCSBDfMA2m4MrY8ylAxmeD3MFEsLxirJdeIP8V6fEB7nkw1wmP4YEHGTJ8+v4sk+QRDnS4nftkb8AhTFHHTiLsjD1rLUgbUQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BBmxcXk/; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BBmxcXk/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727620989; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=qbjSc/cnZvendjL1YV2TjPPnNKKr87Pa0C3JPTQomg0=; b=BBmxcXk/opf4jFuQf/d+PigU+NN3LxlycuVm3020FgaWdOXCPjljUFbOt4rD+mxHe5ndns 9FPwSNtQOO8Xe5yDShWDW5u2G1WRmrzSojZnoiIA0rkyHjd8ehbk+RDF0RdQaNgfBVd0hw /Iw4DGMwB5tOWLyFo+2l3qazTg0hGV4= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-556-J86Hbp8iMtqg-Jva_QujWQ-1; Sun, 29 Sep 2024 10:43:02 -0400 X-MC-Unique: J86Hbp8iMtqg-Jva_QujWQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3FC73190ECE8; Sun, 29 Sep 2024 14:43:01 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id 8A1D01979060; Sun, 29 Sep 2024 14:42:58 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:42:47 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:44 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 4/7] uprobes: simplify xol_take_insn_slot() and its caller Message-ID: <20240929144244.GA9480@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The do / while (slot_nr >=3D UINSNS_PER_PAGE) loop in xol_take_insn_slot() makes no sense, the checked condition is always true. Change this code to use the "for (;;)" loop, this way we do not need to change slot_nr if test_and_set_bit() fails. Also, kill the unnecessary xol_vaddr !=3D NULL check in xol_get_insn_slot(), xol_take_insn_slot() never returns NULL. Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index 4619de10772e..bfe106ecad38 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1631,25 +1631,20 @@ void uprobe_dup_mmap(struct mm_struct *oldmm, struc= t mm_struct *newmm) */ static unsigned long xol_take_insn_slot(struct xol_area *area) { - unsigned long slot_addr; - int slot_nr; + unsigned int slot_nr; =20 - do { + for (;;) { slot_nr =3D find_first_zero_bit(area->bitmap, UINSNS_PER_PAGE); if (slot_nr < UINSNS_PER_PAGE) { if (!test_and_set_bit(slot_nr, area->bitmap)) break; - - slot_nr =3D UINSNS_PER_PAGE; continue; } wait_event(area->wq, (atomic_read(&area->slot_count) < UINSNS_PER_PAGE)); - } while (slot_nr >=3D UINSNS_PER_PAGE); + } =20 - slot_addr =3D area->vaddr + (slot_nr * UPROBE_XOL_SLOT_BYTES); atomic_inc(&area->slot_count); - - return slot_addr; + return area->vaddr + slot_nr * UPROBE_XOL_SLOT_BYTES; } =20 /* @@ -1666,12 +1661,8 @@ static unsigned long xol_get_insn_slot(struct uprobe= *uprobe) return 0; =20 xol_vaddr =3D xol_take_insn_slot(area); - if (unlikely(!xol_vaddr)) - return 0; - arch_uprobe_copy_ixol(area->page, xol_vaddr, &uprobe->arch.ixol, sizeof(uprobe->arch.ixol)); - return xol_vaddr; } =20 --=20 2.25.1.362.g51ebf55 From nobody Thu Nov 28 17:34:22 2024 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 246B317C7C6 for ; Sun, 29 Sep 2024 14:43:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620993; cv=none; b=uE7KtVDhM1av9HoPsG4PHOMtL/17VygqFcw3DsEgWQC8cuBY4WcdzoNhmIMANwo5/qZfUsChX6Z9J+i5DpYShGbCRxfLRqha9VwwTWz9VbDOFiM8e2yWgc71Kbw/ztXJCI+4hR15QBcpFozR3/iKZlx51Go0HaL3cn3msKLq0fg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620993; c=relaxed/simple; bh=4yqWNnyhvMBa++hA/O2fKRiM/JnaiH2w9lyssaDVduU=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=H5D7xZUlT7IldEV7tlzlN5wc5DQYe8uhRaNm2Z2Du6XK5bVjsO/VnnSBLlD7VRXs/PtXmyZR4dk0fJ7bHVglk1XAnovsH9Cz9pAB/egUR6MKobSw8boQoHPrXErNW8D1r6aNLRmIdNqpWzywpLgGZacA6Yo0FoVyceWerYDvS+o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AwtUS/Fb; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AwtUS/Fb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727620991; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=1HZRMRvEFHJMDMiUg5OPpXavNPgBWInozzwyVSKhvk4=; b=AwtUS/FbZU6if3usuQd80S6ZFEVd5WPCl4vxXllJ2+24TdK031ACI5s0F5eMBMaUIgNZ4b /zmK15Q8wqdvMZmv4i7tMbVOuekrvxMycMOqnTEb1mdURw0UjBau7WMiPIpruhkth1aXKc FKe7Ek2ZNVwzVt6n4KisIxaf/BsB15A= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-640-b_c1T7tlOgqfbMOZEy7KrQ-1; Sun, 29 Sep 2024 10:43:07 -0400 X-MC-Unique: b_c1T7tlOgqfbMOZEy7KrQ-1 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D218E19030A7; Sun, 29 Sep 2024 14:43:05 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id 2ACA519560AE; Sun, 29 Sep 2024 14:43:02 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:42:52 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:48 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 5/7] uprobes: move the initialization of utask->xol_vaddr from pre_ssout() to xol_get_insn_slot() Message-ID: <20240929144248.GA9483@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This simplifies the code and makes xol_get_insn_slot() symmetric with xol_free_insn_slot() which clears utask->xol_vaddr. Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 22 ++++++++-------------- 1 file changed, 8 insertions(+), 14 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index bfe106ecad38..a7223be5ac2e 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1649,21 +1649,19 @@ static unsigned long xol_take_insn_slot(struct xol_= area *area) =20 /* * xol_get_insn_slot - allocate a slot for xol. - * Returns the allocated slot address or 0. */ -static unsigned long xol_get_insn_slot(struct uprobe *uprobe) +static bool xol_get_insn_slot(struct uprobe *uprobe) { - struct xol_area *area; - unsigned long xol_vaddr; + struct uprobe_task *utask =3D current->utask; + struct xol_area *area =3D get_xol_area(); =20 - area =3D get_xol_area(); if (!area) - return 0; + return false; =20 - xol_vaddr =3D xol_take_insn_slot(area); - arch_uprobe_copy_ixol(area->page, xol_vaddr, + utask->xol_vaddr =3D xol_take_insn_slot(area); + arch_uprobe_copy_ixol(area->page, utask->xol_vaddr, &uprobe->arch.ixol, sizeof(uprobe->arch.ixol)); - return xol_vaddr; + return true; } =20 /* @@ -1951,21 +1949,17 @@ static int pre_ssout(struct uprobe *uprobe, struct pt_regs *regs, unsigned long bp_va= ddr) { struct uprobe_task *utask =3D current->utask; - unsigned long xol_vaddr; int err; =20 if (!try_get_uprobe(uprobe)) return -EINVAL; =20 - xol_vaddr =3D xol_get_insn_slot(uprobe); - if (!xol_vaddr) { + if (!xol_get_insn_slot(uprobe)) { err =3D -ENOMEM; goto err_out; } =20 - utask->xol_vaddr =3D xol_vaddr; utask->vaddr =3D bp_vaddr; - err =3D arch_uprobe_pre_xol(&uprobe->arch, regs); if (unlikely(err)) { xol_free_insn_slot(current); --=20 2.25.1.362.g51ebf55 From nobody Thu Nov 28 17:34:22 2024 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B751F183098 for ; Sun, 29 Sep 2024 14:43:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620998; cv=none; b=nW8ijxR7NebKBOI4RwGqKhXmYwlK/vOkxUXfDZoPdTel9bppyOT12fFlrJVw3K+49pOxUYUCNjkzVP/CJ+xoVBa728my3iDvizpOe+ptNyyGVNJlWHmx5KF1lo+l8dcks4xxvQBSptB92MONcUu3ke4mG27juMVj8fis1/WwaWc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727620998; c=relaxed/simple; bh=mnE4trvkE9/VgExdbnRnWJG5l/jEO8ThaYtZ1F8rCuk=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=jE+tTaK0vIVhE+5p5+FOmliXrYAReNzE+Dc1MLZm0cjopq4JdIFyGlHV3i5/Rvs9fn/Q3KAftCokEet+3FcqDAlg44o3uSZbpmH7QR7aA/o88vapRcOXD0kEdE0psw8pGwKF5f7SZ6cK6tL2If28kMBww52h0eaJmmlHlkRtWhU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=C7Fdbxnb; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="C7Fdbxnb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727620995; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=sQ1CTBKnRWK4uC6arvzLMaDJzn3AO5JvwnwPCBgfSjU=; b=C7FdbxnbFRa9aY3+3spPSqdilvMynsnTm1cml9NvEhK841q1ydf2pdJZ4+VeVsga3e6DYz +uOBQetSSImndxhRWyD8y+RT015lE92HnXddJj+vYf1FkHkLqezr1MwWmD4jerSjpHnzuH yGExQ0TUKhPSZzuGQPeyh9doCBqoDvg= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-284-aVe4cFt4PJiC0Hz1rE7SrA-1; Sun, 29 Sep 2024 10:43:12 -0400 X-MC-Unique: aVe4cFt4PJiC0Hz1rE7SrA-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A353219030A7; Sun, 29 Sep 2024 14:43:10 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id D883B3003E40; Sun, 29 Sep 2024 14:43:07 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:42:57 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:53 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 6/7] uprobes: pass utask to xol_get_insn_slot() and xol_free_insn_slot() Message-ID: <20240929144253.GA9487@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add the "struct uprobe_task *utask" argument to xol_get_insn_slot() and xol_free_insn_slot(), their callers already have it so we can avoid the unnecessary dereference and simplify the code. Kill the "tsk" argument of xol_free_insn_slot(), it is always current. Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index a7223be5ac2e..da45d0e5bcf4 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1650,9 +1650,8 @@ static unsigned long xol_take_insn_slot(struct xol_ar= ea *area) /* * xol_get_insn_slot - allocate a slot for xol. */ -static bool xol_get_insn_slot(struct uprobe *uprobe) +static bool xol_get_insn_slot(struct uprobe *uprobe, struct uprobe_task *u= task) { - struct uprobe_task *utask =3D current->utask; struct xol_area *area =3D get_xol_area(); =20 if (!area) @@ -1667,12 +1666,12 @@ static bool xol_get_insn_slot(struct uprobe *uprobe) /* * xol_free_insn_slot - free the slot allocated by xol_get_insn_slot() */ -static void xol_free_insn_slot(struct task_struct *tsk) +static void xol_free_insn_slot(struct uprobe_task *utask) { - struct xol_area *area =3D tsk->mm->uprobes_state.xol_area; - unsigned long offset =3D tsk->utask->xol_vaddr - area->vaddr; + struct xol_area *area =3D current->mm->uprobes_state.xol_area; + unsigned long offset =3D utask->xol_vaddr - area->vaddr; =20 - tsk->utask->xol_vaddr =3D 0; + utask->xol_vaddr =3D 0; /* * xol_vaddr must fit into [area->vaddr, area->vaddr + PAGE_SIZE). * This check can only fail if the "[uprobes]" vma was mremap'ed. @@ -1954,7 +1953,7 @@ pre_ssout(struct uprobe *uprobe, struct pt_regs *regs= , unsigned long bp_vaddr) if (!try_get_uprobe(uprobe)) return -EINVAL; =20 - if (!xol_get_insn_slot(uprobe)) { + if (!xol_get_insn_slot(uprobe, utask)) { err =3D -ENOMEM; goto err_out; } @@ -1962,7 +1961,7 @@ pre_ssout(struct uprobe *uprobe, struct pt_regs *regs= , unsigned long bp_vaddr) utask->vaddr =3D bp_vaddr; err =3D arch_uprobe_pre_xol(&uprobe->arch, regs); if (unlikely(err)) { - xol_free_insn_slot(current); + xol_free_insn_slot(utask); goto err_out; } =20 @@ -2313,7 +2312,7 @@ static void handle_singlestep(struct uprobe_task *uta= sk, struct pt_regs *regs) put_uprobe(uprobe); utask->active_uprobe =3D NULL; utask->state =3D UTASK_RUNNING; - xol_free_insn_slot(current); + xol_free_insn_slot(utask); =20 spin_lock_irq(¤t->sighand->siglock); recalc_sigpending(); /* see uprobe_deny_signal() */ --=20 2.25.1.362.g51ebf55 From nobody Thu Nov 28 17:34:22 2024 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CD59183098 for ; Sun, 29 Sep 2024 14:43:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727621002; cv=none; b=o7u913SI74dM0CK9AHAqS6pdbiONIsn62gn9Bop0MYwEQfb1/veNarNivFBG3fHWFnnVKf6D3QkbrWXdQFKxJkTZxp2dB8QHBcJRIlj3qRy2QDr2DPvOHJHjbLzmTiiKOOjFg12fiIq0dWNQeBSVTSgQ7U+sRzo39FA5qNnGNQ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727621002; c=relaxed/simple; bh=FfaZhrQq34BZlheLdSwojN4sapArypT72+Y+nLTOLDA=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type: Content-Disposition:In-Reply-To; b=HfBnbHLGkdmbH4HZ9zr6ATiqY5e158mmKy8m0m7hCYUuNM1XpeELRbMX2oARkYbDV97uLyMr+gwfuuZjjSlEmJu+OPB3TioiRk+SMDTIVdMrqD9Y3EDWc92L9QuQWe638r7/+fq6FZTS6rbOtWpQyTFj/Y0/7yt7Y6LUmXDf1CA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fcOlL1X9; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fcOlL1X9" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727621000; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=PFEiL3F5MSjU++Jxe01AvOazddBzZZRjIDhG9rgyzoQ=; b=fcOlL1X9TFa4n8u/GCm8dWpVUOpDmdhiRws/8o/KJTDva0nQyHtKi7KGxGe6lE7ac6hsli 2V3CV4FlleOhUfOy9o/0TLn1BX6sm5N0o6J39/T81V+DJgBi9hH+oQCLEKT0unIZei5kLy NzbyHbJfnuoqjHyCTBZrKuAAnbGMU/k= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-530-2aiClEGVNq6dSkKRgCxqKg-1; Sun, 29 Sep 2024 10:43:16 -0400 X-MC-Unique: 2aiClEGVNq6dSkKRgCxqKg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 42BB0195FE21; Sun, 29 Sep 2024 14:43:15 +0000 (UTC) Received: from dhcp-27-174.brq.redhat.com (unknown [10.45.224.44]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with SMTP id 90E271979060; Sun, 29 Sep 2024 14:43:12 +0000 (UTC) Received: by dhcp-27-174.brq.redhat.com (nbSMTP-1.00) for uid 1000 oleg@redhat.com; Sun, 29 Sep 2024 16:43:01 +0200 (CEST) Date: Sun, 29 Sep 2024 16:42:58 +0200 From: Oleg Nesterov To: Andrii Nakryiko , Jiri Olsa , Masami Hiramatsu , Peter Zijlstra Cc: Liao Chang , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH 7/7] uprobes: deny mremap(xol_vma) Message-ID: <20240929144258.GA9492@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240929144201.GA9429@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" kernel/events/uprobes.c assumes that xol_area->vaddr is always correct but a malicious application can remap its "[uprobes]" vma to another adress to confuse the kernel. Introduce xol_mremap() to make this impossible. With this change utask->xol_vaddr in xol_free_insn_slot() can't be invalid, we can turn the offset check into WARN_ON_ONCE(offset >=3D PAGE_SIZE). Signed-off-by: Oleg Nesterov --- kernel/events/uprobes.c | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index da45d0e5bcf4..20c58b6ee1ad 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1475,9 +1475,15 @@ static vm_fault_t xol_fault(const struct vm_special_= mapping *sm, return 0; } =20 +static int xol_mremap(const struct vm_special_mapping *sm, struct vm_area_= struct *new_vma) +{ + return -EPERM; +} + static const struct vm_special_mapping xol_mapping =3D { .name =3D "[uprobes]", .fault =3D xol_fault, + .mremap =3D xol_mremap, }; =20 /* Slot allocation for XOL */ @@ -1670,21 +1676,19 @@ static void xol_free_insn_slot(struct uprobe_task *= utask) { struct xol_area *area =3D current->mm->uprobes_state.xol_area; unsigned long offset =3D utask->xol_vaddr - area->vaddr; + unsigned int slot_nr; =20 utask->xol_vaddr =3D 0; - /* - * xol_vaddr must fit into [area->vaddr, area->vaddr + PAGE_SIZE). - * This check can only fail if the "[uprobes]" vma was mremap'ed. - */ - if (offset < PAGE_SIZE) { - int slot_nr =3D offset / UPROBE_XOL_SLOT_BYTES; - - clear_bit(slot_nr, area->bitmap); - atomic_dec(&area->slot_count); - smp_mb__after_atomic(); /* pairs with prepare_to_wait() */ - if (waitqueue_active(&area->wq)) - wake_up(&area->wq); - } + /* xol_vaddr must fit into [area->vaddr, area->vaddr + PAGE_SIZE) */ + if (WARN_ON_ONCE(offset >=3D PAGE_SIZE)) + return; + + slot_nr =3D offset / UPROBE_XOL_SLOT_BYTES; + clear_bit(slot_nr, area->bitmap); + atomic_dec(&area->slot_count); + smp_mb__after_atomic(); /* pairs with prepare_to_wait() */ + if (waitqueue_active(&area->wq)) + wake_up(&area->wq); } =20 void __weak arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr, --=20 2.25.1.362.g51ebf55