From nobody Sat Feb 7 16:59:23 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F6F716E883 for ; Fri, 21 Jun 2024 09:16:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718961371; cv=none; b=nbQc5uvAZyLWq9Njm4SMDGYirA3zWnqZx8a19vFnBzqzWCC1iR9/UtQTzEH7X1ntbOG1snhCOWui95RkA44lXzxPAWSs8/SyamSuzkOOlERofaSdyn8RIzzblpc0JldiVjc0t71ELyOXTmCI+bJIQAICKXS3u31hIF1ibWHJBhE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718961371; c=relaxed/simple; bh=QIm0h9NLC7E/SqoC9yvtlRN3pBmKqYZnA6vS14mU9gY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VqVKI/KFmR96tiJRcjLW4hMOtYkI7f99+DR2mqu8Hrdwv8BpMaAwu2vLPOBhl7R6uBDvI/pI8BHH8dbAS07GzU6j5isfMgos2P8/Ku7W5KoALsV8lIggT/5/Iu+6wQohlQb+fgKgVVSO0oeozs63j/8CKfTsD0tLO3WP7M2c/7A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jfUPcfCh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jfUPcfCh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 29C80C32781; Fri, 21 Jun 2024 09:16:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1718961370; bh=QIm0h9NLC7E/SqoC9yvtlRN3pBmKqYZnA6vS14mU9gY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jfUPcfChSiO0xYEl+Jd7zrglwHmLSQzr6q3c4/E+TKK0UfTV7kq6IvvAqw+qhcPoN 3g5HNX5gbfxHTjj3dy6BIahFhhNIyP8I43PqnSNjUxwBsqkalkzkqV9D+GQIc52CCX xa2FOTkaWWtWC5MLVAJAzot+oloWAOeEjoCI4KhbMBnpIhRz342eEf5eVYToGfaVNz VCaMFhbHAUk9ZufcRHrRLbZeV2LDMBQX0djiVajisj74VtzO5Si9TRFCTb7nYVVlZt JrQVSF4FzGblhTy/VJ2cFXUGxxCsqdMDs6GBoOKP4Si6oosUhQnL+nIYuVY++fxGoV uj7E4LMcY0vtQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Sebastian Andrzej Siewior Subject: [PATCH 1/4] task_work: s/task_work_cancel()/task_work_cancel_func()/ Date: Fri, 21 Jun 2024 11:15:58 +0200 Message-ID: <20240621091601.18227-2-frederic@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240621091601.18227-1-frederic@kernel.org> References: <20240621091601.18227-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A proper task_work_cancel() API that actually cancels a callback and not *any* callback pointing to a given function is going to be needed for perf events event freeing. Do the appropriate rename to prepare for that. Signed-off-by: Frederic Weisbecker --- include/linux/task_work.h | 2 +- kernel/irq/manage.c | 2 +- kernel/task_work.c | 10 +++++----- security/keys/keyctl.c | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/include/linux/task_work.h b/include/linux/task_work.h index 795ef5a68429..23ab01ae185e 100644 --- a/include/linux/task_work.h +++ b/include/linux/task_work.h @@ -30,7 +30,7 @@ int task_work_add(struct task_struct *task, struct callba= ck_head *twork, =20 struct callback_head *task_work_cancel_match(struct task_struct *task, bool (*match)(struct callback_head *, void *data), void *data); -struct callback_head *task_work_cancel(struct task_struct *, task_work_fun= c_t); +struct callback_head *task_work_cancel_func(struct task_struct *, task_wor= k_func_t); void task_work_run(void); =20 static inline void exit_task_work(struct task_struct *task) diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c index 71b0fc2d0aea..dd53298ef1a5 100644 --- a/kernel/irq/manage.c +++ b/kernel/irq/manage.c @@ -1337,7 +1337,7 @@ static int irq_thread(void *data) * synchronize_hardirq(). So neither IRQTF_RUNTHREAD nor the * oneshot mask bit can be set. */ - task_work_cancel(current, irq_thread_dtor); + task_work_cancel_func(current, irq_thread_dtor); return 0; } =20 diff --git a/kernel/task_work.c b/kernel/task_work.c index 95a7e1b7f1da..54ac24059daa 100644 --- a/kernel/task_work.c +++ b/kernel/task_work.c @@ -120,9 +120,9 @@ static bool task_work_func_match(struct callback_head *= cb, void *data) } =20 /** - * task_work_cancel - cancel a pending work added by task_work_add() - * @task: the task which should execute the work - * @func: identifies the work to remove + * task_work_cancel_func - cancel a pending work matching a function added= by task_work_add() + * @task: the task which should execute the func's work + * @func: identifies the func to match with a work to remove * * Find the last queued pending work with ->func =3D=3D @func and remove * it from queue. @@ -131,7 +131,7 @@ static bool task_work_func_match(struct callback_head *= cb, void *data) * The found work or NULL if not found. */ struct callback_head * -task_work_cancel(struct task_struct *task, task_work_func_t func) +task_work_cancel_func(struct task_struct *task, task_work_func_t func) { return task_work_cancel_match(task, task_work_func_match, func); } @@ -168,7 +168,7 @@ void task_work_run(void) if (!work) break; /* - * Synchronize with task_work_cancel(). It can not remove + * Synchronize with task_work_cancel_match(). It can not remove * the first entry =3D=3D work, cmpxchg(task_works) must fail. * But it can remove another entry from the ->next list. */ diff --git a/security/keys/keyctl.c b/security/keys/keyctl.c index 4bc3e9398ee3..ab927a142f51 100644 --- a/security/keys/keyctl.c +++ b/security/keys/keyctl.c @@ -1694,7 +1694,7 @@ long keyctl_session_to_parent(void) goto unlock; =20 /* cancel an already pending keyring replacement */ - oldwork =3D task_work_cancel(parent, key_change_session_keyring); + oldwork =3D task_work_cancel_func(parent, key_change_session_keyring); =20 /* the replacement session keyring is applied just prior to userspace * restarting */ --=20 2.45.2 From nobody Sat Feb 7 16:59:23 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EDAFD16EB59 for ; Fri, 21 Jun 2024 09:16:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718961374; cv=none; b=m9Pa4C1QN4KRVrbyUFdcITR+xpQTLKO8bKerWSWrj/VIMQev/+R7fNXie3Ywuovj8NbyGVqJuiry0vV0fdD94yp32LIiyr37SSZJxJ7VGyNVEwlNlEyX3zNvoRlQtn7w/sncOEiTews4Z1oK6+tSIwo+9pQV7coSj/Cxygd+eRk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718961374; c=relaxed/simple; bh=yxEHyP8F59DlEgnpBUShwu2ascF0KGLnYNcwxKRX5ns=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RI1BZ75FxcUfHGNzPLHJNbanlr4gXzj7vsH+Z5jsytEuE4fOqHJyhXB+iWaQajkwD/vzhddTMbyIu0Ahz62A7xA4ln3/RCBI59EwY9JZk/rGhvW57sDyyqbfURDMOtx5gYFZukG2I0MqkRD72UDbh2VkbD+SEhcSh6k7AX8Oo8o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PENtdBkZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PENtdBkZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 277F5C4AF0F; Fri, 21 Jun 2024 09:16:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1718961373; bh=yxEHyP8F59DlEgnpBUShwu2ascF0KGLnYNcwxKRX5ns=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PENtdBkZXrxpozLB69Dm8bpmeyW0THBFbKA1GfpgYQxjyTf6QCWSySA3zZ93EQQwS cFiyrHFfk+ufP4kOcHDcpjNhB7+ERI9l9YQkDK1Gtr5FWXELeDgXI7RqvMD6iFV/U5 RYgzTbbWBW16DIOpMlnp7MtckBaxNNyTcrB1VsWIGQF5/PBqSY5cBz/g3EXO6KIpy7 0w2ftYSvlP7QYtPktXDWkHakCIdiKqF90uUTN2hAPdsFbdGpi7q76er1AqRf5rvS2k /0PldJoxDmxV91Peezzvq8ezR+01Rss5eRh4NDus01wG4tIGJ5J+2TtlA9G62yG6/S fQ9Ey0e8QSS7Q== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Sebastian Andrzej Siewior Subject: [PATCH 2/4] task_work: Introduce task_work_cancel() again Date: Fri, 21 Jun 2024 11:15:59 +0200 Message-ID: <20240621091601.18227-3-frederic@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240621091601.18227-1-frederic@kernel.org> References: <20240621091601.18227-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Re-introduce task_work_cancel(), this time to cancel an actual callback and not *any* callback pointing to a given function. This is going to be needed for perf events event freeing. Signed-off-by: Frederic Weisbecker --- include/linux/task_work.h | 1 + kernel/task_work.c | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+) diff --git a/include/linux/task_work.h b/include/linux/task_work.h index 23ab01ae185e..26b8a47f41fc 100644 --- a/include/linux/task_work.h +++ b/include/linux/task_work.h @@ -31,6 +31,7 @@ int task_work_add(struct task_struct *task, struct callba= ck_head *twork, struct callback_head *task_work_cancel_match(struct task_struct *task, bool (*match)(struct callback_head *, void *data), void *data); struct callback_head *task_work_cancel_func(struct task_struct *, task_wor= k_func_t); +bool task_work_cancel(struct task_struct *task, struct callback_head *cb); void task_work_run(void); =20 static inline void exit_task_work(struct task_struct *task) diff --git a/kernel/task_work.c b/kernel/task_work.c index 54ac24059daa..2134ac8057a9 100644 --- a/kernel/task_work.c +++ b/kernel/task_work.c @@ -136,6 +136,30 @@ task_work_cancel_func(struct task_struct *task, task_w= ork_func_t func) return task_work_cancel_match(task, task_work_func_match, func); } =20 +static bool task_work_match(struct callback_head *cb, void *data) +{ + return cb =3D=3D data; +} + +/** + * task_work_cancel - cancel a pending work added by task_work_add() + * @task: the task which should execute the work + * @cb: the callback to remove if queued + * + * Remove a callback from a task's queue if queued. + * + * RETURNS: + * True if the callback was queued and got cancelled, false otherwise. + */ +bool task_work_cancel(struct task_struct *task, struct callback_head *cb) +{ + struct callback_head *ret; + + ret =3D task_work_cancel_match(task, task_work_match, cb); + + return ret =3D=3D cb; +} + /** * task_work_run - execute the works added by task_work_add() * --=20 2.45.2 From nobody Sat Feb 7 16:59:23 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2574516EC0B for ; Fri, 21 Jun 2024 09:16:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718961377; cv=none; b=l8zUGe4tAm/GBKbHq/NTt4c17ggORfxpmEFDhtdnCrV6yoHcWgbm93fx8Szu1tqDKH1B18ZdmuQKjQDQGDEMfockOgJp4aMpyKeW/UiNi+FgL4E+wCme2sYmztvjYHG3f77EJ8BWk1XXm8ajaXWblZbdXI83rIrDnZd8s/OuP+s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718961377; c=relaxed/simple; bh=5XJeijBFwMIAQTzHMcx3zvbS0cEEi69GXEIlzTUup14=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GaIOL7LCJCJmXnwMswtuiYyjmB1cSgBNqtQZB4AFtkJmpXxyx3a+uoqy+PhVLZXvNWBMccRCg858PYDDGj9G1yvCYuBoH1GAC2pG17b9JYD4QYhgkv0R++dUIoqzT3fudlPg3UWoR3KPe98zA6vhFeWq+CUairZfiun8ZHOt65I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RS5d0SOZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RS5d0SOZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 021D7C32781; Fri, 21 Jun 2024 09:16:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1718961376; bh=5XJeijBFwMIAQTzHMcx3zvbS0cEEi69GXEIlzTUup14=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RS5d0SOZKDThKmdrWuPx65fT4JYdZp3xrJk3KwhRXbvClNnTCj8H1NboF+wRr9waK KosG5uIJWaPc0q0/OwFWCKMortW5EY+nWOut7S3wMcth4HQnsVMBZiCag8/aYthXwM Pn9JNun3+8Z4nYVwmeBOoHJCiOJZYFB5in2mj3cR66+uwXyUI181BA/ZCWLQHI9rI7 21O34WCCRZSRwgQpmgIYJB4jLvmKbGUSG5tE4YPHLJx1qoY3nwlKYGRMSv2yPQ866B 469xO1YUEbqrWp7R2kySxRWrIkec/OLi2cmWVHL6+dJE8VrtOOUXoCz826ygLMpWFj X+wg0pVaZzzkQ== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Sebastian Andrzej Siewior Subject: [PATCH 3/4] perf: Fix event leak upon exit Date: Fri, 21 Jun 2024 11:16:00 +0200 Message-ID: <20240621091601.18227-4-frederic@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240621091601.18227-1-frederic@kernel.org> References: <20240621091601.18227-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a task is scheduled out, pending sigtrap deliveries are deferred to the target task upon resume to userspace via task_work. However failures while adding an event's callback to the task_work engine are ignored. And since the last call for events exit happen after task work is eventually closed, there is a small window during which pending sigtrap can be queued though ignored, leaking the event refcount addition such as in the following scenario: TASK A ----- do_exit() exit_task_work(tsk); perf_event_overflow() event->pending_sigtrap =3D pending_id; irq_work_queue(&event->pending_irq); =3D=3D=3D=3D=3D=3D=3D=3D=3D> PREEMPTION: TASK A -> TASK B event_sched_out() event->pending_sigtrap =3D 0; atomic_long_inc_not_zero(&event->refcount) // FAILS: task work has exited task_work_add(&event->pending_task) [...] perf_pending_irq() // early return: event->oncpu =3D -1 [...] =3D=3D=3D=3D=3D=3D=3D=3D=3D> TASK B -> TASK A perf_event_exit_task(tsk) perf_event_exit_event() free_event() WARN(atomic_long_cmpxchg(&event->refcount, 1, 0) !=3D 1) // leak event due to unexpected refcount =3D=3D 2 As a result the event is never released while the task exits. Fix this with appropriate task_work_add()'s error handling. Fixes: 517e6a301f34 ("perf: Fix perf_pending_task() UaF") Signed-off-by: Frederic Weisbecker --- kernel/events/core.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/kernel/events/core.c b/kernel/events/core.c index 8f908f077935..7c3218d31d1d 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -2284,18 +2284,15 @@ event_sched_out(struct perf_event *event, struct pe= rf_event_context *ctx) } =20 if (event->pending_sigtrap) { - bool dec =3D true; - event->pending_sigtrap =3D 0; if (state !=3D PERF_EVENT_STATE_OFF && - !event->pending_work) { - event->pending_work =3D 1; - dec =3D false; + !event->pending_work && + !task_work_add(current, &event->pending_task, TWA_RESUME)) { WARN_ON_ONCE(!atomic_long_inc_not_zero(&event->refcount)); - task_work_add(current, &event->pending_task, TWA_RESUME); - } - if (dec) + event->pending_work =3D 1; + } else { local_dec(&event->ctx->nr_pending); + } } =20 perf_event_set_state(event, state); --=20 2.45.2 From nobody Sat Feb 7 16:59:23 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DEE5A16F0D2 for ; Fri, 21 Jun 2024 09:16:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718961380; cv=none; b=G99DkSQYTcuHYYyTHfdyn1iEtm8AF332ybvIY7vbIRQyD4by5cVcBu3gQyThiMmaPkTeQk/BtRl35SnIMXHE0JWvNEY54CkSrIzrLMrlKmozmFqN0+pHr8vdqAv3dUaHS83ZHcXnAWPpINoz0YIWFiP6f9a/Bu63uNHHnyWUX2U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718961380; c=relaxed/simple; bh=tIr6xsl9f1qlL3ubydb+4cEtOOda2NPwxbdGg11VU2k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JbOyHU9/1fTozSGNIoW3BDtUutYzm0dHIwb100apG5zLHrmkBE8QOMChD4cdJkYmAxoaDDHHJZV8pTJXAUQaiY7XFRGoeWCRrOf0PCyqb8l1y/kM3d6zNowhMP8UsSKmoSvG0AlSMcglKt1lts7Os0KxRoEWN4gycocQwFFED/k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ohiki8X8; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ohiki8X8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 054FFC4AF09; Fri, 21 Jun 2024 09:16:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1718961379; bh=tIr6xsl9f1qlL3ubydb+4cEtOOda2NPwxbdGg11VU2k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ohiki8X8ilUGBaJ7gIupFINyIDRIkaniT7gVKr6Iwd9XrNDZKfOuVeufXr3czCW4E igxvoZsMarJ2BDeZZXD/HJXuFFsThKi5Ht/NdM3fTHOzphrJs0/2G6n/9Bfq0fQ5uR SwAzvZyNCFd6EcZQTLtBBK7qP+6y2dNSTAlONmafm8eOQCnHyisHgPk2t/+MPZYzM0 SFiV7TWPpaaf6FIA5R3X/XCuCT/ww2qbKzisfr0NWRwbnWWQ/B5gbsVK7VsahRnl1o Dyj3CSxrvdda/NQtta8OUUA6pIV8z71yZo7f1gWBQRwH8awSB4Y8uvDVKWRmgWO94F rFx2bYrgbRR7g== From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , Sebastian Andrzej Siewior Subject: [PATCH 4/4] perf: Fix event leak upon exec and file release Date: Fri, 21 Jun 2024 11:16:01 +0200 Message-ID: <20240621091601.18227-5-frederic@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240621091601.18227-1-frederic@kernel.org> References: <20240621091601.18227-1-frederic@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The perf pending task work is never waited upon the matching event release. In the case of a child event, released via free_event() directly, this can potentially result in a leaked event, such as in the following scenario that doesn't even require a weak IRQ work implementation to trigger: schedule() prepare_task_switch() =3D=3D=3D=3D=3D=3D=3D> perf_event_overflow() event->pending_sigtrap =3D ... irq_work_queue(&event->pending_irq) <=3D=3D=3D=3D=3D=3D=3D perf_event_task_sched_out() event_sched_out() event->pending_sigtrap =3D 0; atomic_long_inc_not_zero(&event->refcount) task_work_add(&event->pending_task) finish_lock_switch() =3D=3D=3D=3D=3D=3D=3D> perf_pending_irq() //do nothing, rely on pending task work <=3D=3D=3D=3D=3D=3D=3D begin_new_exec() perf_event_exit_task() perf_event_exit_event() // If is child event free_event() WARN(atomic_long_cmpxchg(&event->refcount, 1, 0) !=3D 1) // event is leaked Similar scenarios can also happen with perf_event_remove_on_exec() or simply against concurrent perf_event_release(). Fix this with synchonizing against the possibly remaining pending task work while freeing the event, just like is done with remaining pending IRQ work. This means that the pending task callback neither need nor should hold a reference to the event, preventing it from ever beeing freed. Fixes: 517e6a301f34 ("perf: Fix perf_pending_task() UaF") Signed-off-by: Frederic Weisbecker --- include/linux/perf_event.h | 1 + kernel/events/core.c | 38 ++++++++++++++++++++++++++++++++++---- 2 files changed, 35 insertions(+), 4 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index a5304ae8c654..393fb13733b0 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -786,6 +786,7 @@ struct perf_event { struct irq_work pending_irq; struct callback_head pending_task; unsigned int pending_work; + struct rcuwait pending_work_wait; =20 atomic_t event_limit; =20 diff --git a/kernel/events/core.c b/kernel/events/core.c index 7c3218d31d1d..586d4f367624 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -2288,7 +2288,6 @@ event_sched_out(struct perf_event *event, struct perf= _event_context *ctx) if (state !=3D PERF_EVENT_STATE_OFF && !event->pending_work && !task_work_add(current, &event->pending_task, TWA_RESUME)) { - WARN_ON_ONCE(!atomic_long_inc_not_zero(&event->refcount)); event->pending_work =3D 1; } else { local_dec(&event->ctx->nr_pending); @@ -5203,9 +5202,35 @@ static bool exclusive_event_installable(struct perf_= event *event, static void perf_addr_filters_splice(struct perf_event *event, struct list_head *head); =20 +static void perf_pending_task_sync(struct perf_event *event) +{ + struct callback_head *head =3D &event->pending_task; + + if (!event->pending_work) + return; + /* + * If the task is queued to the current task's queue, we + * obviously can't wait for it to complete. Simply cancel it. + */ + if (task_work_cancel(current, head)) { + event->pending_work =3D 0; + local_dec(&event->ctx->nr_pending); + return; + } + + /* + * All accesses related to the event are within the same + * non-preemptible section in perf_pending_task(). The RCU + * grace period before the event is freed will make sure all + * those accesses are complete by then. + */ + rcuwait_wait_event(&event->pending_work_wait, !event->pending_work, TASK_= UNINTERRUPTIBLE); +} + static void _free_event(struct perf_event *event) { irq_work_sync(&event->pending_irq); + perf_pending_task_sync(event); =20 unaccount_event(event); =20 @@ -6828,24 +6853,28 @@ static void perf_pending_task(struct callback_head = *head) struct perf_event *event =3D container_of(head, struct perf_event, pendin= g_task); int rctx; =20 + /* + * All accesses to the event must belong to the same implicit RCU read-si= de + * critical section as the ->pending_work reset. See comment in + * perf_pending_task_sync(). + */ + preempt_disable_notrace(); /* * If we 'fail' here, that's OK, it means recursion is already disabled * and we won't recurse 'further'. */ - preempt_disable_notrace(); rctx =3D perf_swevent_get_recursion_context(); =20 if (event->pending_work) { event->pending_work =3D 0; perf_sigtrap(event); local_dec(&event->ctx->nr_pending); + rcuwait_wake_up(&event->pending_work_wait); } =20 if (rctx >=3D 0) perf_swevent_put_recursion_context(rctx); preempt_enable_notrace(); - - put_event(event); } =20 #ifdef CONFIG_GUEST_PERF_EVENTS @@ -11959,6 +11988,7 @@ perf_event_alloc(struct perf_event_attr *attr, int = cpu, init_waitqueue_head(&event->waitq); init_irq_work(&event->pending_irq, perf_pending_irq); init_task_work(&event->pending_task, perf_pending_task); + rcuwait_init(&event->pending_work_wait); =20 mutex_init(&event->mmap_mutex); raw_spin_lock_init(&event->addr_filters.lock); --=20 2.45.2