From nobody Mon Sep 28 23:12:47 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23C974195A3; Sat, 15 Aug 2026 14:56:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786805801; cv=none; b=h+JiO/VtMNRs3fJuKQ5kyBTXneGaYEsduxiLvNaoatC5GCCjyMsZlRD0LrhsufFRtQ5tVCro2wrNQvmxRI2lOD+8f8AkjEnFWthroC/DE+cLn9TkBW2XUIIjA8f41jxBVPTdP1/rs//jjsM2JPZtgGX4FrN6opYvUmYDSG1V7bM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786805801; c=relaxed/simple; bh=Jr8is1Cti6P50cWYYbyd906COEhy8VmjZtYEHUFgX5I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dPjhdFlIJ9GSibvRVSl3GWHfOTGw2Jw+8a/wTEd26gzeY1LAADPTRVTnJtywLSNiVHY+a2nsUx+pfUtGXNkc27B7pUdk0K1nJosB/eh7qj1HXWxCYZ6wduuxOHBKVcVJHm3MZiUiCmRqoUWyC0B/ACPCYDKFTBcfP+XdfLL8/5w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TucNt55R; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TucNt55R" Received: by smtp.kernel.org (Postfix) with ESMTPS id C4280C2BCF6; Sat, 15 Aug 2026 14:56:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1786805800; bh=Jr8is1Cti6P50cWYYbyd906COEhy8VmjZtYEHUFgX5I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=TucNt55Rwd3xg40KzTHHgyQlcNGDkAG3Z8r3ErecVloGJL7p5VSXp4XHN1bo2CRy0 Drdhui+w+/Xp29XbTqFCwQZf7B66ktEaq14+8VJ4r84/sP6iA1zxv/LA+XAJuB2HYU TWgyJ3OiSziW4j/l2nIoYQMYO8hoviiQuKXYH0n47/jLxjS1/IxfkpPgvkstS4zYsN nnDsyocCxCF8etwZgqmTBp4HwQuQST3ufD9JIxCu4dwJ5CqDc/3HoE6rCixOsXMnWu uhFpMM+PNMw+BqQ86YsH1pB85HYbZe5Z8Xwzvht3RASO/f6xn1QYlomAF3MMm/VWTK ysQQ6kk2dPwCQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A367EC5DF67; Sat, 15 Aug 2026 14:56:40 +0000 (UTC) From: Junrui Luo via B4 Relay Date: Sat, 15 Aug 2026 22:56:35 +0800 Subject: [PATCH 1/4] media: v4l2-mem2mem: serialize REMOVE_BUFS with job execution Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260815-vicodec-fixes-v1-1-12101572eba7@outlook.com> References: <20260815-vicodec-fixes-v1-0-12101572eba7@outlook.com> In-Reply-To: <20260815-vicodec-fixes-v1-0-12101572eba7@outlook.com> To: Mauro Carvalho Chehab , Hans Verkuil , Benjamin Gaignard , Dafna Hirschfeld Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Junrui Luo , Yuhao Jiang , stable@vger.kernel.org X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=5309; i=moonafterrain@outlook.com; h=from:subject:message-id; bh=6krpIkSHmwa+ABnniq+DXrpDxKlpmMs7gAcsjsK/D8g=; b=owJ4nJvAy8zAJVb4wiKgu++DA+NptSSGrIY6tVVRuvkbzB4+ivjOJrHObdPfyF1+2tvX3Y1lq T5Qp2TVs7ajlIVBjItBVkyR5XjBpW8Wvlt0t/hsSYaZw8oEMoSBi1MAJnJwBcM/rZ1hWpt75+wr SvZzuKb7sL5b0W6t21aRN5NTWx+6njV+xfC/8qRIYFB9seOVRd+P70lYcvt3j83ak9ss853yvr+ VU5FhBwDiDE13 X-Developer-Key: i=moonafterrain@outlook.com; a=openpgp; fpr=C770D2F6384DB42DB44CB46371E838508B8EF040 X-Endpoint-Received: by B4 Relay for moonafterrain@outlook.com/default with auth_id=909 X-Original-From: Junrui Luo Reply-To: moonafterrain@outlook.com From: Junrui Luo VIDIOC_REMOVE_BUFS can free a buffer that an in-flight job is still decoding from, giving a use-after-free of the reference frame. For stateless decoders a reference frame sits in VB2_BUF_STATE_DEQUEUED, which is exactly what REMOVE_BUFS frees: userspace dequeues the decoded capture buffer and names its timestamp as the reference for the next frame. device_run() resolves that timestamp with vb2_find_buffer() and keeps the vb2_plane_vaddr() result for the whole decode without holding a reference. Nothing serializes the two paths: REMOVE_BUFS runs under m2m_ctx->q_lock and q->mmap_lock, while the job path holds neither and reaches device_run() from schedule_work() outside any ioctl lock. Hold the instance off the job queue and wait for its in-flight job before removing the buffers. The pause is per-instance rather than device-wide because the buffers belong to that instance's own queues and the caller is inside one of its own ioctls, so the context cannot be released while the wait is in progress. A losing race degrades to vb2_find_buffer() returning NULL and the job failing with -EINVAL, which is already handled. Reproduced on vicodec under KASAN: BUG: KASAN: vmalloc-out-of-bounds in add_deltas+0x450/0xcc0 Read of size 1 at addr ffffc90000a77bc0 by task trigger_bin/71 Call Trace: add_deltas+0x450/0xcc0 decode_plane+0x1799/0x34b0 fwht_decode_frame+0x173/0x620 v4l2_fwht_decode+0x595/0xea0 device_run+0x6bb/0x1850 media_request_ioctl+0x2fb/0x450 __se_sys_ioctl+0xb1/0x110 do_syscall_64+0x12c/0x360 Fixes: 2f2419502f69 ("media: v4l2: Add mem2mem helpers for REMOVE_BUFS ioct= l") Reported-by: Yuhao Jiang Assisted-by: Claude:claude-opus-5 Cc: stable@vger.kernel.org Signed-off-by: Junrui Luo --- drivers/media/v4l2-core/v4l2-mem2mem.c | 60 ++++++++++++++++++++++++++++++= +--- 1 file changed, 56 insertions(+), 4 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-co= re/v4l2-mem2mem.c index a65cbb124cfe..10a0a74f782c 100644 --- a/drivers/media/v4l2-core/v4l2-mem2mem.c +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c @@ -41,6 +41,8 @@ module_param(debug, bool, 0644); #define TRANS_RUNNING (1 << 1) /* Instance is currently aborting */ #define TRANS_ABORT (1 << 2) +/* Instance must not be scheduled while its buffers are being removed */ +#define TRANS_PAUSED (1 << 3) =20 =20 /* The job queue is not running new jobs */ @@ -309,9 +311,9 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m= 2m_dev, =20 spin_lock_irqsave(&m2m_dev->job_spinlock, flags_job); =20 - /* If the context is aborted then don't schedule it */ - if (m2m_ctx->job_flags & TRANS_ABORT) { - dprintk("Aborted context\n"); + /* If the context is aborted or paused then don't schedule it */ + if (m2m_ctx->job_flags & (TRANS_ABORT | TRANS_PAUSED)) { + dprintk("Aborted or paused context\n"); goto job_unlock; } =20 @@ -1388,16 +1390,66 @@ int v4l2_m2m_ioctl_create_bufs(struct file *file, v= oid *priv, } EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_create_bufs); =20 +/* + * Keep @m2m_ctx off the job queue and wait for its in-flight job, if any. + * + * Only this instance is held back: the buffers about to be removed belong= to + * its own queues, so no other instance can be reading them. Waiting on th= is + * instance is also what makes the wait safe -- the caller is inside one o= f its + * ioctls, so v4l2_m2m_ctx_release() cannot free it here. + */ +static void v4l2_m2m_pause_ctx(struct v4l2_m2m_ctx *m2m_ctx) +{ + struct v4l2_m2m_dev *m2m_dev =3D m2m_ctx->m2m_dev; + unsigned long flags; + + spin_lock_irqsave(&m2m_dev->job_spinlock, flags); + m2m_ctx->job_flags |=3D TRANS_PAUSED; + if (m2m_ctx->job_flags & TRANS_RUNNING) { + spin_unlock_irqrestore(&m2m_dev->job_spinlock, flags); + wait_event(m2m_ctx->finished, + !(m2m_ctx->job_flags & TRANS_RUNNING)); + return; + } + if (m2m_ctx->job_flags & TRANS_QUEUED) { + list_del(&m2m_ctx->queue); + m2m_ctx->job_flags &=3D ~TRANS_QUEUED; + } + spin_unlock_irqrestore(&m2m_dev->job_spinlock, flags); +} + +static void v4l2_m2m_resume_ctx(struct v4l2_m2m_ctx *m2m_ctx) +{ + struct v4l2_m2m_dev *m2m_dev =3D m2m_ctx->m2m_dev; + unsigned long flags; + + spin_lock_irqsave(&m2m_dev->job_spinlock, flags); + m2m_ctx->job_flags &=3D ~TRANS_PAUSED; + spin_unlock_irqrestore(&m2m_dev->job_spinlock, flags); + + v4l2_m2m_try_schedule(m2m_ctx); +} + int v4l2_m2m_ioctl_remove_bufs(struct file *file, void *priv, struct v4l2_remove_buffers *remove) { struct v4l2_fh *fh =3D file_to_v4l2_fh(file); struct vb2_queue *q =3D v4l2_m2m_get_vq(fh->m2m_ctx, remove->type); + int ret; =20 if (q->type !=3D remove->type) return -EINVAL; =20 - return vb2_core_remove_bufs(q, remove->index, remove->count); + /* + * Removal is only allowed for DEQUEUED buffers, but that is exactly + * the state a stateless decoder's reference frame is in while a job + * resolves it by timestamp and reads its memory. + */ + v4l2_m2m_pause_ctx(fh->m2m_ctx); + ret =3D vb2_core_remove_bufs(q, remove->index, remove->count); + v4l2_m2m_resume_ctx(fh->m2m_ctx); + + return ret; } EXPORT_SYMBOL_GPL(v4l2_m2m_ioctl_remove_bufs); =20 --=20 2.51.2 From nobody Mon Sep 28 23:12:47 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23BF53AAF69; Sat, 15 Aug 2026 14:56:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786805801; cv=none; b=RKemrEW4/jxV3jWkJodkZVvxki3Efn2eYipKWQ+NcXtui0d2FuWKF7sroGLTape7l+Bud2I3P5nU16CmY8HAp2+8JQtf2EneM4hxXuWv9ZK/yAHl0OnbmAJPsDiuceJaEZ+hsi+ese1h6PvwruaWluOiGcONHO4dvQBkZPSPVBo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786805801; c=relaxed/simple; bh=9gbx3w5N0uBUyse1NRCNxyglDGHce1r+QyRriAJm4yg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=XFFvIF8bAjYpibtQCEIVWqjJ4JsjkjzseAe59+VtdhEkm08QDzp5T0kl+FqvlRWBpAMMj2T6710CJPKn8Fi84ba3oeK6QtwWoCuOgu6Eh4BpOfQTF2SHUIdeuN4vOZjvqgCF7gy2E3RPCrTUjd9CK4/bORtoPu2rfmwqdbaxkP4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HmQpcuxm; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HmQpcuxm" Received: by smtp.kernel.org (Postfix) with ESMTPS id D1926C2BCC7; Sat, 15 Aug 2026 14:56:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1786805800; bh=9gbx3w5N0uBUyse1NRCNxyglDGHce1r+QyRriAJm4yg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=HmQpcuxmNZ4Ek3P5U6Y92bL6EAWd55LNTKWHu/bGdSJaJKXDNB9bpb07cdN+cZwUT 0utjMxrXPOmqo2noRxZd6LNW337u5kJIU1GM1Do+xW8FC72f4l+0B7PPZ8bcA9TOOv az+mLgKZCeAHmVpDMC20SQRN3XkDT2Kpmdy23qxRJWLiJtXE/+izF1AileRjLpZpqx fCZpfARjaQZ1F8BwhvUUV0uwKGBPh5jsCGtsspHRy/G8BKfdI2NXbsHTFyGro0sCKD LizZk86Yli7pK1sohI5GP/baxOXg3Z1iHHoxPQNkgMh1rvXXgvjFj5oqX1Q1N8Z1hE +PH3txb2+H4UQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B29C6C5AD5A; Sat, 15 Aug 2026 14:56:40 +0000 (UTC) From: Junrui Luo via B4 Relay Date: Sat, 15 Aug 2026 22:56:36 +0800 Subject: [PATCH 2/4] media: vicodec: fix out-of-bounds write on under-rounded coded dimensions Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260815-vicodec-fixes-v1-2-12101572eba7@outlook.com> References: <20260815-vicodec-fixes-v1-0-12101572eba7@outlook.com> In-Reply-To: <20260815-vicodec-fixes-v1-0-12101572eba7@outlook.com> To: Mauro Carvalho Chehab , Hans Verkuil , Benjamin Gaignard , Dafna Hirschfeld Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Junrui Luo , Yuhao Jiang , stable@vger.kernel.org X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=2300; i=moonafterrain@outlook.com; h=from:subject:message-id; bh=/Cgn2Z+/GhQ6lI+TCHOr+FQEmXhNn+WmP8yMOCGBoj0=; b=owJ4nJvAy8zAJVb4wiKgu++DA+NptSSGrIY6NbW0R/sbpLbrFcQ7OscatP0RN7MrW7yg+4fLv V3fe3QmG3eUsjCIcTHIiimyHC+49M3Cd4vuFp8tyTBzWJlAhjBwcQrARN5LMPyV/RazkIk94+/x mphdDLE/rlnJCrnMOfBxg/m2m6/YVibdZ/jvfvJiVlzAVbmL+3dv6HOsUjoyc0Lnne5VZi67p83 69/8PCwBrn05+ X-Developer-Key: i=moonafterrain@outlook.com; a=openpgp; fpr=C770D2F6384DB42DB44CB46371E838508B8EF040 X-Endpoint-Received: by B4 Relay for moonafterrain@outlook.com/default with auth_id=909 X-Original-From: Junrui Luo Reply-To: moonafterrain@outlook.com From: Junrui Luo A capture format whose width or height is one more than a multiple of 16 makes the FWHT codec write one 8x8 block row, or column, past the end of the capture plane, and read the same block out of the reference frame for P-coded frames. vic_round_dim() is documented to round a frame dimension up so that both the luma and the chroma plane end up a multiple of 8, but it only rounds the chroma plane: round_up((dim) / (div), 8) * (div) For div =3D=3D 2 the result is a multiple of 16 in every case but one: when dim % 16 =3D=3D 1, dim / div is already a multiple of 8 and the macro retur= ns dim - 1. encode_plane() and decode_plane() round the same dimension with round_up(dim, 8), which yields dim + 7 -- one block more than the coded dimension that sized the buffer. On a KASAN-enabled kernel, a 641x360 YUYV P-frame triggers: BUG: KASAN: slab-out-of-bounds in add_deltas+0x450/0xcc0 Read of size 1 at addr ffff888009070800 by task trigger_bin/70 Call Trace: add_deltas+0x450/0xcc0 decode_plane+0x1916/0x3390 fwht_decode_frame+0x173/0x620 v4l2_fwht_decode+0x751/0x1120 device_run+0x6bb/0x1850 Round the dimension itself up to a multiple of 8 * div. The rounding changes only for div =3D=3D 2 and dim % 16 =3D=3D 1, and MAX_WIDTH and MAX_= HEIGHT are both multiples of 16, so the rounded value still fits the advertised limits. Fixes: 3b15f68e19c2 ("media: vicodec: Add support for resolution change eve= nt.") Reported-by: Yuhao Jiang Assisted-by: Claude:claude-opus-5 Cc: stable@vger.kernel.org Signed-off-by: Junrui Luo --- drivers/media/test-drivers/vicodec/codec-fwht.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/media/test-drivers/vicodec/codec-fwht.h b/drivers/medi= a/test-drivers/vicodec/codec-fwht.h index 0eab24020e9e..4b4d39031089 100644 --- a/drivers/media/test-drivers/vicodec/codec-fwht.h +++ b/drivers/media/test-drivers/vicodec/codec-fwht.h @@ -61,7 +61,7 @@ * both luma and chroma components resolutions are rounded up to * a multiple of 8 */ -#define vic_round_dim(dim, div) (round_up((dim) / (div), 8) * (div)) +#define vic_round_dim(dim, div) round_up(dim, 8 * (div)) =20 struct fwht_cframe_hdr { u32 magic1; --=20 2.51.2 From nobody Mon Sep 28 23:12:47 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23DFF41D13F; Sat, 15 Aug 2026 14:56:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786805801; cv=none; b=O+0Y5jRdOxSOLCz7dJvgAYnjIhlU/kKcqD80uYaPLWMYxzLN2r2IiRKpMzjgCQ8P9LNmiU50ml39NwqhgZGI655L3bSPkWXvg8hmuHHls3Zj3TL0c/vjZsECTL3MPEju5vfBQ6wd/BgWf7uDz9mpemyqrDyQt79Dg4hSlXwfpNk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786805801; c=relaxed/simple; bh=3quwtnblckAjT2ds5UWdPCagF676MycNoyOJ/zGFrFA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=XzJ68y+X+Bl4LWGnVxRcFXArPzQ88waaek2pmyOqqeXHGh25Dk2R5JJPDGT/bIPf/XP7bIJlzOGHBpNsK6laXgxW6CNwSAXa8qDIOIAu/uFbOAUnLgEt8Q2aEzUkHZ8a5ci1OSHcR/47jejv+uEteiBjM8S9FAcLQZ6ZErTZ0W4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=EgCTsBV9; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="EgCTsBV9" Received: by smtp.kernel.org (Postfix) with ESMTPS id DFEF0C2BCFA; Sat, 15 Aug 2026 14:56:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1786805800; bh=3quwtnblckAjT2ds5UWdPCagF676MycNoyOJ/zGFrFA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=EgCTsBV96q2ihFKObcUTovF68ovbyQcrFODA9A9n5bLDAzqGDcNank/KFjRnrAoSY 9mYZKH/IgZ2hlj2+b6/ySXLXKHvPS4CCiC1gF6O7mpYreUwaMggbSm36nHE0FBkS2S K8ig3ejDUY0d+Kvbl2ccVv3cqU+tNDnSKx98II0GmwG31DycAArlZNmdKidPwpJ3Rx mffLXxxzEUlwiP08TpbJd8aKn7uBxclmTVbgMQyou4NnjI87IoRGSLRwS8CSFyT7cR pun6iRgZx6AGkYeBYRzr3Tw9CSF1AMlBGCCF/i+UZN/Cz/DtyFJq9Tccr9ZM3QBCA6 cP/wy/W0GSXlw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C216CC5DF6E; Sat, 15 Aug 2026 14:56:40 +0000 (UTC) From: Junrui Luo via B4 Relay Date: Sat, 15 Aug 2026 22:56:37 +0800 Subject: [PATCH 3/4] media: vicodec: clamp visible dimensions on S_FMT to coded bounds Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260815-vicodec-fixes-v1-3-12101572eba7@outlook.com> References: <20260815-vicodec-fixes-v1-0-12101572eba7@outlook.com> In-Reply-To: <20260815-vicodec-fixes-v1-0-12101572eba7@outlook.com> To: Mauro Carvalho Chehab , Hans Verkuil , Benjamin Gaignard , Dafna Hirschfeld Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Junrui Luo , Yuhao Jiang , stable@vger.kernel.org X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=2305; i=moonafterrain@outlook.com; h=from:subject:message-id; bh=xw+SM0NXC9kyoKDEQVYsDRhjEC52Kz8+luL+uJhz9SQ=; b=owJ4nJvAy8zAJVb4wiKgu++DA+NptSSGrIY6tc5t0XfZ2C7auOxbVPr91OOvv2fP+X1y5uyfL jy3pHLM17R1lLIwiHExyIopshwvuPTNwneL7hafLckwc1iZQIYwcHEKwETa1jL8T1zPKLn76u4U l2tvfv5apnHY/splvr+OzNr+t5p2u3WEpzAy7EjtnTrvrK6c7Fxe6fiyBq9e/fsG3Hp/z1vZLV5 x+OIZZgAcyk2d X-Developer-Key: i=moonafterrain@outlook.com; a=openpgp; fpr=C770D2F6384DB42DB44CB46371E838508B8EF040 X-Endpoint-Received: by B4 Relay for moonafterrain@outlook.com/default with auth_id=909 X-Original-From: Junrui Luo Reply-To: moonafterrain@outlook.com From: Junrui Luo Setting a smaller CAPTURE format on the stateful decoder lets the visible dimensions exceed the coded ones, and decode_plane() then writes past the end of the capture buffer -- a kernel heap out-of-bounds write controllable from unprivileged userspace. vidioc_s_fmt() writes coded_width, coded_height and sizeimage but never touches visible_width or visible_height, which vicodec_open() initialised to 1280x720, so shrinking the coded pair leaves the visible pair larger. job_ready() compares the bitstream header against the stale visible dimensions, so no source-change event fires and decode_plane() iterates over the visible height at a stride derived from coded_width. On a KASAN-enabled kernel, a 1280x720 I-frame decoded into a 640x368 CAPTURE buffer triggers: BUG: KASAN: vmalloc-out-of-bounds in decode_plane+0x1bb8/0x34b0 Write of size 1 at addr ffffc900004b8080 by task trigger_bin/69 Call Trace: decode_plane+0x1bb8/0x34b0 fwht_decode_frame+0x173/0x620 v4l2_fwht_decode+0x595/0xea0 Clamp visible_width and visible_height to the new coded bounds after every S_FMT, maintaining the invariant visible <=3D coded that update_capture_data_from_header() relies on. Fixes: 3b15f68e19c2 ("media: vicodec: Add support for resolution change eve= nt.") Reported-by: Yuhao Jiang Assisted-by: Claude:claude-opus-5 Cc: stable@vger.kernel.org Signed-off-by: Junrui Luo --- drivers/media/test-drivers/vicodec/vicodec-core.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/media/test-drivers/vicodec/vicodec-core.c b/drivers/me= dia/test-drivers/vicodec/vicodec-core.c index 318e8330f16a..36b92f68ac42 100644 --- a/drivers/media/test-drivers/vicodec/vicodec-core.c +++ b/drivers/media/test-drivers/vicodec/vicodec-core.c @@ -1029,6 +1029,11 @@ static int vidioc_s_fmt(struct vicodec_ctx *ctx, str= uct v4l2_format *f) return -EINVAL; } =20 + q_data->visible_width =3D min(q_data->visible_width, + q_data->coded_width); + q_data->visible_height =3D min(q_data->visible_height, + q_data->coded_height); + dprintk(ctx->dev, "Setting format for type %d, coded wxh: %dx%d, fourcc: 0x%08x\n", f->type, q_data->coded_width, q_data->coded_height, --=20 2.51.2 From nobody Mon Sep 28 23:12:47 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E09341D635; Sat, 15 Aug 2026 14:56:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786805801; cv=none; b=H2YsDArs53Jd6iw/c6PP6XVeOSxT/yNGEau5ouf/MuSiKmlSkWG3GXI9aPHvjoSQOrI9Ihv7BlqUafweLxDRS1XeltnidGMp7Ap4Pqhe44V64hwF8y0JamPMzewAUFtMHSAGOOn8UksQcA0ImIW8BJGj2ifXsqfn244+zAWvKvs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786805801; c=relaxed/simple; bh=E7cbrRaKnfEnuB6TrYEj9kr/oZV+HBllIAUlhUhevAs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=SOoPI2VFXDhe5dWBjpaCNt6mDgpe3s7ZnQRwI8IgvjI7EeOV0MaJVoN580hM2AVI/QZUgXtNoiI3cY8mMnsLOyembBz75XrNXJNtp3uqP3/oxF4LqQPFJMK37oz7KGPb3XTTYnsg3AYPeJjaSDmO0rctb+53zwTNRlDxPcojkgM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=flroNpZZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="flroNpZZ" Received: by smtp.kernel.org (Postfix) with ESMTPS id E6746C2BCFD; Sat, 15 Aug 2026 14:56:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1786805801; bh=E7cbrRaKnfEnuB6TrYEj9kr/oZV+HBllIAUlhUhevAs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=flroNpZZeds8B09mO7QvwTXMy28jOQ0Ju9xzui3R0QYzLw8M1XZFeNFpYdp4Bzii6 XUACkU3o47B2AxQn623rEF4KKH+4VDPz/bpqQj3kZE705dfDbdIJlZNqZlIdFMQCWW wW33o28uR9FmOi7h/+JC3Rroz1XRXzL0kuw27W8TS3iAzIC7+jIePqPEWKGfkVz2W2 HbItKN/rV4v+Z4miSSFh3Fv+PIbK9tg2uMfthtA6OJev4tHGxEw/uC1luo55wDoASA vuNw6mPVbEBSNpRNhOkRmDOU+a51/jcX0Hu224y6n0z5zIWoZKiRokBzMaGSkQ182p mfMi1U43Nl9gg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D200FC5DF6A; Sat, 15 Aug 2026 14:56:40 +0000 (UTC) From: Junrui Luo via B4 Relay Date: Sat, 15 Aug 2026 22:56:38 +0800 Subject: [PATCH 4/4] media: vicodec: make encoder CAPTURE dimensions read-only Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260815-vicodec-fixes-v1-4-12101572eba7@outlook.com> References: <20260815-vicodec-fixes-v1-0-12101572eba7@outlook.com> In-Reply-To: <20260815-vicodec-fixes-v1-0-12101572eba7@outlook.com> To: Mauro Carvalho Chehab , Hans Verkuil , Benjamin Gaignard , Dafna Hirschfeld Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, Mauro Carvalho Chehab , Junrui Luo , Yuhao Jiang , stable@vger.kernel.org X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=2745; i=moonafterrain@outlook.com; h=from:subject:message-id; bh=39hTfONQcomxQ23TBhyVDkf40Z5LuSFAHoiUH91wz3M=; b=owJ4nJvAy8zAJVb4wiKgu++DA+NptSSGrIY69f9S97dv0M2N3STy3Zft8LqiJd3Lz3grX9ijw hmzknf1sn8dpSwMYlwMsmKKLMcLLn2z8N2iu8VnSzLMHFYmkCEMXJwCMBHDlwz/rO7x5Nace7V/ 7X/By3KZN7hWZv8MaTzD90bn18z5U1heVTAy7Os44KpTX2Pb+tek3lbsZje7wiLGI8fc9X7/zDw udt2ZEwAzA0ya X-Developer-Key: i=moonafterrain@outlook.com; a=openpgp; fpr=C770D2F6384DB42DB44CB46371E838508B8EF040 X-Endpoint-Received: by B4 Relay for moonafterrain@outlook.com/default with auth_id=909 X-Original-From: Junrui Luo Reply-To: moonafterrain@outlook.com From: Junrui Luo Setting the encoder's compressed CAPTURE format to a smaller resolution than the raw OUTPUT format makes the encoder write past the end of the CAPTURE buffer. For a stateful encoder the CAPTURE width and height are not client-settable; Documentation/userspace-api/media/v4l/dev-encoder.rst specifies them as "ignored (read-only)" on VIDIOC_S_FMT. vicodec only implements half of that: vidioc_s_fmt_vid_out() derives the CAPTURE coded size and sizeimage from the OUTPUT format, but S_FMT on the CAPTURE queue overwrites them. The encoder then takes its geometry from the OUTPUT queue alone, and v4l2_fwht_encode() gets no destination length. Overwrite the requested width and height with the OUTPUT queue's coded dimensions in vidioc_try_fmt_vid_cap(), making them read-only as the interface requires. Fixes: efec9c815e5d ("media: vicodec: pass on enc output format to capture = side") Reported-by: Yuhao Jiang Assisted-by: Claude:claude-opus-5 Cc: stable@vger.kernel.org Signed-off-by: Junrui Luo --- drivers/media/test-drivers/vicodec/vicodec-core.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/media/test-drivers/vicodec/vicodec-core.c b/drivers/me= dia/test-drivers/vicodec/vicodec-core.c index 36b92f68ac42..6d20a749635c 100644 --- a/drivers/media/test-drivers/vicodec/vicodec-core.c +++ b/drivers/media/test-drivers/vicodec/vicodec-core.c @@ -890,6 +890,8 @@ static int vidioc_try_fmt_vid_cap(struct file *file, vo= id *priv, struct v4l2_format *f) { struct vicodec_ctx *ctx =3D file2ctx(file); + struct vicodec_q_data *q_data_out =3D + get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT); struct v4l2_pix_format_mplane *pix_mp; struct v4l2_pix_format *pix; =20 @@ -900,6 +902,10 @@ static int vidioc_try_fmt_vid_cap(struct file *file, v= oid *priv, pix =3D &f->fmt.pix; pix->pixelformat =3D ctx->is_enc ? V4L2_PIX_FMT_FWHT : find_fmt(f->fmt.pix.pixelformat)->id; + if (ctx->is_enc) { + pix->width =3D q_data_out->coded_width; + pix->height =3D q_data_out->coded_height; + } pix->colorspace =3D ctx->state.colorspace; pix->xfer_func =3D ctx->state.xfer_func; pix->ycbcr_enc =3D ctx->state.ycbcr_enc; @@ -911,6 +917,10 @@ static int vidioc_try_fmt_vid_cap(struct file *file, v= oid *priv, pix_mp =3D &f->fmt.pix_mp; pix_mp->pixelformat =3D ctx->is_enc ? V4L2_PIX_FMT_FWHT : find_fmt(pix_mp->pixelformat)->id; + if (ctx->is_enc) { + pix_mp->width =3D q_data_out->coded_width; + pix_mp->height =3D q_data_out->coded_height; + } pix_mp->colorspace =3D ctx->state.colorspace; pix_mp->xfer_func =3D ctx->state.xfer_func; pix_mp->ycbcr_enc =3D ctx->state.ycbcr_enc; --=20 2.51.2