From nobody Sat Sep 21 03:15:35 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A112C761AF for ; Mon, 27 Mar 2023 03:14:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232259AbjC0DOG (ORCPT ); Sun, 26 Mar 2023 23:14:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232208AbjC0DN4 (ORCPT ); Sun, 26 Mar 2023 23:13:56 -0400 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9ACC14C22; Sun, 26 Mar 2023 20:13:45 -0700 (PDT) X-UUID: 5d9403c0cc4d11edb6b9f13eb10bd0fe-20230327 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=5dmmAN53knGP8A38vOJ+8chP7YSe5P6lPPiDtcUaDnc=; b=OjrRAIex9hIVUVhmnj8ikieVk4Vi3v3eqYWTIXrFKj8RzWB2lLxY0y5fl5M2mt08+p6YmkItzlhw4nXpvdCrlK6XTrwl9rYmJudhFgpkDmSRMTq+30HYXAv3r+Fu82Efjmev2X30lJaICWhNmBObRJclhUgKv8JMHs4LbL05y6A=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.22,REQID:7f3e2305-ce9d-4763-8e59-dba96ab0db00,IP:0,U RL:0,TC:0,Content:0,EDM:0,RT:0,SF:95,FILE:0,BULK:0,RULE:Release_Ham,ACTION :release,TS:95 X-CID-INFO: VERSION:1.1.22,REQID:7f3e2305-ce9d-4763-8e59-dba96ab0db00,IP:0,URL :0,TC:0,Content:0,EDM:0,RT:0,SF:95,FILE:0,BULK:0,RULE:Spam_GS981B3D,ACTION :quarantine,TS:95 X-CID-META: VersionHash:120426c,CLOUDID:b498fef6-ddba-41c3-91d9-10eeade8eac7,B ulkID:230327111339RI6GJ9G6,BulkQuantity:0,Recheck:0,SF:38|29|28|17|19|48,T C:nil,Content:0,EDM:-3,IP:nil,URL:0,File:nil,Bulk:nil,QS:nil,BEC:nil,COL:0 ,OSI:0,OSA:0,AV:0 X-CID-BVR: 0,NGT X-CID-BAS: 0,NGT,0,_ X-UUID: 5d9403c0cc4d11edb6b9f13eb10bd0fe-20230327 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 944995957; Mon, 27 Mar 2023 11:13:37 +0800 Received: from mtkmbs13n2.mediatek.inc (172.21.101.108) by mtkmbs10n2.mediatek.inc (172.21.101.183) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.25; Mon, 27 Mar 2023 11:13:36 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs13n2.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.25 via Frontend Transport; Mon, 27 Mar 2023 11:13:36 +0800 From: Moudy Ho To: Mauro Carvalho Chehab , Matthias Brugger , Hans Verkuil , "Ping-Hsun Wu" CC: , , , , , Moudy Ho Subject: [PATCH v7 01/12] media: platform: mtk-mdp3: fix potential frame size overflow in mdp_try_fmt_mplane() Date: Mon, 27 Mar 2023 11:13:24 +0800 Message-ID: <20230327031335.9663-2-moudy.ho@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230327031335.9663-1-moudy.ho@mediatek.com> References: <20230327031335.9663-1-moudy.ho@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Fix overflow risk when setting certain formats whose frame size exceeds a RGB24 with 7723x7723 resolution. For example, a 7723x7724 RGB24 frame: 1. bpl (byte per line) =3D 7723 * 3. 2. Overflow occurs when bpl * 7724 * depth. Fixes: 61890ccaefaf ("media: platform: mtk-mdp3: add MediaTek MDP3 driver") Signed-off-by: Moudy Ho --- drivers/media/platform/mediatek/mdp3/mtk-mdp3-regs.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/drivers/media/platform/mediatek/mdp3/mtk-mdp3-regs.c b/drivers= /media/platform/mediatek/mdp3/mtk-mdp3-regs.c index 4e84a37ecdfc..36336d169bd9 100644 --- a/drivers/media/platform/mediatek/mdp3/mtk-mdp3-regs.c +++ b/drivers/media/platform/mediatek/mdp3/mtk-mdp3-regs.c @@ -4,6 +4,7 @@ * Author: Ping-Hsun Wu */ =20 +#include #include #include #include @@ -428,14 +429,15 @@ const struct mdp_format *mdp_try_fmt_mplane(struct v4= l2_format *f, u32 bpl =3D pix_mp->plane_fmt[i].bytesperline; u32 min_si, max_si; u32 si =3D pix_mp->plane_fmt[i].sizeimage; + u64 di; =20 bpl =3D clamp(bpl, min_bpl, max_bpl); pix_mp->plane_fmt[i].bytesperline =3D bpl; =20 - min_si =3D (bpl * pix_mp->height * fmt->depth[i]) / - fmt->row_depth[i]; - max_si =3D (bpl * s.max_height * fmt->depth[i]) / - fmt->row_depth[i]; + di =3D (u64)bpl * pix_mp->height * fmt->depth[i]; + min_si =3D (u32)div_u64(di, fmt->row_depth[i]); + di =3D (u64)bpl * s.max_height * fmt->depth[i]; + max_si =3D (u32)div_u64(di, fmt->row_depth[i]); =20 si =3D clamp(si, min_si, max_si); pix_mp->plane_fmt[i].sizeimage =3D si; --=20 2.18.0